back
user profile
tang8330
102karma·43submissions·April 13, 2016
about
co-founder of Artie (YC S23)
recent activity (43 total)
comment
Great question - there's no Debezium under the hood. Artie has its own Reader and Transfer layers, built from scratch. TOAST columns: Artie has automatic detection built in. If a TOAST column has…
comment
Thanks for the flag. We'll get that fixed!
comment
Artie OSS: https://github.com/artie-labs/transfer
Artie Cloud: https://www.artie.so/ …
comment
On a single unbounded (CPU + mem) Debezium running on a VM extracting Postgres, I was able to clock in about 7-10m/hr. You could increase the # of tasks, but then it'll hinder your DB perf. …
comment
Hi! Yes, our next 2 connectors are going to be S3 and DynamoDB. GitHub Issue: https://github.com/artie-labs/transfer/issues/157 If you'd like to be another design …
comment
Definitely. For staging temporary tables to merge for our Redshfit destination, we're uploading it to S3. We will be creating a S3 destination with TSV, Avro, Parquet format support verily soon.…
comment
For Postgres, we have our own custom snapshotter that is capable of doing parallel snapshots against your read replica and not incur WAL growth. More details here: https://news.ycombinator.…
comment
Thank you so much for the support!
comment
Thanks for the comment! Your comment regarding DDL is interesting. Today, this is what happens: 1/ Column doesn't exist in the destination, we'll create it based on our typing inference…
comment
Hm, perhaps I wasn't being clear, apologies for that. What I am proposing above is ways to provide a view to teams that do not want real-time data while keeping your underlying dataset in real-ti…
comment
Thanks for your feedback! > Something McKinley doesn't address is that it's quite advantageous if the values in your data warehouse don't change intra-day because this lets business …