back
user profile

levkk

2,540karma·349submissions·February 9, 2022
about
Sharding Postgres. <https://pgdog.dev>

lev@pgdog.dev

recent activity (349 total)
comment
We also do that! But it's not well documented at the moment.
2mo ago·view thread
comment
Thanks! Glad we made it relatively easy to migrate!
2mo ago·view thread
comment
For sure. They should be used for "metadata"-style tables only. High throuput writes should be direct-to-shard.
2mo ago·view thread
comment
Yeah good callout. We'll add rendezvous soon enough. Until then, being compatible with Postgres partitions has been advantageous -- while we build everything out, people were able to migrate to P…
2mo ago·view thread
comment
Not yet, but actively working on this as we speak.
2mo ago·view thread
comment
fwiw, we support cross-shard transactions. They are not magic though, just good old 2pc and a bit of coordination.
2mo ago·view thread
comment
Thanks!
2mo ago·view thread
comment
This reminds me of college. We had to cite our own papers from prior semesters or risk getting kicked out for plagiarism. I don't miss those days :)
2mo ago·view thread
comment
Thanks! Yup...to be expected. If you know, you know, and have the scars to prove it :)
2mo ago·view thread
comment
Yup!
2mo ago·view thread
comment
Two big ones: 1. Control plane to manage multi-node deployments; "works out of the box" experience to make PgDog easy to deploy and use 2. QoS (quality of service): automatically block bad q…
2mo ago·view thread
comment
Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/ Load balancer with health checks and failover, works out of the box. :) Battle-tested at thi…
2mo ago·view thread
comment
I see you met Sage, our newest founding engineer :) If you're not having fun at your job... In all seriousness, we review every single line of code that goes in and only people who work for PgDog…
2mo ago·view thread
comment
Yup. We support schema-based sharding: https://docs.pgdog.dev/configuration/pgdog.toml/sharded_sche... …
2mo ago·view thread
comment
Not the place and not the time, but we are building an enterprise edition that "just works" out of the box. Not saying that the open source experience cannot be better - it always can and we…
2mo ago·view thread
comment
Always is. Marketing is not our strong suit (only engineers here). We'll get better at it.
2mo ago·view thread
comment
> 8 smaller boxes handling ~500GB each and then one medium box for the proxy? That's exactly right. Get in touch (lev@pgdog.dev), happy to help or at the very least tell you what current works…
2mo ago·view thread
comment
This is not an extension, it's a proxy! Very different. You can deploy it anywhere already without having to wait for upstreaming or your cloud provider adding support for it. It's one of th…
2mo ago·view thread
comment
Yes, except it doesn't have any cross-dependencies on the same volume, so the uptime here should be higher.
2mo ago·view thread
comment
OLAP means different things to different people. For us, it's just making sure your admin dashboard keeps working basically: SELECT tenant_id, COUNT(clicks) FROM users GROUP BY tenant_id …
2mo ago·view thread
comment
Two schools of thought: 1. Let it crash. Increase the RAM, try again. 2. Page to disk (swap), make it slow but ultimately work. Both have their trade-offs. There is no free lunch here.
2mo ago·view thread
comment
The docker compose example is just a demo. I don't know anyone who runs Postgres with docker compose / swarm in prod :) But yes, happy to add volumes so it seems more real.
2mo ago·view thread
comment
We do, just buried deep in our blog: https://pgdog.dev/blog/pgdog-vs-citus The same old processes vs. threads debate, plus having the ability to scale the coordinator past a sing…
2mo ago·view thread
comment
Depends. Only pooling, very little. Load balancing/sharding needs to parse queries, so a bit more. Could go up to a GB per pod, sometimes more if you have a lot of unique SQL queries (unique by t…
2mo ago·view thread
comment
Old benchmark, but still good: https://pgdog.dev/blog/pgbouncer-vs-pgdog
2mo ago·view thread
comment
We should add it to brew/apt/etc for sure. Also, we could add it to crates.io so you could do something like `cargo install pgdog`. Distribution, distribution, distribution.
2mo ago·view thread
comment
Sorry, out walking the dog (not a pun). I'll post more details in a few.
2mo ago·view thread
comment
Getting there! Cross-shard writes do because of 2pc. Reads are eventually consistent.
2mo ago·view thread
comment
Crap! Missed opportunity.
2mo ago·view thread
comment
Not quite. The performance gain is to bring those features to Postgres! Edit: Performance gains are from having the ability to load balance reads (horizontal scaling for read queries) and scale out wr…
2mo ago·view thread