back
user profile
levkk
2,540karma·349submissions·February 9, 2022
about
Sharding Postgres. <https://pgdog.dev>
lev@pgdog.dev
recent activity (349 total)
comment
We also do that! But it's not well documented at the moment.
comment
Thanks! Glad we made it relatively easy to migrate!
comment
For sure. They should be used for "metadata"-style tables only. High throuput writes should be direct-to-shard.
comment
Yeah good callout. We'll add rendezvous soon enough. Until then, being compatible with Postgres partitions has been advantageous -- while we build everything out, people were able to migrate to P…
comment
Not yet, but actively working on this as we speak.
comment
fwiw, we support cross-shard transactions. They are not magic though, just good old 2pc and a bit of coordination.
comment
Thanks!
comment
This reminds me of college. We had to cite our own papers from prior semesters or risk getting kicked out for plagiarism. I don't miss those days :)
comment
Thanks! Yup...to be expected. If you know, you know, and have the scars to prove it :)
comment
Yup!
comment
Two big ones: 1. Control plane to manage multi-node deployments; "works out of the box" experience to make PgDog easy to deploy and use 2. QoS (quality of service): automatically block bad q…
comment
Good thing we support HA as well: https://docs.pgdog.dev/features/load-balancer/ Load balancer with health checks and failover, works out of the box. :) Battle-tested at thi…
comment
I see you met Sage, our newest founding engineer :) If you're not having fun at your job... In all seriousness, we review every single line of code that goes in and only people who work for PgDog…
comment
Yup. We support schema-based sharding: https://docs.pgdog.dev/configuration/pgdog.toml/sharded_sche... …
comment
Not the place and not the time, but we are building an enterprise edition that "just works" out of the box. Not saying that the open source experience cannot be better - it always can and we…
comment
Always is. Marketing is not our strong suit (only engineers here). We'll get better at it.
comment
> 8 smaller boxes handling ~500GB each and then one medium box for the proxy? That's exactly right. Get in touch (lev@pgdog.dev), happy to help or at the very least tell you what current works…
comment
This is not an extension, it's a proxy! Very different. You can deploy it anywhere already without having to wait for upstreaming or your cloud provider adding support for it. It's one of th…
comment
Yes, except it doesn't have any cross-dependencies on the same volume, so the uptime here should be higher.
comment
OLAP means different things to different people. For us, it's just making sure your admin dashboard keeps working basically: SELECT tenant_id, COUNT(clicks)
FROM users
GROUP BY tenant_id
…
comment
Two schools of thought: 1. Let it crash. Increase the RAM, try again. 2. Page to disk (swap), make it slow but ultimately work. Both have their trade-offs. There is no free lunch here.
comment
The docker compose example is just a demo. I don't know anyone who runs Postgres with docker compose / swarm in prod :) But yes, happy to add volumes so it seems more real.
comment
We do, just buried deep in our blog: https://pgdog.dev/blog/pgdog-vs-citus The same old processes vs. threads debate, plus having the ability to scale the coordinator past a sing…
comment
Depends. Only pooling, very little. Load balancing/sharding needs to parse queries, so a bit more. Could go up to a GB per pod, sometimes more if you have a lot of unique SQL queries (unique by t…
comment
Old benchmark, but still good: https://pgdog.dev/blog/pgbouncer-vs-pgdog
comment
We should add it to brew/apt/etc for sure. Also, we could add it to crates.io so you could do something like `cargo install pgdog`. Distribution, distribution, distribution.
comment
Sorry, out walking the dog (not a pun). I'll post more details in a few.
comment
Getting there! Cross-shard writes do because of 2pc. Reads are eventually consistent.
comment
Crap! Missed opportunity.
comment
Not quite. The performance gain is to bring those features to Postgres! Edit: Performance gains are from having the ability to load balance reads (horizontal scaling for read queries) and scale out wr…