levkk

Sounds like we both got hit with this one :)

My last big co, we had a team of 10 who's entire job was to sync data from Postgres into Elastic. It would take weeks and fallover regularly due to traffic. If we could have a DB that could do se…

I want to use this right now, but two issues: - Bash script from Internet requiring sudo, no way - VSCode plugin? I don't use VSCode. I'm not switching from Zed (literally built in Rust for …

That's good to know. Don't hesitate to reach out if you ever want to chat about this stuff. My email: lev@pgdog.dev

Thanks!

Interesting. I've been thinking about it as well. I wrote some code to prototype it here: https://github.com/pgdogdev/pgdog/blob/main/pgdog/src/backe…

That's a good question. To echo the comment above/below, we would have to define what a "direct connection" is. In my interpretation (and as the founder, I think that matters becau…

Definitely. I wouldn't be such a big fan of Postgres if this wasn't the case. That being said, if sharding was natively supported and worked well, they would of done it ages ago. All massive…

From an infra management perspective, a multi-TB table is not fun to maintain. If you can comfortably shard before you get there, you'll be happy you did. Caveat being, sharding has to be ergonom…

It sounds like we just need to add a config: cross_shard_queries = off What else am I missing?

Echoing the comment below (above), since we can fingerprint queries using the Postgres parser, we can create an allow list and a more fine-grained ruleset.

Everything is fine. You never have to release your code for anything that's not PgDog itself. If you're running PgDog as a service, anyone who connects to it can ask you for the source code …

Thank you! We used Makara for this at Instacart, but eventually grew to using other languages, e.g. Python, Go, etc. and reinventing the wheel every time was pretty tedious.

Hey Jake! I would love to collaborate. Email me: lev@pgdog.dev. Config management is a solved problem, we can use K8s or any number of CD tools. PgDog config reloading can be synchronized. Best effort…

Small consolation prize is we can generate unique primary keys: https://docs.pgdog.dev/features/sharding/primary-keys/ . I would like to implement cross-shard unique ind…

Check this out and let me know what you think: https://pgdog.dev/blog/multi-tenant-pg-can-be-easy I think there are a few good solutions for multi-tenant safety. We just need erg…

Thanks! I never considered this. What would that look like?

I've been thinking about other algorithms as well, like range-based. We can definitely override the algorithm for hot shards. Ideally, we match what Postgres does with partitions: range, hash, an…

Thanks! I'm curious: > I’d want friction on breaking this boundary Why do you want friction? > implications of joining across shards are not the same That's usually well understood and…

Thank you! Good feedback. The query parser is cached, so if you're using prepared statements or just the extended protocol with placeholders, it's almost free: cost of a mutex + hash lookup.…

It's config driven, so no split brain. All proxies have the same config and deployments are synchronized: 1. pause traffic 2. reload config 3. resume traffic This can be done in under a second. R…

No issues. PgDog is a company, email me if you want to use it internally and we'll work it out. lev@pgdog.dev

Absolutely. Join our Discord, if you'd like: https://discord.com/invite/CcBZkjSJdd

No. If you're using it internally though, there are no issues. You don't have to share anything. If you're using it externally, e.g. building a PgDog cloud service, you'll need to …

I don't think Supavisor actually does sharding.

Thanks! Will do. 15 year journey starts now.

Thank you. Years in the making.