back
user profile
levkk
2,540karma·349submissions·February 9, 2022
about
Sharding Postgres. <https://pgdog.dev>
lev@pgdog.dev
recent activity (349 total)
comment
I would say, over 100 Postgres connections, consider getting a connection pooler. Requests per second is highly variable. Postgres can serve a lot of them, as long as you keep the number of server con…
comment
The current behavior unfortunately is to just let it through and return an incorrect result. We are adding more checks here and rely heavily on early adopters to have a decent test suite before launch…
comment
So many, where to begin. 1. People don't design schemas to be sharded, although many gravitate towards a common key, e.g. user_id or country_id or tenant_it or customer_id. Once that happens, sha…
comment
PostgREST is a translation layer: you use HTTP methods, inputs and outputs, to interact with Postgres, the database. It's a replacement for SQL, the language, which happens to also have a load ba…
comment
1. Yup, we support online resharding, so you don't need to deploy this until you have to. 2. That's right, we broadcast the DDL to all shards in the configuration. If two-phase commit [1] is…
comment
Absolutely. We do weekly releases here: https://github.com/pgdogdev/pgdog/releases . Each release includes a detailed changelog. I believe you can use an RSS reader if those …
comment
I think we may have fixed this 3 weeks ago: https://github.com/pgdogdev/pgdog/pull/744 Might be worth another try. If not, a GitHub issue with more specifics would be g…
comment
I'll need a bit more info about your use case to answer. We use logical replication to move data between shards, with the intention of creating new shards. This is managed by PgDog. We are buildi…
comment
Not currently, but we can add this. One thing we have to be careful of is to not retry requests that are executing inside transactions, but otherwise this would be a great feature.
comment
It can be silent, but usually it's loud and confusing because people do something like this (Rails example): user = User.create(email: "test@test.com")
SendWelcomeEmail.perfor…
comment
It shards it as well. We handle schema sync, moving table data (in parallel), setting up logical replication, and application traffic cutover. The zero-downtime resharding is currently WIP, working on…
comment
Not really, replication lag is generally an accepted trade-off. Sync replication is rarely worth it, since you take a 30% performance hit on commits and add more single points of failure. We will add …
comment
We have all kinds, it's not specific to any particular sector. That's kind of the beauty for building for Postgres - everyone uses it in some capacity! My general advice is, once you see mor…
comment
Subms typically, yeah. We measured the average latency between nodes in the same AZ (e.g., AWS availability zone) to be less than one ms, so you need to account for one extra hop and processing time b…
comment
It feels better now, but we still need to add crash protection - in case PgDog itself crashes, we need to restore in-progress 2pc transaction records from a durable medium. We will add this very soon.
comment
I'm curious about your reasoning. Jira/Trello etc. are like $10/mo/seat, why bother rewriting them from scratch? You'll spend more in tokens doing so. Same for gmail/goog…
comment
I think the right way to handle this as a repository owner is to close the PR and block the "contributor". Engaging with an AI bot in conversation is pointless: it's not sentient, it ju…
comment
One of the many problems PgDog will solve for you!
comment
More power to you! Good luck!
comment
Him! That settles the Turing test debate.
comment
So... I already tell Claude Code to do this. Just run kubectl for me please and figure out why my helm chart is broken. Scary? A little but it's doing great. Not entirely sure why a specialized t…
comment
This happens routinely every other Monday or so.
comment
I believe the science, but I've been using it daily and it's been getting worse, noticeably.
comment
Ok so Arch apparently has an install script that does everything[0]. I tried it the other day and it's pretty flawless, albeit terminal-based so not for everyone I guess. Pacman is _amazing_. Apt…
comment
I was terrified until it worked. The Postgres "ABI" is relatively stable - the parser only really changes between major versions and we bake the whole code into the same executable - largely…
comment
That's the experimental feature I was talking about! :) Thank you so much for the kind words!
comment
Thank you!
comment
That makes sense. For example, your redis instance will have fixed RAM, so might as well pre-allocate it at boot and avoid fragmentation. Memcached works similarly (slabs of fixed size), except they a…