back
user profile

levkk

2,540karma·349submissions·February 9, 2022
about
Sharding Postgres. <https://pgdog.dev>

lev@pgdog.dev

recent activity (349 total)
comment
I would say, over 100 Postgres connections, consider getting a connection pooler. Requests per second is highly variable. Postgres can serve a lot of them, as long as you keep the number of server con…
5mo ago·view thread
comment
The current behavior unfortunately is to just let it through and return an incorrect result. We are adding more checks here and rely heavily on early adopters to have a decent test suite before launch…
5mo ago·view thread
comment
So many, where to begin. 1. People don't design schemas to be sharded, although many gravitate towards a common key, e.g. user_id or country_id or tenant_it or customer_id. Once that happens, sha…
5mo ago·view thread
comment
PostgREST is a translation layer: you use HTTP methods, inputs and outputs, to interact with Postgres, the database. It's a replacement for SQL, the language, which happens to also have a load ba…
5mo ago·view thread
comment
1. Yup, we support online resharding, so you don't need to deploy this until you have to. 2. That's right, we broadcast the DDL to all shards in the configuration. If two-phase commit [1] is…
5mo ago·view thread
comment
Absolutely. We do weekly releases here: https://github.com/pgdogdev/pgdog/releases . Each release includes a detailed changelog. I believe you can use an RSS reader if those …
5mo ago·view thread
comment
I think we may have fixed this 3 weeks ago: https://github.com/pgdogdev/pgdog/pull/744 Might be worth another try. If not, a GitHub issue with more specifics would be g…
5mo ago·view thread
comment
I'll need a bit more info about your use case to answer. We use logical replication to move data between shards, with the intention of creating new shards. This is managed by PgDog. We are buildi…
5mo ago·view thread
comment
Not currently, but we can add this. One thing we have to be careful of is to not retry requests that are executing inside transactions, but otherwise this would be a great feature.
5mo ago·view thread
comment
It can be silent, but usually it's loud and confusing because people do something like this (Rails example): user = User.create(email: "test@test.com") SendWelcomeEmail.perfor…
5mo ago·view thread
comment
It shards it as well. We handle schema sync, moving table data (in parallel), setting up logical replication, and application traffic cutover. The zero-downtime resharding is currently WIP, working on…
5mo ago·view thread
comment
Not really, replication lag is generally an accepted trade-off. Sync replication is rarely worth it, since you take a 30% performance hit on commits and add more single points of failure. We will add …
5mo ago·view thread
comment
We have all kinds, it's not specific to any particular sector. That's kind of the beauty for building for Postgres - everyone uses it in some capacity! My general advice is, once you see mor…
5mo ago·view thread
comment
Subms typically, yeah. We measured the average latency between nodes in the same AZ (e.g., AWS availability zone) to be less than one ms, so you need to account for one extra hop and processing time b…
5mo ago·view thread
comment
It feels better now, but we still need to add crash protection - in case PgDog itself crashes, we need to restore in-progress 2pc transaction records from a durable medium. We will add this very soon.
5mo ago·view thread
comment
I'm curious about your reasoning. Jira/Trello etc. are like $10/mo/seat, why bother rewriting them from scratch? You'll spend more in tokens doing so. Same for gmail/goog…
5mo ago·view thread
comment
I think the right way to handle this as a repository owner is to close the PR and block the "contributor". Engaging with an AI bot in conversation is pointless: it's not sentient, it ju…
6mo ago·view thread
comment
One of the many problems PgDog will solve for you!
6mo ago·view thread
comment
More power to you! Good luck!
6mo ago·view thread
comment
Him! That settles the Turing test debate.
6mo ago·view thread
comment
So... I already tell Claude Code to do this. Just run kubectl for me please and figure out why my helm chart is broken. Scary? A little but it's doing great. Not entirely sure why a specialized t…
6mo ago·view thread
comment
This happens routinely every other Monday or so.
6mo ago·view thread
comment
I believe the science, but I've been using it daily and it's been getting worse, noticeably.
6mo ago·view thread
comment
Ok so Arch apparently has an install script that does everything[0]. I tried it the other day and it's pretty flawless, albeit terminal-based so not for everyone I guess. Pacman is _amazing_. Apt…
6mo ago·view thread
comment
I was terrified until it worked. The Postgres "ABI" is relatively stable - the parser only really changes between major versions and we bake the whole code into the same executable - largely…
6mo ago·view thread
comment
That's the experimental feature I was talking about! :) Thank you so much for the kind words!
6mo ago·view thread
comment
Thank you!
6mo ago·view thread
comment
That makes sense. For example, your redis instance will have fixed RAM, so might as well pre-allocate it at boot and avoid fragmentation. Memcached works similarly (slabs of fixed size), except they a…
7mo ago·view thread