back

by levkk·3y ago·view on hn ↗
I used pg_repack at a very large scale, and one issue I ran into was write amplification. Since it used triggers to keep a log of rows changed, writes during repack were 2x normal.

pg_squeeze use of replication slots seems cool because it just tells Postgres to save WAL it already wrote, no 2x amplification.

1 comments
Good point.

I wonder if pg_squeeze would work well in a large-scale heavy-writes case. There is a very sad and hard limit of single-threaded walsender that can saturate a single core (depends on CPU and schema / writes complexity; for example, I saw systems where it happened at ~2 WAL/s, or roughly 1-2 TiB of WAL data generated per day). So at some scale, the process might not converge.

Also, we apply changes in the same system, there are 2x writes anyway – though, it happens asynchronously with pg_squeeze, unlike with triggers (though, that "delta" table pg_repack temporarily writes to, it's very light in terms of indexes, and we do only INSERTs, so this amplification is not 2x in terms of IO and timing overhead, right?)