back

by sixhobbits·6y ago·view on hn ↗
> This is all working fine

Are there any specific reasons that you think Postgres won't be able to scale?

I don't have personal experience solving the problem you're describing but if you do choose to use something other than Postgres I think Apache Kafka might be interesting for some parts of your infrastructure at least.

Segment [0] has written quite a lot about their tech in a similar space.

[0] https://segment.com/blog/the-10m-engineering-problem/

2 comments
Cloudflare documented their PostgreSQL to ClickHouse journey here: https://blog.cloudflare.com/http-analytics-for-6m-requests-p...

It covers a lot of the scaling bottlenecks in their original PG-based pipeline and how they eliminated them.

(Disclaimer: I work on ClickHouse.)

I think you are right, I'm kind of looking for a technical challenge where I actually should just improve the current setup. Thanks for the link!