abelanger

It can be used as an alternative to dagster or airflow but doesn't have the prebuilt connectors that airflow offers. And yes, there are ways to reuse tasks across workflows, but the docs for that…

Nice, Waka looks cool! I've talked a bit about the tradeoffs with library-mode pollers, for example here: https://news.ycombinator.com/item?id=39644327 . Which isn't to say t…

We'd like to make money off Hatchet Cloud, which is in early access - some more on that here [1] and here [2]. Pricing will be transparent once we're open access. Like I mention in that comm…

I'm assuming specifically you mean Nex functions? Otherwise NATS gives you connectivity and a message queue - it doesn't (or didn't) have the concept of task executions or workflows. Wi…

Really have an axe to grind with this comment...

The underlying queue is very similar. See this comment, which details how we're different from a library client: https://news.ycombinator.com/item?id=39644327 . We also have the c…

This depends on the SDK - both Typescript and Golang support a `registerAction` method on the worker which basically let you register a single step to only run on that worker. You would then call `put…

We did it because there are methods that should be accessed which don't map to `args` cleanly. For example, we let users call `context.log`, `context.done` (to determine whether to return on canc…

Pueue looks cool, it's not an alternative to Hatchet though - looks like it's meant to be run in the terminal or by a user? We're very much meant to run in an application runtime. Like …

To clarify - you're right, this is a long time in a message/event queue. It's not an eternity in a task queue which supports DAG-style workflows with concurrency limits and fairness str…

This reminds me of: https://news.ycombinator.com/item?id=28234057 If you're saying that the scheduling in Hatchet should be a separate library, we rely on go-cron [1] to run cron…

To be clear, the 25ms isn't a guarantee. We have a load testing CLI [1] and the secondary steps on multi-step workflows are in the range of 25ms, while the first steps are in the range of 50ms, s…

Not at the moment - the biggest ask has been Rails, but on the other hand Sidekiq is so beloved that I'm not sure it makes sense at the moment. We have our hands very full with the 3 SDKs, though…

The heartbeat duration (5s) is not the same as the inactive duration (60s). If a worker has been down for 60 seconds, we reassign to provide some buffer and handle unstable networks. Once someone asks…

It wouldn't be suitable for that at the moment, but might be after some refactors coming this weekend. I wrote a very quick scheduling API which pushes schedules as workflow triggers, but it'…

Thank you! > Do you publish pricing for your cloud offering? Not yet, we're rolling out the cloud offering slowly to make sure we don't experience any widespread outages. As soon as we&#…

Yep, we're backed by YC in the W24 batch - this is evident on our landing page [1]. We're both second time CTOs and we've been on both sides of this, as consumers of and creators of OSS…

This isn't built specifically for generative AI, but generative AI apps typically have architectural issues that are solved by a good queueing system and worker pool. This is particularly true on…

You can execute a new workflow programmatically, for example see [1]. So people have triggered, for example, 50 child workflows from a parent step. As you've identified the difficult part there i…

I like that idea, basically the first HTTP request ensures the worker gets spun up on a lambda, and the task gets picked up on the next poll when the worker is running. We already have the underlying …

Appreciate it, thank you! We've spent quite a bit of time in the Celery Flower console. Admittedly it's been a while, I'm not sure if they've added views for chains/groups…

It's very similar - I used Temporal at a previous company to run a couple million workflows per month. The gRPC networking with workers is the most similar component, I especially liked that I on…

Yep, exactly - Gabe has also been thinking about providing per-user signed URLs to task executions so clients can subscribe more easily without a long-lived token. So basically, you would start the wo…

I haven't used pg-boss, and feature-wise it looks very similar and is an impressive project. The core difference is that pg-boss is a library while Hatchet is a separate service which runs indepe…

Thank you, appreciate the kind words! What boxes are you looking to check? Yes, I'm not a fan of the RabbitMQ dependency either - see here for the reasoning: https://news.ycombinator.c…

This should be fixed now, here's the direct link: https://docs.hatchet.run/self-hosting .

I think we might have had a dead link in the README to our self-hosting guide, here it is: https://docs.hatchet.run/self-hosting . The component which needs the highest uptime is our i…

When I started on this codebase, we needed to implement some custom exchange logic that maps very neatly to fanout exchanges and non-durable queues in RabbitMQ and weren't built out on our Postgr…

Each task in Hatchet is backed by a workflow [1]. Workflows are predefined steps which are persisted in PostgreSQL. If a worker dies or crashes midway through (stops heartbeating to the engine), we re…

Done [1]. We'll expand this section over time. There are also definite tradeoffs to our architecture - spoke to someone wanting the equivalent 1.5m PutRecord/s in Kinesis, which we're d…