Item — HN Reader

Wow, this is a great topic. I don't really have specific suggestions, but I'd like to contribute some thoughts on the matter.

Monetizing anything isn't inherently problematic; the challenge lies in defining what should be paid for and what should be offered for free.

In the realm of open-source products and SaaS, the common practice is to provide free self-hosting options while charging for cloud hosting or enterprise-specific features, such as access control and authentication integrations.

However, the landscape becomes significantly more challenging for LLMOps (assuming you are still focusing on training as a major aspect of your business, which can be categorized as LLMOps).

Historically, there haven't been many success stories in this area (with exceptions like wand.ai, which focusing on tracking experiments). I believe this difficulty arises from the largely ad-hoc nature of training and fine-tuning processes, making standardization a challenge, coupled with the infrequency of these tasks.

That being said, training/finetuning is a valuable technique. However, transforming it into a company that offers products is really challenging. Successful examples in this realm typically depend heavily on solution customization or consulting-oriented business models.

1 comments

Thanks for the points! I agree monetization in the LLM Ops space is hard and complex. Agreed fully on customizing solutions or consulting.

Yep self hosting solutions like Redhat, or DBs like MongoDB or Gitlab's dashboard style approach could work - the issue is now as you mentioned we offer training and finetuning.

We do plan to offer inference as well, plus the data gathering process, and the final prompt engineering side - but we thought why not have a shot?

It's possible best to make a training and inference platform - maybe some sort of personal ChatGPT training for the public - everyone can train their own personal ChatGPT not via ChatGPT's in context learning or RAG, but coupled with actual fast 30x finetuning, a personal bot can truly be possible.

Thaks for the suggestions!

You have companies that are spending good money on fine-tuning and will start spending money on fine-tuning. It seems like it would almost be easier to just go directly to these companies by looking at their blog posts--they're telling you that they're doing it in some way or another. I know Plaid and friends are doing it.

It's costing them x. you can shave y off. you can get improvements to market faster and cheaper.

Interesting points! I shall try this with my bro!!

I was thinking along the lines of say the cost of A100s or H100s * electricity cost and engineering costs then how much we save, and some discounting factor.

I think the time savings will be more appealing.

It allows for fast iteration and shorter go-to-market, which can generate virtually infinite value, as opposed to saving electricity, which is a limited game.

Fair point - I forgot to mention the time savings LOLL!!!

You may want to look sideways to companies such as hedge funds. They have DNN teams and experiment with LLMs, you may find interesting optimisation opportunities with such teams. Charge according to opportunity that you open up, not electricity saved!

Interesting! Hedge funds - very interesting.

Oh no yep your right on time saved and what opportunities it gives them not just the electricity and capital costs :))

You can now experiment 30 different models instead of 1 - if you have 100 GPUs, we magically made it 3000!