Item — HN Reader

Ignore the naysayers!

Any article, even the really good ones on HN, while they get positive comments, for whatever reason, always get a lot of negative ones, too...

That is, the negative comments are absolutely unavoidable, even for people accomplishing great things!

I personally think that what you've done is brilliant, absolutely brilliant!

I can't wait to see more in this space...

Brilliant, absolutely brilliant!

2 comments

Very nice indeed. Model weights in RAM blocks distributed all over a big FPGA: should be super helpful at minimizing RAM bandwidth bottlenecks. To say nothing of latency.

But model(s) implemented are clearly too small to be useful as a 'chat partner'. Tried a couple of sentences - replies is just some gibberish coming out.

This really needs a bigger FPGA, or some other application(s) where a tiny LLM does actually useful work. Barring that, generated tokens/sec is kind of a meaningless measure imho.

Obviously at 20k tokens per second your primary goal would be some sort of time series model running at 20KHz for processing sensor data and I'd say the model might even be too big for that. You could probably run a bunch of sensors at once.

I am also very cautious with people who tell me something impossible when I can trust my engineering skills and get a good sense that there is potentially a good outcome. In my experience, it simply means they don’t know how to do it, or are frustrated they couldn’t do it themselves and get into the spotlight.