HenryNdubuaku

Ok we'd be adopting Little Language Model officially haha

Now you can try running the python package directly on your Mac to see proper speedup. WebAssembly has some limitations, running on pure hardware, especially ARM-based to see thousands (which 99% of c…

Please give it a try and let us know!

Haha, my rent is cheap lol

yes, that's what we had in mind while building

Ok, this is genuinely funny, we will fix these as we iterate, thanks lol.

Insane!

Your thinking is correct haha

The problem is the target device, an LLM can't run on an average TV well.

Thanks! Let us know how it goes :)

SO when confused, it gives you a low confidence score as a signal, in which case its recommended to finetune using th python package, train on your mac/PC

You explained it better, we've done a bad job at communicating its nice :(

Yep

Exactly the use cases we designed Needle for!

Thanks!

7 for now, English, German, Polish, French, Dutch, Latin, Italin.

A lot of Needle 1 users deployed on microcontrollers, we couldn't just throw them under the bus only t chase benchmarks, so its a price we are willing to pay :(

Haha, its not a generalist model, cheap devices cannot run LLM in production, so we brig on-device intelligence to them in the size they can. Needle is niche, not a Gemma or GPT replacement.

So TFLite is an inference engine while Needle is a model, which you can port on TFLite btw.

thanks, give the playground a go and let us know how to improve!

Yes, though for better results in production, after creating your tool json, use the provided data synthesis and fine-tuning pipeline. It tunes on on your mac.

Ah, another failure point on our end! So a simple "5 + 7" and "add 5 and 7" works. But to handle ambiguity, the python package ships pipelines to synthesize augmentations and fine-…

That demo was Needle 1 indeed and we are creating the guide for ESP32 now as we speak.

Thanks, we shall improve this for the next release.

Technically, you could do that, but we trained this one from the ground up!

It does better with clearer tool description, but we are taking note of these complaints for future improvements.

Try the following tool description: "Calculate the sum of two numbers. Use for any arithmetic or math question." instead of "Add two numbers." Let me know how it goes, thanks!

around +60% confidence threshold is cool from experiments, the problem is that you gotta test on your own workload, no existing benchmark could honestly paint the full picture, so we exposed the confi…