back
user profile
vforno
413karma·78submissions·May 18, 2026
recent activity (78 total)
comment
if you like, colibrì always needs to improve so if you have ideas or anything else you are welcome for pull request issues and also benchmarks!
comment
It’s a good question. In theory MPI could distribute experts across nodes. In practice, for small clusters the added network latency usually hurts more than it helps. Better suited for big clusters w…
comment
Really thanks!
comment
No because I have only 32gb of ram too low
comment
Maybe we can see some integration!
comment
Really thanks!!
comment
In the readme you can see benchmark which everyone with different hardware is running Colibrì, and I have to say I've seen great times! I'm always doing more to improve!
comment
I'd love to! Right now I only have a very consumer-grade computer that I've had fun with! We'll see!
comment
What causes problems is the rewriting in this case are only read while writing is the cache! However, I'm working to improve more and more and make some parts lighter!
comment
Antirez is the number one!thanks really thanks!
comment
Thank you so much, it's true! It all started with this spirit!
comment
I'm trying it, really interesting!
comment
Thanks to you! I will update the model later to make it more and more optimized but you will see everything you need in the readme.
comment
Absolutely yes! With nanoeuler I learned so much by testing every little detail of the project. Every little part you see has been tested and proven several times so that it could be understood and wo…
comment
Yes, because it has many separate kernels instead of aggressive merges like PyTorch (with Torch Compile). Each pass (norm, matmul, residual, RoPE, etc.) launches its own kernel, which increases launch…
comment
Hi, in nanoeuler I use cuBLAS (NVIDIA's super-optimized library) for all matrix multiplications, with the tensor cores in TF32 mode. It's the same thing PyTorch uses underneath, so it's…
comment
Really thanks If you need any help or have any questions I'm here.
comment
Hi, thanks for the comment. Nanoeuler is starting as a study and research project that will obviously improve over time. I'll do my best to make the readme and other things more readable. Thank y…
comment
Most part of trasformer and sft!
comment
Hi, the uploads are one after the other because it was a long, step-by-step research project where I tested the code on another machine. I admit that I'm slowly making up for the commits on all t…
comment
Hi, a couple of hours, not too much! Including sft!
comment
yes yes tested on a 4070 ti 16gb everything worked without problems!