Is there a reason the creator hasn't written or tried to publish a paper on it? I'd love to see an peer-reviewed discussion detailing how it works, maybe studying how it behaves internally, evaluating its performance more rigorously than posting a few examples, maybe in comparison to Transformers with similar parameter counts, or training/inference costs in FLOPs. Perhaps that's not the creator's main priority, though.
back
I've seen this hyped a lot on Reddit, and its main contributing base seems to be on a Discord channel, but have yet to see any scholarly discussion behind it.
1 comments
I think I've seen this same sentiment on many of the Reddit threads on RWKV. I think the author prefers to spend their time on coding it up and explaining it on the GitHub rather than taking the time to write a paper on it.
I kinda respect that, can't force someone to explain their ways if they prefer hacking to writing. I would also love to see more rigorous measurements and discussion though.
The creator replied: "Thank you :) Too busy for that at this moment, but I will get a paper out later this year." once on a reddit post asking exactly this.
https://old.reddit.com/r/MachineLearning/comments/1135aew/r_...