Research in the LLM space is moving so fast, I doubt any architecture will "stick".