Old but relevant: if you read the recently-released Kimi K3 paper[0], you'll see that it's heavily based on Kimi Linear discussed here, scaling it up and adding a bunch more things (like native vision and RL improvements).
The main contribution of the K3 paper is Stable LatentMoE. Like some other models it compresses data sent between layers, which puts certain requirements on the router. K3 improves performance by using a more balanced expert selection strategy.
Compared to the Opus 5 "model card", which read like a standard Anthropic set of alignment principles and safety concerns, this presents a plethora of useful technical details that advances the state of the art.
[0] https://arxiv.org/abs/2607.24653
As it's 9 months old and they just had a major model release
The main contribution of the K3 paper is Stable LatentMoE. Like some other models it compresses data sent between layers, which puts certain requirements on the router. K3 improves performance by using a more balanced expert selection strategy.