i agree it would be interesting but apart from the fact that its be harder to measure and automate, theres real alpha in being the best truly async, hands off model/agent, which is what cog has been working on for 2 years now. its not that im opposed to steering or interaction mid task, its just that 1) it mostly Just Works, 2) it doesnt parallelize/scale well, 3) including on proactive agents (
https://docs.devin.ai/product-guides/automations).
see my “semi async valley of death” post. people are pursuing both sides but per bitter lesson only one side scales indefinitely with compute
that said, multistage rollouts and synhetic rubrics (using grpo advantage? see dr tulu paper) somewhat approximate human intervention and interaction, so theres known ways to model that, its just not thaaaat valuable