this discussion I had with alex earlier this year re-shaped a lot how I’m seeing these new breed of frontier models that are specialist at coding + multi-agent + long running.
the concepts of compositional generalization and sub tasks being locally in-distribution makes it very very obvious why LLM + harness are doing so well right now.
and also explain why tweaking the harness a bit sometime make a whole set of tasks that were previously unatainable for LLM a breeze.
I strongly believe now that processing tasks into a “coding-friendly” format will be a continuous vector of breakthrough for the months to come.
check out the full interview below and big thanks to everyone that submitted questions was an awesome session!
table of content:
* 0:00 - RLM are compositional generalizers
* 4:40 - overview of what is an RLM
* 10:00 - updated info and experiments on RLM
* 22:40 - Big Boss Alex L. Zhang
* 27:45 - what is “locally in-distribution”?
* 34:00 - do we even need long context inside an LLM?
* 36:50 - yacine is rambling again…
* 39:35 - is locally in-distribution computation enough for scientific breakthrough
* 46:40 - why are transformers poor at compositional generalization
* 52:50 - what is missing in this soup of capability from the transformers side.
* 57:25 - [at gun point] improve the harness or train a new model for compositional generalization???
* 1:03:00 - how early RLM should enter the training pipeline?
* 1:07:00 - how do you validate such a major architectural change btw?
* 1:10:00 - what does Alex thinks about long-running persistent ai agents?
* 1:14:00 - is code enough for compositional generalization across modalities?
* 1:18:52 - how are we doing at the frontier for long horizon tasks?
* 1:22:00 - speculative programmatic tool call (sPTC)
* 1:28:00 - does the RLM impose an abstraction the model should learn for itself?
* 1:32:00 - what’s next for Big Boss Alex L. Zhang (and what’s not next)?
