How scaling changes model behavior
Some trends are reasonable to extrapolate, some are not. Even for the trends we are succeeding at extrapolating, it is not clear how that signal translates into different AI behaviors.
Read it here: https://www.interconnects.ai/p/how-scaling-changes-model-behavior
[00:00] How scaling changes model behavior
[05:03] Metaphors for what scaling may solve
[08:45] Short-term scaling is already de-risked
Fig. 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/openai-predictions.webp
Fig. 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/scaling-laws.webp
Fig. 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/situational-awareness.webp
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit
www.interconnects.ai/subscribe
