“Latent world models that are kept frozen at test time can fail at planning when their predictions become inaccurate, especially under test-time distribution shift.”