David Bolin

Apr 18, 2023 | lesswrong.com | Steven Byrnes |Thomas Sepulchre |David Bolin |Roman Leventov

OK, I think I'm now seeing what you're saying here (edit: see my other reply for additional perspective and addressing particular statements made in your comment): In order to predict well in complicated and diverse situations the model must include general-purpose modelling machinery which generates an internal, temporary model. The next token can then be predicted, perhaps, by simply reading it off this internal model.