Boris Kashirin

Jul 29, 2024 | lesswrong.com | Boris Kashirin

I was thinking about how prompt differs from training data in terms of tokenization. If i am to prompt with "solution:" as opposed to "solution: " it seems like it can influence the result, as in training data last token contain some information about next token. If there is token ": T" but my prompt ended in ": " it can be inferred than next token can't be "T[something]". Is this real effect, or I just misunderstand how tokenization works?

May 29, 2023 | lesswrong.com | Wei Dai |Christopher King |Boris Kashirin |Jacy Reese Anthis

Short version: Sentient lives matter; AIs can be people and people shouldn't be owned (and also the goal of alignment is not to browbeat AIs into doing stuff we like that they'd rather not do; it's to build them de-novo to care about valuable stuff). Context: Writing up obvious points that I find myself repeating.