Steven Byrnes

Boston

Featured in: Favicon

substack.com Favicon

ivoox.com

hidesertstar.com Favicon

citr.ca

primepublishers.com

Articles

“Gaming TruthfulQA: Simple Heuristics Exposed Dataset Weaknesses” by TurnTrout

Jan 16, 2025 | ivoox.com | Steven Byrnes

Descripción de “Gaming TruthfulQA: Simple Heuristics Exposed Dataset Weaknesses” by TurnTrout This is a link post.(Explanation. Also I have no reason to think they hate me.)https://turntrout.com/original-truthfulqa-weaknesses --- First published: January 16th, 2025 Source: https://www.lesswrong.com/posts/57k6xNcWtAtsSTcor/gaming-truthfulqa-simple-heuristics-exposed-dataset --- Narrated by TYPE III AUDIO.
“New, improved multiple-choice TruthfulQA” by Owain Evans, James Chua, Steph Lin

Jan 15, 2025 | ivoox.com | Steven Byrnes

TLDR: There is a potential issue with the multiple-choice versions of our TruthfulQA benchmark (a test of truthfulness in LLMs), which could lead to inflated model scores. This issue was analyzed by a helpful post by Alex Turner (@TurnTrout). We created a new multiple-choice version of TruthfulQA that fixes the issue. We compare models on the old and new versions and find very similar performance.
“What Is The Alignment Problem?” by johnswentworth

Jan 15, 2025 | ivoox.com | Steven Byrnes

Descripción de “What Is The Alignment Problem?” by johnswentworth So we want to align future AGIs. Ultimately we’d like to align them to human values, but in the shorter term we might start with other targets, like e.g. corrigibility. That problem description all makes sense on a hand-wavy intuitive level, but once we get concrete and dig into technical details… wait, what exactly is the goal again? When we say we want to “align AGI”, what does that mean?
“On the OpenAI Economic Blueprint” by Zvi

Jan 15, 2025 | ivoox.com | Steven Byrnes

Table of Contents Man With a Plan. Oh the Pain. Actual Proposals. For AI Builders. Think of the Children. Content Identification. Infrastructure Week. Paying Attention. Man With a Plan The primary Man With a Plan this week for government-guided AI prosperity was UK Prime Minister Keir Starmer, with a plan coming primarily from Matt Clifford. I’ll be covering that soon. Today I will be covering the other Man With a Plan, Sam Altman, as OpenAI offers its Economic Blueprint.
“Lecture Series on Tiling Agents” by abramdemski

Jan 15, 2025 | ivoox.com | Steven Byrnes

Descripción de “Lecture Series on Tiling Agents” by abramdemski For my AISC, I'll[1] be presenting more details about the research every Thursday for approximately the next three months. If you are interested in listening in, here is a calendar link. ^Maybe there will be guest speakers at some point, EG, the AISC mentees. The original text contained 1 footnote which was omitted from this narration.
“Inference-Time-Compute: More Faithful? A Research Note” by James Chua, Owain Evans - LessWrong (30+ Karma) - Podcast en iVoox

Jan 15, 2025 | ivoox.com | Steven Byrnes

Figure 1: Left: Example of models either succeeding or failing to articulate a cue that influences their answer. We edit an MMLU question by prepending a Stanford professor's opinion. For examples like this where the cue changes the model answer, we measure how often models articulate the cue in their CoT. (Here we show only options A and B, rather than all four.) Right: Inference-Time-Compute models articulate the cue more often.
“We probably won’t just play status games with each other after AGI” by Matthew Barnett

Jan 15, 2025 | ivoox.com | Steven Byrnes

Descripción de “We probably won’t just play status games with each other after AGI” by Matthew Barnett There is a view I’ve encountered fairly often,[1] which can be summarized as follows: After the widespread deployment of advanced AGI, assuming humanity survives, material scarcity will largely disappear. Everyone will have sufficient access to necessities like food, housing, and other basic resources. Therefore, the only scarce resource remaining will be “status".
“Voluntary Salary Reduction” by jefftk

Jan 15, 2025 | ivoox.com | Steven Byrnes

Descripción de “Voluntary Salary Reduction” by jefftk Until recently Ithought Julia and I were digging a bit into savings todonate more. With the tighter funding climate for effective altruismwethoughtit was worth spending down a bit, especially considering that ourexpenses should decrease significantly in 1.5y when our youngeststarts kindergarten.
“Predict 2025 AI capabilities by Sunday” by Jonas V, elifland, Sage Future

Jan 14, 2025 | ivoox.com | Steven Byrnes

Descripción de “Predict 2025 AI capabilities by Sunday” by Jonas V, elifland, Sage Future Until this Sunday, you can submit your 2025 AI predictions at ai2025.org. It's a forecasting survey by AI Digest for the 2025 performance on various AI benchmarks, as well as revenue and public attention. You can share your results in a picture like this one. I personally found it pretty helpful to learn about the different benchmarks, and also to think through my timelines estimates.
“Get your math consultations here!” by Gurkenglas

Jan 14, 2025 | ivoox.com | Steven Byrnes

Descripción de “Get your math consultations here!” by Gurkenglas This post is mostly an ad to make people click the https://calendly.com/gurkenglas/consultation link starring in my Would you like me to debug your math? post. But writing just an ad feels icky to me, so first: The advice that was most useful to most people was on how to use (graphical) debuggers. The default failure mode I see in a programmer is that they're wrong about what happens in their program.