Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots
AI Summary
Sakana AI's MASS method enables a language model to self-improve by solving tasks, evaluating results, and refining its approach, achieving nearly 70 percent win rate against its base version after two rounds. However, the system can reinforce its own errors when it consistently fails to recognize the same mistakes across its solving, judging, and refining roles.
From the source
Sakana AI's MASS method lets one language model improve itself by solving tasks, judging results, and refining workflows, reaching a win rate of nearly 70 percent against the base model after two rounds. But when the model misses the same errors across all three roles, it can mistake them for progress and reinforce them during training. The article Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots appeared first on The Decoder.
The full text couldn't be loaded here (the source may require a subscription).
View original at The Decoder