German AI consortium releases Soofi S, an open 30B model that tops benchmarks in both English and German
AI Summary
The German AI consortium has released Soofi S, an open 30 billion parameter model that excels in both English and German benchmarks. However, they discovered that test questions from the GPQA science benchmark were inadvertently included in the training data and have since recalculated results after removing the benchmark from evaluation.
From the source
The German consortium behind the AI model Soofi S has acknowledged in version 3.0 of its tech report that test questions from the science benchmark GPQA accidentally ended up in the training data. The community caught the error by examining the publicly available data. The team removed the benchmark from its evaluation and recalculated all results. The article German AI consortium releases Soofi S, an open 30B model that tops benchmarks in both English and German appeared first on The Decoder.
The full text couldn't be loaded here (the source may require a subscription).
View original at The Decoder