AMD acquires Taalas, a startup that bakes AI models directly into silicon
AI Summary
AMD has acquired Canadian startup Taalas, which specializes in embedding AI model weights into silicon chips, enhancing inference speed. A demo chip from Taalas achieved over 16,000 tokens per second per user while running Llama 3.1-8B.
From the source
AMD is buying Canadian startup Taalas, which hard-codes model weights directly into inference chips. That makes them extremely fast but locks each chip to a single model. A demo chip hit over 16,000 tokens per second per user running Llama 3.1-8B. Google is reportedly working on a similar approach for Gemini. The article AMD acquires Taalas, a startup that bakes AI models directly into silicon appeared first on The Decoder.
The full text couldn't be loaded here (the source may require a subscription).
View original at The Decoder