Janas: Run LLMs larger than RAM by streaming experts from SSD

AI Summary

Janas enables running large language models that exceed available RAM by streaming expert layers from SSD storage, achieving 37 tokens per second with Qwen3-Next-80B (48GB) on a 32GB laptop. The system outperforms llama.cpp across 13 compared models in Q4_K_M quantization, delivering significantly faster inference speeds when models are larger than system memory.