Sakana AI

Sakana AI

19,138 Twitter followers

Follow

We are a Tokyo-based R&D company on a quest to create a new kind of foundational AI model based on nature-inspired intelligence. https://sakana.ai/careers

Posts

We are pleased to announce a strategic investment from Citi! sakana.ai/citi This milestone marks Citi’s first such investment in a Japanese company. The investment reflects their high regard for our advanced technical capabilities and our proven track record of implementing AI within the financial sector. We are focused on developing new enterprise-grade AI solutions using nature-inspired intelligence. Our goal has consistently been to bridge the gap between cutting-edge research and practical business applications. Building on our work developing highly specialized AI agents for financial domains, we are ready to take the next step. Through this partnership, we aim to accelerate our international expansion and drive innovation in global financial services, originating from Japan.

Introducing RePo: Language Models with Context Re-Positioning Website: pub.sakana.ai/repo Paper: arxiv.org/abs/2512.14391 Standard language models process information as a rigid linear sequence where the only signal for structure is a fixed token index, forcing them to treat physical proximity as semantic relevance. Cognitive Load Theory suggests this is inefficient. Just as humans struggle when key facts are buried in noise, models waste finite capacity managing disorganized inputs instead of focusing on deep reasoning. RePo breaks this bottleneck by allowing models to actively reorganize their context. Instead of using a fixed index, our module learns to assign positions based on content relevance. This lets the model dynamically pull relevant distant information closer and push noise away, effectively reshaping the attention geometry to match the problem structure. This flexibility yields significant gains in robustness. RePo outperforms standard encodings on noisy contexts, structured data, and long-range dependencies while maintaining competitive general performance. It represents a step toward models that intelligently curate their own working memory rather than passively accepting input order.

-- END --