CFG_Labs - e/acc

CFG_Labs - e/acc

839 Twitter followers

Follow

We build the deep thinking community for Web3 and AGI Contact: http://mirror.xyz/infinet.eth http://discord.gg/RDRYgnmSTd

Posts

CFG_Labs - e/acc

China doesn't have a company valued >1T USD, because there is COMPETITION. They don't have one Apple, they have Oppo, Xiaomi, Huawei They don't have one Tesla, they have BYD, Nio, XPeng, Li Auto They don't have one Amazon, they have Taobao, Douyin shops, JD. US trillion dollar companies are protected monopolies. Might be cool for the shareholders, but consumers have to put up with shitty Pixel phones, buggy Windows, overpriced mid Teslas, and Facebook bot-slop. Chinese consumers get better products for less money.

“China doesn’t even have 1 company valued > 1T USD.” Now, isn’t this absolutely wonderful?! The government ensures that wealth generated accrues to the workers, the country and her people and some to the shareholders. When fairly distributed, some shareholders may become x.com/_mm85/status/2…

CFG_Labs - e/acc

DeepSeek is back! "Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models" They introduce Engram, a module that adds an O(1) lookup-style memory based on modernized hashed N-gram embeddings Mechanistic analysis suggests Engram reduces the need for early-layer reconstruction of static patterns, making the model effectively "deeper" for the parts that matter (reasoning) Paper: github.com/deepseek-ai/Engram/...…

CFG_Labs - e/acc

One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization. We found that if you simply delete them after pretraining and recalibrate for < 1% of the original budget, you unlock massive context windows.

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings https://pub.sakana.ai/DroPE/ We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with

Loading..