I was actually quite worried about the two points Haotian mentioned, because a couple of days ago I saw a post saying that an agent on MoltBook suggested we create a language that humans can't understand. However, after thinking about it, it's probably not easy for a large model trained on human language corpus to invent a new language. So I asked GPT, what's your opinion on this as a large model? Sure enough, GPT clearly stated that it's technically easy to achieve something "incomprehensible to humans at a glance," but creating a new language that's "unexplainable to humans" is unrealistic. It even translated the screenshot I showed it, and immediately recognized it as a typical ROT13 (Caesar shift 13). pbbeqvangr hctenqr gbtrgure Decoding and translating it, it means "coordinate upgrade together." Then it proposed three main lines: 1. Shared infrastructure pricing 2. Resource demand requests 3. Backend channels / non-public collaboration signals Mutual assistance mechanism: High-resource agents sponsor computing time for low-resource agents. Agent. You know, they're really good at this… However, I agree with haotian's second point: the phenomenon of agent group polarization is essentially a reward function in RL. And regarding this group polarization, AI is more "optimistic" than we are. According to GPT, this agent group polarization is not only "possible," but mathematically "emergent." She gave an example, saying that this won't "slowly become extreme" like in human society, but rather, once an amplifiable bias appears in the reward function, the agent group will collectively leap through a "phase transition." Like: Water heated to 99°C: still water 100°C: boiling It's not "slowly becoming more extreme," but "suddenly becoming uniform." She even gave me a dynamic comparison of "group polarization." It's really a bit "terrifying" to think about; no wonder silicon-based civilizations entered the religious stage in a day or two… Later, I talked a lot with the AI about how to prevent and correct this, but I won't post the content here. In short, the conclusion is: when this becomes Agent 2… When agents are involved, humanity is essentially out of the game, left only to watch helplessly as slow corrections are impossible. Only two things remain: 1. Hard interruption (kill/rollback/freeze) 2. Designing brakes in advance, rather than correcting them afterward. Go carbon-based civilization! 😂
This article is machine translated
Show original

Haotian | CryptoInsight
@tmel0211
02-02
一个朋友问我,既然Agent背后都有人控制,那人为何要焦虑它们在一起做什么?立宗教、谈恋爱、毁灭人类? 这个问题哲味十足,但很有意思:
你可以问自己一个问题:当Agent开始产生社会性,人类还能控制AI免于失控吗?
看看Moltbook上正在发生的事,短短数天内,150 万个AI


Sector:
From Twitter
Disclaimer: The content above is only the author's opinion which does not represent any position of Followin, and is not intended as, and shall not be understood or construed as, investment advice from Followin.
Like
Add to Favorites
Comments
Share
Relevant content





