1. This might mean that Ilya was wrong when he predicted that we’re “moving from scaling the recipe (pre-training) to scaling the thought process (inference)”, if training at scale resulted in a dramatic improvement 2. The costs for other labs to replicate it, means the present is probably the most access most of the public will have to the most advanced models. From this point the cost will diverge into K curves, where the average consumer uses open source level models.
Andrew Curran
@AndrewCurran_
03-29
Three weeks ago there were rumors that one of the labs had completed its largest ever successful training run, and that the model that emerged from it performed far above both internal expectations and what people assumed the scaling laws would predict. At the time these were
Sector:
From Twitter
Disclaimer: The content above is only the author's opinion which does not represent any position of Followin, and is not intended as, and shall not be understood or construed as, investment advice from Followin.
Like
Add to Favorites
Comments
Share
Relevant content
