Update/Let's talk about Geoffrey Hinton and the Nobel Prize in Physics today. Let's go a little further.
Hinton is the undisputed "father of deep learning" in the field of artificial intelligence. His fundamental contribution comes from the theory of the backpropagation algorithm, which is widely used in deep learning. Last fall in San Jose, I was first attracted when I saw Hinton being interviewed on YouTube. Hinton's appearance is unforgettable. He looks very much like Pinocchio in "The Adventures of Pinocchio", especially the pointed nose of the little wooden puppet. At that time, the whole world knew about OpenAI and ChatGPT. But apart from the AI circle, few people knew Hinton's name.
I joked about Pinocchio with the old man.
An innocent and curious little wooden puppet who wants to become a real person. Similarly, Hinton wholeheartedly wants to make artificial intelligence a reality, and has also been ridiculed. People familiar with the history of computer development should know that the road of artificial intelligence has been bumpy, with ups and downs in technology investment, and practitioners have been mocked all the way. If you look at the way Hinton speaks, the look in his eyes and his expression, there is a touch of Pinocchio's elegance.
The Nobel Prize notification call came from Sweden at 2 a.m.
The voice on the phone from Stockholm asked, "Where are you?" "I'm in a cheap hotel in California, where the network environment is not very good." Hinton's answer was a bit embarrassing. At this moment, how do you feel about getting the Physics Prize? Hinton said he thought it was a prank joke, and then used the word "dream amazing" to describe his surprise.
"How do you describe yourself?
Are you a computer scientist or a physicist trying to understand biology?"
Hinton did not give an answer in the two simple choices.
From my observation over the years, those who directly give answers in the choice questions provided by others are not real experts.
No matter how high the academic qualifications, it is useless. Many of us are very used to doing exercises, and once we graduate, we enter a state of weightlessness, with a sense of floating and powerlessness. Because people have reached a stage where it is not about doing exercises, but about finding their own way. In reality, most of the initial problems preset by others are often metaphysical. Moreover, they just ask casually without much thought.
Hinton said, "I have been thinking about how the human brain works all my life. When I tried to understand how the brain works, I found a technique that worked surprisingly well."
This answer is very beautiful.
Hinton's lifelong efforts and perseverance have made artificial intelligence a reality. When he was young, Hinton studied at the University of Cambridge, trying to figure out how the human brain thinks, and attempted to major in both physics and physiology, but abandoned both a year later, then switched to architecture, and then abandoned it again. A year later, he even switched to philosophy, but dropped out two years later due to conflicts with his supervisor, and finally obtained a bachelor's degree in experimental psychology. Then, he was unemployed after graduation.
Many people don't know that after graduation, Hinton actually worked as a carpenter for a year. Haha, the Pinocchio in front has been buried here.
Until he heard about a profession called artificial intelligence.
Hinton decisively went to the University of Edinburgh, obtained a doctorate in artificial intelligence, and after graduation went to teach at Carnegie Mellon University (CMU) in the United States, "surprised that everyone here was in the lab." Hinton once complained that in his circles in the UK, people only knew to go to cafes after get off work. Initially, Hinton said he had almost no campus interactions at CMU, and the only exchanges were with a professor from another American university and a statistician who later joined CMU, with whom he could discuss the academic direction of artificial intelligence. Unable to find a way, unable to find like-minded people in AI, in the most boring times, Hinton even read the Russian writer Dostoevsky's "Crime and Punishment".
Hinton published the backpropagation (BP) algorithm theory in 1986. It's not easy to explain.
In a nutshell, BP provides the ability to train deep learning on large datasets, whether it's image recognition, speech recognition or natural language processing, and can even provide the generalization capability of Transformer large models to unseen data scenarios. Today, the closed-source ChatGPT from OpenAI and the open-source LLaMA from Meta are both based on the Transformer architecture. Through pre-training on a large amount of text data, the backpropagation algorithm can adjust the parameters in the model during this process, so that the model can better capture the statistical patterns of language.
The backpropagation algorithm is definitely a key chapter in the machine learning courses of computer science majors.
I remember that seven years ago, when I read the "Machine Learning" book written by Professor Zhou Zhihua of Tsinghua University (known as the "Watermelon Book" in the circle), there was a special section on the "error backpropagation algorithm" in the "Neural Network" chapter, which is the commonly referred to backpropagation (BP) algorithm. This algorithm is a key technology in neural network training, used to update the network weights through the "gradient descent method" to minimize the network's prediction error.
Compared to the technology hype of personal computers, the Internet, smartphones and cloud computing, Hinton, who has been deeply engaged in the field of computer science, and his pursuit of the artificial intelligence dream, has been sitting on the cold bench for three or four decades.
When he was nearly 70 years old, Hinton saw that the whole world was using the deep learning algorithm he proposed.
Those who are familiar with Hinton must know that Ilya Sutskever, the former Chief Scientist of OpenAI, was a student of Hinton's when he was teaching at the University of Toronto, and can be considered his academic disciple. Hinton highly recognized this student's insight and engineering ability. Hinton told a story that when Ilya was asked to write an API interface for the mathematical calculation tool MatLab to help his team standardize the complex data format, the teacher Hinton said we shouldn't waste research time, this work would take a month, but the student Ilya said he had finished the program that morning.
In 2013, Google acquired the AI startup DNNresearch co-founded by Hinton and Ilya.
It should be said that this company was a spin-off of Hinton's research group, which was conducting machine learning research in the field of image recognition at the time, and Google used this technology to enhance its photo search and other functions.
Hinton is British.
Many people say the British are no longer as vigorous as in the days of the British Empire. This summer when I went to the UK, my friends in London reminded me to be careful of car thieves. Just talking about the two British universities related to Hinton, you'll know we can't be complacent.
Within the University of Cambridge, Trinity College, in theoretical physics research alone, it has produced Newton, Maxwell and Hawking, almost laying the foundation for three important eras of physics. In the field of computer science, the King's College has also produced Alan Turing and Hinton, in addition to the well-known Xu Zhimo. Turing proposed the famous problem of how to test whether a machine has intelligence. Hinton, on the other hand, spent his life solving the problem. In 2018, Hinton and two other AI scientists together won the Turing Award. The Turing of that year was such a genius, but he suffered great mental anguish and took a bite of a cyanide-coated apple and died. Today, the British are starting to print Turing's portrait on the 50-pound banknote.
By the way, many people don't have the concept of the Trinity in European and American culture, and still think it's about the same as Sanlin Heavy Industries.
"Trinity" in Christian doctrine refers to the "Trinity" of the Father, the Son, and the Holy Spirit, which is a purely theological concept. The key universities in the UK, in addition to classrooms and libraries, are also churches, where divine figures emerge.
Hinton found his academic home at the University of Edinburgh.
Edinburgh is so beautiful. The mysterious medieval atmosphere may prompt people to ponder the origin and destination. James Clerk Maxwell, the father of electromagnetism, was also one of the earliest to study mathematics and physics in Edinburgh. A hundred years ago, Gu Hongming, who spoke six foreign languages at Peking University, wore an old-fashioned hat and held a teapot in class, almost opposing the May Fourth Reformists, and he had a doctorate in philosophy from the University of Edinburgh. In those days, Peking University was embroiled in disputes from within the campus to the outside, accommodating both revolutionaries and conservatives like Mr. Gu. Mao Zedong's teacher during his time at Changsha Normal School, Yang Changji, graduated from the Philosophy Department of the University of Edinburgh, and he was very fond of Mao. Later, Yang went to Peking University and arranged for Mao to work in the library. Mao changed China, and his ideological enlightenment came from Yang Changji, who graduated from the University of Edinburgh. Many say it is a bit off-topic for the Nobel Prize to be awarded to computer scientists in physics. In fact, the Nobel Prize has not been the first to recognize the computer field, as Shockley and the Bell Labs team had received it before. In 1958, he was awarded the prize for inventing the transistor, and his contribution directly drove the development of computer chip technology. The Shockley Semiconductor Laboratory should be considered the "Whampoa Military Academy" of Silicon Valley's chip industry, as the scientists were not good at management, and the internal division led to the establishment of Fairchild Semiconductor and Intel by those who left. Moore, who proposed Moore's Law, was an employee of his laboratory and one of the famous "Eight Rebels" of Silicon Valley. Scientist Sutton worked at Google for ten years as a vice president, but ultimately left. It was not due to disagreement, but because he was concerned about the risk of humans being unable to control AI and abandoned his work in the company, leaving the enterprise to freely discuss the risks of artificial intelligence without being constrained by corporate policies, becoming a critic in the field of artificial intelligence. Observing modern history, I find that true scientists, once they enter the business and political spheres, are always full of love and hate. The same divide appears between Ilya and Altman, due to "naivety". During World War II, physicist Oppenheimer helped the U.S. military develop the atomic bomb, but after the war, he spent his life opposing the continued development of the hydrogen bomb. He found that he had used technology to change the world, but this technology could spiral out of control in the hands of some. Oppenheimer had many research achievements that could have won him the Nobel Prize, but how could the Nobel Prize be given to someone who made the atomic bomb? His later years were filled with pain and loneliness. Like Oppenheimer, Sutton also shares this concern, publicly criticizing OpenAI's indifference to the safety of artificial intelligence, as "the greed of capitalism". If we look back, more than a hundred years ago, the science fiction writer George Orwell's "The Time Machine" should have known that we today are already in the science fiction world they wrote about. But how far can humanity go forward today, and why have loss of control, disillusionment, and salvation always been the main themes of science fiction? If I were to write science fiction, the first chapter would be about how, due to a war between two countries, humanity is wiped out by nuclear weapons, and the carbon-based life forms perish, while the silicon-based artificial beings living deep underground use light energy and algorithms to self-upgrade and reproduce, ruling the Earth. The big data we have today has been alienated into the blood and cells of the silicon-based beings. Today's Earth civilization has become prehistoric in the history of the stars. Only a few people board the Noah's Ark and go to Mars, and after the efforts of several generations, they are preparing to return to their homeland. Feeling the script is cliché? I am an ordinary person. Science is the core component of the ladder of human civilization and the leaps and bounds of evolution, without inherent good or evil. But since modern times, the earliest application of science has almost been for military purposes. What is the "heart of science"? I dare not make a definitive judgment. There are many scientists like Sutton. They have a pure heart, strong curiosity, and focus on one thing without being distracted by the outside world. I can feel it. Great scientists, driven by the love of exploring even the smallest things out of curiosity, ultimately come to care about the fate of humanity. The pinnacle of thinking comes from one word: love. There is often tragedy in between. We often say, "Good things take time." But perhaps, great things take a lifetime to grind.
