In late May at the Future of Go Summit in Wuzhen, China, DeepMind’s Go-playing artificial intelligence program AlphaGo won a three-game match over grandmaster Ke Jie and continued its dominance over human players. The Chinese Weiqi Association awarded AlphaGo 9-dan professional status, the highest possible rank for Go players, and Google subsidiary DeepMind announced AlphaGo’s retirement, assigning its project team to other AI projects.
AlphaGo was one of the biggest AI breakthroughs in recent memory because it beat the best human players at a game much more complex than chess and other board games. DeepMind relied on deep learning to program AlphaGo to learn Go in a similar manner to a human. But what makes deep learning deep? Is it an AI concept, or distinct from AI? And how is it different than machine learning? These terms aren’t too technical and are sometimes used interchangeably, so it’s helpful to take a look at how they’re similar and how they differ.
Image credit: Electronics360
Machine learning and deep learning are essentially two different approaches to artificial intelligence. The Venn diagram on this page illustrates this nicely: AI is the broadest concept, machine learning is a subset of AI, and deep learning is a further subset of machine learning. Let’s look a little deeper into each circle.
Artificial Intelligence
Artificial intelligence is intelligence exhibited by a machine. This definition is too broad to be of much use, but AI is generally used to describe when a machine seems to exhibit behavior or functions associated with human cognition, like learning from mistakes or independently solving problems.
AI is the oldest of the three concepts, dating to the mid-1950s and the earliest practical computers. The following strategies can be considered general AI, but not machine learning or deep learning, so they belong in the outermost circle.
- Symbolic logic (rules engines, knowledge graphs): These systems map knowledge so a machine can associate and “understand” facts and other data.
- Evolutionary algorithms: Algorithms inspired by biological evolution. They use repeated application of mechanisms like reproduction, mutation, recombination and selection to determine an ideal answer or solution.
- Bayesian networks: Probabilistic graphs that convert a set of variables into a suspected outcome. For example, a set of medical symptoms could be computed into the probability of having a certain disease.
The mechanisms above allow a machine to appear intelligent, but not learn, “think” or improve on its own. Early AI programs flourished in activities with simple, finite rules like checkers and chess.
Machine Learning
Arthur Samuel coined the term “machine learning” in 1959, shortly after the birth of interest in AI. Samuel envisioned machine learning as the “field of study that gives computers the ability to learn without being explicitly programmed.” Machine learning as we know it arose in the 1980s.
Basic machine learning (ML) uses algorithms to process data, learn from the data, and then make a determination or prediction. In contrast to AI, which needs fully hand-coded instructions to accomplish a task, a machine learning system is fed large amounts of data and uses algorithms to learn the task.
Like AI, though, ML still requires a good deal of hand-coding. Machine learning is used in computer vision and image recognition systems, for example. In this case, programmers write classifiers (like edge detection and shape detection filters) for the system so it can determine where an object starts and stops and determine which shape it takes. These classifiers are then used to develop algorithms so the system can determine if an image is a type of sign or a face.
Another machine learning application is e-mail filtering. This system analyzes incoming messages for specific words or phrases and auto-classifies them into different folders. E-mail system owners can customize these systems by correcting the system when the filter erroneously moves a message to the wrong folder. The system stores that correction and “learns” to more accurately classify messages going forward.
Deep Learning
The AI world experienced a revolution around 2012 with the rise of deep learning. Deep learning is a type of representation learning, meaning that a deep learning system converts a large quantity of raw data into a representation that is then used to learn a task.
It’s fair to say that, if AI and machine learning attempt to model the world, deep learning attempts to model the mind. A common feature of deep learning systems is artificial neural networks (ANN), a computational model loosely based on the biological brain. ANNs involve multiple layers of artificial neurons, as shown in the image here. A machine learning system can be called “deep” if it has at least one “hidden layer” between the input and output layers. As of 2017, neural networks may consist of thousands to a few million neural units with millions of connections.
An artificial neural network. Image credit: John Salatas / CC BY-SA 3.0
Deep learning algorithms use multiplied inputs in the form of large datasets to take guesses as to the product of the inputs. Even when guesses are wrong, a DL system can compare the initial guess to the true answer and gauge how wrong it was, using information about the error to modify future responses. In other words, a deep learning system makes educated guesses after processing large amounts of data, and refines itself based on new knowledge of the right or wrong answer. This is similar to the machine learning of an e-mail filter, but it’s also more complex and subtle.
AlphaGo is a prime example of the power of deep learning. DeepMind “trained” its program by having it compete against itself over and over for years, constantly fine-tuning its neural networks until it could beat the world’s best human Go players. Go is a complex game: an expert game lasts around 150 moves with about 250 choices per move, a game-tree complexity of 10360 (compared to chess’s 10120). With this in mind, it’s easy to see how AlphaGo is impossible without the biologically inspired complexity of neural networks.
After some initial excitement and optimism, machine learning was found to be error-prone and brittle, but deep learning has generated new excitement. AI researchers are now teaching computers to not only act like humans, but think like them as well, which points the way to the exciting prospect of general artificial intelligence on the horizon.
Further Reading
Machine Learning, Deep Learning and AI: What's the Difference?
What Makes Deep Learning 'Deep'
What is Artificial Intelligence, Machine Learning and Deep Learning
