Deep Consequences
Why Artificial Neural Networks are Good for Language Technology and Good for Science
August 2015
I’d like to know how humans learn language and use it to share information. I also want to build computational models that replicate this ability. So I am trying to be a language scientist and a language engineer.
This blog is about how these two ambitions interact. I will try to explain why — when it comes to language technology — science and engineering are mutually complementary. Along the way, I aim to dispel the long held view that language science and engineering should be treated as independent endeavours. Separating language science and language engineering is a mistake, for both language scientists and language engineers.
The emergence of artificial neural networks as a tool for processing and understanding massive data sets should be an important catalyst behind the greater integration of language science and engineering. That neural networks are good for engineering is incontrovertible. They have advanced the state-of-the-art in various tasks that require learning from large datasets, including computer vision, character and handwriting recognition and speech recognition. And the range of practical language processing tasks on which neural networks beat the alternatives is also growing quickly.
But how can neural networks bring science and engineering closer together? For a long time they were considered plain bad for science because they are ‘black boxes’. In other words, they might be good at predicting some natural data, but because so few of the design details of the trained model were specified by humans who designed the model, we never really know how they do this prediction. Consequently, when we use them we can’t learn anything about the natural phenomenon that the data pertain to.
Given that neural networks are much better understood now than a few years ago, I suspect the number of people who agree with this reasoning is dwindling. In particular, there are now numerous effective means for interrogating and visualising neural nets to better understand the knowledge they encode, and how this helps them perform their objective (here, here and here, for a start). However, for me, that’s not even the most important reason why the ‘neural nets are bad for science’ claim is wrong. The main reason neural nets are good for science is that they learn in a comparatively unbiased and objective way from raw data reflecting natural phenomena. Therefore they are resistant to (most of) the subjectivity associated with human data processing and analysis.
In the context of language science this is especially important. Because language data is particularly human, it is very hard (if not impossible) for humans themselves to objectively analyse it.
This blog is in four parts. In the first part, I’ll explain why language science, perhaps more than any other science, needs some good news right now. The idea that it is worth studying language in a scientific way (with typical scientific objectives etc.) is actually not well established, and certainly not universally accepted. And even when people try to study language scientifically, ghosts from the past keep clouding their objectivity. So any help from new techniques, methods and researchers with fresh perspectives is great news for language science.
In the second part, I’ll show some unfortunate consequences of studying language in an unscientific way. Attempts to build technology that understands and processes language have been greatly influenced by nonscientific research on language, even with no clear evidence that it helps. Who said science and technology go hand in hand?
In the third part, I’ll explain a bit about how neural networks have (relatively) recently been applied to large-scale language data. In many cases where there is a lot of data to learn from, these models are the best way to get a machine understanding language a bit like a human does. Maybe that shouldn’t be a surprise. One feature of neural nets is that they combine multiple layers of (objective) data processing and analysis before making decisions or predictions. It’s no great shock to me that a model whose structure is determined by an objective analysis of the data (i.e. scientifically) beats one whose structure isn’t.
Finally, in the fourth part, I’ll show how these advances in (language) technology can also be hugely beneficial for (language) science. By creating models whose objectives are to do things with language that humans find easy, and then allowing them to learn from lots of raw data (even if not as much as humans get), we can for the first time get amazing insights into the nature of language and how humans learn and use it.
Linguistics and Science: an Awkward Relationship
A lot of research in linguistics in the second half of the twentieth century was characterised by symbols and mathematical-style notation. This notation, popularised by, among others, Noam Chomsky, made linguistics look like science. However, if a science is an academic field that follows the scientific method, then much of 20th Century linguistics was not a science. Chomsky certainly didn’t believe in testing linguistic hypotheses with experiments. He thought introspection on the part of the researcher was sufficient, saying things like
and
Unfortunately for those interested in studying language scientifically, Chomsky was very influential. He convinced linguists to adopt new theories and pseudo-scientific notation, while simultaneously dissuading anyone from checking them empirically. This is like a populist leader coming to power and immediately pronouncing that the independent judiciary is actually a waste of time and money, given the current state of the country.
Did nobody think this was a bit fishy? Well, my guess is that linguists loved the new-found respect (and funding) associated with having blackboards covered in maths-like notation (especially in a humanities faculty). It’s also very hard to prove people wrong if they don’t believe in scientific validation, which is ironic given Chomsky’s non-academic views on the dangers of established power structures.
Anyway, whatever the cause, linguists around the world began to adopt a Chomskian idea of language. For his part, Chomsky wrote a set of rules that attempted to determine which combinations of nouns, verbs, adjectives etc. constitute acceptable sentences in all of the world’s languages. These rules were quickly developed and extended by others. Chomsky in his writings often mentioned people like Newton or Einstein, which gave the impression that linguists writing the rules of grammar was like physicists discovering the laws of physics.
But there‘s a big difference. The theories of Newton and Einstein were evaluated by how well they predicted the outcome of experiments. Newton’s laws could explain some of the observed data; later, Einstein’s could explain a little more. For Chomsky, introspection was enough.
As time passed (and scepticism grew), there were some attempts to validate Chomsky’s ideas with experiments. But Chomsky’s proposals refer to abstract linguistic notions that are not manifested in physical reality, so it is very hard to test them objectively.
For instance, to test a rule that a certain sequence of nouns, verbs, adjectives etc. is acceptable but that a different sequence is not, we probably want to take sentences that exemplify both cases and measure how people process or interpret them. But even finding sentences that exemplify the two sequences is a hugely subjective exercise. Categories like noun, verb etc. (parts-of-speech) are not pre-defined by a language authority, and are frustratingly hard to pin down once the full diversity of the world’s language is considered. For English, after detailed instructions, native speaking university graduates schooled in western linguistic conventions still disagree on the tag for almost 10% of English words. This makes it a frustratingly subjective exercise to verify whether or not a given sentence conforms to Chomsky’s predictions. Perhaps for this reason, nobody has ever managed to successfully quantify what proportion of language data his theories explain (you know, like a statistic). Einstein would not be impressed.
Of course, there is some subjectivity in almost all science, and natural categories cannot always be ‘defined’. In a biological theory about hearts it might not be possible to define where a heart stops and the rest of an organism starts, or whether an organ in an organism is a heart or not. Any particular (subjective) classification of hearts might cause results to favour one theory over another. That said, over time, widespread experimentation smooths out the effects of this subjectivity. And the theories that ultimately prosper are those that explain the highest proportion of (everyone’s) empirical observations.
But forgetting all of that, to me it just doesn’t seem likely that a researcher (or even a team of researchers) could ‘discover’ a set of rules that describe the world’s languages. The number of external (cultural, geographical and inter-personal) and internal (~80 billion neurons) factors or variables that language depends on is enormous. I’d be similarly sceptical if someone told me he had discovered the rules of the world’s weather, by paying particular attention to it. Or if someone told me he had discovered a pen that never runs out.
We are a long way from being able to connect the causal interactions between masses of subatomic particles (in the cells of the brains of interacting humans) with effects at a higher level of analysis (like the behaviour of those humans). So it seems to me we should think of language, and cognitive sciences in general, as complex phenomena (like climate science or financial economics), rather than law-based systems (like those found in chemistry or physics). That’s not to say that the physical properties of the brain do not constrain these possible interactions and shape the nature of language. But, the fact that the earth’s physical geography constrains the weather doesn’t make scheduling a wedding in the UK much less of a lottery
Science and Mythology in Language Technology
Why am I harping on about Chomsky when other language tech researchers are talking about new stuff like twitter, or singularities? After all, most researchers in these fields acknowledge that he is sort-of old news. Nevertheless, Chomsky’s ideas still pervade both cognitive science and language technology. And it’s currently Summer 2015.
Some of this influence is undeniably positive. Chomsky promoted an approach to psychology sometimes called cognitivism. He argued that we will not understand how humans behave without trying to understand how the brain works. That might seem obvious, but at the time it was not obvious to many psychologists. And while Chomsky’s particular method of understanding how the brain works (by thinking for a while and then writing down how the brain works) doesn’t really meet the standards of science, the influence of his cognitivism shifted focus (and money) towards cognitive science, cognitive and computational neuroscience, AI and other research fields whose importance is now clear.
However, there is a downside. It is often said that engineers and those working in language technology are sceptical about the benefits of integrating linguistic research into their systems. Nevertheless, without considering the influence of Chomsky it’s hard for me to understand why so much of language technology research in the last 40 years has focused on parsing. Parsing is the enterprise of turning sentences (like the boy with red shorts kicked the ball and scored a goal) into tree-like structures like this one.
The task of mapping sentences to tree structures is a hard problem to define and a hard thing to do automatically. Thanks to a huge investment of effort we now have numerous automatic systems that are very good at it (at least for certain languages). However, this came at the cost of lots of time spent by lots of smart people. And many graduate students in linguistics had to take courses on how to construct such trees consistently to produce the training data needed to train models to do it automatically. That’s a big investment. But how did we know it was the right thing to do?
Let’s put this another way. Assume a language understanding system (like question answering, translation, search etc.) is any engine that takes language input and responds with some behaviour (see the diagram below). This behaviour might involve generating language (as with machine translation) or some other response (as with a voice-controlled phone system) — it’s not really important. On the way to the desired behaviour the system will probably have to transform or represent the input in some way other than simple plain text. That’s pretty uncontroversial. But what is the evidence that representing the input with a tree-like structure is necessary (or even helpful) for getting the desired behaviour?
Of course, if you ask around it is possible to find examples of language understanding systems (English to Chinese translation, for example) where the state-of-the-art model does rely on such a representation. But that does not mean a better system could not be constructed without it. And, more tangibly, any systems (like English-to-Chinese translators) that need a parser necessarily rely on a lot of expensive annotation.
Tree-like representations for Chinese and English designed by trained annotators almost certainly encode connections and delimitations that correlate to some degree with the semantics of the sentences (as defined by neural processes or activations, say). So it is unsurprising that applying a parser might work better than doing nothing. But is this really the best use of expensive and highly educated humans if our goal is to build the best possible system? There are lots of other ways in which humans could label those sentences that might also help to improve the performance of systems. And, on the scientific side, does all the tree stuff have anything to do with what a human brain does when it solves the same problem?
Despite the influence of Chomsky on research in language technology, syntax and other linguistic formalities don’t actually form part of the most widely used language technologies, like Google search or speech recognition, a fact sometimes bemoaned by linguists. It’s been suggested that this is because people working in technology don’t understand the subtleties of language and don’t see the importance of getting language completely right. But I don’t think that’s the reason at all.
If adding linguistics to a language understanding system actually worked — i.e. led to measurable improvements in how they perform — then tech investors and Buzzfeed journalists would be all over it, as would other engineers. The fact that it doesn’t shouldn’t be a surprise though. Linguistics would help language technology if it explained important factors of variation in language data. There is no reason to expect it to do so, however, if linguistic ‘facts’ have not been tested for how well (in terms of statistics) they reflect actual language.
A Simple Neural Language Model
The recent achievements of Deep Learning (the name given to neural networks trained on lots of data) have undeniably changed AI, and an increasing number of these improvements relate to language technology. But even trying to look beyond the hype, I still think deep learning is still great news for both language science and language technology. To see why, it’s best to look at how neural networks (of varying depth) have been applied to some specific language engineering problems.
Perhaps the first application of deep learning to large-scale language engineering came in 2001. Yoshua Bengio and collaborators applied a neural network to the established problem of language modelling. Language modelling is the task of assigning probabilities or likelihoods to sequences of words (or sentences). For example, in English the sequence of words it’s raining in London is likely to occur but the sequence it’s in sunny London is less likely. A model that knows whether sequences are likely or not can be used to improve any systems that produce language as output, like Google Translate or Cortana.
The obvious way to attack this problem is to simply check huge corpora of English text. If a sentence occurs many times, we conclude that its probability is high, and if it occurs fewer times its probability is lower. However, the statistics of language mean that, even with today’s massive corpora, many sentences that we are interested in might never occur in our dataset. So previous methods used to break sentences down into smaller chunks, or sequences of words, and see how often they occur in the corpus, before merging these statistics together to get an estimate for the whole sentence.
Bengio designed a model that doesn’t explicitly count how often sequences of words occur in a corpus, but instead attacks the problem much more like a human would (intuitively at least). If I gave you the word sequence the pig rolled over in the… you could make a decent guess at what the next word is likely to be. But it seems unlikely to me that you would do it by thinking ‘well I’ve read a lot of books and whenever I see the words over in the together the next word was puddle 7 times and mud 9 times’. I don’t know how the brain works, but it is certain to be very inefficient to store an explicit record of these sorts of statistics for all the bazillions of possible different word sequences in a language. That’s more of a fact about physics than brains.
When I try to solve a language modelling problem, my thought process seems to be more like ‘we’re on the subject of pigs, and one thing I know about pigs is that they like mud, and what they tend to do with mud is roll in it’. In other words, I seem to use the meaning of the word sequence as a whole, built up from the meaning of the individual words, to guess the next word. Bengio’s model does exactly the same.
A classical neural language model
Get Felix Hill’s stories in your inbox
Join Medium for free to get updates from this writer.
Importantly, the model has the capacity to store each word in the training corpus as a vector (just a list) of numbers (e.g. [a1, a2 …] for the word pig in the diagram above), so that each word lives at a point in a multi-dimensional space.
When training begins, the model has no idea what any words mean — the numbers in the vector corresponding to each word are just randomly chosen. The model ‘reads’ the first n words in the training corpus (n here is just a number pre-chosen by the model designers, e.g. 5), and retrieves its vector for each of the words.
These n vectors are then combined into a single vector [s1 s2 …] by a mathematical function, which is determined by another set of numbers called weights (w1, w2 etc. in the figure). Again, at first the model doesn’t know what the best function is; these weights (and therefore the function itself) are set at random and unlikely to be correct.
Once the model has the vector [s1 s2 … ] representing the first n words, it uses this vector to make a guess at the next word. This guess is then checked against the actual next word in the training corpus. If the guess is correct, nothing happens- the model doesn’t learn anything from correct guesses. However, if the guess is wrong, the model updates its understanding of language in order to make a better guess the next time. Specifically, it updates its understanding of what the words like pig and rolled mean (the vectors [a1 a2 ..], [b1 b2 .. ] etc) and it also updates its understanding of how to combine word meanings to represent the whole phrase of n words (the weights [w1 w2 ..]). I won’t go into the details of how this updating works, but it involves a method called backpropagation. If you are interested in learning more about ‘backprop’ you could try looking here or here.
So, to recap, the model starts with an entirely random (and probably incorrect) idea of what each word in English means and of how to combine them, so its first guesses at the next word will be terrible. However, after repeating the updating procedure for many sentences it starts to show signs of really understanding language.
Firstly, it becomes very good at predicting the next word. For a fixed amount of English text to learn from, it predicts the next word better than the counting approach I sketched out above.
Second, the way the model comes to understand words really matches how humans understand them. Remember, the model stores a vector for each word in English that encodes its meaning in a high-dimensional space. Here is a 2-d projection of where words appear in such a space after a neural language model has been trained.
If you check out the space in more detail here, you will see that classes of words people would naturally group together, such as pronouns, professions, countries or days of the week, are all clustered together by the model. In fact, the idea that these models learn word meanings that align with human semantic memory has been confirmed in numerous empirical tests.
I think it’s fascinating that a model trained to get better at a simple language exercise (trying to guess the next word in a sentence) also learns to understand the meanings of individual words in a similar way to humans. Maybe that can give a clue about how children learn language so efficiently without anyone telling them explicitly what words mean — a question that has baffled philosophers for decades (no, centuries). After seeing such a model, it certainly seems unnecessary to conclude that all word meanings must be innate.
Solving crosswords and other applications
In the last year or so, quite a few other applications of neural language models have emerged. For instance, if you give a neural net lots of images with accompanying captions to learn from, it can become very good at describing in words exactly what a computer sees in an image. In fact, you can now upload a photo and let a neural language model describe what it sees.
In a different project, KyungHyun Cho and I trained a recurrent neural language model on the information encoded in dictionaries (real dictionaries, like the Oxford English Dictionary). It’s easy to look up words in a dictionary by alphabetical order, but what should we do if we can describe the word we are looking for but can’t remember it? By training a neural language model on a dictionary, we can easily create a ‘reverse dictionary’, where users provide a definition and the model tells you what word best fits the definition. I’m not sure I’m totally convinced, but people who try to sell commercial reverse dictionaries are adamant that they are really useful for professional writers who keep losing words on the tip of their tongue.
Interestingly, we found that after training on just six publicly-available dictionaries, our neural language model works about as well as a commercial reverse dictionary (not using neural language models) that has access to over 1000 published dictionaries. We also found another advantage of using neural language models for this application — they are flexible enough to transfer the knowledge encoded in dictionaries and encyclopaedias to the task of answering general knowledge crosswords. When we trained our model on dictionaries and some entries from Wikipedia, it answered general-knowledge crossword questions better than any commercial crossword-answering system that we could find. And it only took us a month or so to put everything together. The work was featured in The Times, and you can read the full version here.
Neural Networks are Good for Linguistics
So it’s pretty clear that neural networks are important for current language technology and engineering. But, as I mentioned in the intro to this blog, neural networks are sometimes considered bad for science because they are ‘black boxes’. In other words, they don’t typically have a design or features that are specific to the type of data or real-world domain that they are being applied to, and so don’t tell us anything about the world, even if they work well. I think this criticism is completely wrong. The fact that they do not encode such domain-specific rules is precisely why neural networks are better for science than other approaches to computational modelling of natural phenomena.
Let’s take some examples. Traditional language understanding systems encoded hard rules like ‘an adjective cannot come immediately before a verb’, which were prescribed by language experts. Similarly, an approach to building a model for learning the weather might be to specify some latent variables, which, on the advice of geographers, are organised such that e.g. the chance of sunshine is bigger after a red sky the previous evening. Suppose also that these models can both predict new language (or weather) with 80% accuracy. Does this mean that the principles we encoded into the design of our model are what exists in the world (i.e. in weather, or language)? We have no idea whether there was some better model out there that could describe the data with 90% or 100% accuracy, and which relied on totally different hard-rules and assumptions.
With a neural network, we don’t encode any hard principles. The model infers the important structures, properties and relationships directly from raw data, in a way that allows it to best achieve its objective.
Suppose we built two very general neural networks in this way and trained them to predict language or weather data. Even if we observed that they both got only 80% accuracy (i.e. the same accuracy as the more structured models), we could still interrogate the neural networks to answer questions like how did they learn to achieve this accuracy?
As I mentioned in the first section, numerous methods are emerging that allow us to better understand how neural networks learn to do what they do. It’s certainly easier to work out what is going on in an artificial neural network than what is going on in a real one.
When we interrogate an artificial network to see how it works, we learn about the strategies that a very simple computational learning machine uses to transform and combine observed data so as to better make the predictions it needs to.
This in turn helps us to understand the most salient principles characterising variation in the data, which is, after all, the object of science.
And because all of the computations performed by the model are simple and generic, this insight is (relatively) objective and (relatively) unbiased by established wisdom. Given that traditions, paradigms and dogmas can exert great influence on the established wisdom at any particular point in time (especially when it comes to little-understood things like brains) this can only be good for science.
In fact, as algorithms develop and access to data grows, I think that a big part of language science (or linguistics) will be concerned with understanding (at different levels of explanation) the internal machinations of man-made models and machines that successfully analyse, model or interact with naturally occurring data.
]And, as hinted at before, this scientific endeavour is already very much underway. We have already seen how learning rich semantic representations of concepts seems to go hand-in-hand with predicting which word is coming next in a sentence. Further, analysis of trained neural language models suggests that, given a good enough understanding of the meanings of individual words in English, the order of words is comparatively unimportant for computing accurate representations of phrases or sentences.
Of course, this won’t be true for all sentences and all language tasks (in our experiments, it doesn’t seem to be true for neural machine translation). Nevertheless, the extent to which it is true is surprising (at least to me) and suggests that the real heavy-lifting in the linguistic brain may be done by a high resolution conceptual system rather than a bespoke module for getting words in the right order..
Discovering how words in a trained natural network ‘depend’ on each other, taken from Nal Kalchbrenner et. al.’s excellent paper on sentiment classification, and from Kyung Hyun Cho et al.’s paper on neural machine translation.
Conclusions
Phew — that was a long one, so I’m grateful if you’ve got this far. I’ve explained a bit about neural networks and how they are now being applied to interesting problems in language. I’ve also mentioned why I think neural language models may be more of an impactful innovation for language science (aka linguistics) than anything Chomsky wrote. Along the way, I’ve argued that language and science — and linguistics and engineering — didn’t really have much to do with each other for most of the twentieth century. And that the unhelpful influence of non-empirical linguistics on language technology remains acute to this day.
None of this is to say that linguists don’t have a critical role to play in language technology. The insight and conclusions from a science of language that is actually scientific (i.e. one with experiments on large-scale, unbiased linguistic and behavioural data) will be essential to the development systems that understand and use language like humans do.
I hope ‘language people’ (myself included) can contribute by analysing and interpreting both the output and the internal states of models that learn to process language, and relating this to child and adult language processing. We can also determine the best objectives and data for training these models so that their linguistic behaviour resembles humans as much as possible.
Many apparent facts about language are not facts at all, but simply the result of cultural, educational or intellectual traditions and paradigms. They can be so conventionalized, however, that even those working with language every day don’t notice. The criteria for ascribing ontological status to any cognitive or linguistic phenomenon must be empirical. And we must not forget that questions like ‘what is a word?’ or ‘what are the different types of word?’ would have quite different answers among speakers of languages like English and those from very different linguistic cultures.
Ironically, it was Chomsky’s criticism of BF Skinner’s behaviourism that began the cognitive revolution and led to the first neurally-inspired computational models of cognition. However, following Chomsky’s input, Skinner’s methodology was replaced by an approach that was equally limited. Although Chomsky was right that we will not fully understand human behaviour or language unless we try to understand the mind and brain, he was wrong in his refusal to accept the converse. It is equally improbable that we will ever understand the mind or the brain without considering the behaviours and functions for which those mechanisms were optimised. Thankfully, when it comes to understanding human linguistic behaviour, at least, the amount of available data is big and getting bigger all the time.
What do I know anyway?
I’m doing a PhD trying to get computers to understand language at Cambridge University. I speak a couple of languages (English, Spanish) quite well and a couple more (French, Portuguese) quite badly. I find English the easiest. After studying pure maths, which definitely isn’t a language, for my undergrad and masters, and then failing to discover myself, I read some books by Steven Pinker, because he knew how the mind worked, and that got me pretty interested in language. Members of the philosophy faculty at UNAM convinced me that Pinker was totally right, before Ricardo Maldonado, who was one of Ronald Langacker’s students, convinced me he was in fact totally wrong. Around this time I stood on a Yellow Pages and had my picture taken with George Lakoff, which you should be able to see below.
I then did an MPhil at Cambridge in psycholinguistics, with empirical linguists who didn’t much like having arguments, and instead taught me to do experiments. During this time, I spent a lot of time agreeing with Christian Bentz, who does like an argument but pretends not to.
After that, I did a PhD in Computer Science, where, thankfully, Douwe Kiela taught me how to use a computer. Then Yoshua Bengio taught me how to build neural networks. His postdoc KyungHyun Cho taught me how neural machine translation models work, and, in return, I taught him, somewhat facetiously, that ‘syntax’ isn’t a thing.
At just that time, Dima Bahdanau, a Belarussion masters student visiting from a little known university in Germany, who was sat next to me in the lab, showed me a cute idea he had been working on to improve neural machine translation, an idea that Yoshua referred to as attention.
After all this inspiration, I started to think about why neural nets work so well, noticed the comparatively objective way in which they learn from data, realised how much maths I had forgotten and then realised that luckily neural nets are mercifully simple (locally, at least).
Now, I try to work with as many different language-inclined people as possible, including some who I hope have an inkling how the brain works, like Jay McClelland, and some who may not, but are building cutting edge language technology, like Jason Weston and Yann LeCun. I’ve also had some nice recent chats with Demis Hassabis, who has an intriguing startup based in London that is looking to explore some of the questions raised in this blog
Acknowledgements
Thanks to Ricardo for explaining the beauty of non-Chomskyan linguistics; to Douwe, Kyunghyn and Jase for drinking enough beer with me to make writing this seem like a good idea; to Dima for encouraging me to think rigorously about neural networks (despite their simplicity); to Yoshua, for paying Cho enough to afford beer for all of us; and to Demis, for giving me the thing I needed most back in 2015: inspiration for where to go next.
