Language is Not Compositional
Written in a rush, because time flies like an arrow.
Whereas fruit flies like a banana.
Compositionality, Conceptual Blends and DALL-E
Felix Hill, May 2022
When we compose meanings, concepts, semantics or any other ‘elements’ of cognition, the outcome is not easily predictable like it is when we compose functions in mathematics or operations in a computer programme.
We all know, without really even having to think, that a wine hangover is a hangover caused by wine, but a college town is a town that has a college. It seems obvious to us that a honey bee is a bee that produces honey, but that a mountain lodge is a lodge located on a mountain.
We can perform similar magic with snow blindness, bacon grease, glass cubes and daisy chains. And we can do it when we haven’t seen the combination before. Do you know what a zoo gun is? I bet you can make a reasonable guess, and I bet most of our guesses are similar. What about a gun zoo? We all know what a pen knife is, but it’s kind of amusing to imagine what a knife pen would be like.
These cases illustrate an amazing fact about language: that the meanings of two innocent-looking words typically combine in radically diverse and complex ways. Nothing on the surface of the language tells us that we should treat each of the above cases differently, yet we can all cook up these very different recipes in our mind, and, I bet, with surprisingly few differences of opinion.
Unfortunately, there is no known well-understood process or procedure for determining how the meanings of two words should combine to form a single coherent meaning. This simple fact may be the most sound explanation for why the endeavour of modelling language with symbolic or rule-based systems — attempted with great vigour and on a large scale continuously from the end of second world war until at least 2010 — seems to have reached its limits.
Considering the examples above, perhaps the only useful rule that we can infer about composition is that the final result seems to depend on lots of different things. These include, but are perhaps not even limited to:
(1) Our prior language experience with the specific combination or other combinations that we judge to be similar.
(2) Our knowledge of how things might naturally fit together given the physical and social constraints of the world that we live in.
And (3) our intuition for what somebody might be trying to say given the way the current conversation is going.
We may be consciously aware of some of these factors, and unaware of others.
Cognitive linguistics (a branch of linguistics that is largely ignored by the NLP community) has long studied how these, and other, factors impact on our conception of word compounds and phrases. Perhaps the most relevant formalism for the present examples is the notion of conceptual blend popularised by Gilles Fauconnier and Mark Turner.
The newly released text-to-image generation system DALL-E has attracted a lot of attention in recent weeks. DALL-E is trained on a large set of images and corresponding captions taken from the internet. Naturally, it generates good quality, appropriate images when prompted with text that is close to the captions in this data. However, it also produces strikingly plausible images when prompted with phrases such as A Shiba Inu wearing a beret and a black turtleneck
Such examples are used in the paper to showcase the skills of the model. in particular, they are chosen because the prompt seems unlikely to be something that would appear as a caption for any image on the web. In other words, the prompt is probablly not in the model’s training data. DALL-E just ‘knows’ how to make sense of a very exotic-sounding prompt that it has never seen before.
So how does DALL-E achieve such precocious feats of conceptual blending?
Well, it can’t rely on conversational context (factor 3, above) since it’s prompts are isolated phrases. As for factor 2 (above) DALL-E can learn something about about the world from the millions of images in its training data, providing it with a rudimentary ‘world model’ that can facilitate some of the conceptual blending.
However, unlike human learners, this world model must necessarily be entirely based on still images, and, further, likely biased towards the sorts of things that get photographed and then collated into image datasets. DALL-E therefore lacks many of the facets of a human learner’s core world model, such as motion and rudimentary physics. Adding these to DALL-E’s training experience would surely enhance even further its ability to blend concepts.
Despite this poverty of stimulus (relative to human learners), DALL-E makes what viscerally seem to be remarkable attempts to both account for the prototypical meanings of input words and also satisfy the physical and conceptual constraints of the world.
Get Felix Hill’s stories in your inbox
Join Medium for free to get updates from this writer.
Consider a further example. When asked to imagine a lettuce hedgehog, it puts the ends of the lettuce leaves towards the back of the hedgehog where the spikes would be, it merges them into a single contiguous object, it creates eyes and noses (things that a hedgehog has) out of specks of dirt that wouldn’t look too out of place in a salad.
Similarly, when it constructs an avocado chair, it seems to do so in a way that fits with how our subconscious intuitions about how these two things should be combined. The avocado could be sitting on the sofa, or the chair could be located on the avocado, or it could be a chair sitting in a salad, or all manner of other ways of combining the two concepts.
But something about the way DALL-E avocados the chair just feels right to us.
And surely we all must agree that this is the only right and proper way to toucanize a hedgehog.
This is why it makes no sense to criticise DALL-E (or neural networks in general) for their poor composition. It is precisely because their composition is surprisingly good that emotions have been stirred and people are enjoying tweeting and sharing these things so much!
Yeah, using DALL-E is good fun, but we can’t learn anything scientific from this brute-force approach of training a model on an internet’s worth of captions and images
Well, I’m not so sure. Let’s consider a bit more history….
The eccentric, multivariate, non-linear nature of semantic composition was one of the key motivations for a group of psychologists to develop some of the earliest artificial neural networks. Rumelhart, Hinton, McClelland and a few other connectionists understood that these data-driven, learning-based models had the potential to acquire conceptual knowledge of the world while respecting the subtleties and eccentricities of the data. Moreover, because such knowledge is distributed throughout weights of the network, including all levels of input processing, it must necessarily constrain the representations that the model constructs when presented with the inputs like words in the raw form.
In other words, in neural networks, the input and prior knowledge could interact in a way that wasn’t possible in competing, more modular, symbolic or rule-based approaches to modelling semantic cognition.
If we were composing smooth, closed-form mathematical functions, these symbolic approaches would probably work very well. But they do not work very well when modelling language. So, if we are looking for scientific insight, one thing that we can learn from this brute-force approach is that language (and meaning) is not like mathematics. Any hard imposition of encapsulation or modularity seems to be unhelpful; what performs best is a model that can acquire background knowledge from data and that allow this knowledge to interact with input processing at all levels of representation and abstraction.
OK, but what about systematicity?
As pointed out in the OpenAI blog, DALL-E performs worse when asked to imagine a green cube on a red cube on a blue cube than to imagine a lettuce hedgehog. This is a bit disappointing, because we can imagine a green cube on a red cube on a blue cube quite easily. However, as pointed out by Tomer Ullman on Twitter, once we get above three objects, this gets quite tricky for us too. It’s actually quite a headache to imagine (or sketch) a green cube on a red cube on a blue cube on a yellow cube on a purple table.
Interestingly, my bet is that we would observe a clear difference in how well children do at the cube-scene sketching task between the ages of 4 and 12. We would probably also observe differences between the final sketches in cultures where people have a formal education and those that do not, and between populations that received the description in a one-off audible instruction and those who could repeatedly consult a written version. Our ability to learn the sort of systematic mental processes that allow us to construct multi-object scenes probably also benefits from our being physical beings who see, feel and hear the world from a first-person perspective, and practice physically arranging things.
So, DALL-E is not as systematic as young children, and far less systematic than formally-educated adult humans with either perfect memories or a pen and paper. But for DALL-E, many of the environmental factors that give rise to this ability in humans are absent, replaced by a vast amount of uncorrelated views of things that make a good photograph on the internet. Understanding what really causes humans to develop into systematic beings, and what makes some humans much more systematic than others, can provide us with insight into how to create models with the powerful compositional capacity of neural networks and the systematicity we desire as well.
First, we might like to think of ways in which a model like DALL-E could be enabled to actively-perceive and learn to interact with its surroundings. We already know that doing so can make a substantial difference to how systematic a deep neural network is. Second, we could consider what is the best proxy for a formal education for such a network. We know that such networks get more systematic through repeated practice at problems that require systematicity, particularly when the correct answers are contrasted with carefully chosen alternatives. And third, we could attempt to develop better memory systems, so that a model or agent can know, for instance, when it is repeating itself and credit itself for doing so when doing so leads to positive outcomes.
In summary
It’s wrong to say that DALL-E is bad at composition (or compositionality, whatever that even means). DALL-E is probably better than any machine that we’ve encountered before at composition — that’s exactly why everyone is so excited.
It is precisely what appears to be an instinct for combining words in a way that respects many of the constraints of the world — and in doing so matching our shared expectations and intuitions — that is what makes DALL-E’s predictions seem so magical to us.
And it is not a surprise that such a capacity came from a deep neural network; it was exactly for these reasons that a few psychologists in the late 1970s and 80s developed models of this type and applied them to language when few others were interested.
On the other hand, as the example with stacking cubes shows, DALL-E is not particularly systematic.
It is a worthy endeavour to research ways to make models like DALL-E as systematic as the smartest humans, particularly if we would like a system that can also do something like science or maths. But the fact that DALL-E isn’t very systematic is not a fundamental limitation that proves the whole approach is wrong.
That said, to overcome this limitation, advocates of data-driven approaches to learning from vision and language, myself included, will need to become increasingly systematic in the way we educate our models.
Thanks to Diarmuid’s PhD for many of the examples of noun compounds, and lots of great people on Twitter for the fun examples from OpenAI’s DALL-E generator.
