The Word Royal Institution Christmas Lectures

Human language can be complex and bewildering.

PHONE RINGS

Oh, dear. Sorry, I've got to take this.

Hello? I can't talk now.

I'm doing the Christmas lectures!

What?

She said DAVID didn't take his money?

What? She said David didn't take HIS money?

Oh, she SAID David didn't take his money.

Why didn't you just say that, then?

Sorry. Now, what you have there is three very different meanings from

exactly the same sentence.

Will anything other than another human being ever be able to cope

with that level of complexity?

In this lecture, I'm going to find out what makes language the ultimate

communication tool and why humans are absolute masters of it.

Welcome to the third Royal Institution Christmas lecture

of 2017.

I'm Professor Sophie Scott.

Now humans have got an incredibly powerful ability - language.

I can convey very precise meanings to anyone within earshot if they

speak my language. To give you a taste, please let me introduce scientist,

comedian and rapper, Alex Lethbridge.

So I've listened to Doc Brown,

Akala and Syntax, they showed me how to flow off my grammar and syntax.

The RI told me, Alex, what's your language?

I checked my head, do you mean English, Fante or Spanish?

Now, my PhD's crazy, I'll do wordplay till it pays me.

And when I get bored, a language or two.

So while you're getting PTSD from your GCSEs

and wondering should I RSVP to GCHQ?

Now I'm not sure, Sophie says language is complex.

You've got the rules like subjects, verbs and objects.

It's more than words, you've got intonation and context.

Final lecture, we're learning all of these concepts.

APPLAUSE

I don't know about you, when I'm listening to rap music, I like to count all the words.

And I reckon in about 25 seconds there, Alex,

you said about 110 words.

-Yeah, exactly.

-And got over about 15 ideas, does that sound about right?

Amazing. Thank you very much, Alex.

No worries, thanks.

We're used to thinking of telepathy as a science-fiction concept, but

Alex just achieved the exact same result.

We share the content of our minds, our brains,

whenever we want to speak, or rap, to anyone.

Don't worry, I'm not going to rap!

Are we unique in having these skills or will we one day have a full

conversation with another species?

When I was a little girl, I so wanted to be able to talk to animals.

Will that ever happen?

And will computers ever be able to fully get their processors around our

language well enough to understand a joke?

Tonight, I'm going to explore what makes language so amazing

and so very difficult.

But what do we mean when we talk about language?

Languages can come in many forms.

We can talk, we can write, we can sign.

And I've got a very basic form of language here.

MORSE BEEPING

Anybody speak Morse?

That was a cry for help!

Now, I don't speak Morse beyond being able to do that.

But basically you can think of language like Morse code as being

a message which we're sending with a code.

And to make a code,

the first thing you need to do is to produce a signal that's got some

kind of structure. Now, that means a signal

that's not just a random stream of noises without any order,

nor can it be a very simple pattern just repeated again and again.

You need to have a capacity to send information like the short and long

patterns of the Morse.

Now, humans do this when we speak aloud.

We're using the sounds of our voices that we use in our language to

express a code.

Can we find any signs of a similar kind of structure in other animals'

voices? And if so,

could we crack their code and have a proper conversation with them?

There are some animals who are very good candidates for being able to

produce these sorts of sounds with structured elements,

and those are birds.

Now, these guys, who are a couple of zebra finches,

and a couple of canaries, they're songbirds.

Songbirds, when they're babies,

they learn all the songs they're going to sing when they're adults.

The most impressive can learn over 1,000 different songs.

Could these songs contain coded information?

Now, I can hear a couple of cheeps coming out of here,

but I think we have a recording of one of the canaries.

Can we listen to that?

CANARY CHIRPS

It's a beautiful sound.

But does it contain enough structure in that signal that it could be used

to transmit a code?

Well, I've got an example of the canary's song here.

And what I'm showing it to you as, is what's called a spectrogram.

Now, a spectrogram is a way of looking at the structure in a sound.

So, what you have along this direction is time.

So, this is the sound unfurling over time,

how it's changing over time. This direction, we've got frequency.

And that's roughly telling you,

low-pitched sounds up to high-pitched sounds.

And where the colours are warmer and brighter,

that's where there's more energy.

And we can see in these individual elements, these little notes,

and we're seeing some quite structured elements to this.

We've got a similar sequence here and repeating there.

And then these sequences of lower and higher alternating notes.

Now, I need to compare this with another kind of voice.

So, I would like a human volunteer, please.

Can I have you in the middle there, with the penguin?

Thank you very much.

Now, what's your name?

-Ruth.

-Ruth. I'm going to ask you to come over here and say the first two

lines of Humpty Dumpty into my computer.

OK, I'll tell you when to go.

If you could just stand about there.

Brilliant. And go now.

Humpty Dumpty sat on the wall.

Humpty Dumpty had a great fall.

Brilliant. Thank you very much, Ruth.

Exemplary, I think you'd agree.

So, here's Ruth's version of Humpty Dumpty shown in the same way

on a spectrogram. You can see immediately there are some differences.

There's the canary, there's Ruth.

You can also see some similarities.

And I'm talking in the most general sense here,

but Ruth is producing individual rhythm in the syllables of what she's saying,

and you're seeing a pattern of that over the sentence.

And you're seeing something broadly comparable in the canary.

We're seeing structure in those sounds.

The canary and the speech sounds have both got rhythm, pitch,

rate information in there.

So it's at least possible that the songbird

is producing something which has got

similarities to the way we code information in our speech.

Could it actually be a code, though?

Well, probably not.

It doesn't seem to be quite enough.

So if you look at how songbirds use song,

what you find is they don't generally change their songs

once they've learned them.

They will always sing the same whole song.

And the other thing they don't do is chop songs up and rearrange them

to make new songs. We do that all the time.

We can use words in lots of different, very novel orders.

The birds don't do this.

So it's entirely possible that, complex though the songbirds are,

they are not producing something that is conveying a complex,

coded meaning.

But there's another group of birds

that can learn to say human-coded signals, use words.

And those are parrots.

Please meet Mike and his parrot.

-Hello.

-Hello.

-Hi.

-Hi.

-Hi, Mike, who have you brought with you?

-So this is Helly...

-You all right?

-You all right?

-..and Helly is an Amazon parrot.

So a South American bird from the rainforests.

And she's already said hello, hasn't she?

She has, yes. She knows that's an introduction,

so it's the first point of call for a conversation

or attention gathering.

-How many other words does she use?

-You all right?

She uses about 80 different sounds and words.

Yeah, human words, though?

Oh, human words, I would say she knows about five or ten.

-Yeah.

-Can you say bye?

Bye!

So she's doing a set of human words.

Did you teach her those or did she just pick them up from seeing how they were used?

Yeah, she understands that "Hello" is a greeting because it's a

word that we would use on arrival.

And she would then understand that the "Bye" is a departure word.

So she places the words with timings, as well.

-Yeah.

-But throughout her life she's had lots of trainers saying,

"Are you all right, are you all right?"

So if she actually gets worried in something like a studio environment,

-she will say "Are you all right?"

-You all right?

And she knows that goes with almost an emotion, as well.

-Yes.

-So she links these words with timings and emotions.

And she can do other sounds that aren't... she can do more than words, can't she?

Yes, she uses lots of different sounds as well as words.

Can we have a bomb?

WHISTLE, BOOM

LAUGHTER

They can also be copying other birds as well, so local birds in the wild.

And also other birds that we house.

Thank you very much, Mike. And thank you very much, Helly.

Lovely meeting you. Bye-bye.

What gives birds their amazing ability to learn to produce these sounds?

Well, the answer may lie within their brains.

Now, birds are more closely related to dinosaurs than they are to us.

But we share similarities in the ways that our brains control both

the learning and the production of sounds we make with our voices

and the genes that build those parts of the brain.

If we look...

..at a bird brain,

we can identify specific areas which are important

in the learning of song and in the control of singing.

And if we look at a human brain...

..we can see a similarity, in that there are specific networks

recruited when we are speaking.

When we're talking, these are human brains here,

this is the right side of the brain and the left side of the brain.

We see some areas which are strongly associated with the control of all

the work we have to do to make the sounds of speech.

We also find the very specific area just on the left side of the brain

which seems to be very important in planning speech.

And it's also important in learning new things to do with our voices.

How can I find out more about this system?

We can take these snapshots of the brain in action and we can work with

people who have had strokes and have damaged these brain areas.

But there's another technique that we can use where we can investigate

what would happen if we could turn off

that part of the brain in someone, and just that part of the brain.

What I'd like to do is introduce you to my colleague from UCL,

Dr Ricci Hannah, and comedian Robin Ince.

Hello Robin, hello. I've been waiting for this day.

Hello, Ricci.

-Hi.

-Hi.

Now, Robin.

Robin, what we're going to do is sit you down here.

-Right.

-And then I'm going to let Ricci explain what we're going to do next. OK?

I have to emphasise, this is a temporary state of affairs, OK?

We're not about to do some sort of terrible live brain surgery on you!

I was quite worried, because I was watching what else was going on and I thought,

"They're going to prove that I'm less intelligent than a parrot!"

So let's find out what happens.

Ricci, can you tell us what you've got here?

So this is a transcranial magnetic stimulator.

And what we can use it for is to probe

how different parts of the brain play a role

in different aspects of behaviour.

In this case, speech.

OK. So can we start just by pointing out, that the way that we get this,

we change Robin's brain activity by passing electrical current through

-that coil, is that right?

-Mm-hmm.

And actually, according to the principles of electromagnetic forces,

which were first described here by Michael Faraday,

what that lets us do is induce currents inside Robin's brain

without actually having to get inside his brain.

It's absolutely amazing.

-So can we start by getting Robin talking and then seeing if we can stop him talking?

-OK!

People have been trying this for years!

-It's going to be popular!

-Can I get you to shuffle back, Robin?

-Yes.

-There we go.

-Are you OK?

-OK, so I'll just position the coil.

-What I want you to do is to say the months of the year really loudly and clearly.

-OK.

When you're ready.

January, February, March, April, M...

Please start talking again!

That is weird!

That is very...

I think Professor Brian Cox who I do a radio show with is going to want a

beret that I wear with that in there, so he can stop me talking!

I don't know what it looked like to you, but it was like just...

It was like kind of Homer with a doughnut, sort of...

If you could bear it, can we try it again?

Yeah! It's really... I find it amazing.

It is extraordinary, isn't it? And then it just comes back.

I think I prefer quiet me. Let's do it again!

Shall I try, and see how far I can get in Jabberwocky, shall I try that?

Yes, excellent, yes.

-Tell me when you want me to.

-When you're ready.

'Twas brillig, and the slithy toves. Did...

It's fantastic!

That's...

So what Ricci's very specifically

focusing on here is this part of the brain,

it's called the inferior frontal gyrus, on the left.

And in humans it's incredibly important for

planning and controlling speech.

If we move to a slightly different area,

even within the same network,

what we find is that Robin will be able to talk absolutely fine.

Would it be OK to try that?

-Yeah, sure.

-OK.

-OK.

-I didn't even look at the health and safety form for this!

-I sent this to you!

-I know, I didn't look, just in case!

You're fine, you're safe.

OK, when you're ready.

'Twas brillig, and the slithy toves

Did gyre and gimble in the wabe

All mimsy were the borogoves And the mome... Yeah, that's...

There you go, there you go.

I think I preferred the one at the side!

I can't thank you enough for being prepared to come out in front of

everybody and have us try and zap your brain!

Thank you very much, Robin. Thank you very much, Ricci.

That was amazing, thank you.

It was brilliant, thank you.

Thank you.

So you could see how precise the effect of the transcranial magnetic

stimulation was. We were only seeing Robin stopping talking when we were

applying the TMS over his left inferior frontal gyrus.

When we went elsewhere in the brain, other things happened,

but it's not stopping him from talking.

So what we're seeing here in the humans and in the birds

are very dedicated brain regions

that are important in vocal control and vocal learning.

It's a strong hint that we really are seeing some commonality in

the brain areas that are to do with the learning

and the producing of vocal sounds.

But can any of these birds really understand the words they're saying?

Well, parrots don't just say human words.

They'll mimic pretty much anything they hear a lot of.

Car alarms, creaking doors, alarm clocks.

Why do they do this at all?

Well, birds are mimicking to show off to potential mates,

to get attention.

To impress other birds and other humans.

To defend their nesting sites.

Perhaps the more impressive a sound they can make,

the more likely they'll find a companion,

or scare off a rival.

So birds aren't showing a great ability to decode our language.

But what do I mean by decoding?

We humans are exceptionally good at working out what words are and what words mean.

I'd like you to watch and listen to a clip that I'm about to play you,

and there's going to be a short test afterwards.

And this will go down on your permanent record.

Back. Deaf. Gidge.

Hock. Fip. Nop.

Fib. Wreck. Sit.

They. Fip. Fip.

Hock. Fib. Gack.

Gin. Hock. Hock.

Lun. Fip. Nat.

Fip. Ros. Hock.

Yas. Beth.

OK. Short test.

What's this called?

-CROWD:

-Fip.

It's a fip. What's this?

-CROWD:

-Hock.

-Hock, excellent.

Well done. Now, I didn't tell you to work out what was going on there.

What you were doing was decoding the information we gave you.

There were some novel sounds in there and they were being associated

in quite a regular way with visual information.

Even if you don't know that your brain is trying to do it,

you're always trying to spot words and work out what those words mean.

We are not the only animal who has an ability to do this.

There's an animal you're probably all very familiar with who's actually

really very, very good at sharing this ability with us.

And that's dogs.

Please give us a very nice doggy-friendly round of applause

for Gable and his owner, Sally.

-Hello.

-Hello.

-Hello.

Hello. Hey, Gable.

So Sally, what's different about Gable?

Gable's got the ability to identify a large number of objects and toys by name.

He currently knows about 150 different names for toys and objects and articles.

Goodness.

Now, can we see a demonstration of that?

So we've got some of Gable's toys here and what were going to do is

get a random selection of those out and then see.

-OK.

-I think there is somebody...

Hi, Sean. Now, we've spent a little bit of time with Sean,

getting Sean used to being with Gable and Gable used to being with Sean.

Can I bring you down? Thank you.

Now Sean, if you could come over here.

Can you pop these gloves on and can you just randomly select 15 toys

from these buckets and spread them out?

When did you first realise Gable could do this?

When he was a young puppy, actually.

He sort of invented the game.

I was trying to watch telly one evening and he kept pestering me.

He wanted to do something.

And I just remembered oh, his red toy was upstairs.

And I just said, "Oh, go and get your red toy."

Sort of idly just dismissing him,

hoping he'd be gone for ages because he couldn't find it.

And he came back with it and he put it in front of me and looked at me.

And I just thought, "I've actually never told you that's called Red Toy."

And so I just then thought, I wonder what happens

if I do teach him a name, and it's gone from there, really.

-Now...

-Wow, look at all your toys.

-..Sean, don't go anywhere,

I'm going to need you again.

Can I ask you, Sally, to tell Gable to pick one of these toys, please?

OK. Gable...

triceratops.

Get triceratops.

-Yes! Yes, good boy!

-Good boy!

Good boy! Leave it.

-Come round here.

-Fantastic.

So we've seen that Gable is really very good at working out what you're

saying and trying to work out from looking at you which one you mean.

But then Gable is your dog and he might be really,

really familiar with your voice.

It would be very interesting to know if we could see this happen with

someone who's got a very different voice.

So Sean, is it OK if Sean has a go?

-Yes.

-And what you need to do is whisper into Sean's ear

which toy you'd like him to pick up.

Do you remember Sean?

-OK.

-Gable, octopus.

Get octopus.

Go.

Yes!

Good boy, good boy!

-And well done, Sean.

-Leave it.

-Can Sean do one more?

-Of course, yes.

Is that OK, Sean?

Gable, get hammer.

Get hammer.

Yes! Amazing.

Good boy!

So this really does indicate that Gable must have some understanding of what

words mean above and beyond just associating your voice with things.

That's fantastic. Thank you so much, Sean, thank you so much, Sally,

and particularly thank you so much, Gable.

Come on, then!

That's amazing.

So, what's happening in Gable's brain so that he can do this?

Well, there's a group of scientists in Hungary who have been doing some

quite extraordinary experiments.

They've been training dogs to lie very still

and then they've been putting them into brain scanners.

The brain scanners don't work if you move so the dog has to stay very still.

And then what they are doing is taking pictures of the activity inside the

dog's brains while they're listening to different sounds and words.

When I scan people, they're not normally that happy!

So this is called functional magnetic resonance imaging and it lets us

take photographs of the brain in action.

And this is showing the results.

So a dog's brain, as you can see,

it's different in shape to a human brain, but it has some of the same

structures in there. And quite strikingly,

one of the things they are finding in the results is when dogs hear words they understand,

we see greater activation in the left side of the brain,

particularly in brain areas to do with processing sound.

And that is very similar to what you find in our brains.

So it looks like it's at least possible that more than any other animal

we've looked at, dogs might be sharing some of our ability

to decode spoken words and they may even be doing it in similar ways to us.

But of course, amazing as they are, dog's brains do have their limits.

The largest number of words that any dog has been found to understand,

and this was an exceptional dog, is about 1,000 words.

Which sounds fantastic.

Gable knows about 150 words,

but every one of you could understand 1,000 words when you were three years old.

And of course human language is more than just single words.

We don't walk around going, "Steps, camera, man".

We are putting words together into sequences.

And when we put words into sequences,

we actually add in an extra level of coded meaning.

So perhaps, if we want to look for some more humanlike ability to put

symbols into sequences like we do with sentences,

we should be looking closer in the evolutionary tree.

Perhaps we should be looking to our closest cousins, other apes.

Now, because I'm not David Attenborough,

I don't get to play with the chimpanzee.

But we have the next best thing.

Please release the apes.

THEY IMITATE MONKEYS

This didn't happen to David Attenborough!

That's just brilliant.

These are human apes.

Please welcome Neil and Ace.

So you guys have been working a lot with our ape cousins?

We have, yeah. We were actually very fortunate to work with Andy Serkis

on Planet Of The Apes Last Frontier, which is an interactive movie.

Hold on a second, sorry.

OK, just reassurance.

We were very lucky, we got to study apes for quite a while before we

started creating our own characters from ape work.

We watched their movements, their behaviour patterns, hierarchies -

here's some of the work we did - and also the speech they use,

which is five different parts of language.

Grunts, barks, hoots, whimpers and screams, which they do use as chimps.

And this is my ape actually, Bryn.

-That's you?

-Yes, that's me.

-That's amazing.

-Yes.

So what this is letting you do is have ape actors which are based on humans,

so you can get them to do things which you could never actually ask

-another ape to do?

-Absolutely,

and we get to create a bit of drama in the family.

It's actually really about family, this interactive film,

so it is centred on this particular family of apes that have splintered

away from Caesar's group.

So you've had to work really closely with the apes to actually learn

about their body language and movements.

Did you pick up on anything that felt like communication that you were looking at?

Yeah, we studied a lot of their gesticulation and body language,

especially between dominant and subservient male apes.

Because what you get is a lot of different behaviour patterns that

have been formed by a simple sign, for instance, the touch of hands.

If somebody is in the dominant position,

it's quite important to be able to...

So if you were offering,

if you're trying to get my attention and forgiveness, for instance. Go on.

He gives me respect and I give it back to him.

-They are communicating with gestures.

-They are, yes.

Amazing. Thank you so much, thank you.

So Neil and Ace aren't just imitating the apes very well,

they are clearly picking up on some aspects of communication,

but is there really some kind of

ape conversation going on?

And would it look at all like the way we use language?

To find out, please welcome chimpanzee researcher from the University of St Andrews,

Dr Cat Hobaiter.

Hello. Lovely to meet you.

So you've been asking some really interesting questions about ape language

and can you tell us more about your work?

Yes, absolutely,

so what people have done in the past was try to teach apes our language,

so human language or sign language,

or they've looked at their vocalisations.

But what we've been looking at is their gestures.

-OK.

-And it turns out, they've got a lot of different gestures.

So their own natural communication contains 60 or 70

different hand and body movements that they're using every day to ask come here, go away,

I want that, all the little meanings.

And how do you work out what those gestures mean?

What we do is we look for a particular gesture

and then we are looking for what happens next.

And if I were to do this and you responded back to me,

one or two cases it could be anything or you could misunderstand me or I

could keep going. But what I'm looking for is what stops me from

signalling. So if I'm asking you for something,

then the thing you do that makes me happy as a signaller

-is the thing that I wanted.

-Yes, so you have to look at the whole context.

Yes, so we need to look at the signaller, at the recipient, the gesture,

then we need to look at not just one or two but hundreds of cases so we

can see the patterns emerging in the behaviour.

And you've been using the general public to help you with your research as

-well, haven't you?

-Yes,

so we were able over a few years to look at all the other apes but there

was one of them missing, which was us.

And we know we have language but we don't know if we still have access

to some of the communication that the apes also use.

The sort of gestural stuff they are using, yes.

Exactly. So whether or not if we showed a member of the public a

particular gesture, could they guess what it meant.

And what we did was put up lots of videos online and ask everyone to

come along and sort of play the game and have a go.

Can we have a go at that now?

-Yes, please.

-Fantastic, so I think we've got a clip to start with.

We are going to watch this closely because we are going to ask you what you think is going on.

So the gesture there was that kind of arm.

Little arm raise from Charlotte.

OK, and is that a baby?

Yes, she's just a little one.

Do we think that's A - I want that, B - move closer, or C - go away?

ALL: B.

Yes, you guys are all well versed in chimpanzee already.

That was move closer.

OK, can we try another one?

OK, we are just highlighting this because it happens very quickly.

Yes, this is a really subtle one so it's just that little foot movement

as she looks back and gives a little foot wiggle to her son.

What happens next?

Is that play with me, climb on me or come here?

ALL: B.

B, yeah. It's rather sweet, that one. Very, very swift.

-Yes.

-Really subtle.

So this is really fascinating.

It does look like you are seeing a sign language,

you're seeing really gestural use of communication.

They're not just waving their arms around passionately,

there is a very precise meaning being conveyed here.

But of course the real power of human language is that we can combine our

words and symbols into sequences and that gives us a more complex level of meaning.

Can the chimps ever do this?

Do you ever see any kind of examples of sequences?

Definitely. So what we see sometimes are these one-off gestures but they

will also put gestures together.

Sometimes a couple at once,

that was a big scratch and shaking the object and he's actually waiting

there. And whatever that gesture was,

it doesn't seem to have done the trick because Rohara is ignoring him.

-He's going to do it again.

-He's going to do it again.

-And the arm went up.

So we've got a scratch, a tree wiggle and the arm going up.

Yes.

And it worked. Oh, all right then, I'll get down.

She's a bit of an old lady.

So the tree waving is the sort of "Come here, I'm in charge".

-The scratching?

-Scratching usually means groom, let's groom,

let's get together and groom.

Our big question for us now is when they are putting these gestures into

sequences, if it was human words the order of the sequence would make a

difference to the meaning, and we are trying to work out now if that's the same for the chimps.

Amazing.

Thank you so much, Cat, thank you.

Now, why is this so exciting?

Well, if other apes can also combine symbols into a specific order to

produce something with complex meaning,

it unlocks a great deal more of the power of the language.

Combining symbols into sequences allows us to describe the world in

much more complex ways.

It gives us the power to share much more complex ideas.

It's a really useful skill but it's not in any way a simple skill.

And to look at exactly what kind of thing the chimps are up against in

terms of the difficulty of understanding sequences of symbols and decoding them

accurately, I need two volunteers.

Can I have you there in the Los Angeles...

Can I have you in the Santa - sorry, Rudolph.

Fantastic, thank you very much, thank you...

You come round here.

You come round this side. Thank you very much.

Now, what's your name?

-Gracie.

-Gracie, lovely to meet you, Gracie.

-And what's your name?

-Ryan.

-Ryan?

-Yes.

-Fantastic.

Now, Ryan and Gracie,

what we need to do is put some general covers on you.

OK, we are just going to, sort of, from top to toe, cover you up.

Thank you. So what we are going to do is I'm going to ask you one at a

time to read some instructions and to mix some things together.

And, Gracie, I'm going to ask you to go first.

So, Ryan, so that you don't see what's going on,

I need you to pop on a blindfold and some ear defenders, OK?

Just so that you don't give away the game.

Right, so in a second I'm going to step forward with you

and we are going to turn over your instructions.

An important thing to remember, when we get to the end

of the instructions, is you step back with me.

Are you all right?

OK, let's start.

I think you might need to... You've got it, you've got it.

Good code-cracking.

And step back.

To see an alarming change of colour.

Fantastic, Gracie.

OK, now don't go anywhere.

We are now going to turn to Ryan.

OK, so what we are going to do, we are going to turn over your instructions

in just a second and I want

you to follow them with these chemicals here. OK?

Yes. First mix A into B.

A into B.

-So I just put...

-A into B.

So I just put them like this?

Yes, very good.

-OK.

-Add C.

And then...

you step back.

LAUGHTER

Are you OK, there?

Amazing.

Well done.

Thank you so much.

Now, we had two different outcomes there.

We had either a dramatic change of colour or an actual foam explosion,

depending on the order in which the chemicals were mixed.

And the only reason why Gracie and Ryan mixed them in different orders

was just how we'd punctuated the instructions.

So they both have the same instructions,

and they were just understanding them differently based on where we

put the full stops. Thank you very much, Gracie, thank you very much, Ryan.

So you can see that the exact grammar,

in this case the punctuation of the sentences,

has completely changed the meaning of those sentences

and that's why this tiny glimpse of the ability to combine gestures

in the apes is so exciting.

So if we think about the animals that we have talked about this evening,

we've seen birds who are able to produce very complex sounds

and they can mimic many other sounds, but they don't seem to understand the meaning of

those words. We've seen dogs,

now dogs don't talk but dogs are incredibly good at working out

what human words mean and what they refer to.

And we've seen the chimpanzees,

they are both using gestures to have quite precise meanings and they are

also using sequences of those gestures to have more sentence-like

structures. But they are not the same as the kind of codes humans use.

Why are there these distinctions between humans and other animals?

Well, I think a big difference is likely to do with our brains.

I just want you to look at these different ones here.

So here we've got a bird brain, that's actually a chicken brain.

That's a dog's brain.

This is a replica chimpanzee brain, and this is a human brain.

Now, they look different.

One obvious difference is size, and the human brain is enormous,

much larger than any of the other brains on the table.

But big brains aren't everything.

To make an analogy to computers,

it's not perhaps just about the raw processing power of the brain,

maybe we need to think about the operating system as well.

And we seem to have a very efficient operating system

for dealing with symbol-based codes.

I always wanted the superpower of being able to talk to animals,

but maybe our mismatching brains mean that I never will.

Maybe there's something else in our world with which we communicate far more frequently,

and maybe I need to stop thinking so much about how I could have

conversations with other animals and maybe think a bit more about talking

to machines.

This year's big Christmas present gadgets are machines that we can talk to,

digital assistants.

A quarter of us now either command our phones via our voice

or use devices like these around the home.

Hey, computer, please tell me what the weather will be like in London tomorrow.

There will be showers there tomorrow with a high of nine and a low of three.

OK.

Now that's nowhere near being anything like a human brain, but when I was doing my PhD

on speech processing 25 years ago,

I would have never believed that one day we would be walking around with

phones in our pockets that we could talk to.

It's incredibly hard to overestimate

how quickly this field has developed.

What's happening inside that box?

How are computers managing to interact with us?

And is it ever having a proper conversation with us?

Are we ever going to have a meaningful dialogue with a computer?

Will it ever understand the jokes we tell?

Now this is an incredibly complex area and we could easily fill three

lectures just on this topic, so I'm going to give you a tiny insight

into aspects of how this works.

But first, let's break down the question.

What do computers need to do to be able to understand us?

And the first thing is it needs to take the sounds that we make and

decode that into words.

This is called speech recognition and it's what is known in science as

a ridiculously difficult question.

And that's because speech is hard and speech is complex.

I'm going to play you a sentence spoken in Estonian

and what I want you to do is count the words you can hear.

OK? So just listen out for words and see how many there are.

VOICE SPEAKS IN ESTONIAN

OK, how many words did you think were there?

Sorry? 13, that's a good guess.

Pink top, glasses.

-32?

-32, OK, a big jump there, about twice as many.

OK. Let's see the actual sentences.

So we've got 22 words, so actually it's a meaningless question to ask you,

because if you don't speak Estonian why would the words stand out to you?

But this is what the computer is confronted with.

It's hearing this continuous flow of sound and it's got to find some way

of getting a toehold on what those words could be.

Now listen to this sentence.

What I'm going to do is record myself speaking.

Where are the words?

So this is the spectrogram of me saying "Where are the words" that I've just recorded.

But where ARE the words?

There's no individual words there at all.

What you're getting is a sort of smear of energy and what looks like

one thing happening at the end there is actually the "ss" sound at the end of words.

When you hear gaps between words when someone is talking to you,

that's because you understand those words.

If you don't understand the words, you don't hear the gaps,

as we saw with the Estonian.

So how do computers deal with this?

How do computers find the starts and ends of words when there actually

are no physical gaps there necessarily at all?

Well the first thing a computer does is it breaks up the incoming stream

of sound into smaller chunks of sound

and we've got a demonstration of this here.

So this is an incoming sentence and I don't know what this is,

I've just got the spectrogram to work off and I'm a computer for the

purposes of this point.

And what I'm going to do is split this up

into different slices of information.

And what I'm going to do then is take those different slices

and go to look them up in the library that I've got,

which will help me try and get the best estimate of what speech sound I'm probably trying to look at.

And my library has to be very... Has to sort of

have an idealised version of speech sounds because we all talk

differently. So instead of trying to find an exact match for the speech sound,

the computer is looking for almost like a best guess.

So if we do this with the first speech sound in my sequence...

I've got a little slice of sound here.

Now if I look at that in isolation, what I can see

is that there is a broad sort of smear of energy there.

There's no bright hotspots.

Remember when you get a more intense bit of energy in a speech sound or

any sound in the spectrogram, you see it as a brighter colour?

There aren't really any bright colours there, but there's a big stretch of red

and it's going almost the whole length.

If I go over here,

I think I might be looking at a "T" so my best guess for that first sound is

that it is "T". Now who would like to help me guess the next sound?

Can I have you in the blue top, please?

Thank you very much.

-Now what's your name?

-Joe.

-Sorry?

-Joe.

-Right, Joe,

you're going to help me be a computer processor and work out what the next sound is.

We've got one here.

Now, Joe,

you can see this looks quite different to that sound we were just looking for.

Now can we find anything in our library that looks like that?

I think you're right,

I think we are dealing with "EE."

Can you put that there for me?

Now not to labour the point, if we carry on like this,

you're not getting like really whip quick recognition of the speech,

are you? We are taking our time so what I'm going to do is throw more

processors at the problem, just as a computer would.

I'm going to take a couple more volunteers please,

so can I have you with the glasses, please,

and can I have you with the polo neck sweater.

Thank you very much. Can I have you?

Thank you very much. Got to have a unicorn.

OK, so, Unicorn, what's your name?

-Sasha.

-Sasha, lovely to meet you, Sasha.

-Hi.

-Evie.

Evie. And your name is?

-Kit.

-Kit, excellent.

Now there's four speech sounds left.

I want you to each grab one and see if you can match it up with any of

the speech sounds here, OK?

There you go.

And what have you got here?

Look at this one. Can you see any that have got sloping shapes?

-Is it that one?

-I think you might be right there.

So if you can remember, you came from number four.

I think I have an S sound.

I think you're absolutely right.

You grab an S sound and pop it up.

Now you have got the really difficult one here

because when the speaker said it, they really underarticulated it,

so what you have is this sound here

and that's going to have to be our best guess, OK?

OK, so if you can pop that in, we can see what we've got.

Thank you very much.

It could be team mates, it could be tea mate.

We don't know still what the words are but we've now got a good guess

what our speech sounds are and that takes us to our next stage.

What I want to do first is say thank you very much

to Joe, Sasha, Evie, Kit.

Thank you very much, thank you.

So the addition of processing power

has really helped us be able to speed up this process

and that's why you can talk to your phone without it

taking that amount of time.

And what we are doing here is pulling out what the speech sounds are

but we still don't know what those words are,

we don't know where the edges are.

What we need to do is go to another level and what computers do

is go to what's called a language level

to start to break this stream of sounds up into words and into sentences.

I need two new volunteers to decode a stream of speech for me, just like

you were a computer. Can I have...

you in the middle with the blue T-shirt?

Yes, there you go, fantastic.

Can I have you? Thank you very much, thank you.

-Now, what's your name?

-Max.

Lovely to meet you, Max. And your name is?

-Cammy.

-Cammy, fantastic.

What I'm going to give you is the same task a computer has got when

it's worked out the speech sounds but it doesn't know what the words are.

What you're going to do is see a list of speech sounds and I want you

to try and work out what words could be in there.

Sometimes the computer doesn't know what a sound is at all.

You'll just see a question mark, maybe there was a cough,

maybe there was a noise, so sometimes you're going to have to guess, OK?

So I can just ask you to step over here, Max, thank you very much.

OK and we will see it appear on the screen, OK?

So reading down in that direction,

Try reading that aloud.

THEY READ WORDS ON SCREEN SLOWLY

Now, think about the building that we are in.

-The Royal Institution.

-The Royal Institution, absolutely,

so context can help you.

You go back at that, we are at the Royal Institution.

One last time.

TOGETHER: Drums are really

loud, so we have to

use ear

defenders at the Royal Institution.

Well done.

Thank you. Thank you, Cammy, thank you, Max.

Thank you.

So that's an extreme example, but what you are seeing there was essentially

the same problem a computer is trying to solve.

It's trying to find the edges,

it's trying to work out where the words could be,

and it's helped in this because it knows what the words are.

It has a database of thousands and thousands of words and it also knows

something about how sentences go together.

It knows how probable it is that a word will follow another word.

And if a word becomes more likely,

the probability of it occurring actually changes in the computer

so it's more easily activated.

And, in fact, our brains do something similar.

If you are listening to speech and you hear something which is highly

predictable like "the ship sailed across the bay"

then you will understand that sentence more easily.

So we are seeing actually, in terms of a lot of the code of language,

humans and computers are not as differently matched as we used to be.

Computers are catching up with a lot of our linguistic abilities.

Computers have become huge and very powerful,

they can be searching through very large databases very quickly,

and the internet means a little box like that can be connected to those

large online databases.

It doesn't need to have all the information there.

But, of course, this is not the full story of how human language works.

There is an entire level of communication

that we use all the time that we have barely mentioned.

Rather than just what we say when we're talking,

we are always sending out information in how we say words.

Everything I've talked about so far has been, if we think about brains,

associated with properties of the left side of the brain.

The left side of the brain in humans, for most people, is associated with

how we decode speech, how we control our own voices.

And the right side of the brain is a lot less interested in these

linguistic properties of communication like words and sentences,

and a lot more interested in all the other things that are going on when

we are talking to people:

who we are talking to.

Are they being emotional?

Are they telling a brilliant joke like,

what is round and sounds like a trumpet?

A crumpet.

GROANS

So to have a meaningful conversation with a machine or understand my

brilliant jokes, we need to do something much more difficult.

We need to add in to our computer the missing right half of its brain.

Because actually when we are talking, when you're listening to somebody,

there are very important aspects of our communication which you need to

pick up on to really understand what somebody is saying.

Not just when it's written down.

The information when someone is speaking is very often being expressed as

much by how they are talking as what they are saying.

And this often refers to something called intonation.

Intonation is how we vary the pitch, the speed,

the melody of our voice when we are speaking.

An example would be that we don't talk to each other like that.

We would consider it to be quite strange.

We are always using intonation to clarify, enhance,

put in emphasis and emotion.

In our brains,

intonation is processed very differently

from the words that you're listening to.

On the whole, intonation is strongly found to be something

that the right hemisphere deals with and is interested in.

And often, it can be as important if not more important to the real

meaning of what somebody is saying.

We've got some recordings here of someone speaking in an emotional style

and we have stripped out all the auditory information that tells you about

the words they are saying and it's just leaving you with the intonation.

See if you can guess the emotion.

BUZZING

Any guesses, does that sound happy?

Angry? It sounds annoyed, doesn't it?

Yeah, that was angry.

That was very angry.

Another one.

BUZZING

Yeah, lots of downward inflections.

Sounds softer.

That was a sad voice so you are definitely using intonation to get aspects

of emotion out, but that's by no means the whole story.

And to help me demonstrate why intonation is so very important to

communication, please welcome news presenter and journalist

Krishnan Guru-Murthy.

Krishnan, obviously the words you say are absolutely critical to your job,

but how you are saying them must be something that you're always thinking about.

We use intonation to do all sorts of things on the news, all the

time, and it's very, very complex.

At the beginning of the news, we're saying, "This is urgent,

"it's exciting, you've got to watch,

"something really important happened today and I've got to tell you about

-"it."

-Yeah.

-And then once we get into the news,

we are trying to use the right intonation for the type of story

we are telling people, so we've got to try and hit the right note, literally.

So if it's something very surprising,

"The new president of the United States is Donald Trump!"

-That's quite surprising.

-That is quite surprising.

If it's very serious or frightening news, something terrible has happened,

"The new president of the United States is Donald Trump."

It's a different thing.

You wouldn't normally do that, but if you were sort of talking about

an attack or a war or something like that,

this is a very serious thing and you're trying to get people in the

right frame of mind. But you're also trying not to frighten them

and you've got to use intonation for that.

Now we've got a very short experiment to do with Krishnan,

who's going to read out some football results.

Just by the intonation on his voice, I want you to...

He's going to stop before he gets to the final score and we're going to

work out if the last team had a higher score or a lower score than

the first team. OK, shall we try one?

Manchester City 2, West Bromwich Albion...

Higher or lower?

-Nil.

-Nil, there you go, lower.

West Ham United one, Tottenham Hotspur...

-Higher, yes.

-Five.

Chelsea one, Liverpool...

One - exactly, it's a draw.

This is absolutely amazing.

What Krishnan is doing, is over the whole course of the sentence,

he's doing a kind of dance with his intonation

that's completely keeping pace with the meaning of what he's saying.

You've got weaving the intonation in and out of the words and you're

picking up on that, you know what that means.

Can you do an example of it incorrectly so we can hear what that sounds like?

Stoke City four, Huddersfield...

five.

And when you say that, it sounds like it's the words that are wrong, whereas it's the

intonation that's wrong, it's really striking.

Thank you very, very much, Krishnan.

-Thank you.

-Thank you, Sophie.

So we are using intonation all the time in regular conversation,

and sometimes we think that that's all we do to pick up on emotion in the voice.

Actually emotion in the voice is incredibly complex.

There's a great deal going on because emotions change how your bodies work

and how they feel. They can change many different aspects of your voice.

Up to now,

computers have really struggled with this kind of information.

But that's changing.

And people are starting to make more progress.

For our last demonstration,

I'm going to look at something really amazing -

a computer that can read emotions from human voices.

Now the acoustics in here are quite reverberant

and this computer has been built to work in the home,

and so what I'm going to do is just step outside where there's a lovely

carpet and it's a little bit more like being in someone's living room.

Let's go and meet Olly.

-Hi. Hi, Raymond.

-Hello.

Now, tell me about Olly.

So Olly can actually understand what you're saying, as well as

emotions of your voice, the tone of your voice.

Excellent, what kind of thing do you do with Olly?

Olly actually enhances the communication between human and technology.

So he could be a digital assistant who's really understanding how you

feel, not just what you are saying?

-Exactly.

-Yes.

-Can we find out more about how he works?

-Sure.

-Is it easiest to demonstrate that?

Yes, of course. So what you need to do is just stand a sensible distance

from Olly, speak normal as usual,

-and then maybe you can add some emotion to it when you speak.

-OK.

So, are you ready?

One, two, three.

SADLY: Hey, Ollie, what's the weather like in London today?

Oh, there we go, yes.

So is Ollie getting the emotion out of your voice by the sound of your

voice or the words you're saying, or is it both?

Actually it's both.

It recognises some acoustic components of what you said,

and it's also using the syntax and semantics of the words you used.

It's very clever. Thank you very much,

-thank you.

-Thank you very much.

Olly is doing an amazing job of decoding emotion as well as words

from the human voice, and you can see from this just how hard emotion

is to read and how extremely nuanced it can be.

We find it very easy, because we've grown up around people using language,

emotion, social meaning in their voices all the time,

so it seems easy to us but it's absolutely vital to understanding language.

All around the world,

animals are communicating in very simple and very complex ways and we

have so much more to learn.

I've touched on just the surface of animal communication today.

Within this, humans do seem to be extraordinarily complex in their linguistic abilities.

But we're only really using language with each other.

Could there ever be another life form that could crack our code -

perhaps one we haven't even encountered yet?

Watching Carl Sagan describe his work on the Voyager space probes

40 years ago in the 1977 Christmas lectures,

literally set me on course to standing here today -

that's why I became a scientist.

The Voyagers contain golden records,

and the golden records contain the sounds of Earth,

including greetings in 55 different human languages.

GREETINGS IN DIFFERENT LANGUAGES

Now, the sounds of the Earth's languages

have long since left our solar system.

The Voyager probes are now 13 billion miles away from Earth.

They are the most distant objects ever created by humans.

Now, if those spacecraft ever encounter an alien life form,

maybe they can use these sounds to start to decode our language.

And if they have brains that work like ours,

maybe one day we could have conversations with them.

This year's lectures have explored our fundamental urge to communicate.

We've looked at where it comes from and why we're so very good at it,

and we just can't help but reach out to others.

The big question is whether there is another form of life out there

that could crack our code,

and I hope you've realised that it will need to have an incredible brain -

at least as incredible as your brains, to do so.

Thank you.