The Secret Science of Pop

# Oh, baby, baby, the reason I breathe is you

# Boy, you've got me blinded... #

We're told that ours is an age of data,

an age in which practically everything we do,

say and make can be reduced to flows of information by algorithms

spinning away on server farms in remote locations.

Data science already tells us where to go, how to get there,

who to date and what to buy them.

But all that's rather trivial stuff.

I think that even the most glorious, ephemeral and marvellous

products of the human mind

can in fact be measured by science.

And by that, I mean pop.

MUSIC: I Really Like You by Carly Rae Jepsen

Armed with just a few algorithms,

I intend to change the way we understand pop.

This is based upon the music -

the wave forms, the numbers.

I'm gathering a team of data scientists to analyse

five decades of the UK's Top 40 hits.

Together, we'll show the artists what science can do.

# Doo, doo! Doo-doo-doo-doo! #

We're teaming up with pop legend Trevor Horn.

So, this is a thing called M Harmony PCA 4.

Right.

# Into the ocean with me... #

We're going to try to do something never tried before -

use data to give an unsigned artist a potential hit.

I like the end. That was good.

I can't even imagine, like,

what the science is going to say or do to the song.

MUSIC: Hey Ya! by OutKast

I'll also be using my analysis to map the turning points

of pop history.

Love Me Do, right on the average.

Yellow Submarine, right on the average.

Lennon and McCartney are writing ditties for prepubescent girls.

We don't know what the answers will be.

Yeah, it's not exactly an advertisement for machine learning, is it?

-No.

-THEY LAUGH

But that's what happens when science...

-Thank you, sir.

-Thank you for exposing me on national television

-in getting it wrong!

-THEY LAUGH

Hey.

..meets culture.

What exactly are your qualifications?

My PhD was on fruit flies,

I've spent most of my professional life studying worms.

MUSIC: You're Gonna Miss Me by 13th Floor Elevators

Let me begin with a confession.

I'm not much of a music fan.

Worse, as an immigrant that's citizen of nowhere,

what I know about British pop history is distinctly second-hand.

It's not that I don't like the stuff, it's just that I don't

have stacks of vinyl at home.

And if you spent your youth

throwing up on the King's Road or off your head in a field outside of

Reading, then you know more about British popular music than I do.

But to do what I want to do I don't have to be a fan.

That's because what I want to do is science.

You may wonder why an evolutionary biologist should

decide to study the charts, but just as fruit flies and finches evolve,

so too, I believe, does pop.

Every new song comes with its own burden of mutations.

Some of them bad, but a few of them flourish

and get passed on to future generations.

Listen carefully and you can hear the music evolve.

There are countless examples, but one clear primordial ancestor

is Kraftwerk's Autobahn.

Bit of honking, bit of synthesiser.

The weirdness begins right from the start.

# Fahren, fahren, fahren auf der Autobahn... #

"Bahn, bahn, bahn, autobahn?"

I mean, really.

It must've been the weirdest thing possible when people first heard it.

# Fahren, fahren, fahren auf der Autobahn... #

Think of its glorious weirdness as a musical mutation.

Remember, this was 1974.

Bizarre though it may have been,

Kraftwerk's mutation changed the course of pop evolution.

Three years later, record producer Giorgio Moroder heard it,

absorbed it and put it into a song

that he made for disco's ultimate diva.

It begins very Kraftwerk-like.

MUSIC: I Feel Love by Donna Summer

Driving drum machine.

Synthesisers coming in.

But it's got a different feel, this is dance music.

And then.

# Ooh

# Heaven knows, heaven knows... #

Donna Summer having an orgasm, or at least faking one.

I Feel Love was a glorious synthesis of disco,

early electronica and pure sex.

It was the future.

Moroder's formula would become the basis of electronic dance music

and its innumerable subgenres -

house, techno,

not to mention neurofunk, speedcore and cybergrind.

I have no idea what I'm saying

but I do believe that all genres only exist because musical mutations

are passed from one generation to the next.

And it's that inheritance, that lineage,

which is then transmitted and recombined with other elements.

That is the essence of evolution.

That is how pop music evolves.

MUSIC: Hey Ya! by OutKast

MUSIC: Back In Black by AC/DC

MUSIC: Fame by David Bowie

To me, pop should be a science of diversity and change,

competition and conflict.

MUSIC: Heartbreak Hotel by Elvis Presley

MUSIC: Losing My Religion BY R.E.M.

MUSIC: Toxic by Britney Spears

MUSIC: Royals by Lorde

But if we're going to make it a science,

a Darwinian science, we need to do what scientists do -

experiments.

MUSIC: Blue Monday by New Order

And in my first experiment,

I want to find the musical adaptations that define

pop success today and put all of them into just one song.

But to do that, I'll need a music producer who'll let science

into his studio.

# They took the credit for your second symphony

# Rewritten by machine on new technology

# And now I understand the problems you can see

# Oh, oh... #

Trevor Horn is the man behind some of the catchiest tunes in pop.

From writing songs with The Buggles...

# Video killed the radio star... #

..to producing Seal and Frankie Goes To Hollywood,

he certainly knows how to make a hit,

even if he isn't clear as to how he does it.

Well, it's like those little puzzles, you know,

where you've got to get, like, five balls into a hole.

And you kind of manoeuvre it and you get one into

a hole and then you get a second one in and then you're trying to

get the third one in and the first two pop out.

You know? Trying to get a hit record's a bit like that.

Every time I've ever tried to analyse it and get any sort

of hard and fast rule, it always changes and...

# If I couldn't read you... #

Trevor and I will be working with Nike Jemiyo,

an unsigned singer.

We're going to take one of her songs and try to turn it into a hit.

She seems unconvinced.

I think there's a reason why some songs last for decades.

And I think that's more to do with heart, maybe, than science. Maybe.

Imagine if you had a formula and all you had to do was adhere to

this formula and you could churn out hit records, that would be so funny.

I just can't see it, though.

# Relax don't do it

# When you want to suck it to it

# Relax don't do it

# When you want to come. #

But I intend to bring some analytical firepower to bear

on this problem.

I've put together a team of analysts from the BBC R&D

and from Queen Mary and Oxford universities

and brought them to where I work,

the Data Science Institute at Imperial College London.

So if you look through, I've got, erm... Is there anything in there

we recognise? Oh, there's Saturdays. I've heard of The Saturdays.

We're beginning our analysis with the last six years of chart music.

Who else is in there? Kanye West.

Got some Katy Perry.

So once we've got all the songs together,

what we need to do is to extract the information from them.

In effect, you're asking computers to listen to music.

Erm...

Tim Cowlishaw and Mi Tian

have the job of reducing our songs to numbers.

The basic idea is to turn the music, as something humans can hear, into

machine-understandable data with meaningful information stored in it.

MUSIC: Baby by Justin Bieber

We're recording tempos, what instruments are present

and what pitches are being played and when.

We're even measuring the length and structure of each song.

By the end, we've turned sound waves into this.

These are the raw data of a single song,

the information that makes it -

the DNA of that song, if you will.

We start with more than a million numbers per song

and then distil them down to quantify its essence.

I can't pretend that these numbers mean anything to me.

Most of the features we've measured are hard for humans to interpret.

To a scientist, numbers on this scale, on this magnitude,

are beautiful.

The difficulty comes in knowing what they mean.

MUSIC: Ice Ice Baby by Vanilla Ice

But, in truth, I don't have to know what these numbers mean or

how they relate to what makes a song great.

That's because we live in the age of machine learning.

In the old way of doing science, the kind of science

that I've done all my life,

you look for causal associations between variables,

the way in which one thing that you've measured in the world affects

another, and you've got an explicit hypothesis about how that works.

Machine learning doesn't go like that.

Our computers know a track's success.

They know which songs topped the charts for weeks...

..and which scraped in at number 40.

The machines then sift through our millions of data points.

They're looking for those features that tend to be present in hits

but absent in flops.

The machine learning approach is to measure everything that you

can possibly measure about a song and throw it into the pot

and let the algorithm figure out what makes a hit.

# Shake it off

# Shake it off

# Shake it off

# Shake it off

# Shake it off

# Shake it off

# Shake it off

# Shake it off Oh-oh

# Shake it off... #

It'll take a lot of data-bashing to get any results,

so it's a good time to introduce Trevor to my way of making music.

For all the time that I've spent analysing pop music,

I've never actually met a real live producer,

never mind the man who made the '80s.

# They send the heart police to put you under

# Cardiac arrest... #

-Ah, Armand.

-You must be Trevor.

I am. Come in.

-Lovely to meet you.

-Nice to meet you.

Come through.

The reason that this kind of study hasn't been done before is

simply because up till now you've not been able to analyse that

-number of songs...

-Yeah.

-..in a quantitative, scientific way.

-Right.

-And I would claim that I, an evolutionary biologist,

know more about popular music...

-Right.

-..than anybody up till now.

Well, what exactly are your qualifications...

to be the person that knows the most about pop music?

My PhD was on fruit flies,

I've spent most of my professional life studying worms.

Very important worms, as it so happens.

The reason I claim that I think that I know more is because,

in some ways, precisely because I am ignorant.

-Right.

-And because that means that I can just let the numbers talk to me.

'But Trevor's not convinced that the high road to pop stardom

'is paved with data.'

Can I tell you, I have five things that I look at, right?

If you want to be a successful artist, right?

You have to be able to write

or have access to the best material,

you must have a really great voice, two octaves,

you have to have personal charm and charisma...

Computers can't measure personal charm and charisma.

..you have to be physically and mentally strong. OK?

The fifth one is you've got to want it.

OK, forget about making her a star. THEY LAUGH

-But that's what you're up against.

-She may well have the qualities.

-Make her a decent record.

-Produce her, yeah...

-Yeah.

-..a song that could feasibly, plausibly,

be released and not disappear into the void of...

-The void, the black hole of...

-The black hole of YouTube.

And if it works, we're going to go in business as a production company.

-Right!

-TREVOR LAUGHS

And become...just a hit factory.

MUSIC: Mrs Robinson by Simon & Garfunkel

But we haven't just gathered modern music.

I think we can use my techniques to see how pop

has evolved over the years.

The thing that I love about pop music is that it comes with its

own meticulously-documented fossil record, the UK Official Charts.

We've got the songs for about 50 years and we can study them all.

Just like the modern tracks,

these historic songs have been converted into numbers.

But here we're looking at how the music changed over the years.

Take 50 years of music, 17,916 songs,

turn them into millions of numbers,

boil those numbers down into a single variable

and this is what you get.

This is the rate of evolution of the UK charts

over the last 50 years.

This is what actually happened.

When the red line is high, the music is evolving quickly.

When low, slow.

And when it crosses the yellow line,

that's when the UK charts had a revolution.

And it begins on a high, with a revolution.

Our first revolution is centred around 1964,

the year that gave us a new TV channel...

-ARCHIVE:

-BBC Two opening night.

MUSIC: Rockin' Robin by The Hollies

-..pirate radio...

-My name's Simon Dee,

with you for the next two hours.

First one off the top of the pile, The Hollies - Rockin' Robin.

..and a pop chart in which musical evolution was in overdrive.

We've forgotten what the sound of the early 1960s was.

It was big, smooth orchestral numbers.

People like Frank Sinatra, Ella Fitzgerald, Connie Francis

were in the charts.

It's like contemplating an age of dinosaurs

before a mass extinction event.

MUSIC: You Really Got Me by The Kinks

It was music for grown-ups

and it was doomed.

# Girl, you really got me goin'... #

And this is the sort of music that swept it away.

This is The Kinks - You Really Got Me.

Dave Davies is the man behind that crunching guitar.

HE PLAYS GUITAR

It's just G and F, you know.

All the different things you can do with G and F.

# You really got me now... #

This was British rock and roll.

# Oh yeah, you really got me now... #

It was loud and sexy,

nothing like the pretty orchestral stuff it replaced.

# You really got me... #

But if you play just the three bottom of notes on guitar...

..sounds bigger

than if it was played like a full chord, which would be...

prettier but...

When you really dig in...

sounds more powerful, sexier.

More aggressive, I guess.

The word aggressive is key.

These songs tend to have fewer harmonies,

stronger rhythms and more thrashing guitars.

So by combining these features into a single variable -

aggression - we can see how the charts have changed.

Aggression rises rapidly in '63, '64,

moving through to '65,

and we can go and look at the artists

that are actually coming in here.

Pretty Things, The Rolling Stones and, of course, The Who.

# You say I've been in prison

# You say I've got a wife... #

The data shows the birth of British beat music,

a musical mutation that swept all before it.

Now, you will surely not be amazed to hear that there was

a pop revolution around '64.

But we've found it just by feeding the songs to a computer.

The thing to remember is that none of this is based upon

the standard cultural mythologies of pop,

the hazy recollections of journalists and rock stars.

This is based upon the music -

the wave forms, the numbers.

MUSIC: Love Me Do by The Beatles

But the numbers conceal a surprise.

# Love, love me do

# You know I love you... #

You may have noticed that I've not mentioned

a certain four-piece that did quite well.

'64 may have been the peak of Beatlemania

but the data suggests that musically they weren't that important.

And none of their charting singles sit high on our plot of aggression.

Love Me Do, right on the average.

Yellow Submarine, right on the average.

Hey Jude, right on the average.

Penny Lane...ah, a little bit below.

It's the hallmark of The Beatles - average.

Who isn't average?

The Kinks aren't average, The Who aren't average,

The Pretty Things aren't average, The Dave Clark Five aren't average,

The Rolling Stones - they're certainly not average.

The London bands are dragging mean aggression up

and transforming the musical landscape.

Meanwhile, Lennon and McCartney

are writing ditties for prepubescent girls.

Now, before you write into Points Of View,

let me be clear.

I'm not saying that the Fab Four weren't culturally important,

that they didn't have winsome personalities and great haircuts.

And I'll even concede that Sgt Pepper may well be the most

influential album of all time.

Or not.

But the fact remains -

The Beatles sat out the British revolution of 1964.

My team are still searching for the numerical ingredients

of a modern pop hit.

PIANO PLAYS

Meanwhile, Trevor's getting his first listen to the song

we'll be working on.

# If I couldn't read you

# The signs say you've moved on

# Cos when we talk and we walk

# Down the lane of memories

# Dive into the ocean with me

# Cos if you stand still... #

The song's called Dive.

Nike wrote it about taking risks to achieve your dreams.

# What we could be. #

Good.

I was going to say, you probably started singing in church, right?

-Yes, I did.

-Yeah, I can tell from the sound of the way you're singing.

But, I mean, there's nothing wrong with that,

that's where Dionne Warwick started to sing, church.

You know? It's a great place to learn.

I mean, great place to learn music, anyway.

Yeah, it's a pleasant song, so...

There's loads of people with pleasant songs,

you've got to find some way of getting through all of them,

-you know? But there's a couple of things that we can try.

-OK.

And the first thing would be to get the song out of you

with a piano and a click.

OK.

METRONOME CLICKS

PIANO PLAYS

Science can't yet direct a singer to produce a perfect vocal,

so I'm leaving this to Trevor.

# If I couldn't read you

# The signs say you've moved on... #

Stop there. Just one more time.

I think a teeny bit more edge than that, if you can do it.

# Packed your past in a box on a ship

# Sail till dawn. #

Sorry to be a pain, to keep you doing it, but just try...

Can you try it at 65?

# Now you've had your freedom

# You want to stay out in the cold

# You think it's easier

# When you're given much to hold... #

That's good. That's what I meant.

I can't even imagine, like, what the science is going to say

or do to the song but it will definitely be interesting.

# Into the ocean with me

# Cos if you stand still... #

I can't wait to hear what happens when I come back

and what they've done with it.

# What we could be. #

I like the end. That was good.

That was kind of a bit of Minnie Riperton there.

Good. You got through it.

Come here and take five.

Yeah, I think you slowed it down.

Oh, did I? Where do I start to slow down?

At the top.

THEY LAUGH

MUSIC: Can't Get You Out Of My Head by Kylie Minogue

The next job is to see if our analysis can turn Dive

into a chart-topper.

So, what we want to know is, what is the magic ingredient that

makes a pop song a hit, right?

As opposed to a non-hit.

Ben Lambert's been going through the data.

The hope was that through machine learning we'd find musical

features that help us distinguish hits from flops.

But the results are not very promising.

So, 50-50 would be just picking randomly.

And we get an accuracy of about 52% or 53%.

-Right.

-So, slightly better than just randomly picking

but, basically, not.

Yeah, it's not exactly an advertisement for machine learning, is it?

-No.

-THEY LAUGH

If we can't identify the features that predict the relative

success of a song, then it's really not clear what to tell Trevor.

We run all the models we can think of, we tweak them in all

the different ways we can, we subset the data...

in all kinds of ways...

..and we get nothing.

We can't predict it.

We just don't know what makes a hit.

But there's a faint glimmer of hope,

for we have identified one correlation.

The one thing which we did find, and it's not a very,

very strong signal, but it's a statistically significant signal,

so that's something, is that there's an association

between success in the charts and

how close a song is to the average.

MUSIC: Style by Taylor Swift

Imagine a perfectly average song,

one whose every feature sits at the centre of our distributions.

Our analysis shows that such a song should do better than most.

Of course, none of our songs actually hit that statistical

sweet spot, but we can measure how close they are to it,

and the closer a song gets, the better it seems to do.

Perhaps, then, what we have to tell Trevor

is to simply make Dive really average.

I guess I would say that in all my years as a scientist,

the discovery that the most average song tends to be the most successful

song is one of the more depressing results that I have ever found.

And I fear that Trevor won't like the news.

The thing that I've found that is predictive of success

is how average the music is.

This is a force which is sort of driving music at any given time

to some sort of...

I don't want to call it a lowest common denominator, but sort of

the centre of the distribution, a kind of a homogenising force.

Do you think that there's a certain inevitability about that?

Because we're in a unique position at the moment.

We're in a position that no-one's ever been in before,

where we have at least 50 years of, 60 years,

70 years of recorded music.

We can go back and we can listen to all of this music

-from the '60s, the '50s.

-Yes.

Never been able to do that before.

I suspect that Trevor's right.

Today, in a few finger taps, you can hear almost any song

in recorded history.

Perhaps this explains why our algorithms have struggled here.

If modern artists are combining genres promiscuously,

the result will be the songs that are neither one thing nor the other.

But this hasn't always been the case.

Our pop history shows that the charts were once

a bloody battleground in which genres vied for supremacy.

We are back at our rate of evolution plot and you can

see that the rate at which the music is changing in the charts

begins to pick up in the 1970s.

Around 1975 it crosses the line, we are into a revolution.

It peaks in the late 1970s,

when the music is changing with maximum speed.

This revolution is going to become

one of the most important in the history of British pop.

When this revolution came, the country seemed half asleep.

The UK was a sea of brown, orange and mustard.

But something was stirring in the pop charts.

# I am an antichrist

# And I am an anarchist... #

Ask a Brit of a certain age what happened in the late 1970s

and chances are he'll say, "Well, mate, it was all about punk."

There's no doubt that punk rock's cultural impact was immense,

so much so that it's easy to forget just how tiny it all was.

MUSIC: No Future by Sex Pistols

Safety pins, spiky hair, spittle and swearing on national TV.

Punk grabbed all of the headlines.

Go on, you've got another five seconds, says something outrageous.

-You dirty

-BLEEP.

-Go on, again.

-You dirty

-BLEEP.

-What a clever boy(!)

-BLEEP.

But that's not the same as making music that mattered.

This is a network of musical relationships

between some 800 artists,

pretty much everybody who charted in the UK in the 1970s.

And it's based upon the music as measured by our computer

and you can see that the relationships it gives make sense.

The more songs of a particular genre,

the bigger the block of colour.

Up here, for example, we have funk and disco,

they are all grouping together.

Up here, we have vast swathes of soft pop.

James Taylor, Joan Baez, Gordon Lightfoot, people like that.

And down here, in yellow, we've got punk.

And there's not very much of it.

In the entire 1970s, only about 68 songs that could be called punk

by any reasonable definition charted.

MUSIC: Hersham Boys by Sham 69

There just wasn't enough punk to have had a significant impact

upon the evolution of the UK charts.

So what was changing pop?

In the '70s, we see songs becoming faster.

MUSIC: We Are Family by Sister Sledge

And more percusso.

Again, by combining features

we create a new variable, rhythmic intensity.

It's the thing that changes in the 1970s.

It begins to increase in 1972,

climbs rapidly, peaks in 1979.

Whatever's changing in the 1970s isn't punk, it's not rock,

it's something else and it's coming from America.

# We are family

# I've got all my sisters with me... #

It started as a funk invasion

but quickly morphed into the music of glitter balls and flairs.

# Get up everybody and sing... #

Disco may have started in the black and gay clubs of New York City...

MUSIC: Stayin' Alive by Bee Gees

..but in the UK, it was Saturday Night Fever

that sent it stratospheric.

And the irresistible grooves of the Bee Gees ruled the charts.

You can hear, you can feel, you can see

and we can measure that rhythmic intensity, that driving beat.

The data are unambiguous -

a tidal wave of disco flooded the charts

with pulsating four-on-the-floor rhythms.

Punk arose, flourished and vanished in almost an instant.

So why does punk rock seem to matter so much?

I think it's a combination of British chauvinism, nostalgia,

Johnny Lydon's charisma and Vivienne Westwood's clothes.

But the fact of the matter is that as far as the music is concerned,

it was never that special.

# If I choose to believe... #

-MUSIC REWINDS

-# If I choose to believe you... #

MUSIC REWINDS

# If I choose to believe... #

Trevor's working with arranger Julian Hinton.

# Dive into the ocean... #

Their first job is to produce Nike's song as they normally would.

This is more of an organic feel so, because of the nature of the song,

it's got a lot more emotion to it,

it needs to have an ebb and flow.

That's why I am being more painstaking.

I'm going into a lot more detail

so that hopefully it retains its performance

and be the best version of what is essentially there.

I'm not trying to...

fix or change the character of it.

The song is a ballad, so they're putting luscious strings

underneath it.

Whether they be from a keyboard or from an orchestra

they're a very, very warm and expressive sound,

probably one of the...

Strings are probably one of the most expressive things

other than the voice.

# Dive into the ocean with me

# I know there's danger

# But this time I'm braver

# Dive into the ocean with me... #

-See just that bit there?

-MUSIC STOPS

-The dive, can you just make it dive at the right time?

-Mm.

He's hitting the water too late.

# Dive into the ocean with me

# I know there's danger

# But this time I'm braver... #

'With Trevor happy, it's time for me to see if I can get

'Dive's features as close to the average as possible.'

-Hi, guys.

-Hi.

'But to do that, we need a point of reference.

'We need to hear some songs that our data show really are average.'

-Oh, yeah, it's G-Eazy featuring...

-Featuring Bebe Rexha.

-Featuring Bebe Rexha.

-Yeah.

MUSIC: Me, Myself & I by G-Eazy and featuring Bebe Rexha

Yeah.

There's some sonic things going on,

-there's a very electronic percussion.

-Yep.

And a very crisp electronic high-end as well, which is copied

from one track to the next and is a really classic pop structure.

MUSIC: You Don't Own Me by Grace

Next up, Grace's You Don't Own Me.

-Exactly.

-Now here we've got a blossoming,

very overtly uplifting sonic here.

-It's got a '50s sound and it's got an '80s, '90s sound...

-Yep.

..and all kinds of other things in between in terms of,

"Ooh, I can hear '70s influenced drums

"or some strings from the 1950s going on in there."

As Trevor predicted, our average songs seem to be a mix of everything

that's gone before.

In order for that track to become successful,

people have got to be OK with all those sounds and textures.

We come to a place of unbelievably...

believable open-mindedness, actually.

# Dive into the ocean with me... #

'The current charts are a homogenised blend of earlier genres.

'That suggests that we need to make Nike's song...'

Nice.

'..into a mishmash of them.'

If I take that down to the original...

-DRUMS PLAY

-# Dive into the ocean with me... #

It doesn't work...

in that style.

Let's take it up even more.

-TRACK PLAYS FASTER

-# This time I'm braver... #

Now we're into sort of... It feels more like a remix.

'But although we can estimate an average,

'it's quite hard to define it musically.'

TRACK PLAYS

So that's the other groove I have.

# Dive into the ocean with me... #

I'm hearing, sort of, mid-Madonna, a little bit there.

'The features themselves are hard to interpret,

'so to find the centre of the charts, we have to experiment.'

One of the things that a lot of the average tracks have,

and I think this is precisely why they're average,

is that they have this combination of a big melodic segment

and then you've got a... Let's say pop dude, sort of, who comes in,

does a bunch of rap.

Can you give Nike's song that?

In other words...

can we get a rap in there?

Jul?

HE LAUGHS

I'm sorry,

are you just googling "rap a cappella edify bmp"?

And I have 118,000 hits.

RAP PLAYS ON COMPUTER

Er...

-RAPS:

-# So many people give their religion

# The music and church let you know how you're livin'... #

'Now this is Crash DDZ, a rapper from Kentucky.'

DRUMS PLAY

# Everything happens for a reason... #

HE LAUGHS

Who's controlling this, by the way? Is it you or you?

-That was a collaboration, actually.

-That was a collaboration.

-Seriously?

-A classic production collaboration.

# We can sense our defences come down so easily

-RAPS:

-# Man would stand, guitar in his hand

# Recording artists, the history, tell us, travelling bands

# Travelling bands, Johnny Cash and clones... #

'In a few mouse clicks, and for 1,

'we had a rap to pair with Nike's vocals.'

This is going to be one happy rapper.

Until we measure Dive, we won't know how close we've got to our goal

of making its features average.

So it's not clear how useful my analysis will be.

# Dive into the ocean with me... #

'But what is clear is that music production

'is already very reliant upon technology.'

The tools that these guys are using, high-end computers,

various kinds of programmes, different kinds of algorithms,

they're utterly familiar to me and yet their product,

the things that come out of this stuff, it's...

Well, it's like magic.

And there's no doubt that the influence of technology

on the last three decades of the pop charts has been immense.

But it doesn't show up in our data where you might expect.

We've seen a revolution in the 1960s, British rock and roll.

We've seen another revolution in the 1970s,

the rise of disco and funk. But what about the 1980s?

The music's changing, it's always changing,

it's not just changing very fast

and there's nothing resembling a revolution.

MUSIC: Take On Me by A-ha

This may be a surprise - the 1980s was the decade of A-ha...

MUSIC: Karma Chameleon by Culture Club

..Culture Club...

MUSIC: Never Gonna Give You Up by Rick Astley

..and, of course, Rick Astley.

But while there was no pop revolution in the '80s,

I do think that it contained the seeds

of the greatest revolution of them all...

..the rise of the machines.

As soon as technology came along, you could dream something,

you could make it happen, you know?

If you sort of dreamt something

and you tried to make it happen in the '70s, it was harder

because you had to get people to play it.

In the wake of Kraftwerk's pioneering electronica,

the '80s and '90s brought the arrival of increasingly versatile

drum machines and synthesisers.

In a few clicks, producers could make any sound they wanted.

So where is tech's influence on the evolution of pop?

We just need to plot our data in a different way.

If we take 17,916 songs - all of the songs in our data

over the history of the UK charts - and plot them

on a single graph of rhythmic intensity,

what we see is something that looks like this.

The key to this plot is the vertical range of music

at any given point in time.

That tells us how much rhythmic variety is in the charts.

We begin in the 1960s. Down here we've got low-intensity stuff,

Shirley Bassey.

Up here, Chubby Checker.

They sound very different but the range is relatively small.

We move along.

The average is changing but that's not actually the big story here.

As we progress into the early '80s,

we start seeing a new world of high-intensity music.

This is electronica, Kraftwerk,

Cabaret Voltaire, Bam Bam,

dance music, house, techno,

all this stuff is coming in and it expands even further.

This expanding space is the third great story in our history of pop.

It's the relentless advance of electronic dance music.

DJ Annie Mac explains why she thinks it's so successful.

It's the collective experience of 100,

1,000 people standing on a floor...

# The weekend is coming up... #

..all experiencing the same thing.

That kind of physical collective experience

is a very beautiful thing.

If you say so.

But the most striking thing is how many sub genres dance has spawned.

House, techno, funky, grime,

drum and bass, jungle, hard-core, break beat, big beat.

Liquid drum and bass, progressive house.

And it's easy to see why this diversity flourished.

I would say technology is definitely a massive reason for

why dance music, electronic music has been so prone to expansion

and evolution and fragmentation.

It's one of the most exciting things, I think,

about electronic music.

# Dive into the ocean with me

# I know there's danger... #

Back in the studio and we're still trying to make Dive truly average,

a song that mashes everything together

to sit at the statistical centre of the charts.

And we think we need a fair chunk of dance intensity.

-How much bass have you guys put on this thing?

-How much bass?

Yeah.

I mean...

is it the case that we can push this towards more of a dance song?

That's really the nub of the problem.

This is a ballad and ballads are a completely different thing and,

you know, if you think about the best dance songs of all time,

-say something like Boogie Wonderland by Earth, Wind & Fire...

-Yep.

..the tune in the verse is all off the beat.

HE SINGS THE MELODY OF BOOGIE WONDERLAND

It's all off the beat and it dances on top of the track.

Most dance songs don't start out life as ballads, you know?

We're going to try one last push to make Nike's song really,

really average.

# Dive into the ocean with me... #

We've even brought her back to do a fresh, faster vocal.

# But this time I'm braver... #

-Given that you have a math degree from Imperial College...

-Yes.

..can you imagine just...

..setting yourself up as a music analyst from now on?

You know, every song you write, it's just going to be,

"Hmm, a little standard deviation away from the mean,

"got to move in there."

Well, maybe we'll see how this works.

Yes, I think perhaps we should.

If it works then maybe.

DANCE VERSION OF DIVE PLAYS

'We've created a few versions of Dive in the hope

'that one will stick closely to the chart average,

'our key to pop success.

'But some versions just don't work.'

All right...

-MUSIC STOPS

-Enough.

Yeah, we can only apologise to Nike, for doing that to her song.

No, but I would...

'Using data to make a hit is proving to be a challenge.'

But I've got another idea.

Thanks to the rise of technology,

we now live in a world of bedroom producers.

There's an ocean of undiscovered artists out there,

and some of them might even be competent.

Can algorithms find the stars of the future?

It's time for another experiment.

And to conduct it, I don't need to go far.

That's because the BBC is home to Introducing,

a website to which unsigned artists can upload their music,

music that we can analyse.

What I have here is a hard drive containing 1,786 songs.

This is the raw material of evolution,

unfiltered by any company or broadcaster

or any consideration of taste other than the musicians' own.

And what we want to know is, is any of it any good?

Normally each track is vetted by a human being

but I suspect that machines can do their job just as well.

What we do is we teach the computer, using a machine-learning algorithm,

what the charts are now

and then we apply that model to the Introducing data

and we ask which of the Introducing songs are most chart-like,

which does the computer think are most likely to go into the charts?

As before, our computers measure each song.

HE MUTTERS

'This is the same process we went through

'for all the chart music earlier on,'

it's just sort of MP3 files go in one end

and spreadsheets come out the other.

The result, a simple list with the most chart-like songs at the top.

Out of all those songs, our algorithm picked one.

And it's a song called Margarita by a group called The Modern Strangers.

The Modern Strangers were actually soon to play a gig in London.

So, two weeks later,

I found myself heading to a dingy club to hear them.

Margarita was the finale.

# Sit back, Margarita

# Nice and slow

# Oooh

# Baby, all I ever wanted was your love

# Ooh... #

Catchy. It certainly had people dancing.

# Sit back, Margarita... #

It's the first time I've really listened to the song.

The computer just gave us a name.

None of us had actually heard the thing.

And I've got to say, it's amazingly convincing.

Margarita is almost an old school disco track -

it comes straight in with a big beat and melodical.

The lyrics may be lacking but it's very, very danceable.

The computer is dumb.

It doesn't have a sophisticated model of beauty

or danceability.

But here's the thing - this song is great.

And people think it's great.

It seems to have bottled some musical magic.

And our computer algorithm has found that same magic, it's...

It shows that it can be bottled by math.

And that's rather amazing.

# Baby, all I ever wanted was your love

# Ooh. #

Thank you very much for having us! Have a good evening.

But every experiment needs a control.

BBC Music Introducing in Kent with Abbie McCarthy.

Good evening, it's after eight o'clock...

I want to pit my algorithm against some human competition.

..right here in Kent.

Abbie McCarthy is the Introducing DJ at BBC Radio Kent.

We get sent probably about 300 tracks a week,

so there's lots of music to listen through,

and then we're just looking for a song that really stands out,

whether that it's really well produced,

it's got a really good beat to it.

# I am scared... #

Then sometimes we have a moment where we're like,

"Wow, this song's really, really incredible."

# Oh-oh-oh... #

Abbie's also picked a song - Gold by singer-songwriter Shells.

I'm going to play both her choice and mine to Rhys Hughes,

the head of programming at Radio 1.

Now, it's not going to be a proper experiment,

but if I played you two songs, would you warrant that you could

pick out the one that Abbie picked and the one that the machine picked?

I've got a 50% chance, haven't I?

You do, that's why it's not a very good experiment but...

Song number one.

MUSIC: Margarita by The Modern Strangers

# Sit back, Margarita

# Nice and slow... #

Get the idea?

-Get the idea?

-Yeah.

Next up, Abbie's song.

MUSIC: Gold by Shells

# Oh, oh, oh

# Oh, everybody's made of gold

# So put me in black and white... #

MUSIC STOPS

One of those two songs, Margarita and Shells, was chosen by computer,

the other one by DJ.

Which was which?

I would say the computer picked the second one you've played.

Wrong. The computer picked Modern Strangers, it picked the first one.

-Right, OK.

-Yeah?

So you have to concede that at least our algorithm

is doing as well as a DJ.

What do you say to just ditching them all and replacing them

with a computer?

I don't think an algorithm would've picked out a Bob Dylan,

I don't think an algorithm would've picked out a David Bowie.

I don't think a computer can understand the emotional response

that you get to a record.

You know, we all have records that we know that make us happy,

records that make us incredibly sad,

records that make, you know, you want to jump around your bedroom

and, you know, throw shapes in the mirror.

And I don't think an algorithm can do that.

Isn't it really just a matter of time before

this rather romantic vision buckles?

-I think...

-And you will have machine-learning algorithms

-and you are going to become as teched up as Google.

-Yeah.

We are, I mean, but I'm an incurable romantic

and I think that... I think, you know, that the human voice

and the human passion will always, will always win through.

Thank you so much.

Oh, thank you for exposing me on national television

in getting it wrong!

I think I've shown that we can indeed pick fantastic songs

without listening to a note.

But it doesn't look as though I'll be selling my algorithms to Radio 1

any time soon.

As for helping Trevor, well, my team have finished analysing

all the versions of Dive and the results are frankly unimpressive.

Here we've plotted the distance of every one of the songs

in the charts over the last year or so from the centre of the charts,

from their average.

Right at the centre is Taylor Swift's Style.

Maybe that's why she's so successful.

And here you can see the problem,

here we have ever-increasing distance from the centre.

Dive is out here at the edge of musical space,

and it's not as though we didn't try to push it down here somewhere -

we gave it a bit of oomph, we upped the tempo,

we thought, "Maybe it needs more bass,"

so we gave it more of that. But whatever we tried,

it just pushed it further out into musical space.

We're not even getting near where we need to be.

And here's the thing, we can measure them,

we can plot them, but so far,

we don't know how to move them.

# If you stand still... #

All that's left is to break the news to the team.

-RAPS:

-# So many people give to religion

# The music and church let you know how you're livin'

# Save a part of life and give till it hurts

# Creativity... #

Well, erm...

So, er...

I like the song.

It's really nice and, erm, the question that we've asked is,

how far are those songs from the centre of the charts?

And the answer is...

..that they're all really far away.

HE LAUGHS

-So my thinking is...

-Yeah.

..that to the degree that we can tell you anything useful...

..it should be to ignore what we are saying.

'It seems that the recipe for pop success will remain hidden for now.

'Even hit-makers sometimes don't know it until they hear it.'

But sometimes when you're going through the process, things happen.

Just the way the singer sings that line

that you didn't think sounded that great,

but suddenly when this person sings it it just sounds amazing

and then suddenly, man, you have something, you know?

And you might have to work for days to get something really special.

It doesn't happen all that often, it's hard to find.

-We'd call it non-linear effect.

-Non-linear effect, yeah.

And non-linear effects,

-they're the hardest for us to get at nowadays.

-Yeah.

No, I can imagine.

We had hoped to show how science could help

a few really talented musicians how to make a great hit song.

And I have to say that we have singularly failed.

DANCE VERSION OF DIVE PLAYS

That's not to say that Dive isn't lovely,

and in exploring the musical space

I suppose that science did play a part, but only a modest one.

If Dive is any good, and I think it's great,

it's because Nike, Trevor and his team made it so.

# Defences come down so easily

# Dive into the ocean with me... #

Wow.

Wow. That's very different.

I'd never imagined as a dance track, no, but I like it,

I think it's good. I really liked the synths, I liked the rhythms,

I liked some of the chord changes.

My head was kind of bopping to it.

You may doubt that we can capture creativity,

that we can bottle art. And I have to agree,

it hasn't been very convincing. And yet, I still think we can.

After all, we've seen the power of data.

It's shown us how pop evolved...

..and how to find great new music.

There's nothing intrinsically mysterious about this,

it is after all just physics and neurobiology.

# Dive into the ocean with me... #

Given enough computational power, given enough data,

we will work it out.

When?

Not by the end of this programme, but we will.

# Dive into the ocean with me

-# I'm braver

-Yes, I'm braver

-# Braver

-Yes, I'm braver

# Dive into the ocean with me

# Cos if you stand still

# You'll never, ever, ever see what we could be

RAPPING

# What we could be

RAPPING

# What we could be. #