0:00:07 > 0:00:11It sometimes seems we're being deluged with data.
0:00:13 > 0:00:15Wave upon wave of news and messages.
0:00:17 > 0:00:19Submerged by step counts.
0:00:21 > 0:00:26Constantly bailing out to make room for more.
0:00:26 > 0:00:31We buy it, surf it, occasionally drown in it
0:00:31 > 0:00:37and with modern technology quantify ourselves and everything else with it.
0:00:37 > 0:00:41Data is the new currency of our time.
0:00:42 > 0:00:46Data has become almost a magic word for...anything.
0:00:46 > 0:00:53Crime and lunacy and literacy and religion and...drunkenness.
0:00:55 > 0:00:58You name it, somebody was gathering information about it.
0:00:58 > 0:01:01It offers the ability to be transformationally positive.
0:01:03 > 0:01:07It's, in one sense, just the reduction in uncertainty.
0:01:07 > 0:01:10So what exactly is data?
0:01:10 > 0:01:15How is it captured, stored, shared and made sense of?
0:01:16 > 0:01:21The engineers of the data age are people that most of us have never heard of,
0:01:21 > 0:01:25despite the fact that they brought about
0:01:25 > 0:01:29a technological and philosophical revolution,
0:01:29 > 0:01:33and created a digital world that the mind boggles to comprehend.
0:01:35 > 0:01:39This is the story of THE word of our times...
0:01:41 > 0:01:45..how the constant flow of more and better data has transformed society...
0:01:48 > 0:01:53..and is even changing our sense of ourselves.
0:01:53 > 0:01:55I can't believe this is my life now.
0:01:59 > 0:02:01So come on in, because the water's lovely.
0:02:13 > 0:02:14My name is Hannah Fry,
0:02:14 > 0:02:19I'm a mathematician, and I'd like to begin with a confession.
0:02:20 > 0:02:23I haven't always loved data.
0:02:23 > 0:02:27The truth is mathematicians just don't really like data that much.
0:02:27 > 0:02:30And for most of my professional life I was quite happy sitting in
0:02:30 > 0:02:35a windowless room with my equations, describing the world around me.
0:02:35 > 0:02:40You can capture the arc of a perfect free kick or the beautiful
0:02:40 > 0:02:42aerodynamics of a race car.
0:02:42 > 0:02:47The mathematics of the real world is clean and ordered and elegant,
0:02:47 > 0:02:51everything that data absolutely isn't.
0:02:51 > 0:02:54There was one moment that helped to change my mind.
0:02:56 > 0:03:01It was in 2011 when I came across a little game that a teenage Wikipedia
0:03:01 > 0:03:04user called Mark J had invented.
0:03:04 > 0:03:09Now, Mark noticed that if you hit the first link in the main text of any
0:03:09 > 0:03:14Wikipedia page and then do the same for the next page, a pattern emerges.
0:03:14 > 0:03:17So the page for data, for example,
0:03:17 > 0:03:19links from "set" to "maths"
0:03:19 > 0:03:25to "quantity" to "property" and then "philosophy",
0:03:25 > 0:03:28which after a few more links will loop back onto itself.
0:03:28 > 0:03:32Now, the page "egg" ends up in the same place,
0:03:32 > 0:03:37and even that famously philosophical boyband One Direction will take you
0:03:37 > 0:03:39all the way through to "philosophy",
0:03:39 > 0:03:42although you have to go through "science" to get there.
0:03:44 > 0:03:49The same goes for "fungi", or "hairspray",
0:03:49 > 0:03:54"marmalade", even "mice", "dust" and "socks".
0:03:54 > 0:03:59It was a very strange finding and it called for some statistics.
0:04:01 > 0:04:04Another Wikipedia user, Il Mare,
0:04:04 > 0:04:09wrote a computer program to try and investigate this phenomenon.
0:04:09 > 0:04:16Now, he discovered, amazingly, that for almost 95% of Wikipedia pages,
0:04:16 > 0:04:20you will end up getting to "philosophy" eventually.
0:04:20 > 0:04:25Now, that's pretty cool, but how did it change my mind about data?
0:04:25 > 0:04:31Well, the pattern that Mark J discovered and the data that was captured and analysed,
0:04:31 > 0:04:36it revealed a hidden mathematical structure,
0:04:36 > 0:04:42because Wikipedia is just a network with loops and chains hidden all over the place
0:04:42 > 0:04:46and it's something that can be described beautifully using mathematics.
0:04:48 > 0:04:55For me this was the perfect example of how there are two parallel universes.
0:04:55 > 0:04:58There's the tangible, noisy, messy one,
0:04:58 > 0:05:02the one that you can see and touch and experience.
0:05:02 > 0:05:07But there's also the mathematical one, where I think the key to our
0:05:07 > 0:05:09understanding lies.
0:05:09 > 0:05:13And data is the bridge between those two universes.
0:05:15 > 0:05:19Our understanding of everything from cities to crime,
0:05:19 > 0:05:21global trade,
0:05:21 > 0:05:24migration and even disease...
0:05:25 > 0:05:27..it's all underpinned by data.
0:05:30 > 0:05:32Take this for example.
0:05:33 > 0:05:37Rural Wiltshire and a dairy farm
0:05:37 > 0:05:41gathering data from its cows wearing
0:05:41 > 0:05:42pedometers.
0:05:44 > 0:05:46We can't be out here 24-7.
0:05:46 > 0:05:50The pedometers help us to have our eyes and ears everywhere.
0:05:50 > 0:05:53It turns out when cows go into heat
0:05:53 > 0:05:56they move around a lot more than normal.
0:05:56 > 0:06:00Constant monitoring of their steps and some background mathematics
0:06:00 > 0:06:04reveal the prime time for insemination.
0:06:04 > 0:06:06We'll be able to look at the data
0:06:06 > 0:06:09and within 24 hours there'll be a
0:06:09 > 0:06:12greater chance of getting her in calf.
0:06:12 > 0:06:15Data-driven farming is now big business,
0:06:15 > 0:06:19turning a centuries-old way of life into precision science.
0:06:23 > 0:06:26Pretty much every industry you can think of
0:06:26 > 0:06:27now relies on data.
0:06:32 > 0:06:34We all agree that we are undergoing
0:06:34 > 0:06:36a major revolution in human history.
0:06:46 > 0:06:48The digital world replacing the analogue world.
0:06:48 > 0:06:50A world based on data,
0:06:50 > 0:06:55they are made of codes rather than a world made of biological or physical
0:06:55 > 0:06:57data, that is extraordinary.
0:06:57 > 0:06:59Why philosophy at this stage?
0:06:59 > 0:07:02Because when you face extraordinary challenges,
0:07:02 > 0:07:05the worst thing you can do is to get close to it.
0:07:05 > 0:07:07You need to take a long run-up.
0:07:07 > 0:07:09The bigger the gap, the longer the run-up.
0:07:09 > 0:07:11And the run-up is called philosophy.
0:07:12 > 0:07:15In the spirit of taking a long run-up,
0:07:15 > 0:07:17we'll start with the word itself.
0:07:20 > 0:07:23"Data" is originally from the Latin "datum",
0:07:23 > 0:07:25meaning "that which is given".
0:07:25 > 0:07:28Data can be descriptions...
0:07:29 > 0:07:33..counts, or measures...
0:07:35 > 0:07:36..of anything...
0:07:38 > 0:07:40..in any format.
0:07:43 > 0:07:46It's anything that when analysed becomes information,
0:07:46 > 0:07:49which in turn is the raw material for knowledge,
0:07:49 > 0:07:52the only true path to wisdom.
0:07:54 > 0:07:55Look at the data on data.
0:07:58 > 0:08:00And before the scientific and industrial revolution,
0:08:00 > 0:08:03the word barely gets a look in, in English.
0:08:04 > 0:08:07But then it starts to appear in print
0:08:07 > 0:08:11as scientists and the state gather,
0:08:11 > 0:08:13observe and create more and more of it.
0:08:14 > 0:08:18This arrival of the age of data would change everything.
0:08:24 > 0:08:27Industrial Revolution Britain.
0:08:27 > 0:08:28For Victorians,
0:08:28 > 0:08:32booming industry and the growth of major cities were changing both the
0:08:32 > 0:08:36landscape and daily life beyond recognition.
0:08:37 > 0:08:41Into this scene stepped an unlikely man of numbers,
0:08:41 > 0:08:42William Farr,
0:08:42 > 0:08:47one of the first people to manage data on an industrial scale.
0:08:47 > 0:08:50William Farr had a quite unusual upbringing,
0:08:50 > 0:08:53in that he was actually the son of a farm labourer but who had managed to
0:08:53 > 0:08:56get a medical education, which was really very unusual for someone of his class.
0:09:05 > 0:09:08Farr very quickly became absorbed in the study of statistics.
0:09:08 > 0:09:11He was particularly interested, as you might expect for somebody with medical training,
0:09:11 > 0:09:14in public health, life expectancy and about causes of death.
0:09:15 > 0:09:18For anyone interested in statistics,
0:09:18 > 0:09:19there was only one place to be.
0:09:21 > 0:09:24Somerset House in London was home to
0:09:24 > 0:09:26the General Register Office,
0:09:26 > 0:09:27where, in 1839,
0:09:27 > 0:09:30Farr found his dream job.
0:09:30 > 0:09:34From up there in the north wing, William Farr, the apothecary,
0:09:34 > 0:09:37medical journalist and top statistician,
0:09:37 > 0:09:38would really rule the roost.
0:09:38 > 0:09:41Now, this place was almost like a factory.
0:09:41 > 0:09:44Here, they would collect, process and analyse
0:09:44 > 0:09:47vast amounts of data.
0:09:47 > 0:09:50So in would come the census returns, the records of every single birth,
0:09:50 > 0:09:55death and marriage in the country, and out would come the big picture,
0:09:55 > 0:09:58the usable information that could help
0:09:58 > 0:10:00inform policy and reform society.
0:10:01 > 0:10:05I think it's sometimes difficult for us to remember just how little people knew
0:10:05 > 0:10:08in the early 19th century about the changes that Britain was going through.
0:10:08 > 0:10:11So when Farr did an analysis of population density and death
0:10:11 > 0:10:15rate, he was able to show that life expectancy in Liverpool
0:10:15 > 0:10:16was absolutely atrocious.
0:10:16 > 0:10:18It was far, far worse than the surrounding areas.
0:10:18 > 0:10:21This came as a surprise to a lot of people who believed that Liverpool,
0:10:21 > 0:10:24a coastal town, was actually quite a salubrious place to live.
0:10:27 > 0:10:32At Somerset House, Farr spearheaded a revolution in the systematic
0:10:32 > 0:10:34collection of data
0:10:34 > 0:10:37to uncover the real picture of this changing society.
0:10:39 > 0:10:44Its scale and ambition was described in a newspaper at the time.
0:10:44 > 0:10:47"In arched chambers of immense strength and extent
0:10:47 > 0:10:48"are, in many volumes, the
0:10:48 > 0:10:54"genuine certificates of upwards of 28 million persons
0:10:54 > 0:10:59"born into life, married or passed into the grave."
0:10:59 > 0:11:02Here, every person was recorded equally.
0:11:02 > 0:11:04A revolutionary idea.
0:11:07 > 0:11:11"Here are to be found the records of nonentities,
0:11:11 > 0:11:15"side-by-side with those once learned in the law
0:11:15 > 0:11:16"or distinguished in
0:11:16 > 0:11:19"literature, art or science."
0:11:20 > 0:11:25But what really motivated William Farr was not just data collection,
0:11:25 > 0:11:30it was the possibility that data gathered could be analysed to help
0:11:30 > 0:11:33overcome society's greatest ill.
0:11:35 > 0:11:40Cholera was probably the most feared of all of the Victorian diseases.
0:11:40 > 0:11:43The terrifying thing was that you could wake up in the morning and feel absolutely fine,
0:11:43 > 0:11:45and then be dead by the evening.
0:11:47 > 0:11:50Between the 1830s and the 1860s,
0:11:50 > 0:11:53tens of thousands died in London alone.
0:11:56 > 0:11:59The control of infectious diseases like cholera,
0:11:59 > 0:12:02which no-one fully understood,
0:12:02 > 0:12:05became the greatest public-health issue of the time.
0:12:06 > 0:12:08However great London might have looked back then,
0:12:08 > 0:12:12it would have smelled absolutely terrible.
0:12:12 > 0:12:16At that point, the Victorians didn't have really a great way of disposing
0:12:16 > 0:12:19of human waste, so it would have flowed down the gutters
0:12:19 > 0:12:21into open sewers
0:12:21 > 0:12:22and out into the Thames.
0:12:22 > 0:12:25Now, the city smelt so bad that it
0:12:25 > 0:12:29was pretty plausible that the foul air
0:12:29 > 0:12:31was responsible for carrying the disease.
0:12:33 > 0:12:36Farr collected a huge range of data during each
0:12:36 > 0:12:38cholera outbreak to try to
0:12:38 > 0:12:43identify what put people most at risk from the bad air.
0:12:44 > 0:12:47He used income-tax data to try and measure the affluence of the different
0:12:47 > 0:12:49boroughs that were affected by cholera.
0:12:49 > 0:12:53He asked his friends at the Royal Observatory to provide data on the
0:12:53 > 0:12:55temperature and climatic conditions.
0:12:56 > 0:12:59But the one that he thought was most convincing was about the topography.
0:12:59 > 0:13:02It was about the elevation above the Thames.
0:13:02 > 0:13:07Using the data, Farr suggested a mathematical law of elevation.
0:13:07 > 0:13:12Its equations described how cholera mortality falls the higher you live
0:13:12 > 0:13:14above the Thames.
0:13:15 > 0:13:18Now, he published his report in 1852,
0:13:18 > 0:13:20which the Lancet described as one of
0:13:20 > 0:13:22the most remarkable productions of
0:13:22 > 0:13:25type and pen in any age and country.
0:13:27 > 0:13:30The only problem was that Farr's work,
0:13:30 > 0:13:33although elegant and meticulous,
0:13:33 > 0:13:35was fundamentally flawed.
0:13:37 > 0:13:40Farr stuck to the prevailing theory
0:13:40 > 0:13:43that cholera was spread by air.
0:13:43 > 0:13:45Such is the power of the status quo.
0:13:45 > 0:13:47But in 1866,
0:13:47 > 0:13:535,500 people died in just one square mile of London's East End,
0:13:53 > 0:13:56and that data made Farr change his mind.
0:13:58 > 0:14:01When Farr came to write his next report,
0:14:01 > 0:14:04the data told a different story
0:14:04 > 0:14:05which proved the turning point in
0:14:05 > 0:14:07combating the disease.
0:14:07 > 0:14:13The common factor among those who died was not elevation or air
0:14:13 > 0:14:16but sewage-contaminated drinking water.
0:14:18 > 0:14:20With this new report,
0:14:20 > 0:14:22Farr may seem to have contradicted
0:14:22 > 0:14:25much of his own work, but I think that
0:14:25 > 0:14:29this is the perfect example of what data can do.
0:14:29 > 0:14:33It provides that bridge essential to scientific discovery,
0:14:33 > 0:14:36from theory to proof, problem to solution.
0:14:38 > 0:14:41Good data, even in huge volumes,
0:14:41 > 0:14:45does not guarantee that you will arrive at the truth.
0:14:45 > 0:14:49But, eventually, when the weight of the data tips the balance,
0:14:49 > 0:14:52even the strongest-held beliefs can be overcome.
0:14:55 > 0:14:59Of course, it was the weight of the data itself which,
0:14:59 > 0:15:01with the dawn of the 20th century,
0:15:01 > 0:15:04was becoming increasingly hard to manage.
0:15:06 > 0:15:10Data stored long form in things like census ledgers could take the best
0:15:10 > 0:15:13part of a decade to process,
0:15:13 > 0:15:16meaning the stats were often out of date.
0:15:17 > 0:15:19When you're dealing with figures like these, it's one thing.
0:15:22 > 0:15:26But when you're counting the population like this it's quite a different matter.
0:15:26 > 0:15:30A deceptively simple solution got what's now called the
0:15:30 > 0:15:33information revolution under way,
0:15:33 > 0:15:38encoding data as holes punched in cards.
0:15:38 > 0:15:41These cards are passed over sorting machines,
0:15:41 > 0:15:44each of which handles 22,000 cards a minute.
0:15:46 > 0:15:51By the 1950s, data processing and simple calculations were
0:15:51 > 0:15:53routinely mechanised,
0:15:53 > 0:15:55laying the groundwork for the next generation of
0:15:55 > 0:15:57data-processing machines.
0:16:00 > 0:16:04They would be put to pioneering work
0:16:04 > 0:16:06in a rather unlikely place.
0:16:08 > 0:16:11In a grand London dining hall, a group of men and women,
0:16:11 > 0:16:16many in their 80s and 90s, have gathered for a special work reunion.
0:16:18 > 0:16:22At its peak, their employer, J Lyons,
0:16:22 > 0:16:25purveyor of fine British tea and cakes,
0:16:25 > 0:16:27had hundreds of tea shops nationwide.
0:16:28 > 0:16:30There are hundreds of items of food.
0:16:30 > 0:16:34All these in a varying quantity each day are delivered to a precise
0:16:34 > 0:16:35timetable to the tea shops.
0:16:38 > 0:16:43These people aren't former J Lyons bakers or tea-shop managers.
0:16:43 > 0:16:47They were hired for their mathematical skills.
0:16:47 > 0:16:51Lyons had a huge amount of data which has to be processed,
0:16:51 > 0:16:54often very low-value data.
0:16:54 > 0:16:57So, for example, the transaction from a tea shop
0:16:57 > 0:16:58would be a cup of tea.
0:16:59 > 0:17:02But each one had a voucher and had to be recorded,
0:17:02 > 0:17:04and had to go to the accounts
0:17:04 > 0:17:08for business reasons and for management reasons.
0:17:08 > 0:17:12Every calculation you did, not only you had to do it twice,
0:17:12 > 0:17:15but you had to get it checked by someone else as well.
0:17:15 > 0:17:19The handling of these millions and millions of pieces of data,
0:17:19 > 0:17:23the storage of that data, are the key of the business problem.
0:17:25 > 0:17:30The Lyons team took the world by surprise when, in 1951,
0:17:30 > 0:17:33they unveiled the Lyons Electronic Office,
0:17:33 > 0:17:34or Leo for short.
0:17:38 > 0:17:42At this point, only a handful of computers existed,
0:17:42 > 0:17:46and they were used solely for scientific and military research,
0:17:46 > 0:17:51so a business computer was a radical reimagining of what this brand-new
0:17:51 > 0:17:52technology could be for.
0:17:55 > 0:17:59Each manageress has a standing order depending on the day of the week.
0:18:00 > 0:18:03She speaks by telephone to head office, where her variations are
0:18:03 > 0:18:05taken quickly onto cards.
0:18:05 > 0:18:08What the girl hears, she punches.
0:18:08 > 0:18:10The programme is fed first,
0:18:10 > 0:18:14laying down the sequence for the multiplicity of calculations Leo will perform.
0:18:15 > 0:18:16It was the first
0:18:16 > 0:18:19opportunity to process large volumes
0:18:19 > 0:18:20of clerical work,
0:18:20 > 0:18:22take all the hard work out of it,
0:18:22 > 0:18:25and put it on an automatic system.
0:18:25 > 0:18:28Before Leo, working out an employee's pay
0:18:28 > 0:18:31took an experienced clerk eight minutes,
0:18:31 > 0:18:35but with Leo that dropped to an astonishing one and a half seconds.
0:18:39 > 0:18:40It was all so exciting
0:18:40 > 0:18:43because we were breaking new ground the whole time.
0:18:43 > 0:18:48Absolutely everything which we did has never been done before.
0:18:49 > 0:18:51By anybody anywhere.
0:18:51 > 0:18:55I don't think we realised the kind of transformation we were part of.
0:18:59 > 0:19:04The post-war years saw a boom in the application of this new computing
0:19:04 > 0:19:05technology.
0:19:06 > 0:19:09Leo ran on paper, tape and cards,
0:19:09 > 0:19:13but soon machines with magnetic tape and disks were developed,
0:19:13 > 0:19:17allowing for greater data storage and faster calculations.
0:19:19 > 0:19:23As more businesses and institutions adopted these new machines,
0:19:23 > 0:19:27application of mathematics to a whole host of new,
0:19:27 > 0:19:29real-world challenges took off.
0:19:31 > 0:19:36And the word "data" went from relatively obscure to ubiquitous.
0:19:44 > 0:19:48"Data" has become almost a magic word for anything.
0:19:48 > 0:19:51The truth is that it is a kind of interface
0:19:51 > 0:19:54today between us and the rest of the world.
0:19:54 > 0:19:57In fact, between us and ourselves,
0:19:57 > 0:20:00we understand our bodies in terms of data,
0:20:00 > 0:20:03we understand society in terms of data,
0:20:03 > 0:20:06we understand the physics of the universe in terms of data.
0:20:06 > 0:20:09The economy, social science, we play with data,
0:20:09 > 0:20:12so essentially it is what we interact with
0:20:12 > 0:20:14most regularly every day.
0:20:17 > 0:20:21Data underpins all human communication,
0:20:21 > 0:20:23regardless of the format.
0:20:23 > 0:20:28And it was the desire to communicate effectively and efficiently that led
0:20:28 > 0:20:33to one of the most important academic papers of the 20th century.
0:20:35 > 0:20:38A mathematical theory of communication
0:20:38 > 0:20:45has justifiably been called the Magna Carta for the information age.
0:20:45 > 0:20:50It was written by a very young and bright employee of Bell Laboratories,
0:20:50 > 0:20:54the American centre for telecoms research that was founded by one of
0:20:54 > 0:20:58the inventors of the telephone, Alexander Graham Bell.
0:20:58 > 0:21:04Now, this paper was written by Claude Shannon in 1948 and it would
0:21:04 > 0:21:09effectively lay out the theoretical framework for the data revolution
0:21:09 > 0:21:12that was just beginning.
0:21:12 > 0:21:14Those that knew him described
0:21:14 > 0:21:18Shannon as a lifelong puzzle solver and inventor.
0:21:18 > 0:21:23To define the correct path it registers the information in its memory.
0:21:23 > 0:21:27Later, I can put him down in any part of the maze that he has already explored
0:21:27 > 0:21:29and he will be able to go directly to the goal without making a single
0:21:29 > 0:21:32false turn.
0:21:32 > 0:21:36During World War II he worked on data-encryption systems,
0:21:36 > 0:21:40including one used by Churchill and Roosevelt.
0:21:41 > 0:21:43But at Bell Labs,
0:21:43 > 0:21:48Claude Shannon was trying to solve the very civilian problem of noisy
0:21:48 > 0:21:49telephone lines.
0:21:49 > 0:21:51# There's a call, there's a call
0:21:51 > 0:21:53# There's a call for you
0:21:53 > 0:21:56# There's a call on the phone for you. #
0:21:56 > 0:22:00In that analogue world of 20th-century phones,
0:22:00 > 0:22:04your speech was converted into an electrical signal using a handset
0:22:04 > 0:22:08like this and then transmitted down a series of wires.
0:22:08 > 0:22:12The voice signals would travel along the wire,
0:22:12 > 0:22:16be detected by the receiver at the other end and then be converted back
0:22:16 > 0:22:20into sound waves to reach the ear of whoever had picked up.
0:22:20 > 0:22:22The problem was,
0:22:22 > 0:22:24the further the electrical signal travelled down the line,
0:22:24 > 0:22:26the weaker it would get.
0:22:26 > 0:22:28PHONE LINE CRACKLES Eventually you couldn't
0:22:28 > 0:22:32even hear the conversation for the amount of noise on the line.
0:22:32 > 0:22:36And you could boost the signal but it would mean boosting the noise, too.
0:22:37 > 0:22:42Shannon's genius idea was just as simple as it was beautiful.
0:22:43 > 0:22:46The breakthrough was converting speech
0:22:46 > 0:22:49into an incredibly simple code.
0:22:50 > 0:22:51ON PHONE: Hello?
0:22:51 > 0:22:55First the audio wave is detected, then sampled.
0:22:55 > 0:23:00Each point is assigned a code of ones and zeros
0:23:00 > 0:23:03and the resulting long string of digits can then be sent down
0:23:03 > 0:23:07the wire with the zeros as brief low-voltage signals
0:23:07 > 0:23:11and ones as brief bursts of high voltage.
0:23:11 > 0:23:16From this code, the original audio can be cleanly reconstructed and
0:23:16 > 0:23:18regenerated at the other end.
0:23:18 > 0:23:20ON PHONE: Hello?
0:23:20 > 0:23:23Shannon was the first person to publish the name
0:23:23 > 0:23:24for these ones and zeros,
0:23:24 > 0:23:27the smallest possible pieces of information,
0:23:27 > 0:23:30and they are called bits or binary digits,
0:23:30 > 0:23:35and the real power of the bit and the mathematics behind it
0:23:35 > 0:23:37applies way beyond telephones.
0:23:38 > 0:23:41They offered a new way for everything,
0:23:41 > 0:23:48including text and pictures, to be encoded as ones and zeros.
0:23:51 > 0:23:57The possibility to store and share data digitally in the form of bits
0:23:57 > 0:24:00was clearly going to transform the world.
0:24:02 > 0:24:05If anyone has to be identified as
0:24:05 > 0:24:08the genius who developed the
0:24:08 > 0:24:11foundational science of mathematics
0:24:11 > 0:24:14for our age, that is certainly Claude Shannon.
0:24:14 > 0:24:18Now, one thing has to be clarified,
0:24:18 > 0:24:24the theory developed by Shannon is about data transmission and it has
0:24:24 > 0:24:27nothing to do with meaning, truth, relevance,
0:24:27 > 0:24:30importance of the data transmitted.
0:24:30 > 0:24:35So it doesn't matter whether the zero and one represent
0:24:35 > 0:24:40an answer to, "Heads or tails?", or to the question, "Will you marry me?",
0:24:40 > 0:24:46for a theory of information is data anyway and if it is a 50-50 chance
0:24:46 > 0:24:51that you will or will not marry me or that it is heads or tails,
0:24:51 > 0:24:54the amount of information, the Shannon information,
0:24:54 > 0:24:56communicated is the same.
0:24:58 > 0:25:04Shannon information is not information like you or I might think about it.
0:25:04 > 0:25:08Encoding any and every signal using just ones and zeros is a pretty
0:25:08 > 0:25:10remarkable breakthrough.
0:25:10 > 0:25:16However, Shannon also came up with a revolutionary bit of mathematics.
0:25:16 > 0:25:21That equation there is the reason you can fit an entire HD movie on a
0:25:21 > 0:25:26flimsy bit of plastic or the reason why you can stream films online.
0:25:26 > 0:25:30I'll admit, it might not look too nice, but...
0:25:32 > 0:25:36don't get put off yet, because I'm going to explain how this equation
0:25:36 > 0:25:38works using Scrabble.
0:25:40 > 0:25:43Imagine that I created a new alphabet
0:25:43 > 0:25:45containing only the letter A.
0:25:45 > 0:25:48This bag would only have A tiles inside it
0:25:48 > 0:25:52and my chances of pulling out an A tile would be one.
0:25:52 > 0:25:54You'd be completely certain of what was going to happen.
0:25:54 > 0:25:56Using Shannon's maths,
0:25:56 > 0:26:01the letter A contains zero bits of what's called Shannon information.
0:26:03 > 0:26:05Let's say then I got a little bit more creative, but not much,
0:26:05 > 0:26:09and had an alphabet with two letters, A and B,
0:26:09 > 0:26:11and equal numbers of both in this bag.
0:26:11 > 0:26:15Now my chances of pulling out an A are going to be a half
0:26:15 > 0:26:20and each letter contains one bit of Shannon information.
0:26:22 > 0:26:24Of course, when transmitting real messages,
0:26:24 > 0:26:26you'll use the full alphabet.
0:26:28 > 0:26:30But English,
0:26:30 > 0:26:33as with every other language, has some letters that are used
0:26:33 > 0:26:35more frequently than others.
0:26:35 > 0:26:38If you take a quite common letter like H,
0:26:38 > 0:26:42which appear about 5.9% of the time,
0:26:42 > 0:26:46this will have a Shannon information of 4.1 bits.
0:26:48 > 0:26:52And incidentally, a Scrabble score of four.
0:26:52 > 0:26:56Of course, there are some much more exotic and rare letters,
0:26:56 > 0:27:02like Z, for instance, which appears about 0.07% of the time.
0:27:02 > 0:27:05That gives it 10.5 bits
0:27:05 > 0:27:07and Scrabble score of ten.
0:27:09 > 0:27:13Bits measure our uncertainty.
0:27:13 > 0:27:17If you're guessing a three-letter word and you know this letter is Z,
0:27:17 > 0:27:21it gives you a lot of information about what the word could be.
0:27:22 > 0:27:24But if you know it's H,
0:27:24 > 0:27:27because it is a more common letter with less information,
0:27:27 > 0:27:30you're more uncertain about the answer.
0:27:32 > 0:27:35Now if you wrap up all that uncertainty together,
0:27:35 > 0:27:38you end up with this, the Shannon entropy.
0:27:40 > 0:27:42It's the sum of the probability of
0:27:42 > 0:27:44each symbol turning up times the
0:27:44 > 0:27:47number of bits in each symbol.
0:27:47 > 0:27:49And this very clever bit of insight
0:27:49 > 0:27:52and mathematics means that the code
0:27:52 > 0:27:55for any message can be quantified.
0:27:55 > 0:27:58Not every letter, or any other signal for that matter,
0:27:58 > 0:28:01needs to be encoded equally.
0:28:03 > 0:28:07The digital code behind a movie like this one of my dog, Molly,
0:28:07 > 0:28:08for example,
0:28:08 > 0:28:12can usually be compressed by up to 50%
0:28:12 > 0:28:16without losing any information.
0:28:16 > 0:28:17But there's a limit.
0:28:19 > 0:28:24Compressing more might make it easier to share or download,
0:28:24 > 0:28:27but the quality can never be the same as the original.
0:28:27 > 0:28:30DOG BARKS
0:28:32 > 0:28:35You can't really overstate the impact that Shannon's work
0:28:35 > 0:28:40has had, because without it we wouldn't have JPEGs or Zip files
0:28:40 > 0:28:43or HD movies or digital communications.
0:28:43 > 0:28:47But it doesn't just stop there, because while the mathematics of
0:28:47 > 0:28:51information theory doesn't tell you anything about the meaning of data,
0:28:51 > 0:28:54it does begin to open up a possibility
0:28:54 > 0:28:56of how we can understand ourselves
0:28:56 > 0:29:01and our society, because pretty much anything and everything can be
0:29:01 > 0:29:04measured and encoded as data.
0:29:10 > 0:29:13We say that signals flow through human society,
0:29:13 > 0:29:16that people use signals to get things done,
0:29:16 > 0:29:18that our social life is, in many ways,
0:29:18 > 0:29:20the sending back and forth of signals.
0:29:20 > 0:29:22So what is a signal?
0:29:22 > 0:29:26It's, in one sense, just the reduction in uncertainty.
0:29:34 > 0:29:39What it means to receive a signal is to be less uncertain than you were
0:29:39 > 0:29:43before and so, another way to think of measuring or quantifying signal
0:29:43 > 0:29:46is in that change in uncertainty.
0:29:47 > 0:29:50Using Shannon's mathematics to quantify signals
0:29:50 > 0:29:53is common in the world of complexity science.
0:29:53 > 0:29:57It's rather less familiar to historians.
0:29:57 > 0:30:00I love maths, I love its precision, I love its beauty.
0:30:08 > 0:30:11I absolutely love
0:30:11 > 0:30:19its certainty, and that, Simon can bring that mathematical worldview,
0:30:19 > 0:30:22that mathematical certainty to what I work with.
0:30:24 > 0:30:28The reason behind this remarkable marriage between history and science
0:30:28 > 0:30:32is the analysis of the largest single body of digital text
0:30:32 > 0:30:35ever collated about ordinary people.
0:30:36 > 0:30:39It's the Proceedings of London's Old Bailey,
0:30:39 > 0:30:41the central criminal court of England and Wales,
0:30:41 > 0:30:49which hosted close to 200,000 trials between 1674 and 1913.
0:30:49 > 0:30:54There are 127 million words of everyday speech
0:30:54 > 0:31:00in the mouths of orphans and women and servants and ne'er-do-wells,
0:31:00 > 0:31:02of criminals, certainly,
0:31:02 > 0:31:06but also people from every rank and station in society.
0:31:06 > 0:31:09And that made them unique.
0:31:09 > 0:31:13What's exciting about the Old Bailey and the size of the dataset,
0:31:13 > 0:31:15the length and magnitude of it,
0:31:15 > 0:31:18is that not only can we detect a signal,
0:31:18 > 0:31:22but we are able to look at that signal's emergence over time.
0:31:24 > 0:31:28Shannon's mathematics can be used to capture the amount of information in
0:31:28 > 0:31:30every single word,
0:31:30 > 0:31:34and like the alphabet, the less you expect a word,
0:31:34 > 0:31:37the more bits of information it carries.
0:31:37 > 0:31:40Imagine that you walk into a courtroom
0:31:40 > 0:31:43at the time and you hear a single word,
0:31:43 > 0:31:47the question we ask is how much information does that word carry
0:31:47 > 0:31:50about the nature of the crime being tried?
0:31:52 > 0:31:55You hear the word "the".
0:31:55 > 0:32:00It's common across all trials and so gives you no bits of information.
0:32:00 > 0:32:04Most words you hear are poor signals of what's going on.
0:32:06 > 0:32:09But then you hear "purse".
0:32:09 > 0:32:11It conveys real information.
0:32:12 > 0:32:15Then comes "coin",
0:32:15 > 0:32:17"grab" and "struck".
0:32:17 > 0:32:22The more rare a word, the more bits of information it carries,
0:32:22 > 0:32:24the stronger the signal becomes.
0:32:26 > 0:32:29One of the clearest signals that we see in the Old Bailey,
0:32:29 > 0:32:31one of the clearest processes that comes out,
0:32:31 > 0:32:35is something that is known as the civilising process.
0:32:35 > 0:32:42It's an increasing sensitivity to, and attention to, the
0:32:42 > 0:32:46distinction between violent and nonviolent crime.
0:32:46 > 0:32:52If, for example, somebody hit you and stole your handkerchief,
0:32:52 > 0:32:54in the 18th-century context, in 1780,
0:32:54 > 0:32:57you would concentrate on the handkerchief.
0:32:57 > 0:33:01More worried about a few pence worth of dirty linen than the fact that
0:33:01 > 0:33:05somebody just broke your nose or cracked a rib.
0:33:05 > 0:33:09The fact that 100 years later, by 1880,
0:33:09 > 0:33:14every concern, every focus, both in terms of the words used in court,
0:33:14 > 0:33:18but also in terms of what people were brought to court for,
0:33:18 > 0:33:21focus on that broken nose and that cracked rib,
0:33:21 > 0:33:25speaks to a fundamental change in how we think about the world
0:33:25 > 0:33:28and how we think about how social relations work.
0:33:30 > 0:33:35Look at the strongest word signals for violent crime across the period.
0:33:35 > 0:33:38In the 18th century, the age of highwaymen,
0:33:38 > 0:33:41words relating to property theft dominate.
0:33:43 > 0:33:45But by the 20th century,
0:33:45 > 0:33:49it's physical violence itself and the impact on the victim
0:33:49 > 0:33:51that carry the most weight.
0:33:54 > 0:33:56That notion that one can trace change over time
0:33:56 > 0:33:58by looking at language and how it's used,
0:33:58 > 0:34:00who deploys it in what context,
0:34:00 > 0:34:04that I think gives this kind of work its real power.
0:34:04 > 0:34:07There are billions of words, there's all of Google Books,
0:34:07 > 0:34:10there's every printed newspaper,
0:34:10 > 0:34:13there is every speech made in Parliament,
0:34:13 > 0:34:15every sermon given at most churches.
0:34:15 > 0:34:19All of it is suddenly data and capable of being analysed.
0:34:24 > 0:34:27The rapid development of computers in the mid 20th century
0:34:27 > 0:34:31transformed our ability to encode, store and analyse data.
0:34:33 > 0:34:37It took a little longer for us to work out how to share it.
0:34:40 > 0:34:42This place is home to one of the most
0:34:42 > 0:34:46important UK scientific institutions,
0:34:46 > 0:34:49although it's one you've probably never heard of before.
0:34:49 > 0:34:54But since the 1900s, this place has advanced all areas of physics,
0:34:54 > 0:34:59radio communications, engineering, materials science, aeronautics,
0:34:59 > 0:35:02even ship design.
0:35:02 > 0:35:04NPL, the National Physical Laboratory,
0:35:04 > 0:35:08in south-west London is where the first atomic clock was built
0:35:08 > 0:35:13and where radar and the Automatic Computer Engine, or Ace, were invented.
0:35:14 > 0:35:19The Ace computer was the brainchild of Alan Turing,
0:35:19 > 0:35:22who came to work here right after the Second World War.
0:35:22 > 0:35:26Now, Turing's contributions to the story of data are undoubtedly vast,
0:35:26 > 0:35:31but more important for our story is another person who worked here with
0:35:31 > 0:35:36Turing, someone who arguably is even less well known than this place,
0:35:36 > 0:35:38Donald Davies.
0:35:39 > 0:35:43Davies worked on secret British nuclear weapons research during the war...
0:35:45 > 0:35:48..later joining Turing at NPL,
0:35:48 > 0:35:52climbing the ranks to be put in charge of computing in 1966.
0:35:54 > 0:35:56As well as the new digital computers,
0:35:56 > 0:36:02Davies had a lifelong fascination with telephones and communication.
0:36:02 > 0:36:05His mother had worked in the Post Office telephone exchange,
0:36:05 > 0:36:06so even when he was a kid,
0:36:06 > 0:36:10he had a real understanding of how these phone calls were routed and
0:36:10 > 0:36:12rerouted through this growing network,
0:36:12 > 0:36:16and that was the perfect training for what was to follow.
0:36:27 > 0:36:28What was Donald Davies like, then?
0:36:28 > 0:36:32He was a super boss because he was very approachable.
0:36:33 > 0:36:39Everybody realised he'd got huge intellect but not difficult with it.
0:36:39 > 0:36:40Very nice guy.
0:36:40 > 0:36:44Davies' innovation was to develop, with his team,
0:36:44 > 0:36:49a way of sharing data between computers, a prototype network.
0:36:49 > 0:36:53Donald had spotted that there was a need to connect computers together
0:36:53 > 0:36:57and to connect people to computers, not by punch cards or
0:36:57 > 0:37:00paper tape or on a motorcycle, but over the wires,
0:37:00 > 0:37:03where you can move files or programs, or
0:37:03 > 0:37:05run a program remotely on another computer,
0:37:05 > 0:37:09and the telephone network is not really suited for that.
0:37:10 > 0:37:12In the pre-digital era,
0:37:12 > 0:37:16sending an encoded file along a telephone line meant that the line
0:37:16 > 0:37:20was engaged for as long as the transmission took.
0:37:20 > 0:37:23So the opportunity here was because we owned the site,
0:37:23 > 0:37:2878 acres with some 50 buildings, we could build a network.
0:37:28 > 0:37:31Davies' team sidestepped the telephone problem
0:37:31 > 0:37:36by laying high-bandwidth data cables before instituting a new way of
0:37:36 > 0:37:39moving data around the network.
0:37:41 > 0:37:44The technique he came up with was packet switching,
0:37:44 > 0:37:47the idea being that you take whatever it is you're going to send,
0:37:47 > 0:37:52you chop it up into uniform pieces, like having a standard envelope,
0:37:52 > 0:37:56and you put the pieces into the envelope and you post them off and they go
0:37:56 > 0:37:59separately through the network and get reassembled at the far end.
0:37:59 > 0:38:01To demonstrate this idea,
0:38:01 > 0:38:03Roger and I are convening NPL's
0:38:03 > 0:38:07first-ever packet-switching data-dash...
0:38:08 > 0:38:11..which is a bit more complicated than your average sports-day event.
0:38:13 > 0:38:16The course is a data network.
0:38:16 > 0:38:19There are two computers, represented
0:38:19 > 0:38:21here as the start and finish signs.
0:38:21 > 0:38:24Those computers are connected by a
0:38:24 > 0:38:27series of network cables and nodes.
0:38:27 > 0:38:29In our case, cables are lines of
0:38:29 > 0:38:33cones and the connecting nodes are Hula Hoops.
0:38:35 > 0:38:40Having built it, all we need now are some willing volunteers.
0:38:40 > 0:38:41And here they are.
0:38:41 > 0:38:44NPL's very own apprentices.
0:38:47 > 0:38:50So welcome to our packet-switching sports day.
0:38:50 > 0:38:52We've got two teams, red and blue.
0:38:52 > 0:38:55'Both teams are pretending to be data
0:38:55 > 0:38:58'and they're going to have to race.'
0:38:58 > 0:38:59You're going to start over there
0:38:59 > 0:39:01where it says "start", kind of obvious,
0:39:01 > 0:39:05and you're trying to get through to the end as quickly as you possibly
0:39:05 > 0:39:07can. You can't just go anywhere,
0:39:07 > 0:39:12you have to go through these hoops to get to the finish line,
0:39:12 > 0:39:13these little nodes in our network.
0:39:13 > 0:39:17You're only allowed to travel along the lines of the cones,
0:39:17 > 0:39:20but only if there's nobody else along that line.
0:39:20 > 0:39:23All clear? OK, there is one catch.
0:39:23 > 0:39:25All of you who are in the red team,
0:39:25 > 0:39:28we are going to tie your feet together.
0:39:29 > 0:39:34So you've got to travel round our network as one big chunk of data.
0:39:34 > 0:39:38Those of you who are in blue, you are allowed to travel on your own,
0:39:38 > 0:39:39so it's slightly easier.
0:39:39 > 0:39:43'The objective is for both teams to deposit their beanbags
0:39:43 > 0:39:47'in the goal in the right order, one to five.'
0:39:47 > 0:39:51EXCITED CHATTER
0:39:51 > 0:39:53Get in the hoop! Get in the hoop!
0:39:53 > 0:39:55Bring out your competitive spirit here.
0:39:55 > 0:39:58We've got packets versus big chunks of data.
0:39:58 > 0:40:01I'm going to time you. Everyone ready?
0:40:01 > 0:40:02OK, over to you, Roger.
0:40:02 > 0:40:05TOOT!
0:40:07 > 0:40:10Remember, you can't go down the route until it's clear.
0:40:10 > 0:40:12'The red and blue teams are exactly the same size,
0:40:12 > 0:40:15'let's say five megabytes each.
0:40:15 > 0:40:21'But their progress through the network is clearly very different.'
0:40:21 > 0:40:24THEY LAUGH
0:40:32 > 0:40:36OK, blues, you took 13 seconds, pretty impressive.
0:40:36 > 0:40:38Reds, 20 seconds.
0:40:38 > 0:40:40That's a victory for the packet switchers.
0:40:40 > 0:40:43Well done, you guys! Well done, you guys.
0:40:43 > 0:40:47The impact that packet switching has had on the world, I mean,
0:40:47 > 0:40:50it sort of came from here and then spread out elsewhere.
0:40:50 > 0:40:54It did indeed, we gave the world packet switching, and the world,
0:40:54 > 0:40:58of course, being America, they took it on and ran with it.
0:41:01 > 0:41:05This little race, Donald Davies' packet switching,
0:41:05 > 0:41:09was adopted by the people that would go on to build the internet,
0:41:09 > 0:41:13and today, the whole thing still runs on this idea.
0:41:16 > 0:41:19Let's say I want to e-mail you a picture of Molly.
0:41:19 > 0:41:24First, it will be broken up into over 1,000 data packets.
0:41:24 > 0:41:27Each one is stamped with the address of where it's from and where it's
0:41:27 > 0:41:32going to, which routers check to keep the packets moving.
0:41:32 > 0:41:34Regardless of the order they arrive,
0:41:34 > 0:41:38the image is reassembled, and there she is.
0:41:41 > 0:41:42This is quite a cool thing, right,
0:41:42 > 0:41:44that you've got one of the original creators
0:41:44 > 0:41:48of packet switching right here and you can ask him...
0:41:48 > 0:41:51Every time you're like... Well, do anything, really.
0:41:51 > 0:41:54"Why is my internet running so slowly?"
0:41:54 > 0:41:57- THEY LAUGH - Don't ask me!
0:42:01 > 0:42:05We've come a very long way in just a few decades.
0:42:05 > 0:42:10Around 3.4 billion people now have access to the internet at home
0:42:10 > 0:42:13and there are around four times the number of phones
0:42:13 > 0:42:16and other data-sharing devices online,
0:42:16 > 0:42:19the so-called Internet of Things.
0:42:21 > 0:42:24Just by being alive in the 21st century
0:42:24 > 0:42:28with our phones, our tablets, our smart devices, all of us are
0:42:28 > 0:42:30familiar with data.
0:42:30 > 0:42:34Really embrace your inner nerd here, because every time you wander around
0:42:34 > 0:42:38looking at your screen, you are gobbling up and churning out
0:42:38 > 0:42:40absolutely tons of the stuff.
0:42:40 > 0:42:43Our relationship with data has really changed -
0:42:43 > 0:42:47it's no longer just for specialists, it's for everyone.
0:42:48 > 0:42:52There's one city in the UK that's putting the sharing and real-time
0:42:52 > 0:42:57analysis of data at the heart of everything it does -
0:42:57 > 0:42:58Bristol.
0:42:59 > 0:43:03Using digital technology, we take the city's pulse.
0:43:04 > 0:43:10This data is the route to an open, smart, liveable city,
0:43:10 > 0:43:15a city where optical, wireless and mesh networks
0:43:15 > 0:43:20combine to create an open, urban canopy of connectivity.
0:43:20 > 0:43:25Taking the pulse of the city under a canopy of connectivity
0:43:25 > 0:43:30might sound a bit sci-fi, or like something from a broadband advert.
0:43:30 > 0:43:33But if you just hold on to your cynicism for a second,
0:43:33 > 0:43:39because Bristol are trying to build a new type of data-sharing network for its citizens.
0:43:41 > 0:43:44There's a city-centre area which now has next-generation
0:43:44 > 0:43:47or maybe the generation after next
0:43:47 > 0:43:51of superfast broadband and then that's coupled to a Wi-Fi network, as well.
0:43:51 > 0:43:54The question is, what can you do with it?
0:44:01 > 0:44:06We would have a wide area network of very simple Internet of Things
0:44:06 > 0:44:10sensing devices that just monitor a simple signal like air quality
0:44:10 > 0:44:11or traffic queued in a traffic jam.
0:44:11 > 0:44:14Once you've got all this network infrastructure,
0:44:14 > 0:44:17you can get an awful lot, a really huge amount of data
0:44:17 > 0:44:20arriving to you in real time.
0:44:22 > 0:44:26What's happening here is a city-scale experiment
0:44:26 > 0:44:28to try and develop and test
0:44:28 > 0:44:32what's going to be called the programmable city of the future.
0:44:33 > 0:44:36It relies on Bristol's futuristic network,
0:44:36 > 0:44:40vast amounts of data from as many sensors as possible
0:44:40 > 0:44:43and a computer system that can simulate
0:44:43 > 0:44:47and effectively reprogram the city.
0:44:47 > 0:44:49The computer system can intervene.
0:44:49 > 0:44:53It could reroute traffic and we can actually radio out to individuals,
0:44:53 > 0:44:55so maybe they get a message on their smartphone
0:44:55 > 0:44:57or perhaps a wrist-mounted device,
0:44:57 > 0:44:59saying, "If you have asthma, perhaps you should get indoors."
0:45:01 > 0:45:06Once you create that capacity for anything and everything in the city
0:45:06 > 0:45:08to be connected together,
0:45:08 > 0:45:11you can really start to re-imagine how a city might operate.
0:45:11 > 0:45:16We are starting to experiment with driverless cars and, in order for
0:45:16 > 0:45:17driverless cars to work,
0:45:17 > 0:45:21they have to be able to communicate with the city infrastructure.
0:45:21 > 0:45:23So, your car needs to speak to the traffic lights,
0:45:23 > 0:45:27the traffic lights need to speak to the car, the cars to speak to each other.
0:45:27 > 0:45:31All of that requires a completely different set of infrastructure.
0:45:33 > 0:45:38Of course, as the amount of data a city can share grows,
0:45:38 > 0:45:42the computing power needed to do something useful with it must grow, too.
0:45:45 > 0:45:48And for that, we have the cloud.
0:45:49 > 0:45:54For example, imagine trying to analyse all of Bristol's traffic data,
0:45:54 > 0:45:57weather and pollution data on your home computer.
0:45:57 > 0:45:58It could take a year.
0:46:01 > 0:46:07Well, you could reduce that to a day by getting 364 more computers,
0:46:07 > 0:46:09but that's expensive.
0:46:09 > 0:46:14A cheaper option is sharing the analysis with other computers over the internet,
0:46:14 > 0:46:19which Google worked out first, but they published the basics
0:46:19 > 0:46:23and now free software exists to help anyone do the same.
0:46:23 > 0:46:26Big online companies rent their spare computers
0:46:26 > 0:46:28for a few pence an hour.
0:46:29 > 0:46:32So, now anyone like me or you
0:46:32 > 0:46:36can do big data analytics quickly for a few quid.
0:46:41 > 0:46:45Such computing power is something we could never have dreamt of
0:46:45 > 0:46:49just a few years ago, but it will only fulfil its potential
0:46:49 > 0:46:54if we can share our own data in a safe and transparent way.
0:46:55 > 0:47:01If Bristol Council wanted to know where your car was at all times
0:47:01 > 0:47:04but could use that information to sort of minimise traffic jams,
0:47:04 > 0:47:06how would you feel about something like that?
0:47:06 > 0:47:09Er, I'm not sure if I'd particularly like it.
0:47:09 > 0:47:12I think it is up to me where I leave my car.
0:47:12 > 0:47:15I understand the idea of justifying it with all these great other ideas,
0:47:15 > 0:47:18but I still probably wouldn't like it very much.
0:47:18 > 0:47:20If they are using it for a better purpose, then yeah,
0:47:20 > 0:47:24but one should know how they are using it and why they'll be using it, for what purpose.
0:47:24 > 0:47:29I'd like to imagine a world in which all the data that was retained
0:47:29 > 0:47:31was used for the greater good of mankind,
0:47:31 > 0:47:35but I can't imagine a circumstance like that
0:47:35 > 0:47:36in the world that we have today.
0:47:36 > 0:47:40We live in a modern society, where if you don't
0:47:40 > 0:47:43let your data out there, not in the public domain,
0:47:43 > 0:47:45but in a secure business domain,
0:47:45 > 0:47:47then you can't take part in society, really.
0:47:49 > 0:47:54Unsurprisingly, people are pretty wary about what happens to their data.
0:47:55 > 0:47:58We need to be careful that civil liberties are not eroded,
0:47:58 > 0:48:02because otherwise the technology is likely to be rejected.
0:48:02 > 0:48:06I think it's an area where us as a society have yet to sort of fully
0:48:06 > 0:48:11understand what the correct way forward is
0:48:11 > 0:48:13and therefore it is very much a discussion.
0:48:13 > 0:48:16It's not a lecture, it's not a code,
0:48:16 > 0:48:20it's one where we are co-producing and co-forming these sorts of rules
0:48:20 > 0:48:23with people in the city, in order to sort of help us work out what the
0:48:23 > 0:48:26right and wrong things to do are.
0:48:26 > 0:48:31It will be intriguing to watch Bristol grapple with the technological
0:48:31 > 0:48:35and ethical challenges of being our first data-centric city.
0:48:37 > 0:48:40In all these contexts, Internet of Things...
0:48:41 > 0:48:45..new forms of health care, smart cities,
0:48:45 > 0:48:48what we're seeing is an increase in transparency.
0:48:49 > 0:48:52You can see through the body, you can see through the house,
0:48:52 > 0:48:56you can see through the city and the square, you can see through society.
0:48:56 > 0:48:58Now, transparency may be good.
0:48:58 > 0:49:03It's something that we may need to handle carefully in order to extract
0:49:03 > 0:49:06the value from those data to improve
0:49:06 > 0:49:10your lifestyle, your social interactions,
0:49:10 > 0:49:12the way in which your city works and so on.
0:49:12 > 0:49:16But it also needs to be carefully handled, because it's touching
0:49:16 > 0:49:19the ultimate nerve of what it means to be human.
0:49:22 > 0:49:24So how much data should you give away?
0:49:24 > 0:49:29Traffic management is one thing but when it comes to health care,
0:49:29 > 0:49:33the stakes, the risks and benefits are even higher.
0:49:35 > 0:49:38And in Bristol, with a project called Sphere,
0:49:38 > 0:49:40they're pushing the boundaries here, too.
0:49:42 > 0:49:45The population is getting older, and an ageing population needs
0:49:45 > 0:49:50more intense health care, but it's very difficult to pay for that health care
0:49:50 > 0:49:53in institutions, paying for nurses and doctors.
0:49:53 > 0:49:57So, the key insight of the Sphere team was that it's now possible
0:49:57 > 0:50:00to arrange, in a house, lots of small devices
0:50:00 > 0:50:03where each device is monitoring a simple set of signals
0:50:03 > 0:50:05about what's going on in that house.
0:50:05 > 0:50:08There might be monitors for your heart rate or your temperature,
0:50:08 > 0:50:12but there might also be monitors that notice, as you're going up and down stairs,
0:50:12 > 0:50:14whether you're limping or not.
0:50:15 > 0:50:19They've invited me to go and spend a night in this
0:50:19 > 0:50:24very experimental house, but unfortunately, I'm not allowed to tell you where it is.
0:50:24 > 0:50:28The project is a live-in experiment and will soon roll out
0:50:28 > 0:50:30to 100 homes across Bristol.
0:50:30 > 0:50:35It's a gigantic data challenge, overseen by Professor Ian Craddock.
0:50:35 > 0:50:36So, that's one up there, then?
0:50:36 > 0:50:38Yes, that's one of the video sensors
0:50:38 > 0:50:41and we have more sensors in the kitchen.
0:50:41 > 0:50:44We have another video camera in the hall and some environmental sensors,
0:50:44 > 0:50:46and a few more in here.
0:50:47 > 0:50:51The house can generate 3-D video,
0:50:51 > 0:50:55body position, location and movement data
0:50:55 > 0:50:56from a special wearable.
0:50:58 > 0:50:59How much data are you collecting, then?
0:50:59 > 0:51:03So, when we scale from this house to 100 houses in Bristol,
0:51:03 > 0:51:06in total we'll be storing over two petabytes of data for the project.
0:51:06 > 0:51:12Lord. So, on my computer at home, I don't even have a terabyte hard drive
0:51:12 > 0:51:14and you're talking about 20,000 of those.
0:51:14 > 0:51:17Yes. I mean, you know, the interaction of people with their environment
0:51:17 > 0:51:21and with each other is a very complicated and very variable thing
0:51:21 > 0:51:23and that's why it is a very challenging area,
0:51:23 > 0:51:26especially for data analysts,
0:51:26 > 0:51:29machine learners, to make sense of this big mass of data.
0:51:31 > 0:51:34I'm happy to find out that the research doesn't call
0:51:34 > 0:51:37for cameras in the bedroom or bathroom,
0:51:37 > 0:51:40but I do have to be left entirely on my own for the night.
0:51:42 > 0:51:48The very first thing I'm going to do is pour myself a nice bloody big
0:51:48 > 0:51:49glass of wine. There we go.
0:51:52 > 0:51:55So, that nice glass of wine that I'm enjoying isn't completely guilt-free,
0:51:55 > 0:51:59because I've got to admit to it to the University of Bristol.
0:52:00 > 0:52:03I have to keep a log of everything I do,
0:52:03 > 0:52:08so that the data from my stay can be labelled with what I actually got up to.
0:52:08 > 0:52:12In this way, I'll be helping the process of machine learning,
0:52:12 > 0:52:16teaching the team's computers how to automatically monitor things like
0:52:16 > 0:52:19cooking, washing and sleeping,
0:52:19 > 0:52:22signals in the data of normal behaviour.
0:52:27 > 0:52:29In the interests of science.
0:52:29 > 0:52:34'I was also asked to do some things that are less expected.'
0:52:34 > 0:52:37Oh! I spilled my drink.
0:52:37 > 0:52:40'The team need to learn to detect out-of-the-ordinary behaviour, too,
0:52:40 > 0:52:44'if they want to, one day, spot specific signs of ill health.'
0:52:46 > 0:52:50Right, I'm going to run this back to the kitchen now.
0:52:52 > 0:52:55It's a fairly strange experience.
0:52:55 > 0:52:58I think the temperature sensors, the humidity sensors,
0:52:58 > 0:53:01the motion sensors, even the wearable
0:53:01 > 0:53:03I don't have a problem with at all.
0:53:03 > 0:53:08For some reason the body position is the one that's getting me.
0:53:08 > 0:53:13On the flipside, though, I would go absolutely crazy to have this data.
0:53:13 > 0:53:16This is the most wonderful... My goodness me.
0:53:16 > 0:53:20Everything you could learn about humans. It would be so brilliant.
0:53:26 > 0:53:30One thing I wanted to do was to do something completely crazy
0:53:30 > 0:53:33just to see if they can spot it in the data. Just to kind of test them.
0:53:36 > 0:53:38OK, ready?
0:53:44 > 0:53:46I can't believe this is my life now.
0:53:48 > 0:53:52'Anyone can get the data from my stay online if they fancy trying to find
0:53:52 > 0:53:55'my below-the-radar escape.
0:53:55 > 0:53:59'The man in charge of machine learning, Professor Peter Flach,
0:53:59 > 0:54:00'has the first look.'
0:54:02 > 0:54:04Between nine and ten, you were cooking.
0:54:04 > 0:54:05Correct.
0:54:05 > 0:54:09Then you went into the lounge. You had your meal in the lounge.
0:54:09 > 0:54:11You know what? I ate on the sofa.
0:54:11 > 0:54:12And you were watching crap television.
0:54:12 > 0:54:14I was watching crap television?
0:54:14 > 0:54:15I've been found out.
0:54:15 > 0:54:20We didn't switch the crap-television sensor on. That's not on here, but OK.
0:54:20 > 0:54:25- So, you were in the lounge sort of until 11:30.- Correct.
0:54:27 > 0:54:30Then you went upstairs, there's a very clear signal here.
0:54:30 > 0:54:34- And then, from then on, there isn't a lot of movement.- I was in bed.
0:54:34 > 0:54:36So, I guess you were in bed.
0:54:36 > 0:54:37Sleeping.
0:54:37 > 0:54:41Normal activities, like cooking or being in bed, are relatively
0:54:41 > 0:54:43straightforward to spot.
0:54:43 > 0:54:45But what about the weird stuff?
0:54:46 > 0:54:48This is yesterday, again.
0:54:48 > 0:54:51I can see it. I can see the moment.
0:54:51 > 0:54:54- You can see the moment? - I can see it, yeah.
0:54:54 > 0:54:58There's something happening here which is sort of rather quick.
0:54:58 > 0:55:02You've been in the lounge for quite a while and then, suddenly,
0:55:02 > 0:55:06there's a brief move to the kitchen here
0:55:06 > 0:55:10and then very quick cleaning up in the lounge.
0:55:10 > 0:55:14- I wasted good wine on this experiment.- Good wine?
0:55:14 > 0:55:19Humans are extraordinarily good at spotting most patterns.
0:55:20 > 0:55:23For machines, the task is much more challenging,
0:55:23 > 0:55:26but, once they've learned what to look for,
0:55:26 > 0:55:28they can do it tirelessly.
0:55:30 > 0:55:31I suppose, in the long run,
0:55:31 > 0:55:35if you are going to scale this up to more houses,
0:55:35 > 0:55:40you can't have people sifting through these graphs trying to find...
0:55:40 > 0:55:42I mean, you have to train computers to do them.
0:55:42 > 0:55:47You have to train computers to do them. One challenge that we are facing is that our models,
0:55:47 > 0:55:51our machine learning classifiers and models, need to be robust
0:55:51 > 0:55:55against changes in layout, changes in personal behaviour,
0:55:55 > 0:55:58changes in the number of people that are in a house.
0:55:58 > 0:56:02And maybe we are wildly optimistic about what it can do,
0:56:02 > 0:56:06but we are in the process of trying to find out what it can do,
0:56:06 > 0:56:08at what cost, at what...
0:56:09 > 0:56:14..invasion into privacy, and then we can have a discussion about whether,
0:56:14 > 0:56:16as a society, we want this or not.
0:56:18 > 0:56:20If this type of technology rolls out,
0:56:20 > 0:56:24machines will be modelling us in mathematical terms
0:56:24 > 0:56:28and intervening to help keep us healthy in real time -
0:56:28 > 0:56:30and that's completely new.
0:56:32 > 0:56:37It's true that our fascination with machine, or artificial, intelligence
0:56:37 > 0:56:40is as old as computers themselves.
0:56:40 > 0:56:44Claude Shannon and Alan Turing both explored the possibilities
0:56:44 > 0:56:45of machines that could learn.
0:56:47 > 0:56:49But it's only today,
0:56:49 > 0:56:53with torrents of data and pattern-finding algorithms,
0:56:53 > 0:56:56that intelligent machines will realise their potential.
0:57:00 > 0:57:03You'll hear a lot of heady stuff about what's going to happen when we
0:57:03 > 0:57:06mix big data with artificial intelligence.
0:57:06 > 0:57:10A lot of people, understandably, are very anxious about it.
0:57:10 > 0:57:12But, for me, despite how much the world has changed,
0:57:12 > 0:57:16the core challenge is the same as it always was.
0:57:16 > 0:57:20It doesn't matter if you are William Farr in Victorian London trying to
0:57:20 > 0:57:25understand cholera or in one of Bristol's wired-up houses,
0:57:25 > 0:57:29all you're trying to do is to understand patterns in the data
0:57:29 > 0:57:32using the language of mathematics.
0:57:32 > 0:57:36And machines can certainly help us to find those patterns,
0:57:36 > 0:57:38but it takes us to find the meaning in them.
0:57:40 > 0:57:43We should be worried about what we're going to do with these smart technologies,
0:57:43 > 0:57:47not about the smart technologies in themselves.
0:57:47 > 0:57:50They are in our hands to shape our future.
0:57:51 > 0:57:53They will not shape our futures for us.
0:58:00 > 0:58:04In the blink of an eye, we have gone from a world where data,
0:58:04 > 0:58:09information and knowledge belonged only to the privileged few,
0:58:09 > 0:58:12to what we have now, where it doesn't matter if you're trying to work out
0:58:12 > 0:58:17where to go on holiday next or researching the best cancer treatments.
0:58:17 > 0:58:20Data has really empowered all of us.
0:58:20 > 0:58:23Now, of course, there are some concerns about big corporations
0:58:23 > 0:58:28hoovering up the data traces that we all leave behind in our everyday lives,
0:58:28 > 0:58:33but I, for one, am an optimist as well as a rationalist
0:58:33 > 0:58:37and I think that if we can marshal together the power of data,
0:58:37 > 0:58:43then the future lies in the hands of the many and not just the few.
0:58:43 > 0:58:47And that, for me, is the real joy of data.
0:58:47 > 0:58:48MUSIC: Good Vibrations by The Beach Boys
0:59:06 > 0:59:10Subtitles by Ericsson