Developed by Monash Business School’s Dr Simon Angus and Dr Paul Raschky with Klaus Ackermann from the University of Chicago, the database is the culmination of more than three year’s work to process more than one trillion observations of whether computers around the world were online or offline.

The three researchers have recently released a working paper, The Internet as Quantitative Social Science Platform: Insights from a Trillion Observations, in which they demonstrate how the database could shed light on human behaviour.

The researchers also used the database to analyse how much sleep people in different countries enjoy – finding Argentinians are the world’s most rested people, sleeping 10.26 hours per night on average.

And in another application, they examine the local economic implications of internet use, including that local businesses are the main winners of our increased connectivity.

“This database is one of a kind, we now have the richest data set to map internet availability around the globe,” Dr Angus says.

“With this database, any research that has been done on major utilities usage can be applied to the internet.

“For example, we could measure the impact of a thunderstorm if the electricity blacks out. By scanning internet usage during that period we can discover when people lost internet connectivity and how long for.”

Matching observations

The database is formed from University of Southern California’s automated surveys and scans of the entire internet address space, or Internet Protocol (IP) addresses, matched with another database of IP locations. This joined database is then aggregated to show the fraction of devices which were online or offline, across 15-minute time segments, for thousands of cities worldwide.

As IP addresses can change randomly over time, research on internet usage and location has previously been very difficult.

This database matches over a trillion observations of whether a computer was online or offline during 2006-12, to an IP location library.

The data was synthesised, using the Australian Synchrotron’s MASSIVE cluster computing facility, from 1.5 trillion initial observations down to 24 billion observations as to IP location and activity.

The data was then broken into 15-minute time segments across 1600 cities within 122 countries.

Global sleep patterns

To study global sleep patterns, the three researchers looked at whether a device was online or offline across 645 cities over a seven-year period. It found people living in major cities generally sleep more compared to surrounding satellite cities.

Japanese people have the least sleep, generally with 7.27 hours per night, while Argentinians have 10.26 hours sleep per night on average.

Australia ranked number 30 out of 45 countries, averaging 8.68 hours sleep per night – more than the Chinese, English, Canadians, Americans and New Zealanders. Over the period, though North America remained consistent, the amount of sleep enjoyed by Europeans declined whereas in East Asia sleep patterns increased, suggesting, for the first time, a form of global sleep duration convergence.

The working paper also used the database to estimate how internet use drove local economic activity and industry productivity.

It found service sectors facing digital competition through outsourcing such as publishing, news, film production, administrative support and education suffered as local internet usage increased.

Other sectors

Sectors that are constrained by location, prosper with increasing internet reach such as wholesale, retail, real estate, repairs, hairdressing, mining, transportation and accommodation. This is probably due to lowered consumer search costs and efficiency gains.

The team believe the database may be also used to analyse how internet availability impacts on political mobilisation and protests. By bringing other databases of geo-located political activity to their internet activity database they are hoping to be able to shed light on the hot topic of the internet’s role in various forms of political dissent.

“Our work can be considered part of a new, exciting wave of truly multi-disciplinary, integrative research, our research findings span the disciplines of economics, computer science, and biology, while leveraging infrastructure used by physicists,” Dr Angus says.