As the national power outage in Venezuela grinds on, the IP-Observatory is providing real-time insights through big-data.

Just before 5 pm on Thursday 7 March 2019, Venezuela was plunged into darkness. Rush-hour commuters were forced to walk home, hospitals scrambled to set up back-up generators, and all the while, rival political parties rushed to establish radically contrasting perspectives on the cause.

At times of crisis, access to reliable information can be a matter of life and death. However, in a country like Venezuela, starkly rated by Freedom House as ‘not free’ for both its press and internet, the lowest rating of any South American nation, there is little hope of receiving reliable information from government sources.

In this tumultuous context, the IP-Observatory, from SoDa Laboratories at the Monash Business School, offers Venezuelans on the ground and the international community with a trusted source of near real-time observations on the state of the country’s internet which also gives insight to what’s happening with their electricity and infrastructure.

Powerful technology for the public good

Using technology developed over four years, the team comprising Dr Klaus Ackermann, lecturer in the Department of Econometrics and Business Statistics, together with Dr Paul Raschky and Dr Simon Angus, Associate Professors in the Department of Economics, and research assistant, Mr Satya Borgohain, have established a powerful observation, processing and visualisation methodology which can provide detailed insights on internet connectivity anywhere on the globe.

“Our central aim with the IP Observatory is to place very complex scanning, data processing and visualisation technology into the hands of ordinary citizens for the progress of a free and open society everywhere,” Dr Angus says.

“We are only one of two platforms — and the only one housed in a major institution like Monash University — that we are aware of, providing this kind of platform to the public,” he says.

“We feel that it has been particularly important for all Venezuelans to be able to see that both platforms, with completely different teams and equipment, have reached essentially the same conclusions on the state of the Venezuelan ICT infrastructure as this crisis has unfolded.”

Dr Angus explains that what Venezuelans need at this time is a lot of help from the international community in terms of practical aid and reliable information as to the state of their infrastructure to make hour-by-hour decisions before considering plans for the next days and weeks.

“The IP Observatory provides this reliable source, and we will continue to do so until there is some resolution,” he says.

Key to the team’s approach is harnessing the remarkable connectivity of the internet to set up remote observational points in multiple locations worldwide, measure the activity of any particular internet address anywhere on the planet, and then gather these observations together into a central location for processing. The team do not monitor content at all.

Over 9 million observations and counting

“We’ve made over 9 million observations of internet activity in Venezuela,” Dr Angus says.

“Whilst this number can seem a lot, the reality is, this is just a fraction of the observations the IP-Observatory is making every day. Our capability stretches to any major ICT event happening globally. In fact, our infrastructure is capable of examining over 9 million internet connections every 15 minutes.”

“We don’t sweat the details. Yes, it’s a lot of data!” Dr Angus laughs.

Venezuela a difficult context

Previously, the team has focussed the IP Observatory’s lens on the 2018 Russian Presidential Election, and the impact of Hurricane Florence on the east coast of the United States as it made ferocious land-fall in September 2018.

In both of these contexts, ICT infrastructure is well developed, making precise estimates of connectivity much easier for the team.

“Previously, we’ve had tens of millions of internet addresses to sample from to make our observations. However, in Venezuela, although estimates vary, roughly half of the population is not online, and those who are online are concentrated in a relatively small number of jurisdictions,” Dr Angus says.

“So in the case of Venezuela, we’ve had to recalibrate our approach to handle this much more challenging context, a fact that is reflected in our observations currently displayed on the IP Observatory. We are showing only around 17 municipios (local government regions) out of nearly 200 across the country to ensure our observations focus on the regions with the highest quality information.”

Exploring the frontiers of social data science

The IP Observatory team are all founding members of the SoDa Laboratories, a newly established empirical sciences group in the Monash Business School, who apply new tools from data science, machine learning, and beyond to answer social science questions.

Alongside the IP Observatory, the SoDa laboratory projects span empirical work on a range of alternative data including nighttime and daytime satellite imagery, textual corpora from news, reports and state bureaucracies, and of course, internet data.

The IP-Observatory can be accessed at https://ip-obervatory.org and the team will continue to tweet updates on the Venezuelan crisis and other events at @IP_Observatory.