Forecasting: Putting the smart into smart meters

A move to unlock the vast data collected by Victoria’s smart meters could help providers better forecast demand.

Since their rollout in Victoria, smart electricity meters have been quietly producing a vast and detailed data set of usage.

But power providers currently only use aggregate data from the entire network to forecast demand. They have access to smart meter data only for their own customers, not the entire network.

This situation potentially impacts on the accuracy of energy forecasts. But Monash Business School’s pre-eminent statistician Professor Rob Hyndman believes a move by the Victorian Government to release smart meter data – due in the next six months – could substantially improve the accuracy of forecasting by allowing a combination of data sources.

New energy technologies

Professor Hyndman and his team are already responsible for developing the algorithm and long-term electricity forecasting for the Australian Energy Market Operator (Aemo), the body responsible for the administration and operation of national and wholesale electricity markets across Australia.

His forecasting model is used for long-term planning in Victoria’s Vision 2030 energy plan, all regions of the National Energy market (the interconnected power system in Australia’s eastern and south-eastern seaboard) and short-term forecasting comparisons in all regions of the National Energy Market.

These forecasts, however, are based only on network data, as smart meter data has not been available. Now, this looks set to change.

Recently the Victorian Government, in partnership with Monash University, launched the Centre for New Energy Technologies. It is anticipated that this centre will release Victoria’s entire smart meter data log over the next six months with the aim of facilitating better energy decision-making.

“The availability of smart meter data is relatively new and unique to Victoria in that all of that data is held by one organisation, the Victorian Government, which isn’t the case anywhere else in the world,” Professor Hyndman says.

“We are on the cusp of a new era because of the availability of this data for the first time. Currently, power companies only have access to smart meter data for their own customers, not the entire network.”

Smart electricity meters report power usage in real-time 15 minute or half hour intervals back to the electricity network. This detailed data can help identify individual habits and has the potential to better predict overall demand.

Yet the overwhelming problem around analysing this data is its size, volume and complexity.

“You are talking about information on eight million households at a half hour or 15-minute intervals,” he says.


And even just turning household appliances off and on or going on holiday can make a difference to regular statistical models – these rapid changes in energy use cause too much ‘noise’ in the data.

“If power companies only used smart meter data alone, it would result in quite bad forecasting because of the level of detail and the lumpiness of the data,” says Professor Hyndman. “Yet if they use a combination of network and individual, with network demand being much more regular and predictable, they can group customers to get a more accurate average usage among similar households.”

New forecasting models for smart meter data

“Energy forecasting is inherently probabilistic,” says Professor Hyndman.

Network forecasting predictors include time of day, day of the week, season, holidays, weather conditions, price, population and recent demand. These same predictors work for smart meters but have much larger percentage errors.

Professor Hyndman has therefore developed a number of statistical models to forecast electricity demand using smart meter data.

To get around the level of detail and volume of data, Professor Hyndman has already collaborated with Dr Souhaib Ben Taieb, Professor Marc Genton and Assistant Professor Raphael Huser from Saudi Arabia’s King Abdullah University of Science and Technology to develop a machine-learning algorithm that can cope with smart meter data.

This algorithm computes one-day ahead probabilistic forecasts for each meter using time series data combined with other variables around the weather, recent demand, population etc.

In another research paper, Professor Hyndman has put forward an approach of analysing smart meter data in percentiles conditional on the time of the week.

“It provides a unique fingerprint of typical usage for a given household,” he says.

“It also avoids the impact of household events such as a party or going on holidays and allows us to cluster households based on probabilistic behaviour.”

Professor Hyndman has also used a hierarchical probabilistic method to forecast electricity demand with smart meter data. Forecasting across data groups needs to reconcile the sum of the forecasts at the lower levels with forecasts at the group level.

Traditional data forecasting methods use a top-down, bottom-up, or middle-out approach to reconcile data. But Professor Hyndman’s method shows that by taking weighted averages of the forecasts from all different levels of the data set, a more accurate data forecast is achieved.

It uses an algorithm to produce a probability density forecast for each series within the hierarchy. Through combining forecasts, the algorithm can synthesise information from different levels in the hierarchy.

“Network demand is more regular and predictable than smart-meter demand,” he says.

“However smart meter demand provides a lot more detail about individual behaviour. Reconciling these forecasts allows us to combine network forecasting with smart meter forecasting.”

Published on 24 Oct 2018