Information about the built environment at scale provides actionable insight that can transform a broad spectrum of industries. This is the vision behind Geoscape®, a location intelligence product conceived by PSMA Australia and developed in partnership with DigitalGlobe, that delivers detailed information about buildings and their attributes, trees and land cover for every address in Australia.
The data for each building in Geoscape includes the presence or absence of photovoltaic solar panels. This information is important for the insurance industry because solar panels may pose a fire hazard and thus affect insurance premiums. Emergency services can also benefit from it in responding to a fire. In addition, quantifying solar panel adoption can help solar panel manufacturers market to customers, enable utilities to understand and predict distributed energy generation, and drive research on factors that affect renewable energy choices.
How did DigitalGlobe identify solar panels on 15 million buildings spanning 7.6 million km2 of the Australian continent? We first collected a clean data set of hundreds of thousands of examples of rooftops with and without solar panels by running a Tomnod crowdsourcing campaign. Our team then curated the crowd labels and the curated data set was used to train a convolutional neural network binary classifier. The resulting model was containerized on GBDX and deployed over the entire building dataset.
A sample of the results for five different cities is shown in the following map. The presence and absence of a solar panel is indicated by green and red, respectively. (The building footprints were provided by our partner, Ecopia.)
A sample of buildings with and without solar panels in Australian cities. Green/red indicates presence/absence of solar panels.
While maps are nice, it’s numbers that are important. Our team examined a large sample of the model’s classifications and recorded the classification errors. For comparison purposes, we had the Tomnod crowd label the same buildings and recorded the crowd’s classification errors as well. We then computed confidence intervals for both the model’s and the crowd’s precision and recall. (Precision is the percentage of buildings that actually have solar panels as a percentage of the buildings identified as having a solar panel. Recall is the percentage of buildings with solar panels that were positively identified.) The results are shown in the following plot for two sets of buildings in different parts of Australia, with each set numbering in the tens of thousands.
Precision and recall confidence intervals for the model and the Tomnod crowd, on two sets of buildings.
These numbers are astonishing for multiple reasons. First, the model outperformed the crowd. Not only is it orders of magnitude quicker to deploy the model compared to running a crowdsourcing campaign, it is also more accurate. In addition, achieving precision and recall that exceed 90 percent given the complexity of this task is remarkable. Even at 30 cm/pixel resolution solar panels are not easy to distinguish and tell apart from similar features such as solar water heaters and skylights. Here are some representative examples where the model made the correct decision and the crowd did not.
Faintly discernible solar panels; model said yes, crowd said no.
Skylights on the left, a pergola and a water heater on the right; model said no, crowd said yes.
While the model outperformed the crowd, it too delivered some false positives and false negatives shown in the samples below.
Model false positives.
Model false negatives.
Why did the model behave this way? It’s not clear, which is part of the fascination of deep learning! At DigitalGlobe Research and Development, we are committed to looking for these answers, and to harnessing the amazing progress of this technology to build accurate and competitive information products that can provide answers to our customers questions at global scale. Stay tuned for updates!
Geoscape® is a registered trademark of PSMA Australia.