Deep Learning and satellite images to estimate the impact of COVID19

Motivated by the fact that the Coronavirus Disease (COVID-19) pandemic has caused worldwide turmoil in a short period of time since December 2019, we estimate the negative impact of COVID-19 lockdown in the capital of Spain, Madrid, using commercial satellite imagery courtesy of Maxar Technologies©. The authorities in Spain are adopting all necessary measures, including urban mobility restrictions, to contain the spread of the virus and mitigate its impact on the national economy. These restrictions leave signatures in satellite images that can be automatically detected and classified.

Monitoring vehicles

We focus on the development of a car-counting solution to monitor the presence of visible cars within high-resolution images. Recent studies reveal up to a 90% increase when comparing cars traffic between fall of 2018 and 2019 in several hospitals from Wuhan, China. This observation could mean that an infection was growing in the community and people required health care services. Similarly, we hypothesize that the number of vehicles decreased drastically during the COVID-19 lockdown in Madrid, thus we further investigate on how to accurately detect these vehicles using computer vision techniques.

Figure 1: Satellite images courtesy of RSMetrics© suggest that COVID-19 may have been present and spreading through China before the outbreak was first reported to the world. — *Figure 1:* *Satellite images courtesy of RSMetrics© suggest that COVID-19 may have been present and spreading through China before the outbreak was first reported to the world.*

Deep Learning

For this reason, we research on how to accurately detect these vehicles using computer vision techniques. Recently, driven by the success of deep learning-based algorithms, most literature have pursued approaches based on Convolutional Neural Networks (CNNs). The main reason for this popularity is that CNNs can automatically learn feature representation so that there is no need for manual feature extraction. As a result, CNNs are attracting widespread interest because of their robustness to appearance changes under “in-the-wild” conditions.

Current approaches detecting objects typically fail or lose precision due to the relatively small size of the target objects and the vast amount of data to be processed in the presence of multiple “in-the-wild” factors, such as, different cities/countries, viewpoint changes, occlusions, illuminations, blurriness, and so on.

Figure2: *Challenging appearance variability due to different factors including viewpoint changes (nadir angle), shadows, daylight changes marked by weather and seasons, etc.*

Labelled data

We categorize existing approaches into two groups according to whether they estimate the number of cars directly from the image (counting by regression), or they learn to detect individual cars first, and then count occurrences to set an overall number of small vehicles in the image (counting by detection). We leap to the conclusion the latter approach achieves superior performance.

Figure 3: Both supervised approaches need a set of training images with annotations. Counting by regression requires the overall number of cars as label. Counting by detection requires the vehicle position by setting the bounding box coordinates on each instance.

Besides the aforementioned difficulties, the studies of object detection in satellite imagery are also challenged by the data set bias problem, which means that learned models are usually constrained to the same scene on which they were trained. To alleviate such biases, we train our model using also different vehicle annotations at different spatial resolutions from COWC and DOTA benchmarks, reflecting the demands of real-world applications. As far as we know, this is the first time that an algorithm successfully combines images at different resolutions to deal with the lack of satellite data properly annotated.

Madrid dataset

As we expected, we need the highest resolution commercially available in order to detect small vehicles. For this reason, we download 153 satellite images from 22 hot spots around the autonomous community of Madrid with a spatial resolution of 30 cm (WorldView-4 satellite data accessible using SecureWatch©). We select specific areas in Madrid where car-counting is a proxy of activity, such as shopping centres, highway crossings, hospitals, industrial areas and universities, among others.

In the video below we visually observe the reduction in the total number of cars before and during the COVID-19 restrictions. Thus, it seems reasonable to study the overall impact of the lockdown on the traffic volume.

Walkthrough over some processed images to visually perceive the dramatic reduction in the presence of vehicles over Madrid (audio in Spanish with English subtitles).

Results

In the experiments we measure the performance of our proposal, and compute car-counting statistics to quantify the dramatic drop in the number of vehicles during the lockdown. As a result, we corroborate these statistics using additional indicators such as telco data and traffic sensors data respectively. We reach the conclusion that these insights correlate with official statistics on economic activity, thus car-counting statistics can complement traditional measures of economic activity in helping policy makers tailor their responses to flatten the recession curve.

Figure 4: Timeline curves of how the COVID-19 outbreak is evolving in Madrid since 2020. Red, yellow and blue colours compare curves obtained using anonymized and aggregated telco data from Telefónica Movistar antennas, traffic statistics acquired from the City Council of Madrid sensors, and by estimating the presence of visible cars with our satellite technology respectively.

Additional information about the vehicle detection technology, the downloaded high-resolution satellite images, a market analysis, and comparative results for each region of interest are also submitted in the supplementary material.

Written by Roberto Valle Fernández.

Don’t miss the next webinar (in Spanish) “Deep Learning and AI to improve traffic in times of Covid19″ that will take place on November 25th. To schedule this event click here. (Remember to follow the registration steps to watch it live)

Spanish version: