Projects

Deep Snow: Deep Learning for Snow Depth Monitoring

Project partners: ExoLabs, Swiss Federal Institute for Forest, Snow and Landscape Research (WSL)
 

Reliable information on the spatial distribution of snow in mountain ranges are critical for risk assessment, outdoor activities, and water resource management. These water resources from snowmelt are indispensable as they provide drinking water, supply crop production and generate hydroelectric power worldwide. Despite this importance, we lack an accurate and operational spatiotemporal quantification of transient water storage in mountain ranges. Consequently, accurate estimates of snow quantities in space and time are the most important unsolved problem in mountain hydrology.

In this project, we aim to develop novel snow products for Switzerland based on multiple Earth Observation (EO) datasets and deep learning algorithms. Our objective is to provide seamless, timely information on snow cover, snow depth and snow water equivalent (SWE) on a daily basis in a high spatial resolution (20 m pixel spacing) via multiple data services. Successful completion would outperform current standards (weekly information with 1 km pixel spacing) and enable new market opportunities. These include improved outdoor safety standards, hydropower production planning and real-time snow risk assessment among others.

Our team is uniquely poised to meet this challenge, consisting of members from EcoVision, ExoLabs, and WSL with expertise in snow monitoring, remote sensing and deep learning. Together, we propose an ambitious, though realistic, project plan to generate, validate, and deploy snow products and services by harnessing the power of EO with deep learning using scalable cloud computing. These products would be really valuable for water management and tourism safety, and could help us better understand how the snow in the alps is reacting to climate change and evolving over time.

Contacts:
Jan Dirk Wegner, ETH Zurich,
Rodrigo Caye Daudt, ETH Zurich,
Hendrik Wulf, ExoLabs,
Yves Bühler, WSL,

4Real: real-time urban pluvial flood forecasting

Project partners: Eawag, Swiss Federal Institute of Aquatic Science and Technology; SDSC, Swiss Data Science Center
 

Urban pluvial floods, occurring when precipitation cannot be fully absorbed by the drainage system, cause flooding and substantial damages, as well as disruption to socio-economic activities. Their fast occurrence and relatively short duration mean that physically-based models for flood prediction are of limited use, due to their long computational runtime.
In this project, we will develop new Deep Learning (DL) methods to generate real-time flood predictions, such that they can be used to alert the population and to plan mitigation and rescue actions. Additionally, by exploiting hydraulic modelling knowledge and tightly integrating it with DL models, we aim to produce DL-based flood models which return spatially explicit, two-dimensional flood hazard maps with water depth, flood extent and flow velocity information. Tightly coupling the underlying hydraulic equations with a DL-framework will provide both, interpretability and adherence to physical constraints. The input data for the DL flood forecasting model will be rainfall forecasts provided by meteorological services (MeteoSwiss), images and 3D digital surface city models.

Contacts:
Jan Dirk Wegner, ETH Zurich,
Stefania Russo, ETH Zurich,
Stefano D'Aronco, ETH Zurich,
Priyanka Chaudhary, ETH Zurich,
João Leitão, Eawag Zurich,

 

Automated Large-scale High Carbon Stock estimation from Space

Project Partners: Barry Callebaut, Greenpeace

 

The aim of this joint ETH Zurich-Barry Callebaut project is stopping deforestation and building an objective, highly automated tool to guide sustainable agribusiness. Deforestation is a worldwide problem that accelerates climate change, destroys the livelihood of traditional local communities, leads to extinction of precious animal and plant species etc. Although there is raising global awareness of the consequences of deforestation, measuring it in practice at large scale and at sufficient detail to guide local decision making processes is lacking. In general, measuring deforestation is difficult because it has to factor in carbon, climate, biodiversity, and traditional land-use of local communities. The High Carbon Stock Approach, adapted by Greenpeace and supported by many NGOs and companies, is designed to account for all important factors to protect primary rainforest, while ensuring land use rights of traditional communities.

Although some first ideas towards mapping HCS from space have been proposed, these often use data with low temporal and spatial resolution, which previously was the only available imagery with sufficient coverage. Often, they also use legacy technology (e.g., maximum likelihood classifiers) that does not work well in complex scenarios. In this project, we propose to develop an unbiased, objective, and automated system centered on deep learning that uses satellite imagery acquired by optical and SAR sensors to estimate different HCS categories and warn if primary rain forest is cut down.

Contacts:
Jan Dirk Wegner, ETH Zurich,
Nico Lang, ETH Zurich,
Oliver von Hagen, Barry Callebaut,

RegisTree

Project partners: Caltech, MIT, US Forest Service, Université Bretagne Sud


In the RegisTree project we aim at developing large-scale, automated tools to map urban trees. We develop a machine learning-based system that can detect and inventory street trees automatically. From publicly available imagery of a city, our system produces a list of the location, species and trunk diameter of each street tree. Our approach uses deep learning to identify the location of a tree, classify its species, and approximate its trunk diameter, and estimate its health state. It combines publicly available geo-referenced Google Maps aerial and street view images along with map data to provide a comprehensive and accurate catalog of street trees. Our algorithm can be used to build an inventory that allows cities to better manage street trees and measure parameters like mortality rates in the long term. Cities may detect areas where trees are getting old and need replanting, create an inventory of empty locations where trees may be planted, or detect streets where trees need pruning.

For more information: https://www.registree.ethz.ch/index.html

Contacts:           
Jan Dirk Wegner, ETH Zurich,
Nico Lang, ETH Zurich,
Ahmed Nassar, Université Bretagne Sud,

Automated Tree stress estimation from images

Project partners: external pageSwiss Federal Institutefor Forest, Snow andLandscape Research (WSL)


Healthy forests prevent soil erosion, slow rainwater runoff, are essential for high biodiversity and key to clean and ample water supply. In addition, major indicators of climate change are trees. However, the amount of trees, their stress level (e.g., defoliation), species, biomass, and age are often unknown because no up-to-date database exists due to the high cost of in-situ surveys. Much effort and resources go into field surveys to monitor trees and their state of health manually. This manual process is very labor-intensive, time-consuming and scales poorly. Several human observers assess each tree, which leads to a variance in the stress estimation caused by slightly different judgments of the same tree. With this project, a collaboration bof ETH and WSL, we aim at developing an automated, image-based system to track changes of trees' states (e.g., stress level, pests, re-planting events) over time at country-scale. The long-term objective is to measure the impact of climate change on Swiss trees.

The system centers on state-of-the-art supervised deep convolutional neural networks where sparse very high-resolution ground-level images from in-situ surveys are combined with lower resolution aerial and satellite images that cover entire Switzerland. The general idea is to use sparse, but very accurate tree measurements acquired by WSL for training deep machine learning approaches that can then predict tree stress at any other location in Switzerland using satellite images of the Sentinel 2 satellites. The hope is that this system will enable faster, more efficient tree stress estimation at large scale. Moreover, it allows citizen scientists to contribute through sharing images of trees in natural environments that are analyzed by our software.  

Contacts:           
Jan Dirk Wegner, ETH Zurich,
Nico Lang, ETH Zurich,
Arthur Gessler, WSL,


Biodiversity estimation from satellite images

Project partners: Swiss Federal Office for Agriculture (BLW), ETH IAS


Maintaining and protecting biodiversity of plants and animals is essential for our life on earth. Due to climate change, human destruction of precious habitats, and industrial agriculture, biodiversity is decreasing in many countries. A major bottleneck for counter-measures is lack of accurate, dense biodiversity data at large scale. Today, the main technique to monitor biodiversity is field surveys that assess biodiversity in situ by counting the amount of different species per area. These studies are very labor-intensive, costly, and deliver only scarce, point-wise data. Moreover, the temporal resolution is low because revisit cycles of the test sites are long.

In this project, we aim at automating biodiversity estimation in Switzerland using (deep) machine learning and overhead imagery. We combine point-wise, high-quality in-situ data of Swiss institutions like BLW, AgrosScope, BAFU, and WSL that have been conducting field surveys for decades with dense, large-scale satellite imagery. First experiments have shown that the spectral resolution of ESA satellites Sentinel 2, designed for vegetation monitoring, partially allows for plant species recognition on the ground. This project aims at directly mapping different species distributions and biodiversity in Switzerland to help protecting the environment and measuring the impact of agriculture on biodiversity.

Contacts:           
Jan Dirk Wegner, ETH Zurich, Stefano D'Aronco, ETH Zurich,
Riccardo De Lutio, ETH Zurich,
Frank Liebisch, ETH Zurich,
Jérôme Frei, BLW,


Monitoring of agricultural sites and their nitrate emission risk from space

Project partners: Swiss Federal Office for Agriculture (BLW), ETH IAS


Monitoring agricultural crops and their growing status is essential for many different reasons like yield prediction, agricultural management, distribution of subsidies, and food security, for example. Acquisition of agricultural parameters is done through self-reporting of the farmers. This is a significant bureaucratic burden both, for the farmers and for federal administration. In addition, this procedure often delivers inaccurate and inhomogeneous data at low temporal resolution.

The objective of this project is to automate the acquisition of agricultural parameters in Switzerland to the largest extent possible by combining (deep) machine learning, satellite imagery, and auxiliary data about the weather, soil quality, the terrain etc. One major parameter that is vital for ensuring good quality of ground water and the environment in general is the amount of nitrogen fertilization. While nitrogen fertilizer is, on the one hand, essential for plant growth it is, on the other, a risk for water quality if applied in too large quantities. A core research question is thus to what extent we can estimate nitrogen status of crops from images (satellite or UAVs) at large scale to build an emission risk map. This shall allow for better ground water protection and reduction of the emission risk overall.

Contacts:           
Jan Dirk Wegner, ETH Zurich, Stefano D'Aronco, ETH Zurich,
Özgür Türkoglu, ETH Zurich,
Frank Liebisch, ETH Zurich,
Jérôme Frei, BLW,


Yield prediction with satellite images and machine learning

In this project, we propose to classify different plant species in overhead images using supervised machine learning. Monitoring of plants is important for a variety of different applications like protection of the environment, estimation of the acreage per species, and yield forecasting. With an industry partner, we aim at estimating the overall acreage per species for a given area as well as predicting yield.

Our processing pipeline relies on powerful deep learning methods, and free satellite images from ESA's Sentinel 2 satellite pair. This constellation offers 10mx10m pixel size on the ground, several infrared and near-infrared channels that are custom-tailored for plant monitoring, and has a revisit time of only 5 days over the same scene increasing chances of acquiring cloud-free images for time-series analysis. We will develop a semantic segmentation method that recognizes crop species at large scale. Several species may not be recognized directly at the 10m pixel resolution especially if they are intercropped with other species. One direction of work is turning counting objects into density estimation. We will further develop a regression method for yield estimation of several species from a time series of satellite images and auxiliary data (e.g., meteorological data). Much weather data is freely available and can help yield prediction at large scale substantially. Due to frequent cloud coverage, a challenging task will be to make time series analysis robust to irregular imaging intervals. The outcome will be an unbiased, objective, and automated system that uses optical imagery acquired by aerial and/or satellite sensors to estimate tree/crop acreage, and predicts yield.

Contacts:        
Jan Dirk Wegner, ETH Zurich,
Andres Rodriguez, ETH Zurich,


Flood-level monitoring from social media images

Project partners: Swiss Federal Institute of Aquatic Science and Technology (EAWAG)


The availability of flood observation data is critical to the flood modelling process. Unfortunately, flooding processes are not easily measurable with conventional measurement technologies such as ultrasound, pressure sensing, or radar.

The main goal of this project is to use (deep) machine learning and computer vision for quantifying urban flood events. Specifically, partially submerged objects of known dimensions are automatically identified in images to estimate flood water level. Examples of such objects are: cars, bicycles, fire hydrants, or even people. Our system centers on the automatic identification of objects with a deep learning approach to estimate their submersion level. We calibrate and compare results achieved outdoors for real flooding invemts with reference image material available from a recent urban flooding experiment in a controlled setting (where ground truth water level is known). We are continuously extending a collection of real-world images from recent flooding events from social media like Twitter, and adding new capabilities like precise geo-coding of images, combination with digital surface models, and overhead images.

Contacts:           
Jan Dirk Wegner, ETH Zurich, Priyanka Chaudhary, ETH Zurich,
Joao Paulo Leitao, EAWAG,
Matthew Moy de Vitry, EAWAG,


Deep learning on graphs

Project partners: ETH Data Analytics Lab


Deep learning methods have recently demonstrated remarkable achievements on many computer vision tasks such as image segmentation and object detection. However, traditional CNN architectures are still limited to regular, grid-structured graphs. The rigidity of these networks makes it hard to exploit high-level priors about the image content or biophysical indicators that are irregularly spaced across the landscape. Modeling high-level, long-range topology has proven to be difficult with standard architectures that model more local, per-pixel evidence. The aim of this project is to develop novel deep learning methods for graphs of arbitrary structure. We want to depart from the standard paradigm that labels pixels but instead leave the grid and learn graph structures directly.

A major research question is to what extent (image) evidence will have to be computed per-pixel and at what point to depart from the grid-structure. Promising starting points are Polygon RNNs and geometric deep learning techniques like graph CNNs. In this project, we modify and further develop these frameworks to be able to analyse sparse and inhomogeneous distributions of environmental variables across the landscape as well as irregular shapes of objects like buildings and roads.

 

Contacts:           
Jan Dirk Wegner, ETH Zurich,
Aurélien Lucchi, ETH Zurich,

 

Mapping of agroforestry cocoa plantations

Project partners: Barry Callebaut
 

The worldwide cocoa production has been nearly tripled over the last thirty years and naturally, acreage has increased drastically. As cocoa plants are solely growing in the tropical regions, increasing agriculture highly affects precious habitats, biodiversity and climate change. To protect and alleviate the former, countries and companies are enforcing growing restrictions, natural reserves and climate-friendly production chains. However, especially with smallholders, this work is laborious, cost-intensive and time-consuming.
Within this project, we investigate the use of ESA’s Sentinel-1 (SAR) and Sentinel-2 (multi-spectral) satellites for cocoa farm mapping in agroforestry settings. We develop novel deep learning methods to automate the detection of existing and new farms at country-scale. A major difficulty of this project is the acquisition of non-cocoa samples that are needed to train a classifier. Thus, one of the core research questions is how to use the tremendous amount of unlabeled samples to reduce the need of tedious mapping campaigns in West Africa to collect accurate reference data. Possible directions are semi-supervised approaches and positive unlabeled learning.
Our hope is that this project will be of great value to protect precious rainforest, foster sustainably grown cocoa, and help fighting deforestation.

Contacts:
Jan Dirk Wegner, ETH Zurich,
Nikolai Kalischek, ETH Zurich,

 

Shade-tree cover and carbon stock assessment for cocoa agroforests

Project partners: University of Queensland, Lindt Cocoa Foundation
 

Agroforestry – the deliberate inclusion of shade trees in cropping systems – can increase the sustainability of cocoa production by supporting high levels of biodiversity, buffering cocoa from contemporary climate changes, offsetting future climate change through carbon sequestration, and by encouraging agricultural intensification without deforestation. Because of these advantages, and in response to supply-chain and reputational risks, chocolate producing companies are increasingly engaging in efforts to implement cocoa agroforestry in major producing countries.
In this project we will develop methods to rapidly assess shade-tree cover and carbon stocks in existing cocoa farms, across large scales, and repeatedly over time. Our aim is to develop an easy-to-use, cost-effective tool to measure changes in shade-tree cover in cocoa farms, and to monitor progress towards implementing agroforestry commitments. We will come up with spatially-explicit recommendations for optimal levels of shade-tree cover accounting for locally-varying growing conditions across Ghana and the Ivory Coast; and determine the carbon-sequestration potential of cocoa agroforestry.
To meet these goals, we will develop a method to assess shade-tree cover and carbon stocks in cocoa farms using powerful deep machine learning techniques on remote-sensed, satellite images. Moreover, we will combine industry estimates of yield across thousands of cocoa farms, local climatic and edaphic variables from existing GIS map layers, and estimates of shade-tree cover using our newly developed methods, to identify optimal levels of shade-tree cover for different growing regions across Ghana and the Ivory Coast.

Contacts:
Alexander Becker, ETH Zürich,
Wilma Hart, University of Queensland,
Jan Dirk Wegner, ETH Zurich,
 

JavaScript has been disabled in your browser