At ThinkingEarth, we view the Earth as a complex unified and interconnected system. To harness the power of Artificial Intelligence (AI), we use cutting-edge techniques, including deep learning, causality, eXplainable AI, and physics-aware Machine Learning. We leverage the predictive abilities of Self-Supervised Learning and Graph Neural Networks to develop task-agnostic Copernicus Foundation Models and a Graph representation model of the Earth. 

We demonstrate the potential of these assets through small-scale downstream Spotlight Applications, as well as large-scale use cases that integrate distributed industrial and user non-EO datasets. These use cases address ambitious problems with high socio-environmental impact and new business growth opportunities, such as accelerating Europe's clean energy transition and independence from volatile fossil fuels, understanding Earth's processes by modeling causal Earth system teleconnections, and assessing and modeling the impact of current and future Climate emergency in biodiversity and food security. 

Anthropogenic climate changes pose severe threats to the Earth system, its ecosystems and human society. Effective strategies must be developed to mitigate further adverse changes and adapt to inevitable impacts, particularly the risks arising from an increase in extreme events. In doing so, transformation pathways toward sustainable Earth stewardship urgently need to be identified from global to local scales. The TUM Innovation Network EarthCare is an explorative and interdisciplinary study that bridges the knowledge gap between the use of Artificial Intelligence and Earth observation data for our planet’s sustainability. Under this general goal, EarthCare explores three main directions 1). methodologies for retrieving geoinformation from massive amount of Earth observation data, 2). Methodologies for creating hybrid earth system models of compartments of the Earth by combining machine learning and Earth observation data, and 3). Methodologies and use cases of impact models for decision of sustainability action. The vision of EarthCare is to provide key methodologies for science and policy making for a sustainable future. 
Our PI team consists of high profile experts from four disciplines: 1.) Earth observation, 2). AI and data science, 3). Earth system modeling, and 4). Sustainability, which has unique capabilities to unlock the potential of merging highly innovative methods in Earth observation, artificial intelligence, and Earth system modeling with applications in biodiversity and forestry, the urban domain, and climate-induced natural hazards. The cross-cutting framework of EarthCare promotes the exchange between key disciplines of climate science and sustainability, and consolidate the scattered expertise in the Munich area.
 

The main subject of the scientific investigations in this project is the improvement of building information (geometry, attributes) in OpenStreetMap (OSM) using a simulation-based fusion of heterogeneous remote sensing data and to use the updated OSM data for follow-up applications. The working basis is the simulation environment SimGeoI, which enables the modeling of imaging processes of different sensors by exploiting the metadata of remote sensing acquisitions as well as available geometric prior knowledge of the scene under investigation. SimGeoI allows not only a coarse semantic interpretation of the scene, but also an object-related alignment of corresponding scene elements. In this project, SimGeoI is used to compare geometric OSM information with remote sensing data produced under different sensor configurations and at different acquisition times, and to enrich OSM with geometric corrections (position, height) and attributes (e.g. building type, roof structure) gained from a fusion of the different remote sensing data. The scientific investigations of this project aim at three core themes: First, a methodical framework will be developed, which allows the geometric correction of OpenStreetMap data based on the prediction and comparison of building shapes, using a pair of remote sensing images (optical, SAR or mixed). In a second step, geometrically improved OSM information will be used to extract building-related attributes from multi-modal remote sensing data. Finally, the transferability of the developed methods will be experimentally analyzed and interfaces to follow-up applications (Open Event Mapping, Climate Event Portal, Virtual Reality) will be investigated. The methodology will be accompanied by validation in order to evaluate the positional, thematic and temporal accuracy of derived results.

This project is funded by MunichRE, which focuses on automating the recognition of key building attributes from remote sensing images to enhance building insurance analysis. Given the widespread availability and extensive coverage of remote sensing data, this approach offers an efficient alternative to traditional field surveys, significantly reducing time and effort in gathering building-related information for insurance analysis.

The project focuses on three related building attributes: footprints, function, and insurance value. In the first phase, a network for footprint recognition will be developed, laying the foundation for subsequent tasks. Custom deep learning models will be created to address each attribute separately. In the next phase, efforts will be dedicated to analyzing the correlations between different attribute recognition tasks, resulting in a multi-task learning network. This will provide a comprehensive framework for recognizing fine building attributes from remote sensing images.

The project goal is AI-based precipitation forecast for Germany with a particular focus on extreme weather events. In this project, a novel method for precipitation forecast based on GNSS data and the traditional meteorological observation data. More specifically, two types of resource efficient AI methods will be developed, including label efficient AI methods and computation efficient AI methods.

Recently big Earth observation data amounts in the order of tens of Petabytes from complementary data sources have become available. For example, Earth observation (EO) satellites reliably provide large scale geo-information of worldwide cities on a routine basis from space. However, the data availability is limited in resolution and viewing geometry. On the other hand, closer-range Earth observation platforms such as aircrafts offer more flexibility on capturing Earth features on demand. This sub-project is to develop new data science and machine learning approaches tailored to an optimal exploitation of big geospatial data as provided by abovementioned Earth observation platforms in order to provide invaluable information for a better understanding of the built world. In particular, we focus on the 3D reconstruction of the built environment on an individual building level. This research landscape comprises both a large scale reconstruction of built facilities, which aims at a comprehensive, large scale 3D mapping from monocular remote sensing imagery that is complementary to those derived from camera streams (Project Cremers), as well as a local perspective, which aims at a more detailed view of selected points of interest in very high resolution, which can serve as basis for thermal mapping (Project Stilla), semantic understanding (Project Kolbe & Petzold) and BIMs (Project Borrmann). From the AI methodology perspective, while the large-scale stream will put the focus on the robustness and transferability of the deep learning/machine learning models to be developed, for the very high-resolution stream we will particularly research on the fusion of multisensory data, as well as hybrid models combing model-based signal processing methods and data-driven machine learning models for an improved information retrieval. With the experience gained in this sub-project, we will lay the foundation for future Earth observation that will be characterized by an ever-improved trade-off between high coverage and simultaneously high spatial and temporal resolutions, finally leading to the capability of using AI and Earth observation to provide a multi-scale 3D view of our built environment.

By 2050, around three quarters of the world’s population will live in cities. The new dimension of ongoing global urbanization poses fundamental challenges to our societies across the globe. Despite of increasing efforts, global urban mapping still drags behind the geometric, thematic and temporal resolutions of geo-information that would be needed to address these challenges.

Recently, big Earth observation data amounts in the order of tens of Petabytes (PBs) from complementary data sources have become available. For example, Earth observation (EO) satellites of space agencies reliably provide geodetically accurate large scale geo-information of worldwide cities on a routine basis from space. But the data availability is limited in resolution and viewing geometry. On the other hand, constellations of small and less expensive satellites owned by commercial players, like “Planet”, have been providing images for global coverage on a daily basis since 2017, yet with reduced geometric and radiometric accuracy.

As complementary sources of geo-information massive imagery, text messages and GIS data from open platforms and social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality.

This project aims at jointly exploiting big data from social media and satellite observations for urban science.