Reconstructing detailed semantic 3D building models represents a fundamental challenge in photogrammetry, geodesy, computer vision and geoinformatics. Recent advancements have highlighted the potential of using 2D building footprints and aerial imagery to generate building models up to the level of detail (LoD) 2. These LoD 2 models display complex roof shapes but exhibit generalized facades. Such building models have become increasingly ubiquitous, with more than 140M open-access building models available, e.g., in the United States, Poland, Germany, and Singapore.
However, achieving facade-detailed semantic LoD3 building models remains a significant, unresolved challenge. Currently, creating LoD3-specific facade elements, such as windows and doors, often relies on manual modeling. Yet, automated LoD3 reconstruction is essential for numerous digital twin applications, including simulating flood damage, estimating heating demand, calculating facade solar potential, and testing automated driving functions.
Our group's preliminary studies show that mobile mapping data offers the best potential for semantic LoD3 facade modeling. Also, recent years have seen a proliferation of mobile mapping units and street-level images, e.g., Google Street View. These units provide highly accurate and dense street-level images and point cloud data. However, utilizing such data necessitates robust, accurate, and comprehensive semantic segmentation and reconstruction algorithms.
To unlock the applications as mentioned earlier, in this project, we aim to:
1. Develop a robust image-to-model projection method leveraging the existing LoD2 building models for the reconstruction of LoD3 models by projecting street-level images onto the LoD2 surfaces
2. Improve semantic segmentation of facade elements in images by using the model-to-ray conflict analysis
3. Investigate the impact of the proposed method on the model-based reconstruction of the level of detail 3 building models.
This project is funded by TUM Global Incentive Fund. It is a collaboration between
- TUM Chair of Engineering Geodesy,
- TUM Professorship of Photogrammetry and Remote Sensing (Olaf Wysocki) and
- National University of Singapore's Urban Analytics Lab, Filip Biljecki.
The project duration is 2024 to 2025.