Intelligent Autonomous Systems (INTAS)

This new 3-year research project is on visual navigation in challenging and changing environments in the context of civil demining

Congratulations to Jessica Buchner for winning the 1st place jury prize at CV Day for her work "Visual Tracking of Reptiles in a Zoological Context" and to Felix Meyer for winning the 1st place audience award for his work "Camera-based overtaking distance control"!

The Paper "HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios" by Nick Theisen, Robin Bartsch, Dietrich Paulus, and Peer Neubert has been accepted at IROS'24.

Abstract: Semantic segmentation is an essential step for many vision applications in order to understand a scene and the objects within. Recent progress in hyperspectral imaging technology enables the application in driving scenarios and the hope is that the device's perceptive abilities provide an advantage over RGB-cameras. Even though some datasets exist,there is no standard benchmark available to systematically measure progress on this task and evaluate the benefit of hyperspectral data. In this paper, we work towards closing this gap by providing the HyperSpectral Semantic Segmentation benchmark (HS3-Bench). It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols. We use the benchmark to derive two strong baseline models that surpass the previous state-of-the-art performances with and without pre-training on the individual datasets. Further, our results indicate that the existing learning-based methods benefit more from leveraging additional RGB training data than from leveraging the additional hyperspectral channels. This poses important questions for future research on hyperspectral imaging for semantic segmentation in driving scenarios.

The paper "Towards Revisiting Visual Place Recognition for Joining Submaps in Multimap SLAM" by Markus Weißflog, Stefan Schubert, Peter Protzel, and Peer Neubert has been accepted at TAROS'24.

Abstract: Visual SLAM is a key technology for many autonomous systems. However, tracking loss can lead to the creation of disjoint submaps in multimap SLAM systems like ORB-SLAM3. Because of that, these systems employ submap merging strategies. As we show, these strategies are not always successful. In this paper, we investigate the impact of using modern VPR approaches for submap merging in visual SLAM. We argue that classical evaluation metrics are not sufficient to estimate the impact of a modern VPR component on the overall system. We show that naively replacing the VPR component does not leverage its full potential without requiring substantial interference in the original system. Because of that, we present a post-processing pipeline along with a set of metrics that allow us to estimate the impact of modern VPR components. We evaluate our approach on the NCLT and Newer College datasets using ORB-SLAM3 with NetVLAD and HDC-DELF as VPR components. Additionally, we present a simple approach for combining VPR with temporal consistency for map merging. We show that the map merging performance of ORB-SLAM3 can be improved. Building on these results, researchers in VPR can assess the potential of their approaches for SLAM systems.

Preprint: https://arxiv.org/abs/2407.12408

She will work in the project "AI-DPA: Analyse und Interpretation von unstrukturierten Daten und Prozessen in zwei- und dreidimensionalen Anwendungsszenarien mit Machine Learning"

Although I already started in Koblenz in 2022, I will now give my inaugural lecture with the title "Künstliche Intelligenz hat viele Dimensionen: Die Magie hochdimensionaler Vektorrepräsentationen für Roboter und andere autonome Systeme" (the lecture will be in German).

Anyone interested is warmly invited!

February 1, 18:00-20:00, Room M001, University Campus

A joint team from our group and researchers from Sweden and Australia has won the "My Seizure Gauge'' Challenge on forecasting epileptic seizures from non-cerebral signals. A short summary of the competition and our winning approach is provided in a paper that has been accepted for publication in Nature Machine Intelligence in the Challenge Accepted track. Congratulations to Kenny Schlegel who has led this team! The paper is a joint work of our team and the challenge organizers.

Kenny Schlegel, Denis Kleyko, Benjamin H. Brinkmann, Ewan S. Nurse, Ross W. Gayler, Peer Neubert (2024). Lessons from the “My Seizure Gauge” Challenge on Forecasting Epileptic Seizures from Non-Cerebral Signals. Nature Machine Intelligence Challenge Accepted (to appear)

The paper "Local positional graphs and attentive local features for a data and runtime-efficient hierarchical place recognition pipeline" by Fangming Yuan, Stefan Schubert, Peter Protzel, and Peer Neubert has been accepted for publication in the IEEE RA-L journal.

Abstract: Large-scale applications of Visual Place Recognition (VPR) require computationally efficient approaches. Further, a well-balanced combination of data-based and training-free approaches can decrease the required amount of training data and effort and can reduce the influence of distribution shifts between the training and application phases. This paper proposes a runtime and data-efficient hierarchical VPR pipeline that extends existing approaches and presents novel ideas. There are three main contributions: First, we propose Local Positional Graphs (LPG), a training-free and runtime-efficient approach to encode spatial context information of local image features. LPG can be combined with existing local feature detectors and descriptors and considerably improves the image-matching quality compared to existing techniques in our experiments. Second, we present Attentive Local SPED (ATLAS), an extension of our previous local features approach with an attention module that improves the feature quality while maintaining high data efficiency. The influence of the proposed modifications is evaluated in an extensive ablation study. Third, we present a hierarchical pipeline that exploits hyperdimensional computing to use the same local features as holistic HDC-descriptors for fast candidate selection and for candidate reranking. We combine all contributions in a runtime and data-efficient VPR pipeline that shows benefits over the state-of-the-art method Patch-NetVLAD on a large collection of standard place recognition datasets with 15x better performance in VPR accuracy, 54x faster feature comparison speed, and 27x less descriptor storage occupancy, making our method promising for real-world high-performance large-scale VPR in changing environments. Code will be made available with publication of this paper.