Project Page of AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM

Burkhardt, Yannick; Barbas Laina, Sebastián; Boche, Simon; Freißmuth, Leonard; Leutenegger, Stefan

AERO-VIS: Asynchronous Event-based Real-time Onboard Visual-Inertial SLAM

Yannick Burkhardt, Sebastián Barbas Laina, Simon Boche, Leonard Freißmuth, Stefan Leutenegger

Mobile Robotics Lab (ETH Zürich & Technical University of Munich), Munich Center for Machine Learning

IEEE Robotics and Automation Letters (RA-L), 2026

Paper Supplementary Code coming soon...

Video Presentation & Real-World Experiments

Abstract

The robustness of event cameras to high dynamic range and motion blur holds the potential to improve visual odometry systems in challenging environments. Although their high temporal resolution does not require synchronous processing, most event-based odometry methods still run at fixed rates, which simplifies system design but restricts latency and throughput. In this work, we present AERO-VIS, a stereo event-inertial SLAM system with an integrated, data-driven, robust, and performance-optimized keypoint detector. By decoupling event preprocessing from state estimation, our architecture operates asynchronously at the maximum rate supported by available hardware, dynamically adapting to downstream runtime demands and ensuring low-latency, real-time performance. When deploying AERO-VIS on a UAV, we achieve unprecedented accuracy in onboard event-based SLAM. These unique characteristics enable us to present the first purely event-based inertial SLAM system that demonstrates closed-loop UAV control and large-scale state estimation while relying solely on onboard compute.

AERO-VIS system diagram. Stereo event streams
and IMU data enter event preprocessing, which performs event retrieval, internal
event buffer updates and MCTS updates on the GPU into a shared MCTS buffer. The
OKVIS2-SuperEvent+/SuperLitE frontend freezes that buffer for inference, then runs
event keypoint detection, keypoint matching and loop closure detection. The backend
performs factor graph optimization and posegraph construction, and a detected loop
closure triggers a detached full graph optimization. — Stereo event and IMU data are preprocessed through time synchronization and high-frequency MCTS calculation in a shared buffer. The frontend processes this buffer asynchronously, freezing it for SuperLitE inference directly after completing the previous iteration's keypoint matching and loop closure detection. The backend optimizes the factor graph of recent sensor observations and constructs the posegraph on every new keyframe. If a loop closure is detected, a detached thread is initiated to re-optimize the global pose graph in the background.

Results

Keypoint Detection & Description

SuperEvent+ raises AUC@10° by 18.7 % over the SuperEvent baseline through our refined MCTS formulation, while SuperLitE keeps a 15.0 % gain at 10.5× lower inference latency.

Relative pose estimation area-under-curve on the Event Camera Dataset (ECD) and the Event-aided Direct Sparse Odometry dataset (EDS), both on the same scale, and inference time of the captured CUDA graph of the TensorRT-compiled models at 16-bit floating-point precision and 240 × 400 pixels input resolution on an NVIDIA Jetson Orin NX.

Two samples of the boxes_6dof
sequence of ECD under slow and fast scene motion. The fixed-time-window channels of
SuperEvent look very different between the two samples and yield few correct
matches under fast motion, while the fixed-event-count channels of SuperEvent+ and
SuperLitE look similar across both samples and yield many correct matches covering
a larger area. — Two samples of the *boxes_6dof* sequence of ECD: one channel-pair with positive (red) and negative (blue) polarity for MCTS_Δt (fixed time windows) and MCTS_{N_e} (fixed event counts), plus the matches predicted by each model. Green matches have a reprojection error below 5 pixels against ground-truth poses; yellow matches are outliers.

Benchmark Evaluation

AERO-VIS is the only evaluated system capable of reliable onboard state estimation, and on most sequences its real-time Jetson results beat both baselines' unconstrained desktop runs.

Arithmetic mean of the per-sequence RMS absolute trajectory error, each sequence itself the median of 5 runs. ESVO2 is omitted where it diverged or crashed on at least one sequence of a dataset. Note the differing axis scale per dataset: rpg-stereo and TUM-VIE are small-scale, VECtor is large-scale. Per-sequence numbers are in the paper.

UAV Onboard Estimation

Onboard a custom UAV with a Jetson Orin NX and two Prophesee EVK4 cameras: frame-based OKVIS2 wins under normal light, but fails in HDR — crashing the drone — and AERO-VIS cuts the error by 90 % under aggressive motion.

RMS absolute trajectory error against motion-capture ground truth. The baseline is the original frame-based OKVIS2 on an Intel RealSense 455 at 480 × 640 pixels. In the HDR experiment OKVIS2 failed outright — the drone crashed into the wall after five rotations.

Estimated and ground-truth
trajectories of a figure-eight flight, coloured by error, with an inset photo of
the UAV in flight. — (a) Closed-loop control under normal conditions.

Estimated and
ground-truth trajectories of in-place rotations in HDR, with an inset photo of
the UAV in a darkened room. — (a) Closed-loop control under normal conditions.

Large-scale Real-world Loop

Estimated trajectory of a
2 km urban walk overlaid on a satellite image. The loop-closed estimate follows
the street layout closely and returns to its start; the estimate without loop
closure drifts visibly away from it.

A 2 km urban walk in 20 min, estimated entirely onboard; loop closure corrects a drift of 1.8 % of the traversed distance. The route runs through varying lighting (sun, shade, artificial light) and dynamic entities — cars, pedestrians, bicycles. Without ground truth, the accuracy shows in how closely the estimate follows the street layout. The initial pose was manually aligned with the map.

Supplementary Results

HDR Robustness

High dynamic range is one of the defining advantages of event cameras, and our Multi-Channel Time Surface reformulation preserves it. Among the EDS evaluation sequences, one is recorded explicitly for HDR testing — ziggy_and_fuzz_hdr — and on it both SuperEvent+ and SuperLitE improve over the SuperEvent baseline by the same margin as on the remaining sequences. The qualitative example on the right illustrates the increased robustness to HDR compared to frame cameras.

Area-under-curve (AUC) of relative pose estimation on the *ziggy_and_fuzz_hdr* sequence of the EDS dataset. **Bold** marks the best and underline the second best result per column. Both of our variants gain roughly 5 AUC points at every error threshold over the SuperEvent baseline, confirming that the event-inherent robustness to HDR is preserved by the MCTS reformulation.
Method	EDS: AUC [%]
Method	@5°	@10°	@20°
SuperEvent	27.4	39.9	50.8
SuperEvent+ (ours)	31.3	45.0	57.6
SuperLitE (ours)	31.2	45.3	58.3

Three stacked panels of the varying_robust sequence of ViViD++. Top: SuperPoint and SuperGlue on standard frames — the frames are almost entirely black and only a single match is found. Middle and bottom: SuperEvent+ and SuperLitE on the event stream, both producing keypoints distributed across the whole image plane and hundreds of consistent matches. — Two samples of the *varying_robust* sequence of the ViViD++ dataset. The light has just been turned off, rendering the frames severely underexposed and preventing frame-based pose estimation due to insufficient matching features. Our event-based SuperEvent+ and SuperLitE variants still predict well-distributed keypoints across the image plane. Due to the absence of ground-truth poses for this sequence, a quantitative classification of inliers and outliers is not possible; however, the matching patterns remain qualitatively consistent.

Design Space Search

SuperLitE is the outcome of a search over the MCTS formulation, the backbone architecture, the number of MCTS channels and the descriptor dimensionality. We additionally investigated how to reduce operations at large spatial resolutions (the early layers), which are the primary computational bottleneck in the VGG backbones. SuperEvent+ achieves the best overall accuracy, while SuperLitE offers the best accuracy/latency trade-off; reducing the channel count, descriptor dimension or network depth any further degrades prediction accuracy significantly. All 39 evaluated configurations are published here, since the full table did not fit within the page limit of the manuscript.

Every evaluated configuration, plotted as inference latency against relative pose estimation accuracy. The fixed-event-count formulation dominates the fixed-time-window formulation across the whole latency range. SuperLitE sits on the Pareto frontier at the knee: it is 10.5× faster than SuperEvent while still improving AUC@10° by 7.4 %. Hover a point for the full configuration.

Click a column header to sort; click it again to reverse, and a third time to restore the original ordering. Relative pose estimation AUC is reported at error thresholds of 5°/10°/20°. Inference times of the captured CUDA graph of the TensorRT-compiled models with 16-bit floating-point precision and 240 × 400 pixels input resolution are measured on an NVIDIA Jetson Orin NX. ΔAUC@10° and ΔLatency are computed relative to SuperEvent (first row, orange). Rows in blue and green are SuperEvent+ and SuperLitE. Bold marks the best and underline the second best result per column.

Backbone architectures

C denotes the number of input event/MCTS channels. VGG(a→b) denotes a VGG-block consisting of a 3×3 convolution from a to b channels, batch normalization and a ReLU activation; P denotes 2×2 maximum-pooling.

BibTeX

@article{burkhardt2026aerovis,
      title={{AERO-VIS}: Asynchronous Event-based Real-time Onboard Visual-Inertial {SLAM}},
      author={Burkhardt, Yannick and Barbas Laina, Sebasti\'an and Boche, Simon and Frei{\ss}muth, Leonard and Leutenegger, Stefan},
      journal={IEEE Robotics and Automation Letters},
      year={2026}
  }