Domain adaptive visual inertial global localization in GNSS-denied environments

Abstract

This thesis addresses the problem of long-range global localization for small unmanned aerial vehicles (UAVs) operating in Global Navigation Satellite System (GNSS)–denied environments. The central premise is that visual information from an onboard nadir-looking camera can be aligned with geo-referenced satellite imagery to obtain absolute position fixes, provided that (i) short-term motion is estimated reliably, (ii) the global estimator can reason over multi-modal hypotheses without instability, and (iii) the appearance gap between aerial/satellite domains is reduced so that feature-level correspondence becomes feasible. To this end, we propose an integrated pipeline that combines a lightweight visual–inertial odometry (VIO) front-end, a marginalized particle filter (MPF) for state estimation, and a learned image-to-image domain adaptation module based on cycle-consistent generative adversarial networks (CycleGAN). A custom likelihood function is introduced to convert feature-matching evidence into robust particle weights, thereby improving convergence in ambiguous conditions. For VIO, we employ a tightly coupled multi-state constraint formulation (MSCKF-like), in which visual feature residuals are projected into the left null space of per-landmark Jacobians, enforcing constraints across a sliding window of camera poses without explicitly augmenting landmark states. This maintains bounded state dimension and estimator consistency under First-Estimates Jacobians while providing accurate short-term motion and pose estimates at high rate. The VIO output supplies drift-limited motion priors to the global estimator. Global localization is performed with a marginalized particle filter. The state is partitioned into a low-dimensional nonlinear subspace, represented by particles, and a linear–Gaussian subspace, represented by per-particle Kalman filters (Rao–Blackwellization). This decomposition substantially reduces the number of particles required to cover the hypothesis space while preserving linear covariance propagation for velocity, attitude-error, and bias terms. The discrete-time form of the error-state dynamics is adopted for propagation, and observations are produced by rendering a particle-conditioned view (a crop and rotation of the satellite map) using the particle's horizontal position, yaw, and altitude-dependent scale. A key challenge is the domain gap between onboard imagery and satellite references, arising from seasonal appearance, illumination, and acquisition geometry. To mitigate this gap, we train three unpaired image translation models: the original CycleGAN, CUT (contrastive unpaired translation), and Turbo-CycleGAN. CycleGAN learns bidirectional mappings regularized by cycle consistency; CUT replaces cycle consistency with a patchwise contrastive objective to better preserve content; Turbo-CycleGAN enhances texture fidelity and artifact suppression with multi-scale refinements at the cost of higher training and inference complexity. We evaluate these models on UAV/satellite domain adaptation using a task-aligned metric: the mean number of geometrically verified feature correspondences across 100 query images, normalized by the matches obtained from raw (unadapted) UAV images. The proposed framework has been validated through multiple flight tests conducted under varying environmental conditions, altitudes, and scene characteristics. These tests demonstrate that domain adaptation significantly mitigates the appearance discrepancies between UAV and satellite imagery, resulting in more reliable feature correspondences and improved localization accuracy. In challenging cases with large seasonal differences or repetitive textures, adaptation helps maintain estimator stability by preventing divergence and reducing drift. At higher altitudes, where the effective resolution diminishes and feature observability becomes limited, the approach still provides measurable improvements in position accuracy. Across scenarios, the combination of VIO, MPF, and domain adaptation consistently outperforms a purely visual baseline without adaptation, with the largest gains observed when the raw domain gap or the texture-induced ambiguity is severe. The MPF's partitioned formulation enables robust multi-hypothesis reasoning without the instability that can arise when error states are reset at each update, and the proposed likelihood reduces false convergence. At the same time, we identify practical trade-offs. Turbo-CycleGAN offers the highest domain-alignment quality but incurs substantial model size and runtime, which may be challenging for the most resource-constrained airframes. The thesis concludes that accurate, computationally tractable global localization in GNSS-denied settings can be achieved by: employing a consistent, tightly coupled VIO to provide drift-limited motion priors, using a marginalized particle filter to reason over global hypotheses while keeping computation bounded, adapting image appearance across domains with unpaired translation to restore descriptor consistency, and shaping particle likelihoods with a discriminative mapping from verified match statistics. The overall system advances the state of practice toward reliable, real-time-capable UAV navigation in diverse environmental conditions. Future avenues include night-to-day domain adaptation for low-light operations, image rectification strategies for non-gimballed sensors, lightweight generative models suitable for edge deployment, enhanced likelihoods that exploit spatial/geometric structure beyond match counts, and multi-modal fusion (e.g., magnetic field maps) for scenarios that are intrinsically challenging for vision alone.