<!-- PROJECT LOGO -->

<!-- <p align="center">

  <h1 align="center">VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM</h1>
</p> -->

<p align="center">

  <h1 align="center">VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM</h1>
  <p align="center">
    <a href="https://zzh2000.github.io"><strong>Zihan Zhu</strong></a>
    ·
    <a href="https://www.ifp.uni-stuttgart.de/en/institute/team/Zhang-00004/"><strong>Wei Zhang</strong></a>
    .
    <a href="https://www.ifp.uni-stuttgart.de/institut/team/Haala-00001/"><strong>Norbert Haala</strong></a>
    .
    <a href="https://people.inf.ethz.ch/pomarc/"><strong>Marc Pollefeys</strong></a>
    ·
    <a href="https://cvg.ethz.ch/team/Dr-Daniel-Bela-Barath"><strong>Daniel Barath</strong></a>
</p>
<p align="center"> <strong>arXiv 2025</strong></p>
  <h3 align="center"><a href="https://arxiv.org/abs/2512.02293">Paper</a> | <a href="https://www.youtube.com/watch?v=vj4oCx0AU_U">Video</a> | <a href="https://vigs-slam.github.io/">Project Page</a></h3>
  <div align="center"></div>
</p>
<p align="center">
    <img src="./media/teaser.jpg" alt="teaser_image" width="100%">
</p>

<p align="center">
Given a sequence of RGB frames and IMU readings, our VIGS-SLAM robustly tracks the camera trajectory while reconstructing a high-fidelity 3D Gaussian map.
</p>
<br>

<!-- TABLE OF CONTENTS -->
<details open="open" style='padding: 10px; border-radius:5px 30px 30px 5px; border-style: solid; border-width: 1px;'>
  <summary>Table of Contents</summary>
  <ol>
    <li>
      <a href="#getting-started">Getting Started</a>
    </li>
    <li>
      <a href="#data-preparation">Data Preparation</a>
    </li>
    <li>
      <a href="#run-demo">Run Demo</a>
    </li>
    <li>
      <a href="#run-evaluation">Run Evaluation</a>
    </li>
  </ol>
</details>


## Getting Started
1. Clone the repo with submodules
```Bash
git clone --recursive XXXXX
git checkout release
git submodule update --init --recursive
```

2. Create a new Conda environment and then activate it. Here we provide the environment for RTX 5090. Please note that we use the PyTorch version 2.8 compiled by CUDA 12.8 in the `environment_5090.yaml` file.
```Bash
conda env create -f environment_5090.yaml
conda activate imugs_slam
pip install thirdparty/simple-knn
pip install thirdparty/diff-gaussian-rasterization
python -m pip install --no-binary=:all: -U sophuspy
pip install --no-binary=:all: --no-build-isolation --no-cache-dir -v torch-scatter
```

3. Compile the CUDA kernel extensions (takes about 10 minutes). Please note that this process assume you have CUDA 12.8 installed. To look into the installed CUDA version, you can run `nvcc --version` in the terminal.
```Bash
python setup.py install
```

4. Download the pretrained weights of Omnidata models for generating depth and normal priors
```Bash
wget https://zenodo.org/records/10447888/files/omnidata_dpt_normal_v2.ckpt -P pretrained_models
wget https://zenodo.org/records/10447888/files/omnidata_dpt_depth_v2.ckpt -P pretrained_models
```

## Data Preparation
### Euroc Dataset
Please download the dataset from [Euroc](https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) website, for euroc, no need to do preprocessing.

### RPNG AR Table Dataset
Please first download the dataset from [RPNG AR Table Dataset](https://github.com/rpng/ar_table_dataset) website, then do the preprocessing by running (remember to change the folder directory in the python code)
```Bash
python scripts/preprocess_rpngar.py
```

### UTMM Dataset
Directly download the processed [UTMM](https://huggingface.co/datasets/neel1302/UT-MM/tree/main) dataset using the script below
```Bash
bash prep_utmm.sh
```

### FAST-LIVO2 Dataset
Directly download the processed FAST-LIVO2 dataset using the script below
```Bash
bash prep_livo2.sh
```

## Run Demo
After preparing the RPNG AR Table dataset, you can run IMU-GS-SLAM for a demo. The result will be saved in the `outputs/rpng/table_01` folder including the estimated camera poses, the Gaussian map, and the renderings. To visualize the constructing process of the Gaussian map, using the `--gsvis` flag. To visualize the intermediate results e.g. estimated depth and point cloud, using the `--droidvis` flag.

```bash
python demo.py \
--imagedir /home/zihzhu/data/Datasets/rpngar/table_01/rgb \
--calib calib/rpngar.txt \
--config config/rpng.yaml \
--imufile /home/zihzhu/data/Datasets/rpngar/table_01/imu.txt \
--undistort \
--IMU_poseinit_after 20 \
--output outputs/rpng/table_01 \
--gsmapping # enable gaussian mapping
[--gsvis] # Optional: Enable Gaussian map display
[--droidvis] # Optional: Enable point cloud display
[--rerunvis] # Optional: Point cloud and pose visualization using rerun 
```

To replay the rerun recording:
```bash
rerun outputs/rpng/table_01/rerun_stream.rrd
```

## Batched Run & Tracking Evaluation
### Euroc
Run the following script to run all sequences of the Euroc dataset in batch and also do tracking evaluation. (remember to change the dataset folder path directly in the python file)
```bash
python eval_euroc_mono.py
```

### RPNG AR Table Dataset
Run the following script to run all sequences of the RPNG AR Table Dataset in batch and also do tracking evaluation. (remember to change the dataset folder path directly in the python file)
```bash
python eval_rpng_mono.py
```

### UTMM Dataset
Run the following script to run all sequences of the UTMM Dataset in batch and also do tracking evaluation. (remember to change the dataset folder path directly in the python file)
```bash
python eval_utmm_mono.py
```

### FAST-LIVO2 Dataset
Run the following script to run all sequences of the FAST-LIVO2 Dataset in batch and also do tracking evaluation. (remember to change the dataset folder path directly in the python file)
```bash
python eval_livo2_mono.py
```

## Rendering Evaluation

```bash
python scripts/eval_rendering.py
```
