<p align="center">
  <h1 align="center">ZipSplat: Fewer Gaussians, Better Splats</h1>
  <p align="center">
    <a href="https://github.com/veichta">Alexander Veicht</a>
    &middot; Sunghwan Hong
    &middot; Dániel Baráth
    &middot; Marc Pollefeys
    <br>
    <sub>ETH Zürich &nbsp;·&nbsp; Microsoft</sub>
  </p>
  <p align="center">
    <!-- TODO: add arXiv link once live -->
    <a href="https://arxiv.org/abs/XXXX.XXXXX">Paper</a> &middot;
    <a href="https://veichta.com/zipsplat">Project Page</a>
  </p>
</p>

<p align="center">
  <img src="assets/teaser.png" alt="ZipSplat teaser" width="100%">
  <br>
  <em>ZipSplat decouples Gaussian placement from the pixel grid, reconstructing a scene from a few
  unposed images in under a second — higher quality from far fewer Gaussians.</em>
</p>

---

ZipSplat is a feed-forward 3D Gaussian Splatting model that reconstructs a scene from a few
**unposed** images in a single forward pass — no camera poses, no intrinsics, no per-scene
optimization. Unlike pixel-aligned methods that emit one Gaussian per pixel (tying the budget to
image resolution rather than scene complexity), ZipSplat compresses the views into a compact set of
**scene tokens** and decodes each into a small group of Gaussians with free 3D positions. Because
this compression happens at inference, a single trained model spans the entire quality–efficiency
curve through one `compression` knob — reaching state-of-the-art quality with ~6× fewer Gaussians
than pixel-aligned methods.

## Setup

```bash
git clone https://github.com/cvg/ZipSplat.git && cd ZipSplat
python -m pip install -e .
# optional: xformers' fused SwiGLU kernel the released checkpoint was trained with (small speedup)
python -m pip install -e ".[xformers]"
```

Requires Python ≥ 3.10 and a CUDA GPU (rendering uses [gsplat](https://github.com/nerfstudio-project/gsplat)).
The pretrained weights are downloaded automatically on first use.

## Demo

Reconstruct a scene and explore it live in your browser. Drag the **compression** slider to trade
Gaussians for quality in real time, and toggle **token coloring** to see how each token's group of
Gaussians is placed:

```bash
pip install "zipsplat[viewer]"
python -m zipsplat.viewer assets/examples/drone.mp4   # or assets/examples/office/, or your own dir/glob/video
```

<!-- TODO: add assets/viewer.gif — a screen recording dragging the compression slider -->
<p align="center">
  <img src="assets/viewer.gif" alt="ZipSplat interactive viewer" width="90%">
</p>

## Inference

```python
import math, torch
from zipsplat import ZipSplat, Camera, Pose, load_image, viz

model = ZipSplat(weights="zipsplat").cuda().eval()
images = [load_image(p) for p in paths] # raw images, any size (auto-resized)
gaussians = model(images)[0] # feed-forward 3D Gaussians

# render a novel view
camera = Camera.from_fov(math.radians(60), w=512, h=512)
pose = Pose.from_Rt(torch.eye(3), torch.zeros(3)) # identity pose
rgb, info = gaussians.render(camera, pose)

# export
gaussians.save_ply("scene.ply")                             # open in any 3DGS viewer
viz.turntable(gaussians, "turntable.mp4", sweep_deg=None)   # wiggle orbit video
```

`gaussians` is a `Gaussians` object; `gaussians.save_ply(...)` writes a standard 3DGS `.ply` you can
drop into [SuperSplat](https://playcanvas.com/supersplat/editor) or any Gaussian-splat viewer.

<details>
<summary><b>Input: images, a video, or a single image</b></summary>

```python
from zipsplat import load_image, load_video

# multiple images (any sizes — center-cropped to square and resized internally)
gaussians = model([load_image(p) for p in paths])[0]

# a single image
gaussians = model([load_image("photo.jpg")])[0]

# a video clip (evenly samples num_frames across the clip)
gaussians = model(load_video("assets/examples/drone.mp4", num_frames=24))[0]
```

The model handles 1–24+ views; more views give wider coverage, fewer give a tighter reconstruction.
</details>

<details>
<summary><b>Compression — fewer Gaussians</b></summary>

`compression` is the query-sampling ratio in `(0, 1]`. `1.0` (default) uses every token; lower values
run k-means to keep a subset, shrinking the Gaussian count with graceful quality falloff.

```python
gaussians = model(images, compression=1.0)   # full quality
gaussians = model(images, compression=0.25)  # ~4x fewer Gaussians
print(gaussians.num_gaussians)
```
</details>

<details>
<summary><b>Rendering</b></summary>

```python
# a single view (scalar camera/pose) or a batch of views ([V])
rgb, info = gaussians.render(camera, pose)               # mode="RGB" by default
rgbd, _ = gaussians.render(camera, pose, mode="RGB+ED")  # append expected depth
```

Cameras/poses are moved to the scene's device automatically. `info["alphas"]` holds the rendered
opacity. Build cameras with `Camera.from_fov`, `Camera.from_focal`, or `Camera.from_K`.
</details>

<details>
<summary><b>Turntable video</b></summary>

```python
from zipsplat import viz

viz.turntable(gaussians, "orbit.mp4")                  # full 360° turntable
viz.turntable(gaussians, "wiggle.mp4", sweep_deg=None) # gentle wiggle (best for front-facing scenes)
```
</details>

<details>
<summary><b>Camera priors (optional)</b></summary>

If you have calibrated cameras and poses, pass them with `use_priors=True` to inject them into the
backbone. Intrinsics are adjusted automatically for the internal resize.

```python
gaussians = model(images, cameras=cameras, poses=poses, use_priors=True)
```
</details>

## Models

| Name   | Backbone  | Train res | Notes                                      |
| ------ | --------- | --------- | ------------------------------------------ |
| `da3g` | DA3-Giant | 252 px    | Default release checkpoint (RE10K + DL3DV) |

Weights are hosted on the [Hugging Face Hub](https://huggingface.co/veichta/zipsplat) and fetched on
first use. You can also pass a local path or URL: `ZipSplat(weights="path/to/checkpoint.tar")`.

## Training

Training and evaluation live in the `splatfactory` package in this repository.
<!-- TODO: training quickstart / pointer once the splatfactory side is cleaned up -->

## BibTeX

```bibtex
@article{veicht2026zipsplat,
  title   = {ZipSplat: Fewer Gaussians, Better Splats},
  author  = {Veicht, Alexander and Hong, Sunghwan and Bar{\'a}th, D{\'a}niel and Pollefeys, Marc},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}
```

## License

The **code** in this repository (the `zipsplat` and `splatfactory` packages) is released under the
[Apache-2.0 License](LICENSE).

The **pretrained weights** are released separately under
[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — **non-commercial use only**. This
is required by their dependencies: the released checkpoint is initialized from
[DA3-Giant](https://huggingface.co/depth-anything/DA3-GIANT) (CC BY-NC 4.0) and trained on
[DL3DV-10K](https://github.com/DL3DV-10K/Dataset) (CC BY-NC 4.0). See the
[Hugging Face model card](https://huggingface.co/veichta/zipsplat) for the full weights license and
dataset attribution.
</content>
