File size: 3,226 Bytes
7d207b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | # Polymorphic
**Compute-efficient video-to-3D research by Modular Computing.**
This repository contains the Polymorphic video-to-3D experiments and releases. The current public experiment combines **AnySplat** for scene/camera initialization with **Fourier Splatting** for per-scene optimization and novel-view synthesis.
> **Important:** This is an experimental reconstruction pipeline, not a feed-forward model that instantly converts any video into a finished asset. Each new scene requires optimization. The released representation is a renderable splat scene; mesh/GLB conversion is not included as a validated workflow.
## Featured experiment: Fourier Splatting
[**Open the full experiment model card and run instructions →**](experiments/fourier-splatting/README.md)
The experiment directory contains:
- Runnable preparation and evaluation scripts
- Apartment and kitchen scene checkpoints
- PLY exports at Fourier frequency levels K=1–6
- Per-scene metrics, experiment report, and third-party license notices
### Held-out novel-view results
Metrics are from two indoor scenes, using the same VGG LPIPS implementation for both methods. Higher PSNR/SSIM is better; lower LPIPS is better.
| Scene | Method | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | Primitives |
|---|---|---:|---:|---:|---:|
| Apartment | AnySplat baseline | 19.1235 | 0.6612 | 0.3154 | 1,858,896 |
| Apartment | Fourier Splatting K=6 | 20.3469 | 0.7041 | 0.3173 | 99,483 |
| Kitchen | AnySplat baseline | 21.6155 | 0.6660 | 0.2678 | 1,983,229 |
| Kitchen | Fourier Splatting K=6 | 25.3078 | 0.8212 | 0.2586 | 120,000 |
Across these two scenes, Fourier Splatting used approximately **16.5–18.7× fewer primitives** than the AnySplat baselines. Apartment LPIPS was slightly worse, so the results do not indicate uniform improvement. This is a small exploratory evaluation, not a generalization or state-of-the-art claim.
## Quick links
- [Fourier Splatting model card and instructions](experiments/fourier-splatting/README.md)
- [Experiment report](experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md)
- [Metrics JSON](experiments/fourier-splatting/fourier_results.json)
- [Generic video-frame preparation script](experiments/fourier-splatting/scripts/make_fourier_scene_generic.py)
- [Apartment PLY, K=6](experiments/fourier-splatting/models/apartment/lod/fourier_k6.ply)
- [Kitchen PLY, K=6](experiments/fourier-splatting/models/kitchen/lod/fourier_k6.ply)
## Reproducing the experiment
Start with the [experiment README](experiments/fourier-splatting/README.md), which documents dependencies, frame extraction, scene preparation, optimization, rendering, results, and known limitations. The research run was conducted on an NVIDIA H100 80 GB with PyTorch/CUDA 12.9 at 448×448 resolution. The complete workflow has **not yet been validated from a clean environment**, and dependency compatibility may require adjustments.
## Licensing
This is a research release that incorporates upstream AnySplat and Fourier Splatting code. Their respective license terms apply; see the license notices in the experiment directory. In particular, review Fourier Splatting's non-commercial research/evaluation terms before any commercial use.
|