| # Polymorphic |
|
|
| **Compute-efficient video-to-3D research by Modular Computing.** |
|
|
| This repository contains the Polymorphic video-to-3D experiments and releases. The current public experiment combines **AnySplat** for scene/camera initialization with **Fourier Splatting** for per-scene optimization and novel-view synthesis. |
|
|
| > **Important:** This is an experimental reconstruction pipeline, not a feed-forward model that instantly converts any video into a finished asset. Each new scene requires optimization. The released representation is a renderable splat scene; mesh/GLB conversion is not included as a validated workflow. |
|
|
| ## Featured experiment: Fourier Splatting |
|
|
| [**Open the full experiment model card and run instructions →**](experiments/fourier-splatting/README.md) |
|
|
| The experiment directory contains: |
| - Runnable preparation and evaluation scripts |
| - Apartment and kitchen scene checkpoints |
| - PLY exports at Fourier frequency levels K=1–6 |
| - Per-scene metrics, experiment report, and third-party license notices |
|
|
| ### Held-out novel-view results |
|
|
| Metrics are from two indoor scenes, using the same VGG LPIPS implementation for both methods. Higher PSNR/SSIM is better; lower LPIPS is better. |
|
|
| | Scene | Method | PSNR ↑ | SSIM ↑ | VGG LPIPS ↓ | Primitives | |
| |---|---|---:|---:|---:|---:| |
| | Apartment | AnySplat baseline | 19.1235 | 0.6612 | 0.3154 | 1,858,896 | |
| | Apartment | Fourier Splatting K=6 | 20.3469 | 0.7041 | 0.3173 | 99,483 | |
| | Kitchen | AnySplat baseline | 21.6155 | 0.6660 | 0.2678 | 1,983,229 | |
| | Kitchen | Fourier Splatting K=6 | 25.3078 | 0.8212 | 0.2586 | 120,000 | |
|
|
| Across these two scenes, Fourier Splatting used approximately **16.5–18.7× fewer primitives** than the AnySplat baselines. Apartment LPIPS was slightly worse, so the results do not indicate uniform improvement. This is a small exploratory evaluation, not a generalization or state-of-the-art claim. |
|
|
| ## Quick links |
|
|
| - [Fourier Splatting model card and instructions](experiments/fourier-splatting/README.md) |
| - [Experiment report](experiments/fourier-splatting/FOURIER_SPLATTING_REPORT.md) |
| - [Metrics JSON](experiments/fourier-splatting/fourier_results.json) |
| - [Generic video-frame preparation script](experiments/fourier-splatting/scripts/make_fourier_scene_generic.py) |
| - [Apartment PLY, K=6](experiments/fourier-splatting/models/apartment/lod/fourier_k6.ply) |
| - [Kitchen PLY, K=6](experiments/fourier-splatting/models/kitchen/lod/fourier_k6.ply) |
|
|
| ## Reproducing the experiment |
|
|
| Start with the [experiment README](experiments/fourier-splatting/README.md), which documents dependencies, frame extraction, scene preparation, optimization, rendering, results, and known limitations. The research run was conducted on an NVIDIA H100 80 GB with PyTorch/CUDA 12.9 at 448×448 resolution. The complete workflow has **not yet been validated from a clean environment**, and dependency compatibility may require adjustments. |
|
|
| ## Licensing |
|
|
| This is a research release that incorporates upstream AnySplat and Fourier Splatting code. Their respective license terms apply; see the license notices in the experiment directory. In particular, review Fourier Splatting's non-commercial research/evaluation terms before any commercial use. |
|
|
