Photorealistic 3D Reconstruction with NeRFs
Neural Radiance Field pipelines that reconstruct photorealistic 3D scenes from sparse, low-quality 2D imagery.
Machine Learning Intern
- Render quality
- 30+ PSNR
- Training time
- ~2× faster
At MIT Lincoln Laboratory I worked on reconstructing photorealistic 3D models from imagery that was sparse and low quality — the kind of input real-world sensing produces, rather than the clean captures NeRF demos usually assume.
Problem
Neural Radiance Fields produce stunning 3D reconstructions, but they're sensitive to input quality and camera coverage, and training is slow. The goal was reconstructions that hold up under realistic, degraded inputs — without waiting hours per scene.
The project became a pipeline problem: first recover a reliable camera coordinate system, then train a radiance field that could synthesize missing viewpoints, then tune the model until the result was high quality enough to iterate on quickly.
NeRF pipeline
Reconstructing a scene as a learned field of light
A single visual walkthrough of the project: sparse captures become camera poses, rays sample a radiance field, and the renderer converges into a cleaner 3D reconstruction. Click a stage below to hold that moment in the animation.
Sparse inputs
Start with imperfect 2D captures
Limited, low-quality imagery is the constraint. The goal is to recover a coherent 3D scene without assuming clean, dense coverage.
COLMAP / SfM
Recover where each image was taken
Structure from Motion turns unordered photos into estimated camera poses and sparse geometry: the coordinate frame the NeRF needs.
NeRF training
Learn color and density along rays
Rays sample a learned field that predicts volume density and view-dependent color, then renders those samples back into views.
NeRFStudio
Tune the model into a practical renderer
Instant-NGP, Nerfacto, and Nerfacto-Big become engineering tradeoffs: quality, speed, sampling, and model capacity.
Conceptual animation of my MIT Lincoln Laboratory pipeline: COLMAP pose recovery, NeRFStudio model experiments, and renderer optimization for sparse imagery.
Approach
I used COLMAP as the structure-from-motion backbone to estimate each image's camera pose and sparse scene geometry from unordered 2D images. That pose recovery step was critical: without a stable coordinate system, the NeRF has no consistent notion of where light rays originate or what they should explain.
Those posed images then trained the radiance field itself: a neural scene representation that maps 5D camera queries - 3D location plus 2D viewing direction - to volume density and view-dependent color. Rendering becomes a supervised loop: sample rays through the field, synthesize views, compare them against the original captures, and improve the field.
Working in NeRFStudio, I implemented and compared Instant-NGP, Nerfacto, and Nerfacto-Big, tuning model capacity, sampling, and training schedules for sparse, low-quality inputs. Access to MIT Lincoln Lab's compute environment made the work feel like both research and systems engineering: run controlled model variants, monitor quality in real time, and keep tightening the loop between fidelity and training speed.
Results
The pipeline reached 30+ Peak Signal-to-Noise Ratio on the reconstructions while cutting training time roughly in half, making the approach practical to iterate on.
The work gave me a deeper foundation in computer vision, photogrammetry, and neural rendering while balancing research exploration with production-style deliverables. The code remains proprietary to MIT Lincoln Laboratory.