Project Page | Paper | Data
Official PyTorch implementation of NeST: a physics-guided machine-learning framework to leverage polarization for revealing 3D stress tensor fields hidden within 3D transparent objects.
This implementation has been tested on Ubuntu 24 with NVIDIA GPUs with RAM > 20 GB.
Create a new Anaconda environment using the supplied environment.yml
conda env create -f environment.yml
Activate the created environment
conda activate nest
This implementation relies on tiny-cuda-nn for fast neural rendering.
For efficient GPU use set TCNN_CUDA_ARCHITECTURES environment variable in L22 of setup.sh based on your GPU. Find the correct number here.
Run the setup script in the created conda environment to run additional packages including tiny-cuda-nn and nerfacc
bash setup.sh
Download datasets from this link into a data folder inside the main repo. Ensure the folder structure is the same, e.g., data/sim/femur/
sim subfolder contains the measurements rendered from the 3D-TSV using the simulation framework developed in the paper. real subfolder contains polarization images captured experimentally and the corresponding calibration files.
After ensuring correct data has been downloaded,follow this command to reconstruct stress fields from the input polarized measurements. Config files in the config folder enlist the hyperparameters for different scenes.
python nerfacc_scripts/03_reconstruct_stress_from_renders.py --config <config_file name>
Provide full path for the config file, e.g. --config configs/sim/train/femur
(coming soon)