To do this, the researchers leveraged the power of AI, more specifically through the so-called neural radiance fields. NeRFs use neural networks to reconstruct the scene from the images by predicting the color of light radiating in any direction. NVIDIA claims Instant NeRF to be the fastest technology of its kind yet, allowing for a multiplicative speed-up of over 1,000x in some instances so that rendering at 1080p occurs in mere milliseconds. As discussed by Principal Research Scientist Thomas Muller in a GDC 2022 talk titled Instant Neural Graphics Primitives, this compounded effect is the result of three main improvements: a task-specific GPU implementation of the rendering/training algorithm, which uses the GPU’s fine-grain control flow capabilities to be much faster than dense tensors; a fully fused implementation of a small neural network, which is faster than general purpose matrix multiplication routines; lastly, NVIDIA developed a technique called multiresolution hash grid encoding which is task agnostic and has better speed/quality tradeoff than preexisting work. Unsurprisingly, the model for Instant NeRF was developed with the CUDA Toolkit and the Tiny CUDA Neural Networks library. You can dive deeper into the research and even access its code on this page; according to NVIDIA, the neural network is lightweight enough to run easily on a single GPU, especially if it is equipped with Tensor Cores. David Luebke, vice president for graphics research at NVIDIA, said in a statement: The applications for the Instant NeRF technology can be many, from quickly scanning real environments or persons so that game creators can then use the digital scans in their projects to training self-driving cars or robots to understand the shape and size of real objects.
