Neural Radiance Field (NeRF) is an emerging computer graphics task that is used for 3D modeling and rendering in the metaverse, providing a user-friendly and immersive experience. However, it has limitations to be accelerated on mobile AR/VR devices due to its memory-intensive hash encoding and extensive computational load. This paper presents NeuGPU to achieve NeRF-based instant 3D modeling and real-time rendering with 3 key features. 1) Segmented Hashing with Spatial Pruning (SHSP) partitions hash table into multiple segments and reduces 66% of External Memory Access (EMA). Spatial Management Unit (SMU) supports SHSP operation and 3 types of sub-block management. 2) Attention-based Hybrid Interpolation Unit (AHIU) exploits heterogeneous reuse characteristics of hash-based NeRF, and achieves 56.4% power reduction. 3) Similarity-Sparsity Skipping core supports energy-efficient Multi-layer Perceptron (MLP) operation by applying coarse-skip for both similar and sparse data. As a result, NeuGPU achieves 8.7× faster modeling, and 231.4× smaller energy per iteration compared to edge GPU even though it has 31× smaller external memory bandwidth.

  - A NeRF-based instant 3D modeling and real-time rendering processor

  - Segmented Hashing with Spatial Pruning (SHSP)

  - Attention-based Hybrid Interpolation Unit (AHIU)
  - Similarity-Sparsity Skipping core

  - ISSCC 2024