A low-power and real-time 3D neural rendering processor, MetaVRain, is proposed with 3 key features: 1) a visual perception core for 1120× faster rendering, 2) the 1D and 2D hybrid neural engines for 3.7× higher throughput with 2.4× higher energy efficiency during DNN inference, and 3) a modulo based positional encoding unit to minimize HW realization of the sinusoidal function. It finally achieves a maximum of 118 FPS while consuming at least 99.95% lower power compared with modern GPUs and TPU.

  - A new NeRF acceleration architecture: Bundle-frame-familiarity (BuFF) architecture 

    => Imitating human's visual perception stages: 1) Spatial Attention (SA), 2) Temporal Familiarity (TF), and 3) Top-down Attention (TDA)

  - Visual perception core (VPC)

  - 1D-2D hybrid neural engine (HNE)

  - Centrifugal sampling (CS) based output sparsity prediction

  - Dynamic neural network allocation (DNNA)

  - Periodic polynomial approximation for sinusoidal function

  - Modulo-based positional encoding unit (Mod-PEU)

  - 3D stylization demonstration system

  - ISSCC 2023