NuVPU

My Contributions:

• Development of frequency-driven mixed compression
• Design of Memory Management Unit

This work presents NuVPU, a unified neural video processor that supports both streaming and post-processing. Target NVC increases intermediate data by up to 100×, causing large EMA due to pixel shuffle layers. Therefore, NuVPU proposes 2-step memory optimization scheme to reduce EMA. First, FAC introduces frequency-driven mixed quantization which adjusts the quantization level based on the frequency of each tile, improving the cache hit ratio with 2.9× higher compression efficiency than JPEG at the same PSNR degradation (<0.05dB). Second, ATS dynamically optimizes tile processing order to maintain a high warping ratio within cache capacity. When memory thresholds are exceeded, ATS shifts from spatial to temporal processing to prevent overflow. Together, they achieve an 81.3% reduction in EMA.