Background

Processing-in-memory (PIM), an increasingly studied neuromorphic hardware, promises throughput improvements for deep learning inference. Leveraging the massively parallel and efficient analog computing inside memories, PIM circumvents the bottlenecks of data movements in conventional digital hardware. However, the nature of analog computing and the necessity of analog-to-digital converters in PIM lead to a drastically different computing fabric with non-idealities and a huge accuracy loss when directly running networks quantized by existing methods.

Technology Overview

Here, Northeastern University researchers propose a general framework for training quantized networks for general PIM systems. Specifically, the researchers make the following propositions:

(1) PIM-aware training based on a basic assumption of generalized straight-through estimator (GSTE), which is a generalization of the famous straight-through estimator (STE), that has been adopted in conventional quantization.

2) Scaling techniques for both forward and backward propagation during training to tackle convergence problems.

3) Leveraging Batch Normalization (BN) calibration to close gaps between idealized training and real-case inference on real PIM systems with fabrication and run-time variations.

4) Utilizing an adjusted precision training algorithm to study the potential relations between training precision and the effective number of bits (ENOB) of the actual physical PIM system for inference.

This proposed framework was tested on three major PIM schemes (native, bit-serial, differential) that cover the majority of PIM hardware designs. The framework was then extensively evaluated on a silicon prototype of SRAM PIM with realistic non-idealities

Benefits

Applicable to a wide range of PIM systems
Easy to use
Achieves better performance for the final system
Saves the data movement energy and latency involved in conventional architecture

Applications

Hardware-based edge AI
Deep learning acceleration
Processing-in-memory system and non-Von Neuman architectures

Opportunity

Research Collaboration
Licensing
Commercial partner

A General Neural Network Training Method for Processing-In-Memory Systems

IP Status

Seeking