ESDA: A Composable Dynamic Sparse Dataflow Architecture for Efficient Event-based Vision Processing on FPGA

ESDA is a framework for building customized DNN accelerators for event-based vision tasks. It leverages the spatial sparsity of event-based input by a novel dynamic sparse dataflow architecture. This is achieved by formulating the computation of each dataflow module as a unified token-feature computation scheme. To enhance the spatial sparsity, ESDA also constrained Submanifold Sparse Convolution to build our DNN models.

The project mainly consists of three parts

• Software model training on event-based datasets with sparsity and quantization
• Hardware design optimization (use synthesis optimization to search for optimal mapping)
• Hardware synthesis, implementation, and evaluation