Title：Revisit Fast Algorithms for Convolutional Neural Networks
Speaker ：Dr. Zhen Jia，AWS AI, Amazon
Time：December 16, 2019 ,13:30 PM
Venue：345 Lecture Hall, Physics Building
Abstract：As convolutional neural network (ConvNet) regained its popularity, several fast approaches were proposed to speed up the computation of its most time-consuming convolutional layers. Recently, it has been demonstrated that Winograd based convolution can greatly reduce the computation complexity of a convolution. A common belief is that Winograd based algorithms should be faster than FFT-based ones, which were the most frequently used methods to reduce the computation before Winograd proposed.
In this talk, I will first present our work on optimizing the Winograd based convolution. Our implementation achieves high hardware utilization through a series of optimizations and runs on average more than 3x faster than other state-of-the-art CPU implementations on the benchmarked layers. Then, I will revisit FFT. We propose a performance model based on the idea of Roofline model to analyze and compare the performance of FFT and Winograd base convolutions on modern processors. To validate the predictions of our model, we also implement FFT based convolutions with the same optimization methods used in Winograd. The results of our experiments verify our model's prediction and demonstrate that Winograd is not always faster than FFT convolution.
Brief Bio: Zhen Jia is an applied scientist in the AWS AI team of Amazon, CA, USA. Before joining Amazon. He was a Postdoctoral Research Associate at Princeton University in Computer Science Department. His research interests include convolutional neural network accelerating, workload characterization, benchmarking, and performance optimization at system level. He received the Ph.D degree from Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS). He has published over 30 papers in top computer system conferences and journals like HPCA, PPoPP, ICS, PACT, TPDS, etc. His publications have been cited 1,100+ times to date.
