Student Posters Repository

Operation batching is an important way to accelerating the execution of neural networks having dynamic computation graphs. There have been some strategies proposed to batch the operations automatically and execute the batched operations. But there are still opportunities for further optimization in computing performance. In my research, I found there are least 3 further optimizations: 1) The calculation of the parameters’ gradients could be executed in a more efficient way; 2) Overhead coming from data copy could be eliminated if the memory stores the arguments of a batched nodes could be allocated in continuous space in advance; 3) Nodes that don’t depend on each other in the computation graph generated by the automatic operation batching strategies could be executed simultaneously in a task-parallel way. Optimization 1) has already been tried and evaluated on 5 different NLP task benchmarks. Improvements in computing performance have been observed in some cases while decrement also happened in other cases. Profiling has been done and the result shows that Optimization 2) and 3) are promising.