1. 1. Chainer FPGA FPGA 7 2015/08/02 @ / Preferred Networks, Inc.
2. 2. l (Seiya Tokui) @beam2d (Twitter, GitHub) l PFI (2012-2014) -> PFN (2014-) l Deep Learning2012 l 4 Chainer 2
3. 3. l Deep Learning Deep Learning Chainer l FPGA FPGA FPGA 3
4. 4. l Feed-forward Net DAG l Chainer 4
5. 5. Feed-forward Net l l l l 5
6. 6. Feed-forward Net l l elementwise l Feed-forward Net 6
7. 7. Feed-forward Net l l 7
8. 8. Neural Net l l DAG l Recurrent Net, Neural Turing Machine, etc. l feed-forward 8
9. 9. l l l backpropagation l l l 9
10. 10. l l (f + g) = f + g 10
11. 11. l l l l 3 4 3 5 l 11
12. 12. l (fully-connected layer) l l 12 x y bW y = Wx + b x = W y W = y x b = y
13. 13. l l 3 (cuda-convnet) (Cae, Chainer, cuDNN) u cuDNN GEMM (implicit GEMM) FFT FFT (fbcunn) 13 http://deeplearning.net/tutorial/lenet.html
14. 14. Chainer l Deep Learning l Preferred Networks / Preferred Infrastructure l URL: http://chainer.org/ https://github.com/pfnet/chainer http://docs.chainer.org/ l v1.1.1 2 1 14
15. 15. Chainer v1.1.1 l Python l Variable Function Optimizer l Variable NumPy PyCUDA 2 PyCUDA l Variable Function 15
16. 16. Chainer l Function Variables -> Variables l DAG x = chainer.Variable(...) y = chainer.Variable(...) z = x**2 + 2*x*y + y 16 x y _ ** 2 2 * _ _ * _ _ + _ z _ + _ Actually, Split nodes are automatically inserted (they accumulate the gradients on backprop)
17. 17. Backward Unchaining l Chainer l Fine-tuning Truncated Backprop-Through-Time 17 x f y g z y g z y.unchain_backward()
18. 18. Function l 4 forward_cpu / forward_gpu backward_cpu / backward_gpu l CPU/GPU forward / backward l check_type_forward 18
19. 19. l v1.1.1 NumPy PyCUDA/scikit-cuda l CPU NumPy l GPU PyCUDA CUDA Driver API ElementwiseKernel ReductionKernel Chainer Function 19
20. 20. (2) l v1.3.0 (Sep. 2) CuPy l CuPy: Chainer GPU NumPy Chainer CuPy l cupy 20
21. 21. l Variable GPGPU l GPU GPU Chainer GPU 21
22. 22. FPGA 22 l NeuFlow Yann LeCun Torch7 FPGA ConvNet https://github.com/clementfarabet/neuow TeraDeep l Navigating the Design Space of Recongurable Neural Networks Accelerators EPFL FPGA CPU/GPU/ASIC FPGA GPU ASIC
23. 23. l l l l l Chainer l Chainer NumPy PyCUDA v1.3.0 CuPy l l FPGA 23