Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
머신러닝/딥러닝 시스템
MEMOML Software Stack
MEMO
High-level Structure
ML Framework
•
•
•
MEMO
High-level Structure
ML Framework
•
MEMO
Training
ML Framework
Forward pass: logits computationMini-batch
Backward pass: gradients computation
Input pipelineCompute
loss
MEMO
Auto-Differentiation
ML Framework
•
•
•
MEMO
Reverse-mode Autodiff
ML Framework
MEMO
Auto-differentiation
ML Framework
MEMOML Framework Categories
Build a graphExecute the graph
Directly execute statements
TensorFlow, Caffe2, MXNet, CNTK
PyTorch, TensorFlow Eager, MXNet Imperative
* Python: De-facto deep learning programming language
MEMOML Framework Categories
Build a graphExecute the graph
Directly execute statements
TensorFlow, Caffe2, MXNet, CNTK
PyTorch, TensorFlow Eager, MXNet Imperative
Easy to optimize and deploy
Hard to program and debug
Easy to program and debug
Little room for optimization
MEMO
Google TensorFlow
ML Framework Example
•
•
•
MEMO
Facebook PyTorch
ML Framework Example
•
•
•
MEMO
Open Neural Network Exchange (ONNX)
Interoperability between Frameworks
MEMO
ML frameworks
Symbolic ML frameworks and imperative ML frameworks
ML framework examples
Summary
텐서플로우
MEMOTensorFlow
TensorFlow 1.x default mode
•
Express numerical computation as a computation graph
•
•
Tensor: N-dimensional array
•
•
•
Tensors flow through the graph → TensorFlow
MEMOGoogle TensorFlow
The graph’s compiled to CPU / GPU / AIPU code
Salient features of TensorFlow graphs
MEMOTensorFlow Programming Model
Graph: model computation
Session: runs a graph
•
MEMOTensorFlow Programming Model
Symbolic Graph Style
•
•
•
•
MEMOTensorFlow Eager Mode
Statements directly executed without the separation of graph definition and execution
Write code that you can easily execute in a REPL
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
MEMOTensorFlow Programming Example
We can deploy the graph with a session
MEMOTensorFlow Eager Mode
Enabling eager execution requires two lines of code
Lets you write code that you can easily execute in a REPL
MEMOTensorFlow Example
Linear Regression
MEMOTensorFlow Example
Linear Regression
MEMOTensorFlow Example
Linear Regression
MEMOInput Pipeline with TensorFlow Dataset
MEMOInput Pipeline with TensorFlow Dataset
Shuffle, repeat, batch your data
Map each element of your dataset to transform it in a specific way to create a new dataset
MEMOSummary
TensorFlow programming model
TensorFlow basic program example
TensorFlow linear regression example
TensorFlow Dataset