Probability Overview Naive Bayes Classifier · 2018-03-23 · 머신러닝의학습방법들-Gradient descent based learning-Probability theory based learning-Information theory based

Probability Overview

Naive Bayes Classifier

Director of TEAMLABSungchul Choi

머신러닝의 학습 방법들

- Gradient descent based learning

- Probability theory based learning

- Information theory based learning

- Distance similarity based learning

머신러닝의 학습 방법들

- Gradient descent based learning

- Probability theory based learning

- Information theory based learning

- Distance similarity based learning

Probability

P (X) =count(Event X)

count(ALL Event)

P (�1 < x < 1)

Z 1

�1f(x) dx = 1

Source: https://goo.gl/DVNSH2

- 연속형 값

- 이산형 값

Source: https://goo.gl/D9VCSL

Basic concepts of probability

0 P (E) 1

P (S) =NX

i=1

P (Ei) = 1 if all Ei are independent

P (A \B) P (A [B) P (Ac) = 1� P (A)

Conditional probability

P (A|B) =P (A \B)

P (B)

A B

A \B

Conditional probability

P (A|B) =P (A \B)

P (B)

P (B|A) =P (A \B)

P (A)

P (A) = {set of odd number}

P (B) = {less than 4 in a dice}

A B

A \B

Human knowledge belongs to the world.

Bayes’s Theorem



Bayes’s theorem

- 경험에 의한 확률의 업데이트

- 사전 확률(given)에서 사건 발생을 통해 사후 확률로

- 빈도주의 vs. 베이즈주의

- 객관적 확률은 존재하지 않는다!

Queen card game

1

3

1

3

1

3

Probability updated

8

303

30

19

30

Probability updated

8

303

30

19

30

Bayes’s theorem

P (A|B) =P (A \B)

P (B)P (B|A) =

P (A \B)

P (A)

P (A \B) = P (B)P (A|B) = P (A)P (B|A)

P (A|B) =P (A \B)

P (B)=

P (A)P (B|A)

P (B)

Cookie Quiz

쿠키 두 그릇이 있다고 한다. 첫 번째 그릇에는

바닐라 쿠키 30개와 초콜렛 쿠키 10개가 있고, 두

번째 그릇에는 각 쿠키가 20개씩 있다. 임의 쿠키를

집었는데 해당 쿠키가 바닐라 쿠키이다. 이 쿠키가

그릇 1에서 나왔을 확률은?

Source: https://goo.gl/GzopZy

Cookie Quiz쿠키 두 그릇이 있다고 한다. 첫 번째 그릇에는 바닐라 쿠키 30개와 초콜렛 쿠키 10개가 있고, 두 번째 그릇에는 각

쿠키가 20개씩 있다. 임의 쿠키를 집었는데 해당 쿠키가 바닐라 쿠키이다. 이 쿠키가 그릇 1에서 나왔을 확률은?

P (A|B) =P (A \B)

P (B)=

P (A)P (B|A)

P (B)P (Choco)

P (V anilla)

P (B1)

P (B2)


Bayes’s theoremH is Class

D is Data사전 확률 우도사후확률

데이터가 발생할 확률(Evidence)

P (H|D) =P (H)P (D|H)

P (D)<latexit sha1_base64="sUd+TkA9QBTMtePJ5VAVo7nCkmU=">AAACCHicbZBPS8MwGMbT+W/Of1WPXoJDWC+jFUE9CEN32LGCdYOtjDRLt7A0LUkqjG5XL34VLx5UvPoRvPltzLYedPOBwC/P+74k7xMkjEpl299GYWV1bX2juFna2t7Z3TP3D+5lnApMPByzWLQCJAmjnHiKKkZaiSAoChhpBsObab35QISkMb9To4T4EepzGlKMlLa6JnQrjXHdglewEwqEM3213Ep93LAmmuvWpGuW7ao9E1wGJ4cyyOV2za9OL8ZpRLjCDEnZduxE+RkSimJGJqVOKkmC8BD1SVsjRxGRfjbbZAJPtNODYSz04QrO3N8TGYqkHEWB7oyQGsjF2tT8r9ZOVXjhZ5QnqSIczx8KUwZVDKexwB4VBCs20oCwoPqvEA+QTkTp8Eo6BGdx5WXwTquXVef2rFy7ztMogiNwDCrAAeegBhrABR7A4BE8g1fwZjwZL8a78TFvLRj5zCH4I+PzB4jFl0Q=</latexit><latexit sha1_base64="sUd+TkA9QBTMtePJ5VAVo7nCkmU=">AAACCHicbZBPS8MwGMbT+W/Of1WPXoJDWC+jFUE9CEN32LGCdYOtjDRLt7A0LUkqjG5XL34VLx5UvPoRvPltzLYedPOBwC/P+74k7xMkjEpl299GYWV1bX2juFna2t7Z3TP3D+5lnApMPByzWLQCJAmjnHiKKkZaiSAoChhpBsObab35QISkMb9To4T4EepzGlKMlLa6JnQrjXHdglewEwqEM3213Ep93LAmmuvWpGuW7ao9E1wGJ4cyyOV2za9OL8ZpRLjCDEnZduxE+RkSimJGJqVOKkmC8BD1SVsjRxGRfjbbZAJPtNODYSz04QrO3N8TGYqkHEWB7oyQGsjF2tT8r9ZOVXjhZ5QnqSIczx8KUwZVDKexwB4VBCs20oCwoPqvEA+QTkTp8Eo6BGdx5WXwTquXVef2rFy7ztMogiNwDCrAAeegBhrABR7A4BE8g1fwZjwZL8a78TFvLRj5zCH4I+PzB4jFl0Q=</latexit><latexit sha1_base64="sUd+TkA9QBTMtePJ5VAVo7nCkmU=">AAACCHicbZBPS8MwGMbT+W/Of1WPXoJDWC+jFUE9CEN32LGCdYOtjDRLt7A0LUkqjG5XL34VLx5UvPoRvPltzLYedPOBwC/P+74k7xMkjEpl299GYWV1bX2juFna2t7Z3TP3D+5lnApMPByzWLQCJAmjnHiKKkZaiSAoChhpBsObab35QISkMb9To4T4EepzGlKMlLa6JnQrjXHdglewEwqEM3213Ep93LAmmuvWpGuW7ao9E1wGJ4cyyOV2za9OL8ZpRLjCDEnZduxE+RkSimJGJqVOKkmC8BD1SVsjRxGRfjbbZAJPtNODYSz04QrO3N8TGYqkHEWB7oyQGsjF2tT8r9ZOVXjhZ5QnqSIczx8KUwZVDKexwB4VBCs20oCwoPqvEA+QTkTp8Eo6BGdx5WXwTquXVef2rFy7ztMogiNwDCrAAeegBhrABR7A4BE8g1fwZjwZL8a78TFvLRj5zCH4I+PzB4jFl0Q=</latexit><latexit sha1_base64="sUd+TkA9QBTMtePJ5VAVo7nCkmU=">AAACCHicbZBPS8MwGMbT+W/Of1WPXoJDWC+jFUE9CEN32LGCdYOtjDRLt7A0LUkqjG5XL34VLx5UvPoRvPltzLYedPOBwC/P+74k7xMkjEpl299GYWV1bX2juFna2t7Z3TP3D+5lnApMPByzWLQCJAmjnHiKKkZaiSAoChhpBsObab35QISkMb9To4T4EepzGlKMlLa6JnQrjXHdglewEwqEM3213Ep93LAmmuvWpGuW7ao9E1wGJ4cyyOV2za9OL8ZpRLjCDEnZduxE+RkSimJGJqVOKkmC8BD1SVsjRxGRfjbbZAJPtNODYSz04QrO3N8TGYqkHEWB7oyQGsjF2tT8r9ZOVXjhZ5QnqSIczx8KUwZVDKexwB4VBCs20oCwoPqvEA+QTkTp8Eo6BGdx5WXwTquXVef2rFy7ztMogiNwDCrAAeegBhrABR7A4BE8g1fwZjwZL8a78TFvLRj5zCH4I+PzB4jFl0Q=</latexit>

Bayes’s theorem

P (H|D) =P (C)P (D|H)

P (D)

P (A1 \B) + P (A2 \B) + P (A3 \B)

P (A \B) = P (B)P (A|B) = P (A)P (B|A)

P (B) = P (A1)P (B \A1) + P (A2)P (B \A2) + P (A3)P (B \A3)

Example

공1 무작위로 1개 나오는 다음과 같은 완구가 있다.


나온 공이 “큰공”일때,

검은색일 확률은?

Example


P (H|D) =P (C)P (D|H)

P (D)

P (B) = P (A1)P (B \A1) + P (A2)P (B \A2) + P (A3)P (B \A3)


Simple Bayes Classifier



Viagra 스팸필터기

- Viagra라는 단어의 유무를 통해 스팸 여부 확인

- Viagra 단어가 들어가면 무조건 스팸?

- 어느정도 확률로 스팸이라고 해야할까?


number viagra spam number viagra spam1 1 1 11 1 02 0 0 12 0 03 0 0 13 0 04 0 0 14 1 05 0 0 15 0 06 0 0 16 0 07 0 1 17 0 08 0 0 18 0 19 1 1 19 0 110 1 0 20 1 1




NB Classifier Overview



제대로 된 스팸 필터기를 만들어보자

- Viagra 단어외에 영향을 주는 단어들은?

- 오히려 스팸을 제외해주는 단어는 어떻게 찾지?

- 한번에 여러 단어들을 고려하는 필터기를 만들자

Feature의 확장

P (spam|viagra)

P (spam|viagra, hello, lucky,marketing...)

변수가 많을 때, 조건부 확률의 변화

Multivariate multiplication rule

P (Y |X1, X2) =P (Y \X1 \X2)

P (X1 \X2)

P (Y \X1 \X2) = P (Y |X1, X2)P (X1 \X2)

P (X1, X2, X3, ...Xn

)

= P (X1)P (X2|X1)P (X3|X1, X2) . . . P (Xn

|X1 . . . Xx�1)

Problems

- 계산이 어려워짐

- Feature의 차원이 증가하면 Sparse Vector가 생성à 확률이 0이 되는 값이 늘어남

Naïve Bayes Classifier

- 복잡하게 하지말고 단순(naïve)하게 해결하자

- 각 변수의 관계가 독립임을 가정

- 계산이 용이해지고, 성능이 생각보다 좋음

Joint Probability

P (A \B) = P (A)P (B) if A and B are independent

P (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

Naïve Bayes Classifier

P (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

Issues

- 너무 많은 확률값 à 0에 수렴하게 되는 문제

- 곱하지 말고 더하자 à log

log{P (Yc)

nY

i=1

P (Xi|Yc)} = logP (Yc) +

nX

i=1

logP (Xi|Yc)P (Y |X1 \X2) =

P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

Issues

- 확률이 0인 변수들이 존재함 à 전체값 0

- 작게나마 확률이 나올 수 있도록 변경 à 스무딩

P (X|Y ) =

count(X \ Y ) + k

count(Y ) + (k ⇤ |number of class|)

log{P (Yc)

nY

i=1

P (Xi|Yc)} = logP (Yc) +

nX

i=1

logP (Xi|Yc)


NB Classifier Implementation



Dataset – German Credit

- 대출 사기인가? 아닌가를 예측하는 문제

- 데이터를 NB에 맞도록 간단하게 변환

- Binary 데이터들로 이루어진 대출 사기 데이터

Preprocessing

One-Hot Encoding

Modelling

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

<latexit sha1_base64="4550UoA6suVF95oUyO3S5kzKirc=">AAACdHicfVFNb9NAEF2br2K+AlyQENLQuFIiVZHdC3CoVMGFY5AIDYqDNV6v21XXa7M7RkRb/wN+HTf+BheubNIcoEU8aaWnN280s2+KVklLSfIjCK9dv3Hz1s7t6M7de/cfDB4++mCbznAx441qzLxAK5TUYkaSlJi3RmBdKHFcnL1Z14+/CGNlo9/TqhXLGk+0rCRH8lI++BbH0XT0MedwDvM83YesbMjue67HcAhZZZC7jWGctaYpMyVrSTZ38jDtPzndw3Q0z+X52tC7/1nGPWSfOywhI/GVXOw7YpAWEBQWQvVRHOeDYTJJNoCrJN2SIdtimg+++3V5VwtNXKG1izRpaenQkORK9FHWWdEiP8MTsfBUYy3s0m1S62HPKyVUjfFPE2zUPzsc1tau6sI7a6RTe7m2Fv9VW3RUvVw6qduOhOYXg6pOATWwPgGU0ghOauUJciP9rsBP0QdN/lCRDyG9/OWrZHYweTVJ3x0Mj15v09hhT9kuG7GUvWBH7C2bshnj7GfwJHge7Aa/wmfhMNy7sIbBtucx+wvh5DcdlLqP</latexit><latexit sha1_base64="4550UoA6suVF95oUyO3S5kzKirc=">AAACdHicfVFNb9NAEF2br2K+AlyQENLQuFIiVZHdC3CoVMGFY5AIDYqDNV6v21XXa7M7RkRb/wN+HTf+BheubNIcoEU8aaWnN280s2+KVklLSfIjCK9dv3Hz1s7t6M7de/cfDB4++mCbznAx441qzLxAK5TUYkaSlJi3RmBdKHFcnL1Z14+/CGNlo9/TqhXLGk+0rCRH8lI++BbH0XT0MedwDvM83YesbMjue67HcAhZZZC7jWGctaYpMyVrSTZ38jDtPzndw3Q0z+X52tC7/1nGPWSfOywhI/GVXOw7YpAWEBQWQvVRHOeDYTJJNoCrJN2SIdtimg+++3V5VwtNXKG1izRpaenQkORK9FHWWdEiP8MTsfBUYy3s0m1S62HPKyVUjfFPE2zUPzsc1tau6sI7a6RTe7m2Fv9VW3RUvVw6qduOhOYXg6pOATWwPgGU0ghOauUJciP9rsBP0QdN/lCRDyG9/OWrZHYweTVJ3x0Mj15v09hhT9kuG7GUvWBH7C2bshnj7GfwJHge7Aa/wmfhMNy7sIbBtucx+wvh5DcdlLqP</latexit><latexit sha1_base64="4550UoA6suVF95oUyO3S5kzKirc=">AAACdHicfVFNb9NAEF2br2K+AlyQENLQuFIiVZHdC3CoVMGFY5AIDYqDNV6v21XXa7M7RkRb/wN+HTf+BheubNIcoEU8aaWnN280s2+KVklLSfIjCK9dv3Hz1s7t6M7de/cfDB4++mCbznAx441qzLxAK5TUYkaSlJi3RmBdKHFcnL1Z14+/CGNlo9/TqhXLGk+0rCRH8lI++BbH0XT0MedwDvM83YesbMjue67HcAhZZZC7jWGctaYpMyVrSTZ38jDtPzndw3Q0z+X52tC7/1nGPWSfOywhI/GVXOw7YpAWEBQWQvVRHOeDYTJJNoCrJN2SIdtimg+++3V5VwtNXKG1izRpaenQkORK9FHWWdEiP8MTsfBUYy3s0m1S62HPKyVUjfFPE2zUPzsc1tau6sI7a6RTe7m2Fv9VW3RUvVw6qduOhOYXg6pOATWwPgGU0ghOauUJciP9rsBP0QdN/lCRDyG9/OWrZHYweTVJ3x0Mj15v09hhT9kuG7GUvWBH7C2bshnj7GfwJHge7Aa/wmfhMNy7sIbBtucx+wvh5DcdlLqP</latexit><latexit sha1_base64="4550UoA6suVF95oUyO3S5kzKirc=">AAACdHicfVFNb9NAEF2br2K+AlyQENLQuFIiVZHdC3CoVMGFY5AIDYqDNV6v21XXa7M7RkRb/wN+HTf+BheubNIcoEU8aaWnN280s2+KVklLSfIjCK9dv3Hz1s7t6M7de/cfDB4++mCbznAx441qzLxAK5TUYkaSlJi3RmBdKHFcnL1Z14+/CGNlo9/TqhXLGk+0rCRH8lI++BbH0XT0MedwDvM83YesbMjue67HcAhZZZC7jWGctaYpMyVrSTZ38jDtPzndw3Q0z+X52tC7/1nGPWSfOywhI/GVXOw7YpAWEBQWQvVRHOeDYTJJNoCrJN2SIdtimg+++3V5VwtNXKG1izRpaenQkORK9FHWWdEiP8MTsfBUYy3s0m1S62HPKyVUjfFPE2zUPzsc1tau6sI7a6RTe7m2Fv9VW3RUvVw6qduOhOYXg6pOATWwPgGU0ghOauUJciP9rsBP0QdN/lCRDyG9/OWrZHYweTVJ3x0Mj15v09hhT9kuG7GUvWBH7C2bshnj7GfwJHge7Aa/wmfhMNy7sIbBtucx+wvh5DcdlLqP</latexit>

ModellingY의 Index

𝑃(𝑋$|𝑌')

Classifier

P(Y=1|X)의 확률과 P(Y=0|X)의 확률 비교


Multinomial NB



Multinomial Naïve Bayes

- X값이 Binary가 아니라 1 이상의 값을 가지는 문제

- 일반적으로 Text 문제를 분류할 때 많이 쓰임

- 단어의 존재 유무가 아닌 단어의 출현횟수 Feature로

Text의 Feature 표현?

문자 è Feature

문자를 Vector로 – One-hot Encoding하나의 단어를 Vector의 Index로 인식, 단어 존재시 1 없으면 0

Bag of words

단어별로 인덱스를 부여해서한 문장(또는 문서)의 단어의 개수를 Vector로 표현

Bag of words

단어별로 인덱스를 부여해서한 문장(또는 문서)의 단어의 개수를 Vector로 표현

다시 돌아가서..

Multinomial Naïve BayesP (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

기본식!!

Multinomial Naïve BayesP (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

Likelihood만 바뀜


- 식 계산하는 방식이 다름

http://sebastianraschka.com/Articles/2014_naive_bayes_1.html

nY

i=1

P (Xi|Yc)<latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="X/BbPPQRM1pmBhxdK1enSbL+gJw=">AAAB2HicbZDNSgMxFIXv1L86Vq1rN8EiuCozbtSd4MZlBccW2qFkMnfa0ExmSO4IpfQFXLhRfDB3vo3pz0KtBwIf5yTk3pOUSloKgi+vtrW9s7tX3/cPGv7h0XGz8WSLygiMRKEK00u4RSU1RiRJYa80yPNEYTeZ3C3y7jMaKwv9SNMS45yPtMyk4OSszrDZCtrBUmwTwjW0YK1h83OQFqLKUZNQ3Np+GJQUz7ghKRTO/UFlseRiwkfYd6h5jjaeLcecs3PnpCwrjDua2NL9+WLGc2uneeJu5pzG9m+2MP/L+hVl1/FM6rIi1GL1UVYpRgVb7MxSaVCQmjrgwkg3KxNjbrgg14zvOgj/brwJ0WX7ph0+BFCHUziDCwjhCm7hHjoQgYAUXuDNG3uv3vuqqpq37uwEfsn7+AaqKYoN</latexit><latexit sha1_base64="2E/bCqjC2D2g5woPSZvkI083ILM=">AAAB/nicbZC9TsMwFIVvyl8pBQori6FCKkuVsAADEhILY5EoLWpK5LhOa9WxI9tBqkJmFl6FhQEQj8HG2+D+DNByJEufzrF1fU+YcKaN6347haXlldW14nppo7y5tV3ZKd9qmSpCm0Ryqdoh1pQzQZuGGU7biaI4DjlthcPLcd56oEozKW7MKKHdGPcFixjBxlpBZd9PlOz5nMXM6CBj515+n4kcNWrtgD3eBeQoqFTdujsRWgRvBlWYqRFUvvyeJGlMhSEca93x3MR0M6wMI5zmJT/VNMFkiPu0Y1HgmOpuNlklR4fW6aFIKnuEQRP394sMx1qP4tDejLEZ6PlsbP6XdVITnXYzJpLUUEGmg6KUIyPRuBfUY4oSw0cWMFHM/hWRAVaYGNteyZbgza+8CM3j+lndu3ahCHtwADXw4AQu4Aoa0AQCT/ACb/DuPDuvzse0rYIzq20X/sj5/AGtyZi+</latexit><latexit sha1_base64="2E/bCqjC2D2g5woPSZvkI083ILM=">AAAB/nicbZC9TsMwFIVvyl8pBQori6FCKkuVsAADEhILY5EoLWpK5LhOa9WxI9tBqkJmFl6FhQEQj8HG2+D+DNByJEufzrF1fU+YcKaN6347haXlldW14nppo7y5tV3ZKd9qmSpCm0Ryqdoh1pQzQZuGGU7biaI4DjlthcPLcd56oEozKW7MKKHdGPcFixjBxlpBZd9PlOz5nMXM6CBj515+n4kcNWrtgD3eBeQoqFTdujsRWgRvBlWYqRFUvvyeJGlMhSEca93x3MR0M6wMI5zmJT/VNMFkiPu0Y1HgmOpuNlklR4fW6aFIKnuEQRP394sMx1qP4tDejLEZ6PlsbP6XdVITnXYzJpLUUEGmg6KUIyPRuBfUY4oSw0cWMFHM/hWRAVaYGNteyZbgza+8CM3j+lndu3ahCHtwADXw4AQu4Aoa0AQCT/ACb/DuPDuvzse0rYIzq20X/sj5/AGtyZi+</latexit><latexit sha1_base64="e6DUYda8XEsDP8La08SlORC32NI=">AAACCXicbVA9T8MwEHX4LOUrwMhiqJDKUiUswIBUwcJYJEKLmhA5jtNadZzIdpCqkJmFv8LCAIiVf8DGv8FtM0DLk056eu9Od/eClFGpLOvbmJtfWFxarqxUV9fWNzbNre0bmWQCEwcnLBGdAEnCKCeOooqRTioIigNG2sHgYuS374mQNOHXapgSL0Y9TiOKkdKSb+65qUhCl9GYKunn9Mwu7nJewFa949OHWx8f+mbNalhjwFlil6QGSrR888sNE5zFhCvMkJRd20qVlyOhKGakqLqZJCnCA9QjXU05ion08vErBTzQSgijROjiCo7V3xM5iqUcxoHujJHqy2lvJP7ndTMVnXg55WmmCMeTRVHGoErgKBcYUkGwYkNNEBZU3wpxHwmElU6vqkOwp1+eJc5R47RhX1m15nmZRgXsgn1QBzY4Bk1wCVrAARg8gmfwCt6MJ+PFeDc+Jq1zRjmzA/7A+PwBTGyaLg==</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit><latexit sha1_base64="z0TZisTl0vBnvyDzK5wJsHlUxbs=">AAACCXicbVBNS8NAEN3Ur1q/oh69rBahXkoignoQil48VjC20sSw2WzbpZtN2N0IJebsxb/ixYOKV/+BN/+N2zYHbX0w8Hhvhpl5QcKoVJb1bZTm5hcWl8rLlZXVtfUNc3PrRsapwMTBMYtFO0CSMMqJo6hipJ0IgqKAkVYwuBj5rXsiJI35tRomxItQj9MuxUhpyTd33UTEoctoRJX0M3pm53cZz2Gz1vbpw62PD3yzatWtMeAssQtSBQWavvnlhjFOI8IVZkjKjm0lysuQUBQzklfcVJIE4QHqkY6mHEVEetn4lRzuayWE3Vjo4gqO1d8TGYqkHEaB7oyQ6stpbyT+53VS1T3xMsqTVBGOJ4u6KYMqhqNcYEgFwYoNNUFYUH0rxH0kEFY6vYoOwZ5+eZY4h/XTun11VG2cF2mUwQ7YAzVgg2PQAJegCRyAwSN4Bq/gzXgyXox342PSWjKKmW3wB8bnD02smjI=</latexit>

P (Xi | Yc) =

Ptf(xi, d 2 Yc) + ↵PNd2Yc + ↵ · V

<latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit>

•x_i: A word from the feature vector x of a particular sample.

•∑tf(xi,d∈y_c): The sum of raw term frequencies of word x_i from all documents in the

training sample that belong to class y_c.

•∑Nd∈y_c: The sum of all term frequencies in the training dataset for class y_c.

•α: An additive smoothing parameter (α=1α=1 for Laplace smoothing).

•V: The size of the vocabulary (number of different words in the training set).


http://slideplayer.com/slide/10998986/

P (Xi | Yc) =

Ptf(xi, d 2 Yc) + ↵PNd2Yc + ↵ · V

<latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit><latexit sha1_base64="vHPCkempjFBBycNXPtZebX4fLfk=">AAACR3icbZDNSxwxGMYzq211beuqRy8vLsKWlmVGhNaDIHrxJCu4umVnGTKZjBtMMkPyTnEZ5s/z4tGbf4MXD614NPuB+NEHAg/P874k+cW5FBZ9/9arzc1/+PhpYbG+9PnL1+XGyuqpzQrDeJdlMjO9mFouheZdFCh5Lzecqljys/jiYNyf/eHGikyf4CjnA0XPtUgFo+iiqBF1Wr1IQKhEAr8j9g12IUwNZWVoCwWYti4j8QMSCIWe9t8hpDIf0mo6cRSVz2X1XELIkgzhtKpHjabf9ieC9yaYmSaZqRM1bsIkY4XiGpmk1vYDP8dBSQ0KJnlVDwvLc8ou6DnvO6up4nZQTkBUsOmSBNLMuKMRJunLjZIqa0cqdpOK4tC+7cbh/7p+gemvQSl0XiDXbHpRWkjADMZUIRGGM5QjZygzwr0V2JA6jujYjyEEb7/83nS32jvt4Hi7ubc/o7FA1skGaZGA/CR75JB0SJcwckXuyF/yz7v27r0H73E6WvNmO2vklWreE7Iwrm8=</latexit>

P (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label


http://slideplayer.com/slide/10998986/

P (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label


Gaussian NB



Gaussian Naïve Bayes

- Category 데이터가 아닌경우에 NB 의 적용

- Continuous 데이터의 적용을 위해 y의 분포를 정규분포(gaussian)으로 가정함

- 확률 밀도 함수 상의 해당 값 x 가 나올 확률로 NB를구현함


P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,

<latexit sha1_base64="QmSLBdtKNmcRKvHn/LEWSlhfXwI=">AAACYXicbVFNaxsxENVukzZ102STHnMRMQUbErNrCm0PhdBeenShzgeWs2jlWVtE2t1KsyFG7J/sLadc+kMq23tokg4IHm/em5GeskpJi3F8H4QvtrZfvtp53Xmz+3ZvPzo4PLdlbQSMRalKc5lxC0oWMEaJCi4rA1xnCi6ym2+r/sUtGCvL4icuK5hqPi9kLgVHT6XRctS7S51sKNNyRq9S0adfKMsNFy5pHLO/DLohqySzcq759TB1XtM0Xg93FVOQY+90I2/nnPpJ9UbVvx423vzYyYycL7B/0kmjbjyI10Wfg6QFXdLWKI1+s1kpag0FCsWtnSRxhVPHDUqhoOmw2kLFxQ2fw8TDgmuwU7eOqKHvPTOjeWn8KZCu2X8djmtrlzrzSs1xYZ/2VuT/epMa809TJ4uqRijEZlFeK4olXeVNZ9KAQLX0gAsj/V2pWHAfGPpfWYWQPH3yczAeDj4Pkh8fumdf2zR2yBE5Jj2SkI/kjHwnIzImgjwE28FesB/8CTthFB5upGHQet6RRxUe/QWuvbZR</latexit><latexit sha1_base64="QmSLBdtKNmcRKvHn/LEWSlhfXwI=">AAACYXicbVFNaxsxENVukzZ102STHnMRMQUbErNrCm0PhdBeenShzgeWs2jlWVtE2t1KsyFG7J/sLadc+kMq23tokg4IHm/em5GeskpJi3F8H4QvtrZfvtp53Xmz+3ZvPzo4PLdlbQSMRalKc5lxC0oWMEaJCi4rA1xnCi6ym2+r/sUtGCvL4icuK5hqPi9kLgVHT6XRctS7S51sKNNyRq9S0adfKMsNFy5pHLO/DLohqySzcq759TB1XtM0Xg93FVOQY+90I2/nnPpJ9UbVvx423vzYyYycL7B/0kmjbjyI10Wfg6QFXdLWKI1+s1kpag0FCsWtnSRxhVPHDUqhoOmw2kLFxQ2fw8TDgmuwU7eOqKHvPTOjeWn8KZCu2X8djmtrlzrzSs1xYZ/2VuT/epMa809TJ4uqRijEZlFeK4olXeVNZ9KAQLX0gAsj/V2pWHAfGPpfWYWQPH3yczAeDj4Pkh8fumdf2zR2yBE5Jj2SkI/kjHwnIzImgjwE28FesB/8CTthFB5upGHQet6RRxUe/QWuvbZR</latexit><latexit sha1_base64="QmSLBdtKNmcRKvHn/LEWSlhfXwI=">AAACYXicbVFNaxsxENVukzZ102STHnMRMQUbErNrCm0PhdBeenShzgeWs2jlWVtE2t1KsyFG7J/sLadc+kMq23tokg4IHm/em5GeskpJi3F8H4QvtrZfvtp53Xmz+3ZvPzo4PLdlbQSMRalKc5lxC0oWMEaJCi4rA1xnCi6ym2+r/sUtGCvL4icuK5hqPi9kLgVHT6XRctS7S51sKNNyRq9S0adfKMsNFy5pHLO/DLohqySzcq759TB1XtM0Xg93FVOQY+90I2/nnPpJ9UbVvx423vzYyYycL7B/0kmjbjyI10Wfg6QFXdLWKI1+s1kpag0FCsWtnSRxhVPHDUqhoOmw2kLFxQ2fw8TDgmuwU7eOqKHvPTOjeWn8KZCu2X8djmtrlzrzSs1xYZ/2VuT/epMa809TJ4uqRijEZlFeK4olXeVNZ9KAQLX0gAsj/V2pWHAfGPpfWYWQPH3yczAeDj4Pkh8fumdf2zR2yBE5Jj2SkI/kjHwnIzImgjwE28FesB/8CTthFB5upGHQet6RRxUe/QWuvbZR</latexit><latexit sha1_base64="QmSLBdtKNmcRKvHn/LEWSlhfXwI=">AAACYXicbVFNaxsxENVukzZ102STHnMRMQUbErNrCm0PhdBeenShzgeWs2jlWVtE2t1KsyFG7J/sLadc+kMq23tokg4IHm/em5GeskpJi3F8H4QvtrZfvtp53Xmz+3ZvPzo4PLdlbQSMRalKc5lxC0oWMEaJCi4rA1xnCi6ym2+r/sUtGCvL4icuK5hqPi9kLgVHT6XRctS7S51sKNNyRq9S0adfKMsNFy5pHLO/DLohqySzcq759TB1XtM0Xg93FVOQY+90I2/nnPpJ9UbVvx423vzYyYycL7B/0kmjbjyI10Wfg6QFXdLWKI1+s1kpag0FCsWtnSRxhVPHDUqhoOmw2kLFxQ2fw8TDgmuwU7eOqKHvPTOjeWn8KZCu2X8djmtrlzrzSs1xYZ/2VuT/epMa809TJ4uqRijEZlFeK4olXeVNZ9KAQLX0gAsj/V2pWHAfGPpfWYWQPH3yczAeDj4Pkh8fumdf2zR2yBE5Jj2SkI/kjHwnIzImgjwE28FesB/8CTthFB5upGHQet6RRxUe/QWuvbZR</latexit>

- 특정 에 대한 의 평균, 표준편차를 에 대입µ,�<latexit sha1_base64="LWamUpo1EfsFSMWYIsfcBrsyNWI=">AAAB8XicbVBNS8NAEN3Ur1q/qh69LBbBg5REBPVW9OKxgrGFJJTNdtMu3c2G3YlQQn+GFw8qXv033vw3btsctPXBwOO9GWbmxZngBlz326msrK6tb1Q3a1vbO7t79f2DR6NyTZlPlVC6GxPDBE+ZDxwE62aaERkL1olHt1O/88S04Sp9gHHGIkkGKU84JWClIJT5GQ4NH0jSqzfcpjsDXiZeSRqoRLtX/wr7iuaSpUAFMSbw3AyigmjgVLBJLcwNywgdkQELLE2JZCYqZidP8IlV+jhR2lYKeKb+niiINGYsY9spCQzNojcV//OCHJKrqOBplgNL6XxRkgsMCk//x32uGQUxtoRQze2tmA6JJhRsSjUbgrf48jLxz5vXTe/+otG6KdOooiN0jE6Rhy5RC92hNvIRRQo9o1f05oDz4rw7H/PWilPOHKI/cD5/ACJUkLM=</latexit><latexit sha1_base64="LWamUpo1EfsFSMWYIsfcBrsyNWI=">AAAB8XicbVBNS8NAEN3Ur1q/qh69LBbBg5REBPVW9OKxgrGFJJTNdtMu3c2G3YlQQn+GFw8qXv033vw3btsctPXBwOO9GWbmxZngBlz326msrK6tb1Q3a1vbO7t79f2DR6NyTZlPlVC6GxPDBE+ZDxwE62aaERkL1olHt1O/88S04Sp9gHHGIkkGKU84JWClIJT5GQ4NH0jSqzfcpjsDXiZeSRqoRLtX/wr7iuaSpUAFMSbw3AyigmjgVLBJLcwNywgdkQELLE2JZCYqZidP8IlV+jhR2lYKeKb+niiINGYsY9spCQzNojcV//OCHJKrqOBplgNL6XxRkgsMCk//x32uGQUxtoRQze2tmA6JJhRsSjUbgrf48jLxz5vXTe/+otG6KdOooiN0jE6Rhy5RC92hNvIRRQo9o1f05oDz4rw7H/PWilPOHKI/cD5/ACJUkLM=</latexit><latexit sha1_base64="LWamUpo1EfsFSMWYIsfcBrsyNWI=">AAAB8XicbVBNS8NAEN3Ur1q/qh69LBbBg5REBPVW9OKxgrGFJJTNdtMu3c2G3YlQQn+GFw8qXv033vw3btsctPXBwOO9GWbmxZngBlz326msrK6tb1Q3a1vbO7t79f2DR6NyTZlPlVC6GxPDBE+ZDxwE62aaERkL1olHt1O/88S04Sp9gHHGIkkGKU84JWClIJT5GQ4NH0jSqzfcpjsDXiZeSRqoRLtX/wr7iuaSpUAFMSbw3AyigmjgVLBJLcwNywgdkQELLE2JZCYqZidP8IlV+jhR2lYKeKb+niiINGYsY9spCQzNojcV//OCHJKrqOBplgNL6XxRkgsMCk//x32uGQUxtoRQze2tmA6JJhRsSjUbgrf48jLxz5vXTe/+otG6KdOooiN0jE6Rhy5RC92hNvIRRQo9o1f05oDz4rw7H/PWilPOHKI/cD5/ACJUkLM=</latexit><latexit sha1_base64="LWamUpo1EfsFSMWYIsfcBrsyNWI=">AAAB8XicbVBNS8NAEN3Ur1q/qh69LBbBg5REBPVW9OKxgrGFJJTNdtMu3c2G3YlQQn+GFw8qXv033vw3btsctPXBwOO9GWbmxZngBlz326msrK6tb1Q3a1vbO7t79f2DR6NyTZlPlVC6GxPDBE+ZDxwE62aaERkL1olHt1O/88S04Sp9gHHGIkkGKU84JWClIJT5GQ4NH0jSqzfcpjsDXiZeSRqoRLtX/wr7iuaSpUAFMSbw3AyigmjgVLBJLcwNywgdkQELLE2JZCYqZidP8IlV+jhR2lYKeKb+niiINGYsY9spCQzNojcV//OCHJKrqOBplgNL6XxRkgsMCk//x32uGQUxtoRQze2tmA6JJhRsSjUbgrf48jLxz5vXTe/+otG6KdOooiN0jE6Rhy5RC92hNvIRRQo9o1f05oDz4rw7H/PWilPOHKI/cD5/ACJUkLM=</latexit>

Yc<latexit sha1_base64="SpNegus/j04PlU+GX72eLZjdcY4=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltpQ9lsN+3SzSbsToQS+hO8eFDx6j/y5r9x2+ag1QcDj/dmmJkXplIYdN0vp7S0vLK6Vl6vbGxube9Ud/fuTZJpxn2WyES3Q2q4FIr7KFDydqo5jUPJW+Hoauq3Hrk2IlF3OE55ENOBEpFgFK10+9BjvWrNrbszkL/EK0gNCjR71c9uP2FZzBUySY3peG6KQU41Cib5pNLNDE8pG9EB71iqaMxNkM9OnZAjq/RJlGhbCslM/TmR09iYcRzazpji0Cx6U/E/r5NhdB7kQqUZcsXmi6JMEkzI9G/SF5ozlGNLKNPC3krYkGrK0KZTsSF4iy//Jf5J/aLu3ZzWGpdFGmU4gEM4Bg/OoAHX0AQfGAzgCV7g1ZHOs/PmvM9bS04xsw+/4Hx8A5XwjYc=</latexit><latexit sha1_base64="SpNegus/j04PlU+GX72eLZjdcY4=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltpQ9lsN+3SzSbsToQS+hO8eFDx6j/y5r9x2+ag1QcDj/dmmJkXplIYdN0vp7S0vLK6Vl6vbGxube9Ud/fuTZJpxn2WyES3Q2q4FIr7KFDydqo5jUPJW+Hoauq3Hrk2IlF3OE55ENOBEpFgFK10+9BjvWrNrbszkL/EK0gNCjR71c9uP2FZzBUySY3peG6KQU41Cib5pNLNDE8pG9EB71iqaMxNkM9OnZAjq/RJlGhbCslM/TmR09iYcRzazpji0Cx6U/E/r5NhdB7kQqUZcsXmi6JMEkzI9G/SF5ozlGNLKNPC3krYkGrK0KZTsSF4iy//Jf5J/aLu3ZzWGpdFGmU4gEM4Bg/OoAHX0AQfGAzgCV7g1ZHOs/PmvM9bS04xsw+/4Hx8A5XwjYc=</latexit><latexit sha1_base64="SpNegus/j04PlU+GX72eLZjdcY4=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltpQ9lsN+3SzSbsToQS+hO8eFDx6j/y5r9x2+ag1QcDj/dmmJkXplIYdN0vp7S0vLK6Vl6vbGxube9Ud/fuTZJpxn2WyES3Q2q4FIr7KFDydqo5jUPJW+Hoauq3Hrk2IlF3OE55ENOBEpFgFK10+9BjvWrNrbszkL/EK0gNCjR71c9uP2FZzBUySY3peG6KQU41Cib5pNLNDE8pG9EB71iqaMxNkM9OnZAjq/RJlGhbCslM/TmR09iYcRzazpji0Cx6U/E/r5NhdB7kQqUZcsXmi6JMEkzI9G/SF5ozlGNLKNPC3krYkGrK0KZTsSF4iy//Jf5J/aLu3ZzWGpdFGmU4gEM4Bg/OoAHX0AQfGAzgCV7g1ZHOs/PmvM9bS04xsw+/4Hx8A5XwjYc=</latexit><latexit sha1_base64="SpNegus/j04PlU+GX72eLZjdcY4=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltpQ9lsN+3SzSbsToQS+hO8eFDx6j/y5r9x2+ag1QcDj/dmmJkXplIYdN0vp7S0vLK6Vl6vbGxube9Ud/fuTZJpxn2WyES3Q2q4FIr7KFDydqo5jUPJW+Hoauq3Hrk2IlF3OE55ENOBEpFgFK10+9BjvWrNrbszkL/EK0gNCjR71c9uP2FZzBUySY3peG6KQU41Cib5pNLNDE8pG9EB71iqaMxNkM9OnZAjq/RJlGhbCslM/TmR09iYcRzazpji0Cx6U/E/r5NhdB7kQqUZcsXmi6JMEkzI9G/SF5ozlGNLKNPC3krYkGrK0KZTsSF4iy//Jf5J/aLu3ZzWGpdFGmU4gEM4Bg/OoAHX0AQfGAzgCV7g1ZHOs/PmvM9bS04xsw+/4Hx8A5XwjYc=</latexit>

Xi<latexit sha1_base64="/ixP9j8uraY/iBHcsLp/e60NNEo=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsJ+3SzSbsboQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T941EmmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6mfqtJ1SaJ/LBjFMMYjqQPOKMGivdt3u8V625dXcGsky8gtSgQLNX/er2E5bFKA0TVOuO56YmyKkynAmcVLqZxpSyER1gx1JJY9RBPjt1Qk6s0idRomxJQ2bq74mcxlqP49B2xtQM9aI3Ff/zOpmJLoOcyzQzKNl8UZQJYhIy/Zv0uUJmxNgSyhS3txI2pIoyY9Op2BC8xZeXiX9Wv6p7d+e1xnWRRhmO4BhOwYMLaMAtNMEHBgN4hld4c4Tz4rw7H/PWklPMHMIfOJ8/nX2NjA==</latexit><latexit sha1_base64="/ixP9j8uraY/iBHcsLp/e60NNEo=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsJ+3SzSbsboQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T941EmmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6mfqtJ1SaJ/LBjFMMYjqQPOKMGivdt3u8V625dXcGsky8gtSgQLNX/er2E5bFKA0TVOuO56YmyKkynAmcVLqZxpSyER1gx1JJY9RBPjt1Qk6s0idRomxJQ2bq74mcxlqP49B2xtQM9aI3Ff/zOpmJLoOcyzQzKNl8UZQJYhIy/Zv0uUJmxNgSyhS3txI2pIoyY9Op2BC8xZeXiX9Wv6p7d+e1xnWRRhmO4BhOwYMLaMAtNMEHBgN4hld4c4Tz4rw7H/PWklPMHMIfOJ8/nX2NjA==</latexit><latexit sha1_base64="/ixP9j8uraY/iBHcsLp/e60NNEo=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsJ+3SzSbsboQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T941EmmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6mfqtJ1SaJ/LBjFMMYjqQPOKMGivdt3u8V625dXcGsky8gtSgQLNX/er2E5bFKA0TVOuO56YmyKkynAmcVLqZxpSyER1gx1JJY9RBPjt1Qk6s0idRomxJQ2bq74mcxlqP49B2xtQM9aI3Ff/zOpmJLoOcyzQzKNl8UZQJYhIy/Zv0uUJmxNgSyhS3txI2pIoyY9Op2BC8xZeXiX9Wv6p7d+e1xnWRRhmO4BhOwYMLaMAtNMEHBgN4hld4c4Tz4rw7H/PWklPMHMIfOJ8/nX2NjA==</latexit><latexit sha1_base64="/ixP9j8uraY/iBHcsLp/e60NNEo=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsJ+3SzSbsboQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T941EmmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6mfqtJ1SaJ/LBjFMMYjqQPOKMGivdt3u8V625dXcGsky8gtSgQLNX/er2E5bFKA0TVOuO56YmyKkynAmcVLqZxpSyER1gx1JJY9RBPjt1Qk6s0idRomxJQ2bq74mcxlqP49B2xtQM9aI3Ff/zOpmJLoOcyzQzKNl8UZQJYhIy/Zv0uUJmxNgSyhS3txI2pIoyY9Op2BC8xZeXiX9Wv6p7d+e1xnWRRhmO4BhOwYMLaMAtNMEHBgN4hld4c4Tz4rw7H/PWklPMHMIfOJ8/nX2NjA==</latexit>


P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,


https://en.wikipedia.org/wiki/Naive_Bayes_classifier


P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,




P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,




P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,




P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,



P (Y |X1 \X2) =P (Y )P (X1 \X2|Y )

P (X1 \X2)=

P (Y )P (X1|Y )P (X2|Y )

P (X1)P (X2)

P (Yc|X1, . . . , Xn) =

P (Yc)nQ

i=1P (Xi|Yc)

nQi=1

P (Xi)Yc is a label

P (xi | Yc) =1q

2⇡�

2Yc

exp

✓� (xi � µYc)

2

2�

2Yc

◆,



Naïve Bayes with Sklearn



CountVectorizer in Scikit-learn

- 문서에서 Bag of Words Vector를 뽑아주는 class

http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html


- 다른 전처리 모듈 처럼 생성 à 적용의 과정을 거침


- 다른 전처리 모듈 처럼 생성 à 적용의 과정을 거침

NB classifier family in scikit-learn

- Scikit-learn에서 제공하는 NB classifier

- Bernoulli Naïve Bayes

- Multinomial Naïve Bayes

- Gaussian Naïve Bayes

Bernoulli Naïve Bayes

Bernoulli Naïve Bayes




Documents

Probability Overview Naive Bayes Classifier · 2018-03-23 · 머신러닝의학습방법들-Gradient descent based learning-Probability theory based learning-Information theory based