Tensor Decomposition with Missing Indices

Tensor Decomposi-on with Missing Indices

Yuto Yamaguchi and Kohei Hayashi

17/08/22 IJCAI2017@Melbourne 1

Tensor data


#

#

(userA, #movie, Melbourne): 1 (userB, #tennis, Sydney): 2 (userC, #dinner, Canberra): 1 (userB, #beer, Brisbane): 1 (userA, #dinner, Melbourne): 2

e.g., TwiNer data (user, hashtag, loca-on)

Tensor data = mul--‐dimensional data

value

Tensor decomposi-on


e.g., CP decomposi-on [Carroll and Chang, 1970]

+ + … =

Applica-ons •  Recommenda-ons, noise reduc-on, data compression, …

Xijk = UirVjrWkrr∑

X V:, 1

U:, 1

W:, 1

V:, 2

U:, 2

W:, 2

[Our problem] what if indices are missing?


#

#

(userA, #movie, Melbourne): 1 (userB, #tennis, Sydney): 2 (userC, #dinner, Canberra): 1 (userB, #beer, -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐): 1 (userA, -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐, Melbourne): 2

Conven5onal tensor decomposi5on algorithms do not apply to these “incomplete samples” L

value

[Our problem] what if indices are missing?


#

#

(userA, #movie, Melbourne): 1 (userB, #tennis, Sydney): 2 (userC, #dinner, Canberra): 1 (userB, #beer, -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐): 1 (userA, -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐, Melbourne): 2

Conven5onal tensor decomposi5on algorithms do not apply to these “incomplete samples” L

value

Values are not missing

PROPOSED MODEL


Basic idea


(userA, #movie, Melbourne) (userB, #tennis, Sydney) (userC, #dinner, Canberra) (userB, #beer, Brisbane) (userA, #dinner, Melbourne)

+ + … e.g., CPD

infer

construct

decompose

Solve tensor decomposi5on and missing indices inference repeatedly

Proposed model (1/2)


Handle indices as unobserved variables

in ∈ 1,2,…I,φ{ }Observed (can be missing) indices

True (unobserved) indices

missing

Tensor elements

Decomposi5on parameters

[3rd-‐order case]



1. Generate decomposi-on parameters depending on the decomposi-on model

Θ = U,V,W{ } Uir = N ⋅ 0, 1λ

"

#$

%

&' for all i and r

e.g., CPD



2. Generate N indices (in, jn, kn) Delta if not missing

Uniform if missing

in ~



3. Generate N tensor elements depending on decomposi-on model

e.g., CPD

Xin jnkn= Uinr

VjnrWknr

r∑

Proposed model is a natural extension of the conven-onal tensor decomposi-on


where MLE Θ of the proposed model

Parameter inference

Varia-onal MAP-‐EM algorithm

•  E-‐step – Missing indices are inferred using learnt tensor decomposi-on

•  M-‐step – Tensor decomposi-on is learnt using inferred indices


See the paper for details if interested J

Time Complexity (Mth-‐order tensor)


Proposed algorithm for CPD

Conven-onal CPD NNm-

RIm

: # of samples

: # of missing indices for mth mode

: # of latent dimensions

: # of dimensions for mth mode

Only addi5onal term

EXPERIMENTS


Compared algorithms


[MAP-‐EM]: Proposed algo. with q inferred

[Uniform]: Proposed algo. with q fixed as uniform

[Prior]: Proposed algo. with q fixed as data histogram

[Minimal]: CPD with only complete samples

[Complete]: CPD with only complete modes

[CMTF]: Coupled matrix tensor factoriza-on [Acar+, 2011]

Approx. distribu5on on varia5onal inference Proposed

Baselines

Results


Lower beNer Lower beNer Upper beNer

Proposed model (red) works well if •  the number of samples is large, or •  missing ra-o is not very large

Synthe5c data generated by our model TwiZer data (user, hashtag, loca5on)

sample size large (n=10) sample size small (n=1)

Summary •  [New problem] –  Defined a new tensor decomposition problem where

the indices are partially missing

•  [Model] –  Proposed a probabilistic generative model to handle

missing indices •  [Algorithm] –  Developed a parameter inference algorithm


Github: yamaguchiyuto/missing_tensor_decomposition

Technology

Tensor Decomposition with Missing Indices