Embed Size (px)
Citation preview
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Learning Safe Prediction for Semi-Supervised Regression
Yu-Feng Li Han-Wen Zha Zhi-Hua Zhou
Lamda,Nanjing University;AAAI 2017
陈永恒,2017.5.22
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 1 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 2 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 3 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
Semi-supervised learning (SSL) concerns the problem on how toimprove learning performance via the usage of additional unlabeleddata.
Such a learning framework has received a great deal of attentionowing to immense demands in real-world applicationsDespite the success of SSL, studies reveal that SSL with theexploitation of unlabeled data might even deteriorate learningperformance.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 4 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
Semi-supervised learning (SSL) concerns the problem on how toimprove learning performance via the usage of additional unlabeleddata.Such a learning framework has received a great deal of attentionowing to immense demands in real-world applications
Despite the success of SSL, studies reveal that SSL with theexploitation of unlabeled data might even deteriorate learningperformance.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 4 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
Semi-supervised learning (SSL) concerns the problem on how toimprove learning performance via the usage of additional unlabeleddata.Such a learning framework has received a great deal of attentionowing to immense demands in real-world applicationsDespite the success of SSL, studies reveal that SSL with theexploitation of unlabeled data might even deteriorate learningperformance.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 4 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
It is highly desirable to study safe SSL scheme that on one side couldoften improve performance, on the other side will not hurtperformance
Recently several proposals have been developed to alleviate such afundamental challenge for semi-supervised classification (SSC), whilethe efforts on semi-supervised regression (SSR) remain to be limited.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 5 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
It is highly desirable to study safe SSL scheme that on one side couldoften improve performance, on the other side will not hurtperformanceRecently several proposals have been developed to alleviate such afundamental challenge for semi-supervised classification (SSC), whilethe efforts on semi-supervised regression (SSR) remain to be limited.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 5 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Motivation
It is highly desirable to study safe SSL scheme that on one side couldoften improve performance, on the other side will not hurtperformanceRecently several proposals have been developed to alleviate such afundamental challenge for semi-supervised classification (SSC), whilethe efforts on semi-supervised regression (SSR) remain to be limited.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 5 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 6 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)
Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)
A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)
Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)
Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Related Work
SSCSafe semi-supervised SVMs(Li and Zhou 2011; 2015)Safe semi-supervised classifier based on semi-supervised least squareclassifiers.(Krijthe and Loog 2015)A generic safe SSC framework for variants of performance measures(Li, Kwok, and Zhou 2016)Judging the quality of graph in graph-based SSC(Li, Wang, and Zhou2016)
SSRCo-training style algoithm for the learning of two semi-supervisedregressors (Zhou and Li 2005)Coregularization framework using multi-view training data (Brefeld etal.2006)
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 7 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 8 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
Let f1, f2, ..., fb be multiple SSR predictions and f0 be the predictionof certain direct supervised learner, where fi ∈ Ru, i = 0, ..., b,b and urefer to the number of regressors and unlabeled instances.f∗ refers tothe groud-truth assignment
Final safe prediction g(f1, f2, ..., fb, f0), which often outperforms f0,meanwhile could not be worse than f0.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 9 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
Let f1, f2, ..., fb be multiple SSR predictions and f0 be the predictionof certain direct supervised learner, where fi ∈ Ru, i = 0, ..., b,b and urefer to the number of regressors and unlabeled instances.f∗ refers tothe groud-truth assignmentFinal safe prediction g(f1, f2, ..., fb, f0), which often outperforms f0,meanwhile could not be worse than f0.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 9 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
We start with a simple scenario to alleviate this challenge, where theweights of SSR regressors are known. Specifically:
Let α=[α1;α2; ...;αb] ≥ 0 be the weights of individual regressors fi’s.Larger the weight,closer the regressor is to the ground-truth
We use error to measure the performance gain againstf0,i.e,(∥f0 − f∗∥2 − ∥f − f∗∥2)Butf∗ is obviously unknown,so we optimize the following functional instead:
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 10 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
We start with a simple scenario to alleviate this challenge, where theweights of SSR regressors are known. Specifically:
Let α=[α1;α2; ...;αb] ≥ 0 be the weights of individual regressors fi’s.Larger the weight,closer the regressor is to the ground-truthWe use error to measure the performance gain againstf0,i.e,(∥f0 − f∗∥2 − ∥f − f∗∥2)
Butf∗ is obviously unknown,so we optimize the following functional instead:
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 10 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
We start with a simple scenario to alleviate this challenge, where theweights of SSR regressors are known. Specifically:
Let α=[α1;α2; ...;αb] ≥ 0 be the weights of individual regressors fi’s.Larger the weight,closer the regressor is to the ground-truthWe use error to measure the performance gain againstf0,i.e,(∥f0 − f∗∥2 − ∥f − f∗∥2)Butf∗ is obviously unknown,so we optimize the following functional instead:
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 10 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
In reality,the explicit weights of individual regressors is difficult to know.Forthe sake of simplicity, one assumes that α is is from a convex linear setM = α|ATα ≤ b; a ≥ 0,which is a general form that reflects the relationof individual learners in ensemble learning,where A and b aretask-dependent coefficients.
Without further knowledge to determine the weights of individualregressors, one aim to optimize the worst-case performance gain as follow:
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 11 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Problem Setting and Formulation
In reality,the explicit weights of individual regressors is difficult to know.Forthe sake of simplicity, one assumes that α is is from a convex linear setM = α|ATα ≤ b; a ≥ 0,which is a general form that reflects the relationof individual learners in ensemble learning,where A and b aretask-dependent coefficients.Without further knowledge to determine the weights of individualregressors, one aim to optimize the worst-case performance gain as follow:
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 11 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 12 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
Note that Eq.(2) is concave to f and convex to α, and thus it isrecognized as saddle-point convex-concave optimization.
However, it is non-trivial to be solved efficiently because of poorconvergence rate.In order to alleviate the computational overload and understand howEq.(2) works, we in the following show that Eq.(2) can be formulatedas a geometric projection issue that help address the above concerns.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 13 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
Note that Eq.(2) is concave to f and convex to α, and thus it isrecognized as saddle-point convex-concave optimization.However, it is non-trivial to be solved efficiently because of poorconvergence rate.
In order to alleviate the computational overload and understand howEq.(2) works, we in the following show that Eq.(2) can be formulatedas a geometric projection issue that help address the above concerns.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 13 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
Note that Eq.(2) is concave to f and convex to α, and thus it isrecognized as saddle-point convex-concave optimization.However, it is non-trivial to be solved efficiently because of poorconvergence rate.In order to alleviate the computational overload and understand howEq.(2) works, we in the following show that Eq.(2) can be formulatedas a geometric projection issue that help address the above concerns.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 13 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
By setting the derivative of Eq.(2) w.r.t. f to zero, Eq.(2) has aclosed-form solution w.r.t. f as
By substituting Eq.(3) into Eq.(2),we have
By expanding the quadratic form in Eq.(4), it can be rewritten as:
Where F∈ Rb×b is a linear kernel matrix of fi’s,Fij = fTi fj, ∀1 ≤ i, j ≤ b andv= [2fT1 f0; ...; 2fTb f0]
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 14 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
By setting the derivative of Eq.(2) w.r.t. f to zero, Eq.(2) has aclosed-form solution w.r.t. f as
By substituting Eq.(3) into Eq.(2),we have
By expanding the quadratic form in Eq.(4), it can be rewritten as:
Where F∈ Rb×b is a linear kernel matrix of fi’s,Fij = fTi fj, ∀1 ≤ i, j ≤ b andv= [2fT1 f0; ...; 2fTb f0]
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 14 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
By setting the derivative of Eq.(2) w.r.t. f to zero, Eq.(2) has aclosed-form solution w.r.t. f as
By substituting Eq.(3) into Eq.(2),we have
By expanding the quadratic form in Eq.(4), it can be rewritten as:
Where F∈ Rb×b is a linear kernel matrix of fi’s,Fij = fTi fj, ∀1 ≤ i, j ≤ b andv= [2fT1 f0; ...; 2fTb f0]
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 14 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
By setting the derivative of Eq.(2) w.r.t. f to zero, Eq.(2) has aclosed-form solution w.r.t. f as
By substituting Eq.(3) into Eq.(2),we have
By expanding the quadratic form in Eq.(4), it can be rewritten as:
Where F∈ Rb×b is a linear kernel matrix of fi’s,Fij = fTi fj, ∀1 ≤ i, j ≤ b andv= [2fT1 f0; ...; 2fTb f0]
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 14 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 15 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
It is not hard to find that Eq.(4) is a geometric projectionproblem.Specifically, let Ω = f|
∑bi=1 αifi, α ∈ M,Eq.(4) can be
rewritten as,
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 16 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Representation to Geometric Projection
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 17 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Outline
1 IntroductionMotivationRelated Work
2 The Proposed MethodProblem Setting and FormulationRepresentation to Geometric ProjectionHow the Proposal Works
3 Experiments
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 18 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
Before going into the detail analysis,we notice from Figure 1 that ∥f − f∗∥should be smaller than ∥f0 − f∗∥ if f∗ ∈ Ω.Such an observation motivatesus to derive the following results.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 19 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
Before going into the detail analysis,we notice from Figure 1 that ∥f − f∗∥should be smaller than ∥f0 − f∗∥ if f∗ ∈ Ω.Such an observation motivatesus to derive the following results.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 19 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
To understand our proposal more comprehensively, in the following weinvestigate how the performance of our proposal is affected when thecondition previously mentioned is violated.
Specifically,let λ∗ = [λ∗1, ..., λ
∗b] ∈ M be the optimal solution of the
following functional:
and ϵ be the residual, i.e., ϵ = f∗ −∑b
i=1 λ∗i fi, reflecting the degree of
violation.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 20 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
To understand our proposal more comprehensively, in the following weinvestigate how the performance of our proposal is affected when thecondition previously mentioned is violated.Specifically,let λ∗ = [λ∗
1, ..., λ∗b] ∈ M be the optimal solution of the
following functional:
and ϵ be the residual, i.e., ϵ = f∗ −∑b
i=1 λ∗i fi, reflecting the degree of
violation.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 20 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
To understand our proposal more comprehensively, in the following weinvestigate how the performance of our proposal is affected when thecondition previously mentioned is violated.Specifically,let λ∗ = [λ∗
1, ..., λ∗b] ∈ M be the optimal solution of the
following functional:
and ϵ be the residual, i.e., ϵ = f∗ −∑b
i=1 λ∗i fi, reflecting the degree of
violation.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 20 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
Suppose fi’s are normalized into [0 : 1],we then have the followingresult for the proposed method.
Theorem 3 discloses that when the required safeness condition isviolated, the worst-case increased loss of our proposed method is onlyrelated to the norm of the residual (in other words, the quality ofregressors),and has nothing to do with other factors,e.g.,the quantityof regressors.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 21 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
Suppose fi’s are normalized into [0 : 1],we then have the followingresult for the proposed method.
Theorem 3 discloses that when the required safeness condition isviolated, the worst-case increased loss of our proposed method is onlyrelated to the norm of the residual (in other words, the quality ofregressors),and has nothing to do with other factors,e.g.,the quantityof regressors.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 21 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
How the Proposal Works
Suppose fi’s are normalized into [0 : 1],we then have the followingresult for the proposed method.
Theorem 3 discloses that when the required safeness condition isviolated, the worst-case increased loss of our proposed method is onlyrelated to the norm of the residual (in other words, the quality ofregressors),and has nothing to do with other factors,e.g.,the quantityof regressors.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 21 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 22 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 23 / 24
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
...
.
Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou ( Lamda,Nanjing University;AAAI 2017 )Learning Safe Prediction for Semi-Supervised Regression 陈永恒,2017.5.22 24 / 24
![Geriatry Prediction[1]](https://img.pdfslide.tips/doc/110x75/577ce6bb1a28abf103936e6e/geriatry-prediction1.jpg)
