# jeffreys' and bdeu priors for model selection

Post on 29-Jan-2018

225 views

Category:

## Science

Embed Size (px)

TRANSCRIPT

1. 1. Jeffreys' and BDeu Priors for Model Selection WITMSE 2016 Helsinki, Finland, September 20 Joe Suzuki (prof-joe) Joe Suzuki (Osaka Univ., Japan)
2. 2. Goal and Contributions [Goal] Compare for model selection BDeu (Bayesian Dirichlet equivalent uniform) Jeffreys prior (T-K estimator) [Contribution] Mathematically Proves
3. 3. Road Map 1. Bayesian Dirichlet Scores 2. BDeu and Jeffreys Scores 3. A Found Property and its Proof 4. Main Theorem 5. Regularity in Model Selection 6. Summary
4. 4. Assign a Prob. to each Seq.
5. 5. Express a Prob. by the product of Cond. Probs.
6. 6. Simultaneous Probs.
7. 7. Cond. Probs.
8. 8. BDeu and Jeffreys Prior
9. 9. Example 1 : Bayesian Network Structure Learning (BNSL)
10. 10. Example 2: Independence Testing
11. 11. A Motivating Example
12. 12. A Found Property
13. 13. Sketch of J(n)>0 for BDeu
14. 14. Sketch of J(n)0 for Jeffreys
15. 15. An Intuitive Reasoning
16. 16. Main Theorem
17. 17. Examples more likely unlikely
18. 18. Regularity in Model Selection Fitness + Simplicity optimal (-1) x Likelihood + Penalty Term min Newtons Law of Motion Maxwell Equations If model A is better than model B w.r.t. fitness and simplicity, model A should be chosen (regularity). Information Criteria LASSO
19. 19. BDeu violates regularity in model selection Z XZ X Y Y X
20. 20. B&B for efficient BNSL (Depth First Search)
21. 21. Those bounds utilize regularity Campos and Ji 2011 figured out one (=nice) but the bound is not efficient (experiments). Designing Pruning rules for BDeu is HARDer. because regularity cannot be assumed
22. 22. Bayes Prior Based on his/her Belief: Nobody should reject it from a general point of view. BDeu violates regularity contradicts with Newton, Maxwell, Information Critreria, LASSO, etc. People might notice that their beliefs have been wrong, after knowing the new result in this paper.
23. 23. Summary The prior behind BDeu might have been based on a wrong belief That contradicts regularity in model selection Future Work: Consider NML and others in a similar way