Upload File

semisupervised learning a brief introduction. semisupervised learning introduction types of...

  • Home
  • Documents
  • Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References
36
Semisupervis ed Learning A brief introduction

Upload: rosalyn-greene

Post on 30-Dec-2015

242 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning

A brief introduction

Page 2: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning

• Introduction• Types of semisupervised learning• Paper for review• References

Page 3: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Introduction

This is an extension to supervised learning. We have two sets of data:

Motivation: labeled data is sometimes hard to obtain.

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 4: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

An example from Mars Data Analysis

Digital Elevation MapGeomorphic Map

Martian landscape

Manually drawn geomorphic map of this landscape

Geomorphic map shows landforms chosen and defined by a domain expert.

Page 5: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Segmentation

Page 6: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Segmentation: Results.

Displayed on an elevation background.

2631 segments homogeneous in slope, curvature and flood.

Page 7: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Classification: Labeling.

A representative subset of objects are labeled as one of the following six classes:o Plaino Crater Flooro Convex Crater Wallso Concave Crater Wallso Convex Ridgeso Concave Ridges

Labeled segments.

Page 8: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Introduction

How can we learn from unlabeled data at all?

The answer lies in the set of assumptions about theunlabeled data distribution.

If assumptions are right, an advantage can be obtainedusing unlabeled data

But a decrease in performance is possible if assumptions are incorrect.

Page 9: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Introduction

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 10: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Introduction

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 11: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning

• Introduction• Types of semisupervised learning• Paper for review• References

Page 12: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Types of semi-supervised learning:

• Self-Training

• Generative Models

• Graph-Based Algorithms

• Multi-View Algorithms

• SVMs

Page 13: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Self-Training

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 14: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Self-Training Variations

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 15: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 16: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 17: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 18: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 19: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 20: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 21: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 22: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Graph-Based Models

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 23: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: TypesGraph-Based Models

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 24: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Multi-View Algorithms

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 25: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Multi-View Algorithms

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 26: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

SVMs

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 27: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

Figure obtained from X. Zhu. Semi-Supervised Learning Tutorial. ICML 2007

Page 28: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Types

A comparison of different approaches:

Self-Training: Method is simple, but early mistakes propagate down and can be very harmful.

Generative Models: great method if prob. model is correct, but it isdifficult to verify model correctness. In fact, unlabeled data may yielda decrease in accuracy if model is wrong.

Graph-Based Models: good solid mathematical solution if a graph is a good representative of the data distribution.

Multi-view Method: simple method that is less sensitive to errors inclassification, but there may not be a natural split of the features. SVMs: it can be used wherever SVM is applicable; but may fall into a local maxima, and optimization is hard.

Page 29: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning

• Introduction• Types of semisupervised learning• Paper for review• References

Page 30: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Paper

Unlabeled data: Now it helps, now it doesn’t by Singh, et. al.

Problem: analyze when semi-supervised learning helps to improve generalizationperformance.

Figure obtained from: Singh, et. al. Unlabeled data: now it helps, now it doesn’t. NIPS (2008).

Page 31: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Paper

Unlabeled data: Now it helps, now it doesn’t by Singh, et. al.

Some terminology that is necessary for the paper follows:

“The cluster assumption” means that the distributions of classes in the feature space is smooth on each set D ∈ D. The sets in D are called decision sets.

Figure obtained from: Singh, et. al. Unlabeled data: now it helps, now it doesn’t. NIPS (2008).

Page 32: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Paper

Unlabeled data: Now it helps, now it doesn’t by Singh, et. al.

Figure obtained from: Singh, et. al. Unlabeled data: now it helps, now it doesn’t. NIPS (2008).

Page 33: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Paper

Unlabeled data: Now it helps, now it doesn’t by Singh, et. al.

Text obtained from: Singh, et. al. Unlabeled data: now it helps, now it doesn’t. NIPS (2008).

Main Result: “…if the sets D are discernible using unlabeled data (the margin is large enough compared to average spacing between unlabeled data points), then there exists a semi-supervised learner that can perform as well as a supervised learner with clairvoyant knowledge of the decision sets, provided m ≫ n…”

Page 34: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning: Paper

Unlabeled data: Now it helps, now it doesn’t by Singh, et. al.

Figure obtained from: Singh, et. al. Unlabeled data: now it helps, now it doesn’t. NIPS (2008).

Page 35: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning

• Introduction• Types of semisupervised learning• Paper for review• References

Page 36: Semisupervised Learning A brief introduction. Semisupervised Learning Introduction Types of semisupervised learning Paper for review References

Semisupervised Learning


Learning Introduction

SemiSupervised: Scalable Semi-Supervised Routines …...The literature is replete with semi-supervised learning tech niques including greedy graph cut approaches [31], logistic tree

Laplacian Welsch Regularization for Robust Semisupervised

Machine Learning: Machine Learning: Introduction Introduction

An Introduction to Machine Learning - GitHub Pages · Introduction Machine learning The machine learning process What’s machine learning History Supervised learning Non-supervised

Label propagation - Semisupervised Learning with Applications to NLP

Distance Metric Learning: A Comprehensive Surveyrongjin/semisupervised/dist-metric-survey.pdf · distance metric learning methods is presented in section 4. In section 5, we will

Semisupervised Multiview Distance Metric Learning for Cartoon Synthesis

Clustering UNSUPERVISED LEARNING INTRODUCTION Machine Learning

A Lazy Man's Approach to Benchmarking: Semisupervised ......A Lazy Man’s Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration Peter Welinder*‡ Max

Learning Organisations. Introduction Introduction Concept and Definition Concept and Definition Learning Organisation - An Active Philosophy Learning

Research Article Semisupervised Learning Based Disease …downloads.hindawi.com/journals/bmri/2016/3594937.pdf · 2019. 7. 30. · models for extracting the relations between the

Research Article Semisupervised Learning Based …downloads.hindawi.com/journals/acisc/2013/910706.pdfHowever, it is important to note that the task of sum-marizing online product

A Semisupervised Learning Method to Merge Search Engine Results

Semisupervised Learning on Heterogeneous Graphs and its ... · Graph-based semi-supervised learning is widely used in network analysis, for prediction/clustering tasks over nodes

Machine Learning - University of Calgarypages.cpsc.ucalgary.ca/~mrichter/ML/ML 2010/Semisupervised Learnin… · Machine Learning Michael M. Richter ... consists of genes, blocks

July 7–10, 2017 Oudemanhuispoort 4–6 University of ...15:10 Effective Semisupervised Learning on Manifolds. Amir Globerson, Roi Livni and Shai Shalev-Shwartz. 15:20 Quadratic Upper

A Semisupervised Learning Method to Merge Search Engine ...callan/Papers/tois03-lsi.pdf · a more effective method of acquiring resource descriptions. Resource descrip-tions are created

600 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA … · Learning Semi-Riemannian Metrics for Semisupervised Feature ... 1041-4347/11/$26.00 2011 IEEE Published by the ... METRICS FOR SEMISUPERVISED

Semisupervised Learning on Heterogeneous Graphs and its … · 2018-07-06 · „ere are multiple factors that in…uence the ranking of a story on a person’s News Feed. A comprehensive

Semisupervised Learning for Computational Linguisticsmooney/cs388/slides/baldridge_semisup_nlp.pdf · Semisupervised Learning for Computational Linguistics Natural Language Processing

Presentation ISNN 2011: Semisupervised Learning Guided by the Modularity Measure in Complex Networks

214 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE … · assessment, remotely sensed (RS) imagery, semilabeled sam-ples, semisupervised learning, supervised learning, unsupervised learning

A semisupervised segmentation model for collections of images.bak

Semisupervised SVM Based on Cuckoo Search Algorithm and Its … · 2019. 7. 30. · ResearchArticle Semisupervised SVM Based on Cuckoo Search Algorithm and Its Application ZipingHe

Crystallization classification semisupervised

Learning Layers Introduction - Informal Learning @ Work

A Semisupervised Learning Method to Merge Search …callan/Papers/tois03-lsi.pdfA Semisupervised Learning Method to Merge Search Engine Results LUO SI and JAMIE CALLAN Carnegie Mellon

Learning Semi-Riemannian Metrics for Semisupervised Feature Extraction

A Semisupervised Learning Method to Merge Search …callan/Projects/P2P/Papers/tois03-lsi.pdf · A Semisupervised Learning Method to Merge Search Engine Results LUO SI and JAMIE CALLAN

Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning

On efficient large margin semisupervised learning: method ...HL65462 and 1R01GM081535-01, and a UM AHC FRD grant. 1 1 Introduction Semisupervised learning occurs in classification,

Learning: cosa cambia AI, Machine Learning & Deep · Outline History of AIs Introduction to artificial intelligence Introduction to machine learning Introduction to deep learning

Application of three graph Laplacian based semisupervised learning methods to protein function prediction problem

Machine Learning - 北京大学数学科学学院 · Semisupervised learning: between supervised & unsupervised learning Reinforcement learning : occasional rewards – tries to maximize