determinantal point processesapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfdeterminantal point...
TRANSCRIPT
![Page 1: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/1.jpg)
DETERMINANTAL POINT PROCESSESFOR
NATURAL LANGUAGE PROCESSING
Jennifer GillenwaterJoint work with Alex Kulesza and Ben Taskar
![Page 2: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/2.jpg)
OUTLINE
![Page 3: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/3.jpg)
OUTLINE
Motivation & background on DPPs
![Page 4: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/4.jpg)
OUTLINE
Motivation & background on DPPs
Large-scale settings
![Page 5: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/5.jpg)
OUTLINE
Motivation & background on DPPs
Large-scale settings
Structured summarization
![Page 6: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/6.jpg)
OUTLINE
Motivation & background on DPPs
Large-scale settings
Structured summarization
Other potential NLP applications
![Page 7: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/7.jpg)
MOTIVATION &BACKGROUND
![Page 8: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/8.jpg)
SUMMARIZATION
![Page 9: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/9.jpg)
SUMMARIZATION
...
![Page 10: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/10.jpg)
SUMMARIZATION
...
![Page 11: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/11.jpg)
SUMMARIZATION
...
Quality: relevance to
the topic
![Page 12: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/12.jpg)
SUMMARIZATION
...
Quality: relevance to
the topic
Diversity: coverage of core ideas
![Page 13: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/13.jpg)
SUBSET SELECTION
![Page 14: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/14.jpg)
SUBSET SELECTION
![Page 15: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/15.jpg)
SUBSET SELECTION
![Page 16: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/16.jpg)
SUBSET SELECTION
![Page 17: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/17.jpg)
AREA AS SET-GOODNESS
![Page 18: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/18.jpg)
feature space
AREA AS SET-GOODNESS
![Page 19: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/19.jpg)
feature space
Bi
Bj
AREA AS SET-GOODNESS
![Page 20: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/20.jpg)
feature spacequality =
pB>
i Bi
similarity = B>i Bj
Bi
Bj
AREA AS SET-GOODNESS
![Page 21: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/21.jpg)
feature spacequality =
pB>
i Bi
similarity = B>i Bj
Bi +Bj
Bi
Bj
AREA AS SET-GOODNESS
area
=q kB
ik2 2kB
jk2 2�(B
>iBj)2
![Page 22: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/22.jpg)
feature spacequality =
pB>
i Bi
similarity = B>i Bj
Bi +Bj
Bi
Bj
AREA AS SET-GOODNESS
area
=q kB
ik2 2kB
jk2 2�(B
>iBj)2
![Page 23: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/23.jpg)
feature spacequality =
pB>
i Bi
similarity = B>i Bj
Bi +Bj
Bi
Bj
AREA AS SET-GOODNESS
area
=q kB
ik2 2kB
jk2 2�(B
>iBj)2
![Page 24: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/24.jpg)
VOLUME AS SET-GOODNESS
area =q
kBik22kBjk22 � (B>i Bj)2
![Page 25: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/25.jpg)
VOLUME AS SET-GOODNESS
area =q
kBik22kBjk22 � (B>i Bj)2
![Page 26: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/26.jpg)
VOLUME AS SET-GOODNESS
area =q
kBik22kBjk22 � (B>i Bj)2
length = kBik2
![Page 27: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/27.jpg)
VOLUME AS SET-GOODNESS
area =q
kBik22kBjk22 � (B>i Bj)2
length = kBik2 volume = base ⇥ height
![Page 28: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/28.jpg)
VOLUME AS SET-GOODNESS
area =q
kBik22kBjk22 � (B>i Bj)2
= ||B1||2vol(proj?B1(B2:N ))
vol(B) = height⇥ base
length = kBik2 volume = base ⇥ height
![Page 29: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/29.jpg)
AREA AS A DETarea =
qkBik22kBjk22 � (B>
i Bj)2
![Page 30: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/30.jpg)
AREA AS A DETarea =
qkBik22kBjk22 � (B>
i Bj)2
||Bi||22
B>i Bj
B>i Bj( )= det
||Bj ||22
12
![Page 31: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/31.jpg)
AREA AS A DETarea =
qkBik22kBjk22 � (B>
i Bj)2
||Bi||22
B>i Bj
B>i Bj( )= det
||Bj ||22
12
= det( )Bi
Bj Bi
Bj
12
![Page 32: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/32.jpg)
VOLUME AS A DET
= det( )Bi
Bj Bi
Bj
12
vol(B{i,j})
![Page 33: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/33.jpg)
VOLUME AS A DET
= det( )Bi
Bj Bi
Bj
12
vol(B{i,j})
vol(B) = det
12B1
BN
... B1
BN. . .( )
vol(B)
2= det(B>B) = det(L)
![Page 34: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/34.jpg)
COMPLEX STATISTICS
![Page 35: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/35.jpg)
COMPLEX STATISTICS
![Page 36: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/36.jpg)
COMPLEX STATISTICS
![Page 37: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/37.jpg)
COMPLEX STATISTICSP
![Page 38: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/38.jpg)
COMPLEX STATISTICSP
![Page 39: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/39.jpg)
COMPLEX STATISTICSP
![Page 40: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/40.jpg)
COMPLEX STATISTICSP
![Page 41: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/41.jpg)
COMPLEX STATISTICSP
![Page 42: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/42.jpg)
COMPLEX STATISTICSP
![Page 43: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/43.jpg)
COMPLEX STATISTICSP
![Page 44: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/44.jpg)
COMPLEX STATISTICSP
![Page 45: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/45.jpg)
COMPLEX STATISTICS
N items =) 2N sets
![Page 46: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/46.jpg)
EFFICIENT COMPUTATION
![Page 47: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/47.jpg)
det12
EFFICIENT COMPUTATION
![Page 48: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/48.jpg)
EFFICIENT COMPUTATION2
det
![Page 49: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/49.jpg)
EFFICIENT COMPUTATION2
P
det
![Page 50: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/50.jpg)
EFFICIENT COMPUTATION2
det
O(N3)
![Page 51: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/51.jpg)
POINT PROCESSESY = {1, . . . , N}
![Page 52: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/52.jpg)
POINT PROCESSESY = {1, . . . , N}
![Page 53: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/53.jpg)
POINT PROCESSESY = {1, . . . , N}
( )P
![Page 54: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/54.jpg)
POINT PROCESSESY = {1, . . . , N}
( )P = 0.2
![Page 55: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/55.jpg)
DETERMINANTAL
![Page 56: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/56.jpg)
DETERMINANTALP({2, 3, 5}) /
![Page 57: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/57.jpg)
DETERMINANTAL
L11 L12 L13 L14 L15
L21 L22 L23 L24 L25
L35L34L33L32L31
L41 L42 L43 L44 L45
L55L54L53L52L51
P({2, 3, 5}) /
![Page 58: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/58.jpg)
DETERMINANTAL
L11 L12 L13 L14 L15
L21 L22 L23 L24 L25
L35L34L33L32L31
L41 L42 L43 L44 L45
L55L54L53L52L51
P({2, 3, 5}) /
![Page 59: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/59.jpg)
DETERMINANTALL22 L23 L25
L35L33L32
L55L53L52
P({2, 3, 5}) /
![Page 60: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/60.jpg)
DETERMINANTALL22 L23 L25
L35L33L32
L55L53L52
det( )P({2, 3, 5}) /
![Page 61: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/61.jpg)
DETERMINANTALL22 L23 L25
L35L33L32
L55L53L52
det( )P({2, 3, 5}) =
![Page 62: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/62.jpg)
DETERMINANTALL22 L23 L25
L35L33L32
L55L53L52
det( )P({2, 3, 5}) =
det(L+ I)
![Page 63: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/63.jpg)
EFFICIENT INFERENCE
![Page 64: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/64.jpg)
EFFICIENT INFERENCE
PL(Y = Y )Normalizing:
![Page 65: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/65.jpg)
EFFICIENT INFERENCE
PL(Y = Y )
P(Y ✓ Y)
Normalizing:
Marginalizing:
![Page 66: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/66.jpg)
EFFICIENT INFERENCE
PL(Y = Y )
PL(Y = B | A ✓ Y)
PL(Y = B | A \Y = ;)
P(Y ✓ Y)
Normalizing:
Marginalizing:
Conditioning:
![Page 67: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/67.jpg)
EFFICIENT INFERENCE
PL(Y = Y )
PL(Y = B | A ✓ Y)
PL(Y = B | A \Y = ;)
Y ⇠ PL
P(Y ✓ Y)
Normalizing:
Marginalizing:
Conditioning:
Sampling:
![Page 68: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/68.jpg)
EFFICIENT INFERENCE
PL(Y = Y )
PL(Y = B | A ✓ Y)
PL(Y = B | A \Y = ;)
Y ⇠ PL
P(Y ✓ Y)
O(N3)
Normalizing:
Marginalizing:
Conditioning:
Sampling:
![Page 69: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/69.jpg)
LARGE-SCALE SETTINGS
![Page 70: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/70.jpg)
DUAL KERNELKULESZA AND TASKAR (NIPS 2010)
![Page 71: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/71.jpg)
DUAL KERNEL
B1
BN
...
B2
B3
B1
BN. . .B2
B3
LKULESZA AND TASKAR (NIPS 2010)
![Page 72: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/72.jpg)
DUAL KERNEL
B1
BN
...
B2
B3
B1
BN. . .B2
B3
L
N ⇥N
=
KULESZA AND TASKAR (NIPS 2010)
![Page 73: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/73.jpg)
DUAL KERNEL
B1
BN
...
B2
B3
B1
BN. . .B2
B3
C
N ⇥N
=
KULESZA AND TASKAR (NIPS 2010)
![Page 74: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/74.jpg)
DUAL KERNEL
B1
BN
...
B2
B3
B1
BN. . .B2
B3
C
=
KULESZA AND TASKAR (NIPS 2010)
![Page 75: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/75.jpg)
DUAL KERNEL
B1
BN
...
B2
B3
B1
BN. . .B2
B3
= D ⇥D
CKULESZA AND TASKAR (NIPS 2010)
![Page 76: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/76.jpg)
DUAL INFERENCE
![Page 77: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/77.jpg)
DUAL INFERENCE
L = V ⇤V > C = V̂ ⇤V̂ >
![Page 78: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/78.jpg)
DUAL INFERENCE
L = V ⇤V > C = V̂ ⇤V̂ >V = B>V̂ ⇤� 12
![Page 79: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/79.jpg)
DUAL INFERENCE
L = V ⇤V > C = V̂ ⇤V̂ >V = B>V̂ ⇤� 12
O(D3)P
Y det(LY )Normalizing
![Page 80: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/80.jpg)
DUAL INFERENCE
L = V ⇤V > C = V̂ ⇤V̂ >V = B>V̂ ⇤� 12
O(D3)P
Y det(LY )Normalizing
O(D3 +D2k2)Marginalizing & Conditioning
![Page 81: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/81.jpg)
DUAL INFERENCE
L = V ⇤V > C = V̂ ⇤V̂ >V = B>V̂ ⇤� 12
O(ND2k)Y ⇠ PLSampling
O(D3)P
Y det(LY )Normalizing
O(D3 +D2k2)Marginalizing & Conditioning
![Page 82: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/82.jpg)
EXPONENTIAL N
![Page 83: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/83.jpg)
EXPONENTIAL N
N = O({sentence length}{sentence length})
We want to select a diverse set of parses.
![Page 84: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/84.jpg)
EXPONENTIAL N
N = O({sentence length}{sentence length})
We want to select a diverse set of parses.
N = O({node degree}{path length})
![Page 85: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/85.jpg)
i =
STRUCTURE FACTORIZATIONKULESZA AND TASKAR (NIPS 2010)
![Page 86: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/86.jpg)
i =
STRUCTURE FACTORIZATION
Bi = q(i)�(i)
KULESZA AND TASKAR (NIPS 2010)
quality similarity
![Page 87: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/87.jpg)
i =
STRUCTURE FACTORIZATION
Bi = q(i)�(i)
i = {i↵}↵2F↵
c = 1
KULESZA AND TASKAR (NIPS 2010)
quality similarity
![Page 88: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/88.jpg)
i =
STRUCTURE FACTORIZATION
Bi = q(i)�(i)
i = {i↵}↵2F↵
c = 2
KULESZA AND TASKAR (NIPS 2010)
quality similarity
![Page 89: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/89.jpg)
i =
STRUCTURE FACTORIZATION
i = {i↵}↵2F
Bi =
Q↵2F
q(i↵)
��(i)
↵
c = 2
KULESZA AND TASKAR (NIPS 2010)
![Page 90: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/90.jpg)
i =
STRUCTURE FACTORIZATION
i = {i↵}↵2F
Bi =
Q↵2F
q(i↵)
� P↵2F
�(i↵)
�
↵
c = 2
KULESZA AND TASKAR (NIPS 2010)
![Page 91: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/91.jpg)
i =
STRUCTURE FACTORIZATION
i = {i↵}↵2F
Bi =
Q↵2F
q(i↵)
� P↵2F
�(i↵)
�
↵
c = 2
O(ND2k)Y ⇠ PL
KULESZA AND TASKAR (NIPS 2010)
![Page 92: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/92.jpg)
STRUCTURE FACTORIZATION
Bi =
Q↵2F
q(i↵)
� P↵2F
�(i↵)
�
M = R =
↵
c = 2
Y ⇠ PL O(D2k3 +Dk2M cR)
KULESZA AND TASKAR (NIPS 2010)
![Page 93: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/93.jpg)
STRUCTURE FACTORIZATION
Bi =
Q↵2F
q(i↵)
� P↵2F
�(i↵)
�
M = R =
↵
c = 2
Y ⇠ PL O(D2k3 +Dk2M cR)
KULESZA AND TASKAR (NIPS 2010)
M cR = 42 ⇤ 12 = 192 ⌧ N = 412 = 16,777,216
![Page 94: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/94.jpg)
LARGE FEATURE SETS?
![Page 95: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/95.jpg)
Large Exponential
Small dualdual +
structure
Large ? ?
LARGE FEATURE SETS?N = # of items
D=
#offeatures
![Page 96: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/96.jpg)
Large Exponential
Small dualdual +
structure
Large ? ?
LARGE FEATURE SETS?N = # of items
D=
#offeatures
![Page 97: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/97.jpg)
RANDOM PROJECTIONSGILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
![Page 98: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/98.jpg)
RANDOM PROJECTIONS
N
D
�
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
![Page 99: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/99.jpg)
RANDOM PROJECTIONS
D
�
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
M cR
![Page 100: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/100.jpg)
RANDOM PROJECTIONS
D d
D⇥�
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
M cR
![Page 101: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/101.jpg)
RANDOM PROJECTIONS
D d
D⇥
d
=�
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
M cR M cR
![Page 102: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/102.jpg)
RANDOM PROJECTIONSd
D
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
![Page 103: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/103.jpg)
RANDOM PROJECTIONSd
D
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
![Page 104: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/104.jpg)
VOLUME PRESERVATION
![Page 105: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/105.jpg)
VOLUME PRESERVATIONJOHNSON AND LINDENSTRAUSS (1984)
![Page 106: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/106.jpg)
VOLUME PRESERVATIONJOHNSON AND LINDENSTRAUSS (1984)
logN
![Page 107: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/107.jpg)
VOLUME PRESERVATIONJOHNSON AND LINDENSTRAUSS (1984)
logN
![Page 108: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/108.jpg)
VOLUME PRESERVATIONMAGEN AND ZOUZIAS (2008)
logN
![Page 109: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/109.jpg)
VOLUME PRESERVATIONMAGEN AND ZOUZIAS (2008)
logN
![Page 110: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/110.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATIONvol
2= det
![Page 111: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/111.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
⇡
k = 1
vol
2= det
![Page 112: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/112.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
⇡
k = 1
⇡
k = 2
vol
2= det
![Page 113: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/113.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
⇡
k = 1
⇡
k = 2
⇡
k = 3
vol
2= det
![Page 114: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/114.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
![Page 115: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/115.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
d = O⇣
max
n
k✏ ,
log(1/�)+log(N)
✏2 + ko⌘
![Page 116: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/116.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
d = O⇣
max
n
k✏ ,
log(1/�)+log(N)
✏2 + ko⌘
total # of itemssubset size
![Page 117: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/117.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
d = O⇣
max
n
k✏ ,
log(1/�)+log(N)
✏2 + ko⌘
w.p. 1� � : kPk � P̃kk1 e6k✏ � 1
total # of itemssubset size
![Page 118: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/118.jpg)
GILLENWATER, KULESZA, AND TASKAR (EMNLP 2012)
DPP PRESERVATION
d = O⇣
max
n
k✏ ,
log(1/�)+log(N)
✏2 + ko⌘
0 50 100 1500
0.2
0.4
0.6
0.8
1
1.2
L1 v
aria
tiona
l dis
tanc
e
Projection dimension
0
1
2
3
4x 108
Mem
ory
use
(byt
es)
w.p. 1� � : kPk � P̃kk1 e6k✏ � 1
total # of itemssubset size
![Page 119: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/119.jpg)
STRUCTUREDSUMMARIZATION
![Page 120: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/120.jpg)
NEWS THREADING
![Page 121: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/121.jpg)
NEWS THREADING
![Page 122: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/122.jpg)
NEWS THREADING
March 28: Health officials confirm Ebola outbreak in Guinea’s capital
![Page 123: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/123.jpg)
NEWS THREADING
March 28: Health officials confirm Ebola outbreak in Guinea’s capital
August 8: World Health Organization declares Ebola epidemic an
international health emergency
![Page 124: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/124.jpg)
NEWS THREADING
March 28: Health officials confirm Ebola outbreak in Guinea’s capital
August 8: World Health Organization declares Ebola epidemic an
international health emergency
September 2: GlaxoSmithKlein begins Ebola vaccine drug trial
![Page 125: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/125.jpg)
NEWS THREADING
10360
March 28: Health officials confirm Ebola outbreak in Guinea’s capital
August 8: World Health Organization declares Ebola epidemic an
international health emergency
September 2: GlaxoSmithKlein begins Ebola vaccine drug trial
M ⇡ 35,000
![Page 126: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/126.jpg)
PROJECTING NEWS FEATURES
![Page 127: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/127.jpg)
PROJECTING NEWS FEATURES
�(i)
D = 36,356
![Page 128: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/128.jpg)
PROJECTING NEWS FEATURES
G
G�(i)�(i)
D = 36,356 d = 50
![Page 129: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/129.jpg)
30
![Page 130: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/130.jpg)
31
![Page 131: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/131.jpg)
DPP THREADS
![Page 132: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/132.jpg)
DPP THREADS
Jan 08 Jan 28 Feb 17 Mar 09 Mar 29 Apr 18 May 08 May 28 Jun 17
pope vatican church parkinson
israel palestinian iraqi israeli gaza abbas baghdad
owen nominees senate democrats judicial filibusters
social tax security democrats rove accounts
iraq iraqi killed baghdad arab marines deaths forces
![Page 133: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/133.jpg)
DPP THREADS
Jan 08 Jan 28 Feb 17 Mar 09 Mar 29 Apr 18 May 08 May 28 Jun 17
pope vatican church parkinson
israel palestinian iraqi israeli gaza abbas baghdad
owen nominees senate democrats judicial filibusters
social tax security democrats rove accounts
iraq iraqi killed baghdad arab marines deaths forces
Feb 24: Parkinson's Disease Increases Risks to PopeFeb 26: Pope's Health Raises Questions About His Ability to LeadMar 13: Pope Returns Home After 18 Days at HospitalApr 01: Pope's Condition Worsens as World Prepares for End of PapacyApr 02: Pope, Though Gravely Ill, Utters Thanks for PrayersApr 18: Europeans Fast Falling Away from ChurchApr 20: In Developing World, Choice [of Pope] Met with SkepticismMay 18: Pope Sends Message with Choice of Name
![Page 134: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/134.jpg)
System k-means DTM DPP
ROUGE-1F 16.5 14.7 17.2
R-SU4F 3.76 3.44 3.98
Coherence 2.73 3.2 3.3
QUANTITATIVE RESULTS
![Page 135: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/135.jpg)
System k-means DTM DPP
ROUGE-1F 16.5 14.7 17.2
R-SU4F 3.76 3.44 3.98
Coherence 2.73 3.2 3.3
QUANTITATIVE RESULTS
![Page 136: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/136.jpg)
System k-means DTM DPP
ROUGE-1F 16.5 14.7 17.2
R-SU4F 3.76 3.44 3.98
Coherence 2.73 3.2 3.3
QUANTITATIVE RESULTS
![Page 137: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/137.jpg)
System k-means DTM DPP
ROUGE-1F 16.5 14.7 17.2
R-SU4F 3.76 3.44 3.98
Coherence 2.73 3.2 3.3
QUANTITATIVE RESULTS
![Page 138: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/138.jpg)
System k-means DTM DPP
ROUGE-1F 16.5 14.7 17.2
R-SU4F 3.76 3.44 3.98
Coherence 2.73 3.2 3.3
Runtime (s) 626 19,434 252
QUANTITATIVE RESULTS
![Page 139: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/139.jpg)
OTHER POTENTIALNLP APPLICATIONS
![Page 140: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/140.jpg)
POTENTIAL APP: RE-RANKING
![Page 141: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/141.jpg)
• Parser: simple model with local features defines basic scores for all possible parse trees
POTENTIAL APP: RE-RANKING
![Page 142: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/142.jpg)
• Parser: simple model with local features defines basic scores for all possible parse trees
• Re-ranker: more complex model with non-local features provides more refined scores
POTENTIAL APP: RE-RANKING
![Page 143: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/143.jpg)
• Parser: simple model with local features defines basic scores for all possible parse trees
• Re-ranker: more complex model with non-local features provides more refined scores
• Typical pipeline: find the k highest-scoring parses under the simple model, then score these k with the more complex model and output the best
POTENTIAL APP: RE-RANKING
![Page 144: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/144.jpg)
• Parser: simple model with local features defines basic scores for all possible parse trees
• Re-ranker: more complex model with non-local features provides more refined scores
• Typical pipeline: find the k highest-scoring parses under the simple model, then score these k with the more complex model and output the best
• Issue: the k may be largely redundant, so re-ranker does not get to consider significantly different parses
POTENTIAL APP: RE-RANKING
![Page 145: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/145.jpg)
IDEA: USE DPPS FORSELECTING RE-RANKER INPUT
![Page 146: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/146.jpg)
IDEA: USE DPPS FORSELECTING RE-RANKER INPUT
N = O({sentence length}{sentence length})
We want to select a diverse set of parses.
![Page 147: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/147.jpg)
IDEA: USE DPPS FORSELECTING RE-RANKER INPUT
N = O({sentence length}{sentence length})
We want to select a diverse set of parses.
Quality: standard
parser scores
![Page 148: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/148.jpg)
IDEA: USE DPPS FORSELECTING RE-RANKER INPUT
N = O({sentence length}{sentence length})
We want to select a diverse set of parses.
Quality: standard
parser scores
Diversity: edge lengths,
POS pairs, etc.
![Page 149: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/149.jpg)
POTENTIAL APP:WORD SENSE INDUCTION
![Page 150: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/150.jpg)
POTENTIAL APP:WORD SENSE INDUCTION• Goal: identify all possible senses of ambiguous
words (e.g. river bank vs bank deposit)
![Page 151: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/151.jpg)
POTENTIAL APP:WORD SENSE INDUCTION• Goal: identify all possible senses of ambiguous
words (e.g. river bank vs bank deposit)
• Typical approach: unsupervised clustering, cluster centers represent word senses
![Page 152: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/152.jpg)
POTENTIAL APP:WORD SENSE INDUCTION• Goal: identify all possible senses of ambiguous
words (e.g. river bank vs bank deposit)
• Typical approach: unsupervised clustering, cluster centers represent word senses
• Why DPPs fit: can re-express finding cluster centers as the problem of finding a high-quality, diverse set
![Page 153: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/153.jpg)
POTENTIAL APP:WORD SENSE INDUCTION• Goal: identify all possible senses of ambiguous
words (e.g. river bank vs bank deposit)
• Typical approach: unsupervised clustering, cluster centers represent word senses
• Why DPPs fit: can re-express finding cluster centers as the problem of finding a high-quality, diverse set
Quality: centrality (density of points nearby)
![Page 154: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/154.jpg)
POTENTIAL APP:WORD SENSE INDUCTION• Goal: identify all possible senses of ambiguous
words (e.g. river bank vs bank deposit)
• Typical approach: unsupervised clustering, cluster centers represent word senses
• Why DPPs fit: can re-express finding cluster centers as the problem of finding a high-quality, diverse set
Quality: centrality (density of points nearby)
Diversity: same as standard
WSI features
![Page 155: DETERMINANTAL POINT PROCESSESapparikh/nips2014ml-nlp/slides/dpps-nips2014.pdfDETERMINANTAL POINT PROCESSES FOR NATURAL LANGUAGE PROCESSING Jennifer Gillenwater Joint work with Alex](https://reader034.vdocuments.mx/reader034/viewer/2022042402/5f10c4ad7e708231d44ab954/html5/thumbnails/155.jpg)
QUESTIONS?