apache lucene informacijos paieška
DESCRIPTION
Lucene Informacijos paieškaTRANSCRIPT
![Page 1: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/1.jpg)
Informacijos paieškaTrumpas įvadas į Apache Lucene
Mindaugas ŽakšauskasVilnius Kaunas Java User Group
![Page 2: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/2.jpg)
assert this != null
![Page 3: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/3.jpg)
Kalbėsiu apie● Probleminę sritį● Informacijos paieškos teoriją● Lucene● Solr (jei liks laiko)● ?
![Page 4: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/4.jpg)
Who are we?
![Page 5: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/5.jpg)
![Page 6: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/6.jpg)
![Page 7: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/7.jpg)
![Page 9: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/9.jpg)
Sukaposiu gabalais (tokenization)
Dokumentas #1: "The quick brown fox jumps over the lazy dog.”
Word the quick brown fox
Documents 1 1 1 1
Offsets 1[0] 1[4] 1[10] 1[16]
![Page 10: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/10.jpg)
Sukaposiu gabalais (tokenization) #2Dokumentas #2: "I saw a brown fox yesterday. It ran away quickly.”
Word the quick brown fox
Documents 1 1 1, 2 1, 2
Offsets 1[0] 1[4] 1[10], 2[8] 1[16], 2[13]
![Page 11: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/11.jpg)
Stopwords
Anglų:- artikeliai a, the
Lietuvių:- prielinksniai: į, nuo- ištiktukai: oi! - ?
![Page 12: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/12.jpg)
![Page 13: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/13.jpg)
Quick| ⇒ quick
quick|ly ⇒ quick
Stemming, lowercasing
![Page 14: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/14.jpg)
Apache Lucene● ACID (+2 phase commit)● NoSQL (rimtai!)● Concurrency● Java (.NET, Python, Ruby)● Bendruomenė ● Plačiai naudojama
![Page 15: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/15.jpg)
Ne caro laikų Lucene!
Indeksavimo greitis, Lucene v4
![Page 16: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/16.jpg)
![Page 17: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/17.jpg)
Sinonimai● Vanduo - H2O● Reikia spec. žodyno (SynonymMap)
quick brown fox ⇒ "quick", "fast", "brown", "fox"
![Page 18: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/18.jpg)
Fonetinis kodavimashttp://en.wikipedia.org/wiki/Metaphone
0BFHJKLMNPRSTWXY
Stephen Smith ⇒STFN SM0
![Page 19: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/19.jpg)
![Page 20: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/20.jpg)
Boosting (indexing, query)
![Page 21: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/21.jpg)
![Page 22: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/22.jpg)
Užklausos● field: foo bar● field: +foo -bar● field: “foo bar”● field: +“foo bar” AND blah● field: f?o bar*● field: foo~ bar~0.8● date_field: [2000 TO 2001]● field: (foo AND bar) OR bob
http://searchhub.org/dev/2011/12/28/why-not-and-or-and-not/
![Page 23: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/23.jpg)
Rezultatų formulė
org.apache.lucene.search.similarities.TFIDFSimilarity
![Page 24: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/24.jpg)
Finite state transducer
mop, moth, pop, star, stop, top
10 million Wikipedia index - 69Mb
![Page 25: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/25.jpg)
![Page 26: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/26.jpg)
![Page 27: Apache Lucene Informacijos paieška](https://reader034.vdocuments.mx/reader034/viewer/2022042521/555c251cd8b42a09438b4bdf/html5/thumbnails/27.jpg)