![Page 1: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/1.jpg)
Query Languages
J. H. WangMar. 25, 2008
![Page 2: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/2.jpg)
The Retrieval ProcessUserInterface
Text Operations
Query Operations
Indexing
Searching
Ranking
Index
Text
query
user need
user feedback
ranked docs
retrieved docs
logical viewlogical view
inverted file
DB Manager Module
4, 10
6, 7
5 8
2
8
Text Database
Text
![Page 3: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/3.jpg)
Outline
• Keyword-Based Querying• Pattern Matching• Structural Queries• Query Protocols• Trends and Research Issues
![Page 4: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/4.jpg)
Keyword-Based Querying
• A query is the formulation of a user information need
• Keyword-based queries are popular
1. Single-Word Queries2. Context Queries3. Boolean Queries4. Natural Language
Data Retrieval
Information Retrieval
![Page 5: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/5.jpg)
Single-Word Queries
• A query is formulated by a word• A document is formulated by long
sequences of words• A word is a sequence of letters
surrounded by separators• What are letters and separators?
– e.g, ’on-line’ • The division of the text into words is
not arbitrary
![Page 6: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/6.jpg)
Context Queries
• Definition - Search words in a given context• Types
– Phrase• a sequence of single-word queries• e.g, ‘… enhance the retrieval …’
– Proximity• a sequence of single words or phrases, and a
maximum allowed distance between them are specified
• e.g, within distance (enhance, retrieval, 4) will match ‘…enhance the power of retrieval…’
![Page 7: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/7.jpg)
Definition A syntax composed of atoms that retrieve documents,
and of Boolean operators which work on their operands
e.g, translation AND syntax OR syntactic
• Fuzzy Boolean– Retrieve documents appearing in some operands (The AND
may require it to appear in more operands than the OR)
Boolean Queries
![Page 8: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/8.jpg)
Natural Language
• Generalization of “fuzzy Boolean”• A query is an enumeration of words
and context queries• All the documents matching a portion
of the user query are retrieved
![Page 9: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/9.jpg)
Pattern Matching
• Data retrieval– A pattern is a set of syntactic features that
must occur in a text segment• Types
– Words– Prefixes
• e.g ‘comput’->’computer’ , ’computation’, ’computing’, etc
– Suffixes• e.g ‘ters’->’computers’, ’testers’, ’painters’, etc
– Substrings• e.g ‘tal’->’coastal’, ’talk’, ’metallic’, etc
– Ranges • between ‘held’ and ‘hold’ -> ’hoax’ and ‘hissing’
![Page 10: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/10.jpg)
Allowing Errors
• Retrieve all text words which are ‘similar’ to the given word
edit distance: the minimum number of character insertions,
deletions, and replacements needed to make two strings equal, e.q , ‘flower’ and ‘flo wer’
maximum allowed edit distance: the query specifies the maximum number of
allowed errors for a word to match the pattern
![Page 11: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/11.jpg)
Regular expressions
union: if e1 and e2 are regular expressions , then(e1|e2) matches what e1 or e2 matches
concatenation: if e1 and e2 are regular expressions, the occurrences of (e1e2) are formed by the occurrences of e1 immediately followed by those of e2
repetition: if e is a regular expression , then (e*) matches a sequence of zero or more contiguous occurrence of e
‘pro(blem|tein)(s|є)(0|1|2)*’ -> ’problem02’ and ‘proteins’
![Page 12: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/12.jpg)
Structural Queries
• Mixing contents and structure in queries
- contents: words, phrases, or patterns - structural constraints: containment,
proximity, or other restrictions on structural elements
• Three main structures - Fixed structure - Hypertext structure - Hierarchical structure
![Page 13: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/13.jpg)
Fixed Structure
Document: a fixed set of fieldsEx: a mail has a sender, a receiver, a date, a
subject and a body field
Search for the mails sent to a given person with “football” in the Subject field
![Page 14: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/14.jpg)
A hypertext is a directed graph where nodes hold some text (text contents)
the links represent connections between nodes or between positions inside nodes (structural connectivity)
Hypertext
![Page 15: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/15.jpg)
Hypertext : WebGlimpse
WebGlimpse: combine browsing and searching on the Web
![Page 16: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/16.jpg)
Hierarchical Structure
![Page 17: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/17.jpg)
Hierarchical Structure
![Page 18: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/18.jpg)
Hierarchical Structure
• PAT Expressions• Overlapped Lists• Lists of References• Proximal Nodes• Tree Matching
![Page 19: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/19.jpg)
Query Protocols
• Used automatically by software applications to query text database– Not intended for human use– “Protocols” rather than “languages”
• Z39.50• WAIS (Wide Area Information
Service)
![Page 20: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/20.jpg)
Z39.50
• American National Standard Information Retrieval Application Service Definition
• Can be implemented on any platform• Query bibliographical information
using a standard interface between the client and the host database manager
• Z39.50 protocol is part of WAIS
![Page 21: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/21.jpg)
Z39.50 Brief History
• Z39.50-1988(version 1)• Z39.50-1992(version 2)• Z39.50-1995(version 3)• Version 4, development began in
Autumn 1995
![Page 22: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/22.jpg)
Using Z39.50 over the WWW
WWW Client WWW Z39.50
Z39.50 Client
Z39.50Server
RepositoryDigital library
![Page 23: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/23.jpg)
WAIS (Wide Area Information Service)
• Beginning in the 1990s• Query databases through the
Internet
![Page 24: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/24.jpg)
Trends and Research Issues
Model Queries allowed
BooleanVectorProbabilisticBBN
word, set operationswordswordswords
Relationship between types of queries and models
![Page 25: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/25.jpg)
Query Language Taxonomy
The types of queries covered and how they are structured
![Page 26: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/26.jpg)
PAT Tree Expression
• The model allows for the areas of a region to overlap or nest
![Page 27: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/27.jpg)
Overlapped Lists
• The model allows for the areas of a region to overlap, but not to nest
• It is not clear, whether overlapping is good or not for capturing the structural properties
![Page 28: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/28.jpg)
Lists of References
• Overlap and nest are not allowed• All elements must be of the same
type, e.g only sections, or only paragraphs.
• A reference is a pointer to a region of the database.
![Page 29: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/29.jpg)
Proximal Nodes
• This model tries to find a good compromise between expressiveness and efficiency.
• It does not define a specific language, but a model in which it is shown that a number of useful operators can be included achieving good efficiency.
![Page 30: Query Languages J. H. Wang Mar. 25, 2008. The Retrieval Process User Interfac e Text Operations Query Operations Indexing Searching Ranking Index Text](https://reader036.vdocuments.mx/reader036/viewer/2022070412/5697bf741a28abf838c7f8cf/html5/thumbnails/30.jpg)
Tree Matching
• The leaves of the query can be not only structural elements but also text patterns, meaning that the ancestor of the leaf must contain that pattern.