Datu bāzes sistēmas paplašināšanas iespējas(data base system extensebility)

Nākotnes vēlmes un pilnveidojumi realizācijas paplašināšana.

Paplašinājumiem jānodrošina:1) jaunas funkcionalitātes pievienošana (new functionality);2) esošās funkcionalitātes modifikācija (modification of existing

functionality);3) pamatarhitektūras saglabāšana;4) minimālas koda korekcijas.

Paplašinājumam ir minimāla ietekme uz eksistējošo sistēmu.

2

Paplašinājumu veidošanas mehānismiClassification of extensibility mechanisms

There are three different forms of software extensibility:1) white-box extensibility. A software system can be extended by

modifying the source code, and it is the most flexible and the least restrictive form. There are two sub-forms of extensibility:

open-box extensibility. Original source code is directly being hacked into. It requires available source code and the modification permitted source code license.

glass-box extensibility. Glass-box extensibility (also called architecture driven frameworks) allows a software system to be extended with available source code, but may not allow the code to be modified. Extensions have to be separated from the original system in a way that the original system is not affected. One example of this form of extensibility is object-oriented application frameworks which achieve extensibility typically by using inheritance and dynamic binding.

2) gray-box extensibility. Gray-box extensibility is a compromise between a pure white-box and a pure black-box approach, which does not rely fully on the exposure of source code. Programmers could be given the system’s specialization interface which lists all available abstractions for refinement and specifications on how extensions should be developed.

3) black-box extensibility. In black-box extensibility (also called data-driven frameworks) no details about a system’s implementation are used for implementing deployments or extensions; only interface specifications are provided. This type of approach is more limited than the various white-box approaches. Black-box extensions are typically achieved through system configuration applications or the use of application-specific scripting languages by defining components interfaces.

3

Datu bāzes sistēmas paplašinājumi

1. User-defined types - the ability to define text, image and location datatypes. 2. Storage of user-defined type instances - the ability to store and manipulate multi-media type instances. 3. Domain-specific operations - support for user-defined functions/operators like Contains(), Distance(). 4. Domain-specific indexing - support for indexes specific to text data (to evaluate Contains()), spatial data (Distance()) etc., which can be used to speed the query. 5. Optimizer extensibility - support for intelligent ordering of query predicates during evaluation. Since all these operators and functions are user-defined, the optimizer has to be extended to allow type-designers to specify the costs of various operations on the types. In each case where a service is an extensible one, a interface or API provides access to the service.

4

DBS Oracle kodola paplašinājumi (data cartridges)

Data cartridges extend the capabilities of the Oracle server by taking advantage of Oracle Extensibility Architecture framework. This framework lets you capture business logic and processes associated with specialized or domain-specific data in user-defined data types. Data cartridges that provide new behavior without needing additional attributes have the option of using packages rather than user-defined types.

You can customize the indexing and query optimization mechanisms of an extensible database management system and provide specialized services or more efficient processing for user-defined business objects and rich types. When you register your implementations with the server through extensibility interfaces, you direct the server to implement your customized processing instructions instead of its own default processes.The extensibility interfaces consist of functions that the server calls to execute the custom indexing or optimizing behavior implemented for a data cartridge. The interfaces are defined by Oracle; as a cartridge developer, you must implement the functions or interfaces that have the specialized behavior you require in your application. In general, you implement the functions as static methods of an object type. An object type that implements the extensible indexing interface is called an indextype; an object type that implements the extensible optimizing interface is called a statistics type.

5

Data cartridges have the following key characteristics:1. Data cartridges are server-based. Their constituents reside on the server or are accessed from the server. The server runs all data cartridge processes, or dispatches these processes as external procedures.2. Data cartridges extend the server. They define new types and behavior, enabling the server to perform processes that were are otherwise unavailable to it, in component form. Data cartridges can use these new types and behaviors in their applications.3. Data cartridges are integrated with the server. The Oracle Extensibility Framework defines a set of interfaces that integrate data cartridges with the components of the server engine, allowing for domain-specific indexing, domain-specific optimized access to the the CPU resources, and domain-specific optimization of I/O access to cartridge data.4. Data cartridges are packaged. A data cartridge is installed as a unit. Once installed, the data cartridge handles all access issues for each user, including verification of schemas and privileges.

6

DBVS interfeisi un servisi

DBVS Oracle nodrošināja lietotāju ar plašu interfeisu klāstu visdažādāko DBVS kodola paplašināšanas iespēju realizācijai. Noteiktā veidā realizējot iespējamos interfeisus, lietotājs var definēt visdažādākās funkcijas, kuras izpildīs serveris, apstrādājot problēmsfēras datus. Programmēšanas gaitā ir pieejams noteikts datu bāzes servisu klāsts, kas dod iespēju mijiedarboties ar datu bāzes kodolu (darboties ar paplašināmo tipu sistēmu, konstruēt un izpildīt vaicājumus, mijiedarbība starp dažādām programmēšanas valodām u.c.) Kodola paplašinājumu vienkāršoti var uztvert kā neatkarīgi izstrādājamu programmu (bieži vien arī dažādās valodās), kas, pateicoties Oracle paplašināšanas arhitektūrai, tiek cieši integrēta ar kodolu un paplašina to.

Būtiskākais no paplašināšanas aspektiem ir paplašināmās indeksēšanas arhitektūra, kas ir pielāgojama problēmsfēras specifiskajām DBVS lietojuma prasībām. Paplašināmās indeksēšanas mehānisms dod iespēju izjaukt barjeru, kas līdz šim radīja grūtības liela apjoma sarežģītu struktūru datu efektīvai glabāšanai un apstrādei datu bāzē.

7

Extensibility Interfaces

DBMS InterfacesThe DBMS interfaces offer the simplest kind of extensibility services. They can be used through extensions to SQL or to the Oracle Call Interface (OCI). For example, the extensible type manager uses the CREATE TYPE syntax in SQL. Similarly, extensible indexing uses DDL and DML support for specifying and manipulating indexes.

Cartridge Basic Service InterfacesCartridge basic interfaces provide generic services like memory management, context management, internationalization, and cartridge-specific management. They implement behavior for new datatypes in the context of the server's execution environment and provide routines that help developers to implement portable and robust server-side methods.

Data Cartridge InterfacesWhen processing user-defined indextypes, Oracle calls data cartridge functions to perform index search or fetch operations. For user-defined query optimization, the query optimizer calls functions implemented by the data cartridge to compute the cost of user-defined operators or functions.

The database components that make up each cartridge must be installed in a schema that has the same name as the cartridge. If a cartridge uses multiple schemas, the first 10 characters of each schema name must be the same as the cartridge name. Note that the length of schema names in Oracle is limited to 30 bytes, or 30 characters in a single-byte language.

The database components of a data cartridge that must be placed in the cartridge schema include names for types, tables, views, directories, libraries and packages. Because the schema name and username are always the same in Oracle, the choice of a schema name determines the username.

8

Oracle extended architecture

create type ... create type body...

ODCIGet Interfaces() ODCIIndexCreate() ODCIIndexDrop() ODCIIndexStart() ODCIIndexFetch() ODCIIndexClose() ODCIIndexInsert() ODCIIndexUpdate() ODCIIndexDelete() ODCIIndexGetMetadata() ODCIGetInterfaces() ODCIStatsCollect() ODCIStatsDelete() ODCIStatsSelectivity() ODCIStatsIndexCost() ODCIStatsFunctionCost()

User defined

data type

system Inte

rfac

e

User defined

indexes

Inte

rfac

e

Extensions

of

SQL

optimizer

Inte

rfac

e

PL/SQL

cartridge Inte

rfac

e

Oracle

Spatial

cartridge Inte

rfac

e

9

Kodola paplašināšanas process

Problēmu vides analīze

Nepieciešamo objektu noteikšana

Objektu tipu definēšana ar SQL un PL/SQL

Nepieciešamās metodes problēmu vidē

IndekssEksistējošs

indekssJauna indeksa tipa definēšana

multi-domain

Indeksa definēšana

Eksistējošā optimizatora lietošana

Optimizatora paplašināšana

Testēšana

Scenāriju komandu izpilde

Izmaksas

10

DBS Oracle kodola paplašināšanas iespēju klāsts1. Paplašināmo tipu sistēma

Paplašināmo tipu sistēma ir viens no būtiskākajiem Oracle kodola paplašinājumiem. Tā ir ne tikai iespēja papildināt jau definētos tipus, bet arī rīks, ar kura palīdzību tiek realizēti vairāki kodola paplašinājumu interfeisi.Paplašināmo tipu sistēma nodrošina augsta līmeņa (SQL – bāzētu) interfeisu tipu definēšanā. Tipu „uzvedība” var tikt realizēta Java, C/C++ vai arī PL/SQL valodā. Oracle DBVS automātiski nodrošina zema līmeņa infrastruktūras servisus, kuri nepieciešami datu ievadei – izvadei, daudzveidīgai piekļuvei jaunajiem datu tipiem, optimizāciju datu pārraidei starp datu bāzi un lietojumiem u.c. Paplašināmo tipu sistēma nodrošina:1) sarežģītu strukturētu;2) daļēji strukturētu;3) nestrukturētu datu glabāšanas iespējas datu bāzē.

11

DBS Oracle kodola paplašināšanas iespēju klāsts2. Paplašināmā servera izpildvide

Viena no svarīgākajām Oracle paplašināšanas arhitektūras iezīmēm ir iespēja realizēt procedūras un funkcijas dažādās programmēšanas valodās. Oracle piedāvā lietotāja funkcijas un metodes realizēt dzimtajā PL/SQL valodā, Java vai arī C/C++ valodās.Ļoti lielu vērību Oracle piešķir drošības jautājumiem, kas ir saistīti ar tā saucamā „nedrošā” programmas koda izpildi.

1. Gadījumā, kad programma tiek realizēta PL/SQL valodā (vai arī Java valodā, kura ir dziļi integrēta Oracle DBVS), tā var tikt izpildīta Oracle adrešu telpā, jo, pateicoties dažādiem iekšējiem aizsardzības mehānismiem, tās izpilde nevar kaitēt datu bāzes servera darbībai.

2. Savukārt C/C++ valodas kodu varētu nosaukt par „bīstamu”, jo viena no šīs valodas specifikām ir programmētāja iespēja operēt ar atmiņu patvaļīgi, līdz ar to nav garantiju, ka kāda kļūda algoritmā nekaitēs tās atmiņas adrešu telpai, kura ir pieejama programmai. Līdz ar to šajā gadījumā Oracle tādas programmas izpilda ārējā adrešu telpā.

12

Par drošību atbild extproc (external procedure) – speciāls izdalīts ārējās procedūras aģents, kas nodrošina ārējo procedūru izpildi. Šajā shēmā parādās arī Listener process, kurš atbild uz lietotāja procesu prasībām pieslēgties datu bāzei, un līdz ar to darbojas kā starpposms starp lietojumu un datu bāzi. Izmantojot tīkla pieslēgumu, kuru nodibināja Listener process, lietojums nodod ārējās procedūras aģentam bibliotēkas vārdu, ārējās procedūras nosaukumu un visus nepieciešamos parametrus.Līdz ar to var izskatīt procesus, kas notiek katrā no adrešu telpām.Ārējā adrešu telpā:1) ārējo procedūru aģents tiek izsaukts vai nu ar Listener procesu vai arī ar bibliotēku;2) ārējās procedūras aģents izsauc starpvalodu metožu servisu Oracle adrešu telpā.Oracle adrešu telpā:1) starpvalodu metožu serviss piekļūst Oracle datu bāzei caur Oracle serveri;2) starpvalodu metožu serviss izsauc vai nu Listener, vai nu ārējo procedūru aģentu ārējā adrešu telpā.

13

DBS Oracle kodola paplašināšanas iespēju klāsts3. Paplašināmie indeksa tipi

Eksistējošie indeksa tipi:1) B-tree indexes. These indexes are the standard index type. They are

excellent for highly selective indexes (few rows correspond to each index entry) and primary key indexes. Used as concatenated indexes, a B-tree index can retrieve data sorted by the indexed columns. B-tree indexes have the following subtypes:a) index-organized tables. An index-organized table differs from a

heap-organized because the data is itself the index. See "Overview of Index-Organized Tables".

b) reverse key indexes. In this type of index, the bytes of the index key are reversed, for example, 103 is stored as 301. The reversal of bytes spreads out inserts into the index over many blocks. See "Reverse Key Indexes".

c) descending indexes. This type of index stores data on a particular column or columns in descending order. See "Ascending and Descending Indexes".

d) B-tree cluster indexes. This type indexes a table cluster key. Instead of pointing to a row, the key points to the block that contains rows related to the cluster key. See "Overview of Indexed Clusters".

2) Bitmap and bitmap join indexes. In a bitmap index, an index entry uses a bitmap to point to multiple rows. In contrast, a B-tree index entry points to a single row. A bitmap join index is a bitmap index for the join of two or more tables. See "Overview of Bitmap Indexes".

3) Function-based indexes. This type of index includes columns that are either transformed by a function, such as the UPPER function, or included in an expression. B-tree or bitmap indexes can be function-based. See "Overview of Function-Based Indexes".

4) Application domain indexes. A user creates this type of index for data in an application-specific domain. The physical index need not use a traditional index structure and can be stored either in the Oracle database as tables or externally as a file. See "Overview of Application Domain Indexes".

14

Balansētie koki

Balansētie koki (Balanced tree, jeb B-tree) tiek izmantoti skaitlisko tipu indeksēšanai. Balansētais koks pēc būtības ir grafa struktūra koks, un attiecīgo vērtību pārmeklēšana tajā notiek, virzoties uz dziļākajiem koka līmeņiem, līdz netiek sasniegtas koka strupceļa virsotnes. Balansēto koku algoritms nodrošina efektīvu datu ievietošanu un izdzēšanu, rūpīgi balansējot koka plašumu un dziļumu.

15

Index organized tabele

16

Hash indexes

Heša indeksi (hash indexes) ir vairāk paredzēti simbolisko vērtību indeksēšanai. Tie dod ātru pieeju specifiskajam ierakstam, balstoties uz dotā lauka izskaitļoto vērtību, ko sauc par heša vērtību. Katrai indeksējamās kolonnas vērtībai tiek izskaitļota heša vērtība, un rezultāti saglabāti indeksa tabulā. Tādā veidā, meklējot noteiktas vērtības, datu bāzes vadības sistēma var izskaitļot to heša vērtību un veikt attiecīgo atlasi no indeksa tabulas datiem.

17

Overview of Bitmap Indexes

In a bitmap index, the database stores a bitmap for each index key. In a conventional B-tree index, one index entry points to a single row. In a bitmap index, each index key stores pointers to multiple rows.

18

Overview of function-based indexes

A function-based index computes the value of a function or expression involving one or more columns and stores it in an index. A function-based index can be either a B-tree or a bitmap index.The indexed function can be an arithmetic expression or an expression that contains a SQL function, user-defined PL/SQL function, package function, or C callout. For example, a function could add the values in two columns.

Function-based indexes are efficient for evaluating statements that contain functions in their WHERE clauses. The database only uses the function-based index when the function is included in a query.

CREATE INDEX emp_total_sal_idx ON employees (12 * salary * commission_pct, salary, commission_pct);

SELECT employee_id, last_name, first_name, 12*salary*commission_pct AS "ANNUAL SAL"FROM employeesWHERE (12 * salary * commission_pct) < 30000ORDER BY "ANNUAL SAL" DESC;

EMPLOYEE_ID LAST_NAME FIRST_NAME ANNUAL SAL----------- ------------------------- -------------------- ------------------------ 159 Smith Lindsey 28800 151 Bernstein David 28500 152 Hall Peter 27000 160 Doran Louise 27000 175 Hutton Alyssa 26400 149 Zlotkey Eleni 25200 169 Bloom Harrison 24000

19

Application domain indexes

The domain index schema object is an application-specific index that is created, managed, and accessed by routines supplied by an indextype. It is called a domain index because it indexes data in application-specific domains.

Oracle provides extensible indexing to accommodate indexes on complex data types such as documents, spatial data, images, and video clips and to make use of specialized indexing techniques. With extensible indexing, you can encapsulate application-specific index management routines as an indextype schema object and define a domain index (an application-specific index) on table columns or attributes of an object type. Extensible indexing also provides efficient processing of application-specific operators.

The application software, called the cartridge, controls the structure and content of a domain index. The Oracle server interacts with the application to build, maintain, and search the domain index. The index structure itself can be stored in:

1) the Oracle database as an index-organized table;2) externally as a file.

The indextype schema object encapsulates the set of routines that manage and access a domain index. The purpose of an indextype is to enable efficient search and retrieval functions for complex domains such as text, spatial, image, and OLAP data using external application software.

20

The Oracle Data Cartridge Interface (ODCIIndex) specifies all the routines that must be implemented by the index designer. The routines can be implemented as type methods.

Index definition routines: 1) build the domain index when a CREATE INDEX statement references the

indextype; 2) alter the domain index information when a ALTER INDEX statement alters it; 3) remove the index information when a DROP INDEX statement drops it; 4) truncate (saīsināt) the index when the base table is truncated.

Index maintenance routines: maintain the contents of the domain index when the base table rows are inserted, deleted, updated, or loaded.

Index scan routines implement access to the domain index to retrieve rows of the base table that satisfy predicates containing built-in or user-defined operators in the accessing SQL statement. An index scan is specified through three routines:

1) istart, which initializes data structures; 2) ifetch, which fetches rows satisfying the predicate; 3) iclose, which closes the cursor after all rows satisfying the predicate are

returned.

User-defined operators can be invoked anywhere built-in operators can be used, that is, wherever expressions can occur in queries and data manipulation statements, such as: The select list of a SELECT statement or subquery The condition of a WHERE clause The ORDER BY and GROUP BY clauses

The user-defined function bound to an operator can be: A stand-alone function A package function An object member method

An indextype provides index-based implementation for the operators listed in the indextype definition. The Oracle server can invoke routines specified in the indextype to search the domain index to identify candidate rows and then do further processing, such as filtering, selection, and fetching of rows.

21

DBS Oracle kodola paplašināšanas iespēju klāsts4. Paplašināmās indeksēšanas mehānismsJauna indeksa tipa veidošana

Oracle piedāvā universālu paplašināmās indeksācijas mehānismu, ar kura palīdzību ir iespējams:

1) definēt problēmsfēras indeksu kā jaunu indeksa tipu;2) glabāt indeksa datus vai nu Oracle DB (tabulu veidā), vai arī ārpus

tās;3) kontrolēt, izgūt un izmantot indeksu datus, lai optimālāk realizētu

lietotāja vaicājumus.Tādus indeksus sauc par problēmsfēras (domēna) indeksiem, jo tie indeksē datus problēmsfēras specifiskajā veidā.

Kodola paplašinājumam nepieciešams:1) definēt indeksa formātu un saturu;2) izveidot, izdzēst un atjaunot indeksu;3) piekļūt un interpretēt indeksa saturu (vaicājumu izpildes laikā).

Indeksa tipa definēšana notiek pakāpeniski:1) sākumā tiek definēti problēmsfēras operatori;2) tad objektu tips ar tā ķermeni, kas realizē pilnvērtīgai indeksa

darbībai nepieciešamās metodes;3) tad tiek izveidots indeksa tips, balstoties uz dotajiem uz iepriekš

izveidotajiem operatoriem un objektu tipu.

Oracle paplašināmo indeksu darbības pamatprincips ir pamattabulas rindiņu identifikatoru glabāšana kopā ar datiem – darbojoties indeksa mehānismam vaicājuma laikā, datu bāzes vadības sistēmai tiek nodotas to rindiņu identifikatori, kas apmierina specializēto vaicājumu (indeksa mehānisms var tikt iedarbināts, kad vaicājuma WHERE daļā parādās problēmsfēras operators, kas ir piesaistīts indeksa tipam). Tādā veidā, ja vien ir iespējams, rindiņas, kuras var tikt izgūtas, balstoties uz operatora predikāta izteiksmi, tiek izgūtas pateicoties indeksa mehānisma darbībai,- ja vien optimizators nosaka, ka griešanās pie problēmsfēras indeksa ir izdevīgāka.

22

Data cartridge komponenšu veidošana

1. Objektu tipu definēšana.2. Implementing Data Cartridges in PL/SQL3. Implementing Data Cartridges in C, C++ and Java4. Working with Multimedia Datatypes5. Using Extensible Indexing6. Building Domain Indexes7. Defining Operators8. Using Extensible Optimizer9. User-Defined Aggregate Functions10. Using Cartridge Services11.Using Pipelined and Parallel Table Functions12. Designing Data Cartridges

23

Implementing Data Cartridges in C, C++ and Java

To support such special-purpose processing, PL/SQL provides an interface for calling routines written in other languages. This makes the strengths and capabilities of 3GLs like C available through calls from a database server. Such a 3GL routine, called an external procedure, is stored in a shared library, registered with PL/SQL, and called from PL/SQL at runtime to perform special-purpose processing.

External ProceduresUsing Shared LibrariesRegistering an External ProcedureHow PL/SQL Calls an External ProcedureConfiguration Files for External ProceduresDoing CallbacksCommon Potential ErrorsDebugging External ProceduresGuidelines for Using External Procedures with Data CartridgesJava Methods

24

Working with Multimedia Datatypes

Overview of Cartridges and Multimedia DatatypesDDL for LOBsLOB LocatorsEMPTY_BLOB and EMPTY_CLOB FunctionsUsing the OCI to Manipulate LOBsUsing DBMS_LOB to Manipulate LOBsLOBs in External ProceduresLOBs and TriggersUsing Open/Close as Bracketing Operations for Efficient Performance