everything you need to know about collections but were afraid to ask

8/7/2019 Everything You Need to Know About Collections But Were Afraid to Ask

1/48

Copyright 2000-2006 Steven Feuerstein - Page 1

OPP 2007February 28 March 1, 2007

San Mateo Marriott

San Mateo, California

An ODTUG SP* Oracle PL/SQLProgramming Conference

*SP Seriously Practical Conference

For more information visit www.odtug.com or call 910-452-7444

ODTUG KaleidoscopeJune 18 21, 2007

Pre-conference Hands-on Training - June 16 17

Hilton Daytona Beach Oceanfront Resort

Daytona, Florida

WOW-Wide Open World, Wide Open Web!


2/48


Everything you need to know

about collections,but were afraid to ask

Steven FeuersteinPL/SQL Evangelist

Quest Software

[email protected]


3/48


Ten Years Writing Ten Bookson the Oracle PL/SQL Language


4/48


How to benefit most from this seminar

Watch, listen, ask questions.

Download the training materials and supporting scripts: http://oracleplsqlprogramming.com/resources.html

"Demo zip": all the scripts I run in my class available at

http://oracleplsqlprogramming.com/downloads/demo.zip

Use these materials as an accelerator as you venture intonew territory and need to apply new techniques.

Play games! Keep your brain fresh and active by mixinghard work with challenging games

MasterMind and Set (www.setgame.com)

filename_from_demo_zip.sql


5/48


PL/SQL Collections

Collections are single-dimensionedlists ofinformation, similar to 3GL arrays.

They are an invaluable data structure.

AllPL/SQL developers should be very comfortablewith collections and use them often.

Collections take some getting used to.

They are not the most straightforward

implementation of array-like structures. Advanced features like string indexes and multi-

level collections can be a challenge.


6/48


What we will cover on collections

Review of basic functionality

Indexing collections by strings

Working with collections of collections MULTISET operators for nested tables

Then later in the section on SQL:

Bulk processing with FORALL and BULK

COLLECT

Table functions and pipelined functions


7/48


What is a collection?

A collection is an "ordered group of elements,

all of the same type." (PL/SQL User Guide andReference) That's a very general definition; lists, sets, arrays and

similar data structures are all types of collections.

Each element of a collection may be addressed by a unique

subscript, usually an integer but in some cases also a string. Collections are single-dimensional, but you can create

collections of collections to emulate multi-dimensionalstructures.

abc def sf q rrr swq ...1 2 3 4 22 23


8/48


Why use collections?

Generally, to manipulate in-program-memory lists of

information.

Much faster than working through SQL.

Serve up complex datasets of information to non-PL/SQL host environments using table functions.

Dramatically improve multi-row querying, inserting,

updating and deleting the contents of tables.

Combined with BULK COLLE

CT and FORALL.... Emulate bi-directional cursors, which are not yet

supported within PL/SQL.


9/48


Three Types of Collections

Associative arrays (aka index-by tables)

Can be used only in PL/SQL blocks.

Similar to hash tables in other languages, allows you to

access elements via arbitrary subscript values. Nested tables and Varrays

Can be used in PL/SQL blocks, but also can be the

datatype of a column in a relational table.

Part of the object model in PL/SQL.

Required for some features, such as table functions

With Varrays, you specify a maximum number of elements

in the collection, at time of definition.


10/48


About Associative Arrays

Unbounded, practically speaking.

Valid row numbers range from -2,147,483,647 to

2,147,483,647.

This range allows you to employ the row number as anintelligent key, such as the primary key or unique index

value, because AAs also are:

Sparse

Data does not have to be stored in consecutive rows, as isrequired in traditional 3GL arrays and VARRAYs.

Index values can be integers or strings (Oracle9i R2

and above).assoc_array_example.sql


11/48


About Nested Tables

No pre-defined limit on a nested table.

Valid row numbers range from 1 to

2,147,483,647.

Part of object model, requiring initialization.

Is always dense initially, but can become

sparse after deletes.

Can be defined as a schema level type and

used as a relational table column type.

nested_table_example.sql


12/48


About Varrays

Has a maximum size, associated with its type.

Can adjust the size at runtime in Oracle10g R2.

Part of object model, requiring initialization. Is always dense; you can only remove

elements from the end of a varray.

Can be defined as a schema level type andused as a relational table column type.

varray_example.sql


13/48


How to choose your collection type

Use associative arrays when you need to... Work within PL/SQL code only

Sparsely fill and manipulate the collection

Take advantage of negative index values

Use nested tables when you need to... Access the collection inside SQL (table functions, columns in

tables)

Want to perform set operations

Use varrays when you need to... If you need to specify a maximum size to your collection

Access the collection inside SQL (table functions, columns intables).


14/48


Wide Variety of Collection Methods

Obtain information about the collection COUNT returns number of rows currently defined in

collection.

EXISTS returns TRUE if the specified row is defined.

FIRST/LAST return lowest/highest numbers of defined rows. NEXT/PRIOR return the closest defined row after/before the

specified row.

LIMIT tells you the max. number of elements allowed in aVARRAY.

Modify the contents of the collection DELETE deletes one or more rows from the index-by table.

EXTEND adds rows to a nested table or VARRAY.

TRIM removes rows from a VARRAY.


15/48


Useful reminders for PL/SQL collections

Memory for collections comes out of the PGA orProcess Global Area One per session, so a program using collections can

consume a large amount of memory.

Use the NOCOPY hint to reduce overhead of passingcollections in and out of program units.

Encapsulate or hide details of collection management.

Don't always fill collections sequentially. Think about

how you need to manipulate the contents. Try to read a row that doesn't exist, and Oracle raises

NO_DATA_FOUND.mysess.pkg

sess2.sql

nocopy*.*


16/48


FunctionPGA

Data Caching with PL/SQL Tables

First access

Subsequent accesses

PGAFunction

Database

Not in cache;

Request data

from database

Pass Data

to Cache

Application

Application

Requests Data

Data retrievedfrom cache Data returned

to application

Application

Application

Requests Data

Data returned

to application

Data retrieved

from cache

Database

Data found in

cache. Database

is not needed.

emplu.pkg

emplu.tst


17/48


New indexing capabilitiesfor associative arrays

Prior to Oracle9iR2, you could only index byBINARY_INTEGER.

You can now define the index on your associative

array to be: Any sub-type derived from BINARY_INTEGER VARCHAR2(n), where n is between 1 and 32767

%TYPE against a database column that is consistent withthe above rules

A SUBTYPE against any of the above. This means that you can now index on string

values! (and concatenated indexes and...)

Oracle9iRelease 2


18/48


Examples of NewTYPE Variants

All of the following are now valid TYPE declarations inOracle9i Release 2 You cannot use %TYPE against an INTEGER column,

because INTEGER is nota subtype of BINARY_INTEGER.

DECLARE

TYPE array_t1 IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;TYPE array_t2 IS TABLE OF NUMBER INDEX BY PLS_INTEGER;TYPE array_t3 IS TABLE OF NUMBER INDEX BY POSITIVE;TYPE array_t4 IS TABLE OF NUMBER INDEX BY NATURAL;TYPE

array_t5IS TABLE OF NUMBER INDEX BY VARCHAR

2(64);

TYPE array_t6 IS TABLE OF NUMBER INDEX BY VARCHAR2(32767);TYPE array_t7 IS TABLE OF NUMBER INDEX BY

employee.last_name%TYPE;TYPE array_t8 IS TABLE OF NUMBER INDEX BY

types_pkg.subtype_t;

Oracle9iRelease 2


19/48


Working with string-indexed collections

Specifying a row via a string takes some gettingused to, but if offers some very powerful advantages.

DECLARE

TYPE population_type IS TABLE OF NUMBER INDEX BY VARCHAR2(64);

country_populationpopulation_type;continent_populationpopulation_type;

howmany NUMBER;BEGIN

country_population('Greenland') := 100000;country_population('Iceland') := 750000;

howmany := country_population('Greenland');

continent_population('Australia') := 30000000;END;

assoc_array*.sql

assoc_array_perf.tst


20/48


21/48


22/48


Multi-level Collections

Prior to Oracle9i, you could have collections ofrecords or objects, but only if all fields werescalars.

A collection containing another collection was notallowed.

Now you can create collections that containother collections and complex types.

Applies to all three types of collections.

The syntax is non-intuitive and resulting codecan be quite complex.

Oracle9i


23/48


String Tracker Version 2

The problem with String Tracker V1 is that it

only supports a single list of strings.

What if I need to track multiple lists

simultaneously or nested?

Let's extend the first version to support

multiple lists by using a string-indexed, multi-

level collection. A list of lists....


24/48


The String Tracker package (V2)

CREATE OR REPLACE PACKAGE BODY string_trackerIS

TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t;TYPE list_of_lists_aat IS TABLE OF used_aat INDEX BY maxvarchar2_t;g_list_of_listslist_of_lists_aat;

PROCEDURE mark_as_used(

list_in IN maxvarchar2_t,value_in IN maxvarchar2_t,case_sensitive_in IN BOOLEAN DEFAULT FALSE

) ISl_namemaxvarchar2_t :=

CASE case_sensitive_in WHEN TRUE THEN value_inELSE UPPER(value_in) END;

BEGINg_list_of_lists(list_in)(l_name) := TRUE;

END mark_as_used;END string_tracker;

string_tracker2.*


25/48


26/48


Encapsulate these complex structures!

When working with multi-level collections, youcan easily and rapidly arrive at completelyunreadable and un-maintainable code.

What' s a developer to do? Hide complexity -- and all data structures -- behind

small modules.

Work with and through functions to retrieve

contents and procedures to set contents.

cc_smartargs.pkb:

cc_smartargs.next_overloading

cc_smartargs.add_new_parameter


27/48


Nested Tables unveil theirMULTISET-edness

Oracle10g introduces high-level set operations

on nested tables (only).

Nested tables are multisets, meaning that

theoretically there is no order to their elements. Thismakes set operations of critical importance for

manipulating nested tables. .

You can now

Check for equality and inequality

Perform UNION, INTERSECT and MINUS operations

Check for and remove duplicates

Oracle10g


28/48


Check for equality and inequality

Just use the basic operators.

Oracle10g

DECLARE

TYPE clientele IS TABLE OF VARCHAR2(64);group1 clientele := clientele('Customer1','Customer2');group2 clientele := clientele('Customer1','Customer3');group3 clientele := clientele('Customer3','Customer1');

BEGIN

IF group1 = group2 THENDBMS_OUTPUT.put_line('Group1 = Group2');

ELSE

DBMS_OUTPUT.put_line('Group1 != Group2');END IF;

IF group2 != group3 THEN

DBMS_OUTPUT.put_line('Group2 != Group3');ELSE

DBMS_OUTPUT.put_line('Group2 = Group3');END IF;

END;

10g_compare.sql

10g_compare2.sql

10g_compare_old.sql


29/48


UNION, INTERSECT, MINUS

Straightforward, with the MULTISET keyword.

Oracle10g

BEGIN

our_favorites := my_favorites MULTISET UNION dad_favorites;show_favorites('MINE then DAD',our_favorites);

our_favorites := dad_favorites MULTISET UNION my_favorites;show_favorites('DAD then MINE',our_favorites);

our_favorites := my_favorites MULTISET UNION DISTINCT dad_favorites;show_favorites('MINE then DAD with DISTINCT',our_favorites);

our_favorites := my_favorites MULTISET INTERSECT dad_favorites;show_favorites('IN COMMON',our_favorites);

our_favorites := dad_favorites MULTISET EXCEPT my_favorites;show_favorites('ONLY DAD''S',our_favorites);

END;

10g_setops.sql

10g_string_nt.sql

10g_favorites.sql

10g*union*.sql


30/48


Turbo-charged SQL withBULK COLLECT and FORALL

Improve the performance of multi-row SQL

operations by an order of magnitude or more

with bulk/array processing in PL/SQL!

CREATE OR REPLACE PROCEDURE upd_for_dept(dept_in IN employee.department_id%TYPE

,newsal_in IN employee.salary%TYPE)IS

CURSORemp_cur ISSELECT employee_id,salary,hire_dateFROMemployee WHERE department_id = dept_in;

BEGIN

FORrec IN emp_cur LOOPUPDATE employee SET salary = newsal_inWHERE employee_id = rec.employee_id;

END LOOP;

END upd_for_dept;

Conventional

binds (and

lots of them!)


31/48


Oracle server

PL/SQLRuntime Engine SQL Engine

PL/SQL blockProcedural

statementexecutorSQL

statement

executor

FORrec IN emp_cur LOOPUPDATE employee

SET salary = ...WHERE employee_id =

rec.employee_id;END LOOP;

Performance penaltyPerformance penaltyfor many contextfor many context

switchesswitches

Conventional Bind


32/48


Enter the Bulk Bind: FORALL

Oracle server

PL/SQLRuntime Engine SQL Engine

PL/SQL blockProcedural

statementexecutorSQL

statement

executor

FORALL indx INlist_of_emps.FIRST..list_of_emps.LAST

UPDATE employeeSET salary = ...

WHERE employee_id =list_of_emps(indx);

Much less overhead for

Much less overhead forcontext switchingcontext switching


33/48


Use the FORALL Bulk Bind Statement

Instead of executing repetitive, individual DMLstatements, you can write your code like this:

Things to be aware of: You MUST know how to use collections to use this feature!

Only a single DML statement is allowed per FORALL. New cursor attributes: SQL%BULK_ROWCOUNT returns number of

rows affected by each row in array. SQL%BULK_EXCEPTIONS...

Prior to Oracle10g, the binding arraymust be sequentially filled.

Use SAVEEXCEPTIONS to continue past errors.

PROCEDURE upd_for_dept(...) ISBEGIN

FORALL indx IN list_of_emps.FIRST ..list_of_emps.LASTUPDATE employee

SET salary = newsal_inWHERE employee_id = list_of_emps(indx);

END;

bulktiming.sql

bulk_rowcount.sql


34/48


Use BULK COLLECT INTO for Queries

DECLARE

TYPE employees_aat IS TABLE OF employees%ROWTYPEINDEX BY BINARY_INTEGER;

l_employeesemployees_aat;BEGIN

SELECT *BULK COLLECT INTO l_employeesFROMemployees;

FORindx IN 1..l_employees.COUNTLOOP

process_employee(l_employees(indx));END LOOP;

END;

bulkcoll.sql

Declare a

collection of

records to hold

the queried data.

Use BULK

COLLECT to

retrieve all rows.

Iterate through thecollection

contents with a

loop.


35/48


Limit the number of rows returned byBULK COLLECT

CREATE OR REPLACE PROCEDURE bulk_with_limit(deptno_in IN dept.deptno%TYPE)

IS

CURSORemps_in_dept_cur ISSELECT *

FROMempWHERE deptno = deptno_in;

TYPE emp_tt IS TABLE OF emps_in_dept_cur%ROWTYPE;empsemp_tt;

BEGIN

OPEN emps_in_dept_cur;LOOP

FETCH emps_in_dept_curBULK COLLECT INTO empsLIMIT

100;

EXIT WHEN emps.COUNT = 0;

process_emps(emps);END LOOP;

END bulk_with_limit;

Use the LIMIT clause with the

INTO to manage the amount

of memory used with the

BULK COLLECT operation.

WARNING!

BULK COLLECT will notraise

NO_DATA_FOUND if no rows

are found.

Best to check contents of

collection to confirm that

something was retrieved.bulklimit.sql


36/48


Tips and Fine Points

Use bulk binds in these circumstances: Recurring SQL statement in PL/SQL loop. Oracle

recommended threshold: five rows!

Bulk bind rules:

Can be used with any kind of collection; Collectionsubscripts cannot be expressions; The collectionsmust be densely filled (pre-10gR2).

Bulk collects:

Can be used with implicit and explicit cursors

Collection is always filled sequentially, starting atrow 1.

emplu.pkg

cfl_to_bulk*.*


37/48


The Wonder Of Table Functions

A table function is a function that you can call in theFROM clause of a query, and have it be treated as if itwere a relational table.

Table functions allow you to perform arbitrarily

complex transformations of data and then make thatdata available through a query. Not everything can be done in SQL.

Combined with REF CURSORs, you can now more

easily transfer data from within PL/SQL to hostenvironments. Java, for example, works very smoothly with cursor

variables


38/48


Building a table function

A table function must return a nested table orvarray based on a schema-defined type, ortype defined in a PL/SQL package.

The function header and the way it is calledmust be SQL-compatible: all parameters useSQL types; no named notation.

In some cases (streaming and pipelined

functions), the IN parameter must be a cursorvariable -- a query result set.


39/48


Simple table function example

Return a list of names as a nested table, and

then call that function in the FROM clause.

CREATE OR REPLACE FUNCTION lotsa_names(base_name_in IN VARCHAR2,count_in IN INTEGER

)RETURN names_nt

IS

retval names_nt := names_nt();BEGIN

retval.EXTEND (count_in);

FORindx IN 1..count_in

LOOPretval(indx) :=

base_name_in || '' || indx;END LOOP;

RETURN retval;END lotsa_names; tabfunc_scalar.sql

SELECT column_valueFROM TABLE (

lotsa_names('Steven',100))names;

COLUMN_VALUE------------

Steven1...Steven100


40/48


Streaming data with table functions

You can use table functions to "stream" data

through several stages within a single SQL

statement.

Example: transform one row in the stocktable to tworows in the tickertable. CREATE TABLE stocktable(

ticker VARCHAR2(20),trade_date DATE,open_price NUMBER,close_price NUMBER

)/CREATE TABLE tickertable(

ticker VARCHAR2(20),pricedate DATE,pricetype VARCHAR2(1),price NUMBER)

/

tabfunc_streaming.sql


41/48


Streaming data with table functions - 2

In this example, transform each row of the

stocktable into two rows in the tickertable.

CREATE OR REPLACE PACKAGE refcur_pkgIS

TYPE refcur_t IS REF CURSORRETURN stocktable%ROWTYPE;

END refcur_pkg;/

CREATE OR REPLACE FUNCTION stockpivot(datasetrefcur_pkg.refcur_t)RETURN tickertypeset...

BEGIN

INSERT INTO tickertableSELECT *

FROM TABLE (stockpivot(CURSOR(SELECT *FROMstocktable)));

END;

/

tabfunc_streaming.sql


42/48


Use pipelined functions to enhanceperformance.

Pipelined functions allow you to return data

iteratively, asynchronous to termination of the

function. As data is produced within the function, it is

passed back to the calling process/query.

Pipelined functions can be defined to supportparallel execution.

Iterative data processing allows multiple

processes to work on that data simultaneously.

CREATE FUNCTION StockPivot(prefcur_pkg.refcur_t)RETURN TickerTypeSetPIPELINED


43/48


Applications for pipelined functions

Execution functions in parallel.

In Oracle9i Database Release 2 and above, use thePARALLEL_ENABLE clause to allow your pipelined

function to participate fully in a parallelized query. Critical in data warehouse applications.

Improve speed of delivery of data to webpages.

Use a pipelined function to "serve up" data to thewebpage and allow users to being viewing andbrowsing, even before the function has finishedretrieving all of the data.


44/48


Piping rows out from a pipelined function

CREATE FUNCTION stockpivot(prefcur_pkg.refcur_t)RETURN tickertypesetPIPELINED

IS

out_rec tickertype :=tickertype(NULL, NULL, NULL);

in_rec p%ROWTYPE;

BEGINLOOP

FETCH p INTO in_rec;EXIT WHEN p%NOTFOUND;out_rec.ticker := in_rec.ticker;out_rec.pricetype := 'O';out_rec.price := in_rec.openprice;

PIPE ROW(out_rec);

END LOOP;CLOSE p;

RETURN;

END;

tabfunc_setup.sql

tabfunc_pipelined.sql

Add PIPELINED

keyword to header

Pipe a row of data

back to calling block

or query

RETURN...nothing at

all!


45/48


Enabling Parallel Execution

The table function's parameter list must consist only

of a single strongly-typedREF CURSOR.

Include the PARALLEL_ENABLE hint in the program

header. Choose a partition option that specifies how the function's

execution should be partitioned.

"ANY" means that the results are independent of the order

in which the function receives the input rows (through the

REF CURSOR).

{[ORDER | CLUSTER] BY column_list}

PARALLEL_ENABLE ({PARTITION p BY[ANY | (HASH | RANGE)column_list]} )


46/48


Table functions Summary

Table functions offer significant new flexibility

for PL/SQL developers.

Consider using them when you...

Need to pass back complex result sets of data

through the SQL layer (a query);

Want to call a user defined function inside a

query and execute it as part of a parallel query.


47/48


Collections don't start coding without them.

It is impossible to write modern PL/SQL code,

taking full advantage of new features, unless you

use collections.

From array processing to table functions, collections arerequired.

Today I offer this challenge: learn collections

thoroughly and apply them throughout your

backend code.

Your code will get faster and in many cases much

simpler than it might have been (though not always!).


48/48


OPP 2007February 28 March 1, 2007

San Mateo Marriott

San Mateo, California

An ODTUG SP* Oracle PL/SQL

Programming Conference

*SP Seriously Practical Conference

For more information visit www.odtug.com or call 910-452-7444

ODTUG KaleidoscopeJune 18 21, 2007

Pre-conference Hands-on Training - June 16 17Hilton Daytona Beach Oceanfront Resort

Daytona, Florida

WOW-Wide Open World, Wide Open Web!

everything you need to know about collections but were afraid to ask

Documents