oracle sql model clause
DESCRIPTION
Abstract: The session will breakdown the Model clause into its fundamental components and provides some basic real-world examples to demonstrate its greater potential. Though most developers have heard of the SQL Model clause in 10g, many may baulk at the idea of using it - daunted by seemingly foreign syntax that might well have come out of a FORTRAN program. Look a little closer and you'll find it's just like building a spreadsheet. Concise, easy to read syntax that provides the functionality for demanding calculations that would normally require elaborate joins, unions, analytics or PL/SQL. In addition to the development and maintenance burden, we are also faced with the all too familiar problem of business customers duplicating data to an Excel spreadsheet that is shared and erroneously modified around the workplace. This session uses the Model clause as a high performance tool that can simplify approaches to every day problems. It demonstrates that Model is an extension to SQL that forms multi-dimensional arrays with inter-row & inter-array calculations that automatically resolves formula dependencies.TRANSCRIPT
The Model ClauseSpreadsheet Techniques
using SQLScott Wesley
Agenda
• Concepts• Cell Referencing• Focus• Performance• Final thoughts…
Any Model experts?SELECT c, p, m, pp, ipFROM mortgageMODELREFERENCE R ON (SELECT customer, fact, amt FROM mortgage_facts MODEL DIMENSION BY (customer, fact) MEASURES (amount amt) RULES (amt[any, 'PaymentAmt']= (amt[CV(),'Loan']* Power(1+ (amt[CV(),'Annual_Interest']/100/12),amt[CV(),'Payments'])* (amt[CV(),'Annual_Interest']/100/12)) / (Power(1+(amt[CV(),'Annual_Interest']/100/12), amt[CV(),'Payments']) - 1)))DIMENSION BY (customer cust, fact)MEASURES (amt)MAIN amortization PARTITION BY (customer c) DIMENSION BY (0 p) MEASURES (principalp pp, interestp ip, mort_balance m, customer mc) RULES ITERATE(1000) UNTIL (ITERATION_NUMBER+1 =r.amt[mc[0],'Payments']) (ip[ITERATION_NUMBER+1] = m[CV()-1] * r.amt[mc[0], 'Annual_Interest']/1200 ,pp[ITERATION_NUMBER+1] = r.amt[mc[0], 'PaymentAmt'] - ip[CV()] ,m[ITERATION_NUMBER+1] = m[CV()-1] - pp[CV()])ORDER BY c, p;
FizzBuzz
• Write a program that prints the numbers from 1 to 100.
• But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”.
• For numbers which are multiples of both three and five print “FizzBuzz”
http://tkyte.blogspot.com/2007/02/what-is-your-fizzbuzz-factor.html
SQL> WITH data AS (SELECT LEVEL AS n FROM DUAL CONNECT BY LEVEL <= 100)SELECT n, NVL(CASE WHEN MOD(n,3) = 0 THEN 'Fizz' END || CASE WHEN MOD(n,5) = 0 THEN 'Buzz' END, TO_CHAR(n)) AS answerFROM data;
N ANSWER---- ----------- 1 1 2 2 3 Fizz 4 4 5 Buzz 6 Fizz 7 7 8 8 9 Fizz 10 Buzz 11 11 12 Fizz 13 13 14 14 15 FizzBuzz 16 16 17 17 18 Fizz ...
Anthony Wilson said....So... do we win points by using MODEL, or get thrown out of the interview on
mental health grounds?
Nobody in their right mind actually uses this stuff, right?
:)
select id, nfrom all_objectswhere object_id = 1modeldimension by (object_id id)measures (object_name n)rules (n[for id from 1 to 100 increment 1] = to_char(cv(id)),n[mod(id, 3) = 0] = 'fizz',n[mod(id, 5) = 0] = 'buzz',n[mod(id, 15) = 0] = 'fizzbuzz')order by id
SQL Spreadsheets
Craft
Basic Syntax<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES [UPSERT | UPDATE] [AUTOMATIC ORDER | SEQUENTIAL ORDER] [ITERATE (n) [UNTIL <condition>] ] ( <cell_assignment> = <expression> ... )
SELECT country ,year ,sales ,previous_year ,before_thatFROM sales_viewGROUP BY year, country-- Define model structure/optionsMODEL RETURN ALL ROWS PARTITION BY (country) DIMENSION BY (year) MEASURES (SUM(sales) AS sales -- Define 'calculated columns' ,CAST(NULL AS NUMBER) previous_year ,CAST(NULL AS NUMBER) before_that) -- Define rule options RULES AUTOMATIC ORDER ( -- Define actual rules previous_year[ANY] = sales[CV()-1] ,before_that [ANY] = sales[CV()-2] )-- Define order of entire result setORDER BY country, year;
Apply model clause to this query
Concepts
Country Product Year Sales
Partition Dimension Dimension Measure
AUS HAMMER 2006 10
AUS NAIL 2006 200
NZ NAIL 2006 300
NZ DRILL 2007 50
Dimension
Country Product Year Sales
Partition Dimension Dimension Measure
<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (product p, year y) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES …
Measure
Country Product Year Sales
Partition Dimension Dimension Measure
<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (sales) [IGNORE NAV] | [KEEP NAV] [RULES …
Partition
Country Product Year Sales
Partition Dimension Dimension Measure
<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (country)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES …
Array
20
15 5
35 15Hammer Drill
Product
2006
2007
2008Cou
ntry
Yea
r
Aus N
Z
Rules<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES [UPSERT | UPDATE] [AUTOMATIC ORDER | SEQUENTIAL ORDER] [ITERATE (n) [UNTIL <condition>] ] ( <cell_assignment> = <expression> ... )
Rules
COUNTRY PRODUCT YEAR SALESPartition Dimension Dimension MeasureAUS Hammer 2006 20AUS Hammer 2007 15AUS Drill 2007 5NZ Hammer 2006 15NZ Hammer 2007 10NZ Drill 2007 10AUS Hammer 2008 35AUS Drill 2008 15NZ Hammer 2008 25NZ Drill 2008 30
OriginalData
RuleResults
Sales[Hammer,2008] = Sales[Hammer,2006]+Sales[Hammer,2007]Sales[Drill ,2008] = Sales[Drill,2007]*3
SpreadsheetSales[Hammer,2008] = Sales[Hammer,2006]+Sales[Hammer,2007]Sales[Drill ,2008] = Sales[Drill,2007]*3
Return Rows<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES [UPSERT | UPDATE] [AUTOMATIC ORDER | SEQUENTIAL ORDER] [ITERATE (n) [UNTIL <condition>] ] ( <cell_assignment> = <expression> ... )
Rules
COUNTRY PRODUCT YEAR SALESPartition Dimension Dimension MeasureAUS Hammer 2006 20AUS Hammer 2007 15AUS Drill 2007 5NZ Hammer 2006 15NZ Hammer 2007 10NZ Drill 2007 10AUS Hammer 2008 35AUS Drill 2008 15NZ Hammer 2008 25NZ Drill 2008 30
RETURN ALL
ROWS
RETURN UPDATED ROWS
Sales[Hammer,2008] = Sales[Hammer,2006]+Sales[Hammer,2007]Sales[Drill ,2008] = Sales[Drill,2007]*3
Positional Cell ReferencingSELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce', 2000] = 10 ,sales['Bounce', 2008] = 20 );
COUNTRY PROD YEAR SALES------------ ------------ ----- ----------Australia Bounce 2000 10Australia Bounce 2008 20
Symbolic Cell ReferencingSELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales[prod = 'Bounce', year > 1999] = 10 )ORDER BY country, product, year;
COUNTRY PRODUCT YEAR SALES------------ ------------ ----- ----------Australia Bounce 2000 10Australia Bounce 2001 10
Multi-Cell Access on Right SideSELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce',2008] = 100 + MAX(sales) ['Bounce', year BETWEEN 1998 and 2002] ,sales['Y Box',2008] = 1.3 * PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY sales) [product in ('Bounce', 'Y Box'), year < 2003]);
COUNTRY PRODUCT YEAR SALES---------- -------- ----- -------Australia Bounce 2008 3861Australia Y Box 2008 4889
Multi-Cell Access on Right SideSELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce',2008] = 100 + MAX(sales) ['Bounce', year BETWEEN 1998 and 2002] ,sales['Y Box',2008] = 1.3 * PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY sales) [product in ('Bounce', 'Y Box'), year < 2003]);
COUNTRY PRODUCT YEAR SALES---------- -------- ----- -------Australia Bounce 2008 3861Australia Y Box 2008 4889
CV()
SELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce', year BETWEEN 1995 AND 2002] = sales['Mouse Pad', CV(year)] + 0.2 * sales['Y Box', CV()]);
COUNTRY PRODUCT YEAR SALES---------- -------- ----- -------Australia Bounce 1999 6061Australia Bounce 2000 8154Australia Bounce 2001 15211
SELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce', year BETWEEN 1995 AND 2002] = sales['Mouse Pad', CV(year)] + 0.2 * sales['Y Box', CV()]);
COUNTRY PRODUCT YEAR SALES---------- -------- ----- -------Australia Bounce 1999 6061Australia Bounce 2000 8154Australia Bounce 2001 15211
SELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce', year BETWEEN 1995 AND 2002] = sales['Mouse Pad', CV(year)] + 0.2 * sales['Y Box', CV()]);
COUNTRY PRODUCT YEAR SALES---------- -------- ----- -------Australia Bounce 1999 6061Australia Bounce 2000 8154Australia Bounce 2001 15211
SELECT country, product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES ( sales['Bounce', 1999] = sales['Mouse Pad',1999]+0.2*sales['Y Box',1999] sales['Bounce', 2000] = sales['Mouse Pad',2000]+0.2*sales['Y Box',2000] sales['Bounce', 20001] = sales['Mouse Pad',2001]+0.2*sales['Y Box',2001]);
CV(year)-1
SELECT country, product, year, sales, p_year, growth_pctFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSPARTITION BY (country)DIMENSION BY (product, year)MEASURES (sales, 0 p_year, 0 growth_pct)RULES ( p_year[product IN ('Bounce','Y Box'), year BETWEEN 1998 and 2001] = sales[CV(product), CV(year) -1] ,growth_pct[product IN ('Bounce','Y Box'), year BETWEEN 1998 and 2001] = 100* (sales[CV(product), CV(year)] - sales[CV(product), CV(year) -1] ) / sales[CV(product), CV(year) -1])ORDER BY country, product, year;
COUNTRY PRODUCT YEAR SALES P_YEAR GROWTH_PCT---------- --------- ----- ------- ------- ----------Australia Bounce 1999 1878Australia Bounce 2000 3151 1878 67.83Australia Bounce 2001 3761 3151 19.33Australia Y Box 1999 13773Australia Y Box 2000 27007 13773 96.09Australia Y Box 2001 59291 27007 119.54
SELECT country, product, year, sales, p_year, growth_pctFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSPARTITION BY (country)DIMENSION BY (product, year)MEASURES (sales, 0 p_year, 0 growth_pct)RULES ( p_year[product IN ('Bounce','Y Box'), year BETWEEN 1998 and 2001] = sales[CV(product), CV(year) -1] ,growth_pct[product IN ('Bounce','Y Box'), year BETWEEN 1998 and 2001] = 100* (sales[CV(product), CV(year)] - sales[CV(product), CV(year) -1] ) / sales[CV(product), CV(year) -1])ORDER BY country, product, year;
COUNTRY PRODUCT YEAR SALES P_YEAR GROWTH_PCT---------- --------- ----- ------- ------- ----------Australia Bounce 1999 1878Australia Bounce 2000 3151 1878 67.83Australia Bounce 2001 3761 3151 19.33Australia Y Box 1999 13773Australia Y Box 2000 27007 13773 96.09Australia Y Box 2001 59291 27007 119.54
SELECT country, product, year, sales, p_year, growth_pctFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSPARTITION BY (country)DIMENSION BY (product, year)MEASURES (sales, 0 p_year, 0 growth_pct)RULES ( p_year[product IN ('Bounce','Y Box'), ANY] = sales[CV(product), CV(year) -1] ,growth_pct[product IN ('Bounce','Y Box'), year IS ANY] = 100* (sales[CV(product), CV(year)] - sales[CV(product), CV(year) -1] ) / sales[CV(product), CV(year) -1])ORDER BY country, product, year;
COUNTRY PRODUCT YEAR SALES P_YEAR GROWTH_PCT---------- --------- ----- ------- ------- ----------Australia Bounce 1999 1878Australia Bounce 2000 3151 1878 67.83Australia Bounce 2001 3761 3151 19.33Australia Y Box 1999 13773Australia Y Box 2000 27007 13773 96.09Australia Y Box 2001 59291 27007 119.54
Analytics vs Model
LAG(sales,1) OVER (PARTITION BY prod ORDER BY year) p_year
p_year[prod IS ANY, year IS ANY] = sales[CV(prod), CV(year)-1]
SAME!Or is it…?
Model method:
COUNTRY PRODUCT YEAR SALES P_YEAR GROWTH_PCT---------- --------- ----- ------------ ------- ----------Australia Bounce 1999 1,878Australia Bounce 2000 3,151 1878 67.83Australia Bounce 2001 3,761 3151 19.33Australia Bounce 2003 8,790
Analytic method:
COUNTRY PRODUCT YEAR SALES P_YEAR GROWTH_PCT---------- --------- ----- ------------ ------- ----------Australia Bounce 1999 1,878Australia Bounce 2000 3,151 1878 67.83Australia Bounce 2001 3,761 3151 19.33Australia Bounce 2003 8,790 3761 133.73
Rule RepetitionSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year)MEASURES (sales)RULES UPSERT ( sales['Mouse Pad', 2008] = 1.3 * sales['Mouse Pad', 2001] ,sales['Bounce', 2008] = 1.3 * sales['Bounce', 2001] ,sales['Y Box', 2008] = 1.3 * sales['Y Box', 2001]);
PRODUCT YEAR SALES---------- ----- -------Y Box 2008 77078Bounce 2008 4888Mouse Pad 2008 4358
3 rows selected.
Symbolic ReferenceSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year)MEASURES (sales)RULES UPSERT ( sales[product IN ('Mouse Pad', 'Bounce', 'Y Box') ,2008] = 1.3 * sales[CV(product), 2001] );
no rows selected
Positional ReferenceSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year)MEASURES (sales)RULES UPSERT ( sales[FOR product IN ('Mouse Pad', 'Bounce', 'Y Box') ,2008] = 1.3 * sales[CV(product), 2001] );
PRODUCT YEAR SALES---------- ----- -------Y Box 2008 77078Bounce 2008 4888Mouse Pad 2008 4358
3 rows selected.
Constants, Cell References, Queries, et al…
RULES UPSERT ( -- Constants sales[FOR product IN ('Mouse Pad', 'Bounce', 'Y Box') ,2002] = 1.3 * sales[CV(product), CV(year)-1] );
RULES UPSERT ( -- Cell References sales[FOR (product, year) IN ( ('Mouse Pad', 2002), ('Bounce',2002), ('Y Box',2002))] = 1.3 * sales[CV(product), CV(year)-1] );
RULES UPSERT ( -- Queries sales[FOR product IN (SELECT product FROM my_products) ,FOR year IN (2002)] = 1.3 * sales[CV(product), CV(year)-1] );
PRODUCT YEAR SALES---------- ----- -------Y Box 2002 77078Bounce 2002 4888Mouse Pad 2002 4358
et allia – en trappe!RULES UPSERT SEQUENTIAL ORDER ( sales['Bounce', FOR year FROM 2004 TO 2001 DECREMENT 1] = 1.3 * sales[CV(), CV()-1] )
PRODUCT YEAR SALES-------- ----- -------Bounce 2001 4097Bounce 2002 4889Bounce 2003Bounce 2004
4 rows selected.
Ordering FOR loopsRULES UPSERT AUTOMATIC ORDER ( sales['Bounce', FOR year FROM 2004 TO 2001 DECREMENT 1] = 1.3 * sales[CV(), CV()-1] )
PRODUCT YEAR SALES-------- ----- -------Bounce 2001 4097Bounce 2002 5326Bounce 2003 6924Bounce 2004 9001
4 rows selected.
ORDER BY in rulesRULES SEQUENTIAL ORDER (sales[ANY, ANY] = sales[cv(),cv()-1] )
ORA-32637: Self cyclic rule in sequential order MODEL
RULES (sales[ANY, ANY] ORDER BY YEAR ASC = sales[cv(),cv()-1])PRODUCT YEAR SALES NEW_SALES--------- ----- ------------ ----------Bounce 1999 1,878 0Bounce 2000 3,151 0Bounce 2001 3,761 0
RULES (sales[ANY, ANY] ORDER BY YEAR DESC = sales[cv(),cv()-1])
PRODUCT YEAR SALES NEW_SALES--------- ----- ------------ ----------Bounce 1999 1,878 0Bounce 2000 3,151 1877.67Bounce 2001 3,761 3151.35
NULL Measures• Cells that exist in array but are NULL• Cells not in the array at all
(treated as NULL)<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES...
KEEP NAVSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year) MEASURES (sales) KEEP NAVRULES UPSERT (sales['Widget', 2003] = sales['Bounce', 2002] + sales['Bounce', 2001],sales['Widget', 2002] = sales['Bounce', 2002] ,sales['Widget', 2001] = sales['Bounce', 2001]);
PRODUCT YEAR SALES-------- ----- ----------Widget 2001 3760.51Widget 2002Widget 2003
3 rows selected.
IGNORE NAVSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year) MEASURES (sales) IGNORE NAVRULES UPSERT (sales['Widget', 2003] = sales['Bounce', 2002] + sales['Bounce', 2001],sales['Widget', 2002] = sales['Bounce', 2002] ,sales['Widget', 2001] = sales['Bounce', 2001]);
PRODUCT YEAR SALES-------- ----- ----------Widget 2001 3760.51Widget 2002 0Widget 2003 3760.51
3 rows selected.
NULL Defaults
• 0 for numeric data• Empty string '' for character/string data• '01-JAN-2001' for date type data• NULL for all other data types
NULL Cell Reference
• Positional– sales [ANY]– sales [NULL]
• Symbolic– sales [product IS ANY]– sales [product IS NULL]
– sales [product = NULL] -- FALSE
PRESENTV, PRESENTNNVSELECT product, year, salesFROM sales_viewWHERE country = 'Australia'MODEL RETURN UPDATED ROWSDIMENSION BY (product, year) MEASURES (sales) --IGNORE NAV -- Makes no differenceRULES UPSERT (sales['Widget', 2003] = sales['Bounce', 2002] + sales['Bounce', 2001],sales['Widget', 2002] = PRESENTV(sales['Bounce', 2002],1,0) ,sales['Widget', 2001] = PRESENTNNV(sales['Bounce', 2001],1,0));
PRODUCT YEAR SALES-------- ----- ----------Widget 2001 1Widget 2002 0Widget 2003 3760.51
3 rows selected.
Reference Modelsscott@sw10g> select * from exch_rates;
COUNTRY EXCHANGE_RATE---------------- -------------Poland .25France .14Australia .93New Zealand 1.12United Kingdom .45
5 rows selected.
…
COUNTRY YEAR SALES DOLLAR_SALES---------------- ----- ------------ ------------Australia 2001 1,164,871 1,083,330France 2001 1,025,157 143,522United Kingdom 2001 1,674,752 753,638
Reference Models<prior clauses of SELECT statements>MODEL [main] [RETURN {ALL|UPDATED} ROWS] [reference models] [PARTITION BY (<cols>)] DIMENSION BY (<cols>) MEASURES (<cols>) [IGNORE NAV] | [KEEP NAV] [RULES [UPSERT | UPDATE] [AUTOMATIC ORDER | SEQUENTIAL ORDER] [ITERATE (n) [UNTIL <condition>] ] ( <cell_assignment> = <expression> ... )
Reference ModelsSELECT country, year, ROUND(sales) sales, ROUND(dollar_sales) dollar_salesFROM sales_view GROUP BY country, yearMODEL RETURN UPDATED ROWS REFERENCE conv_ref ON (SELECT country, exchange_rate FROM exch_rates) DIMENSION BY (country) MEASURES (exchange_rate)MAIN conversionDIMENSION BY (country, year) MEASURES (SUM(sales) sales, SUM(sales) dollar_sales)RULES(dollar_sales[FOR conversion.country IN ('France','Australia','United Kingdom'), 2001] = sales[CV(country), CV(year)] * 1.0 * conv_ref.exchange_rate[CV(country)]);
COUNTRY YEAR SALES DOLLAR_SALES---------------- ----- ------------ ------------Australia 2001 1,164,871 1,083,330France 2001 1,025,157 143,522United Kingdom 2001 1,674,752 753,638
3 rows selected.
Reference ModelsSELECT country, year, ROUND(sales) sales, ROUND(dollar_sales) dollar_salesFROM sales_view GROUP BY country, yearMODEL RETURN UPDATED ROWS REFERENCE conv_ref ON REFERENCE conv_ref ON (SELECT country, exchange_rate FROM exch_rates)(SELECT country, exchange_rate FROM exch_rates) DIMENSION BY (country) MEASURES (exchange_rate) DIMENSION BY (country) MEASURES (exchange_rate) MAINMAIN conversionconversionDIMENSION BY (country, year) MEASURES (SUM(sales) sales, SUM(sales) dollar_sales)RULES(dollar_sales[FOR conversion.country IN ('France','Australia','United Kingdom'), 2001] = sales[CV(country), CV(year)] * 1.0 * conv_ref.exchange_rate[CV(country)]conv_ref.exchange_rate[CV(country)]);
COUNTRY YEAR SALES DOLLAR_SALES---------------- ----- ------------ ------------Australia 2001 1,164,871 1,083,330France 2001 1,025,157 143,522United Kingdom 2001 1,674,752 753,638
3 rows selected.
Reference ModelsSELECT country, year, SELECT country, year, ROUND(sales) sales, ROUND(dollar_sales) dollar_salesROUND(sales) sales, ROUND(dollar_sales) dollar_salesFROM sales_view FROM sales_view GROUP BY country, yearGROUP BY country, yearMODEL RETURN UPDATED ROWSMODEL RETURN UPDATED ROWS REFERENCE conv_ref ON (SELECT country, exchange_rate FROM exch_rates) DIMENSION BY (country) MEASURES (exchange_rate)MAIN conversionDIMENSION BY (country, year)DIMENSION BY (country, year) MEASURES (SUM(sales) sales, SUM(sales) dollar_sales)MEASURES (SUM(sales) sales, SUM(sales) dollar_sales)RULESRULES(dollar_sales[FOR conversion.country IN ('France','Australia','United Kingdom'), 2001] = sales[CV(country), CV(year)] * 1.0 * conv_ref.exchange_rate[CV(country)]);
COUNTRY YEAR SALES DOLLAR_SALES---------------- ----- ------------ ------------Australia 2001 1,164,871 1,083,330France 2001 1,025,157 143,522United Kingdom 2001 1,674,752 753,638
3 rows selected.
Australia
0.93
Eh?SELECT v.country, v.year ,ROUND(SUM(sales)) sales ,ROUND(SUM(v.sales * r.exchange_rate)) dollar_salesFROM sales_view v, exch_rates r WHERE v.country IN ('France', 'Australia', 'United Kingdom')AND v.year = 2001AND v.country = r.countryGROUP BY v.country, v.yearORDER BY v.country;
COUNTRY YEAR SALES DOLLAR_SALES---------------- ----- ------------ ------------Australia 2001 1,164,871 1,083,330France 2001 1,025,157 143,522United Kingdom 2001 1,674,752 753,638
3 rows selected.
AmortizationSELECT c, p, m, pp, ipFROM mortgageMODELREFERENCE R ON (SELECT customer, fact, amt FROM mortgage_facts MODEL DIMENSION BY (customer, fact) MEASURES (amount amt) RULES (amt[any, 'PaymentAmt']= (amt[CV(),'Loan']* Power(1+ (amt[CV(),'Annual_Interest']/100/12),amt[CV(),'Payments'])* (amt[CV(),'Annual_Interest']/100/12)) / (Power(1+(amt[CV(),'Annual_Interest']/100/12), amt[CV(),'Payments']) - 1)))DIMENSION BY (customer cust, fact)MEASURES (amt)MAIN amortization PARTITION BY (customer c) DIMENSION BY (0 p) MEASURES (principalp pp, interestp ip, mort_balance m, customer mc) RULES ITERATE(1000) UNTIL (ITERATION_NUMBER+1 =r.amt[mc[0],'Payments']) (ip[ITERATION_NUMBER+1] = m[CV()-1] * r.amt[mc[0], 'Annual_Interest']/1200 ,pp[ITERATION_NUMBER+1] = r.amt[mc[0], 'PaymentAmt'] - ip[CV()] ,m[ITERATION_NUMBER+1] = m[CV()-1] - pp[CV()])ORDER BY c, p;
SELECT country, year, ROUND(sales) sales, ROUND(dollar_sales)
dollar_salesFROM sales_view GROUP BY country, yearMODEL RETURN UPDATED ROWS REFERENCE conv_ref ON (SELECT country, exchange_rate FROM exch_rates) DIMENSION BY (country) MEASURES (exchange_rate)
IGNORE NAVMAIN conversionDIMENSION BY (country, year) MEASURES (SUM(sales) sales, SUM(sales)
dollar_sales) IGNORE NAVRULES(conv_ref.country['Australia'] = 0.97);
ERROR at line 1:ORA-03113: end-of-file on communication channel
Reference models are read only
SELECT country, year, SUM(sales) salesFROM sales_view GROUP BY country, yearMODEL RETURN UPDATED ROWSMAIN conversionDIMENSION BY (country, year) MEASURES (SUM(sales) sales) IGNORE NAVRULES(sales[FOR country IN ('France','Australia','United Kingdom'), 2002]
= sales[CV(country), CV()-1] * 1.2);
ERROR at line 1:ORA-00934: group function is not allowed here
Aggregate can’t be in SELECT/ORDER BY
SELECT *FROM sales_viewWHERE country = 'Australia'MODEL DIMENSION BY (product, year)MEASURES (sales)RULES UPSERT(sales['Bounce', 2003] = sales[CV(), CV()-1] + (SELECT SUM(sales) FROM sales_view));
ERROR at line 1:ORA-32620: illegal subquery within MODEL rules
MEASURES (sales, (SELECT SUM(sales) FROM sales_view) AS grand_total)RULES UPSERT(sales['Bounce', 2003] = sales[CV(), CV()-1] + grand_total[CV(), CV()-1]);
Sub-queries
RULES UPSERT(sales[FOR product IN ( WITH kludge_with AS (SELECT prod_name FROM sh.products) SELECT prod_name FROM kludge_with) , 2003] = sales[CV(), CV()-1]);
ERROR at line 1:ORA-32632: incorrect subquery in MODEL FOR cell index
RULES UPSERT(sales[FOR cust_id IN (SELECT cust_id FROM sh.customers), 2003] = sales[CV(), CV()-1]);
ERROR at line 1:ORA-32633: MODEL subquery FOR cell index returns too
many rows
FOR Construct Limitations
SELECT id, n FROM all_objects WHERE object_id=1MODELDIMENSION BY (object_id id)MEASURES (object_name n)RULES ITERATE (:v) (n[iteration_number] =iteration_number)
ERROR at line 5:ORA-32607: invalid ITERATE value in MODEL clause
SELECT id, n FROM all_objects WHERE object_id=1MODELDIMENSION BY (object_id id)MEASURES (object_name n)RULES ITERATE (100) UNTIL :v(n[iteration_number] =iteration_number)
ITERATE UNTIL
Performance
• Replaces multiple joins/unions• Scalable in size & parallelism• Explain plan
Fill Datesscott@sw10g> select * from customer;
NAME AMT DT------ ---------- -----------Scott 117 17-jan-2008Scott 250 17-feb-2008Scott 300 17-apr-2008Scott 50 17-jun-2008Wade 1231 17-mar-2008Wade 2321 17-apr-2008Wade 3122 17-sep-2008Wade 59 17-oct-2008
8 rows selected.
Fill DatesNAME MTH AMT CUM_AMT------ --------- ---------- ----------Scott January 117 117Scott February 250 367Scott March 0 367Scott April 300 667Scott May 0 667Scott June 50 717Scott July 0 717Scott August 0 717Scott September 0 717Scott October 0 717Scott November 0 717Scott December 0 717Wade January 0 0Wade February 0 0Wade March 1231 1231Wade April 2321 3552Wade May 0 3552Wade June 0 3552Wade July 0 3552Wade August 0 3552Wade September 3122 6674Wade October 59 6733Wade November 0 6733Wade December 0 6733
24 rows selected.
NAME AMT DT------ ---------- -----------Scott 117 17-jan-2007Scott 250 17-feb-2007Scott 300 17-apr-2007Scott 50 17-jun-2007Wade 1231 17-mar-2007Wade 2321 17-apr-2007Wade 3122 17-sep-2007Wade 59 17-oct-2007
8 rows selected.
Analytic SolutionSELECT date_fill.name, TO_CHAR(real_dt,'Month') mth, NVL(amt,0) amt ,NVL(SUM(amt) OVER (PARTITION BY date_fill.name ORDER BY real_dt ),0) cum_amtFROM (SELECT name, TRUNC(dt,'mm') dt, SUM(amt) amt FROM customer GROUP BY name, TRUNC(dt,'mm') ) actual_data, (SELECT name, real_dt FROM (SELECT DISTINCT name FROM customer) ,(WITH mths AS (SELECT TRUNC(SYSDATE,'YYYY') real_dt FROM DUAL CONNECT BY LEVEL <= 12) SELECT ADD_MONTHS(real_dt,ROWNUM-1) real_dt FROM mths) ) date_fillWHERE date_fill.real_dt = actual_data.dt(+)AND date_fill.name = actual_data.name(+)ORDER BY date_fill.name, date_fill.real_dt/
Actual Data
Distinct list joined with conjured dates
Outer join actual data with full date list
Analytic ExplainExecution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=13 Card=8 Bytes=31 1 0 SORT (ORDER BY) (Cost=13 Card=8 Bytes=312) 2 1 WINDOW (SORT) (Cost=13 Card=8 Bytes=312) 3 2 HASH JOIN (OUTER) (Cost=11 Card=8 Bytes=312) 4 3 VIEW (Cost=6 Card=8 Bytes=104) 5 4 MERGE JOIN (CARTESIAN) (Cost=6 Card=8 Bytes=104) 6 5 VIEW (Cost=2 Card=1 Bytes=6) 7 6 COUNT 8 7 VIEW (Cost=2 Card=1 Bytes=6) 9 8 CONNECT BY (WITHOUT FILTERING) 10 9 FAST DUAL (Cost=2 Card=1) 11 5 BUFFER (SORT) (Cost=6 Card=8 Bytes=56) 12 11 VIEW (Cost=4 Card=8 Bytes=56) 13 12 SORT (UNIQUE) (Cost=4 Card=8 Bytes=56) 14 13 TABLE ACCESS (FULL) OF 'CUSTOMER' (TABLE) 15 3 VIEW (Cost=4 Card=8 Bytes=208) 16 15 SORT (GROUP BY) (Cost=4 Card=8 Bytes=232) 17 16 TABLE ACCESS (FULL) OF 'CUSTOMER' (TABLE) (Cost=Statistics---------------------------------------------------- 9 recursive calls 30 consistent gets 6 sorts (memory)
Model SolutionSELECT name, TO_CHAR(dt,'DD-MM-YYYY') dt, amt, cum_amt -- Model resultsFROM ( SELECT name, TRUNC(dt, 'MM') dt, SUM(amt) amt FROM customer GROUP BY name, TRUNC(dt, 'MM'))MODELPARTITION BY (name)DIMENSION BY (dt)MEASURES (amt, cast(NULL AS NUMBER) cum_amt) -- Define calculated colIGNORE NAVRULES SEQUENTIAL ORDER( amt[FOR dt FROM TO_DATE('01-01-2007', 'DD-MM-YYYY') TO TO_DATE('01-12-2007', 'DD-MM-YYYY') INCREMENT NUMTOYMINTERVAL(1, 'MONTH') ] = amt[CV(dt)] -- Apply amt for given date, if found ,cum_amt[ANY] = SUM(amt)[dt <= CV(dt)] -- Calculate cumulative)ORDER BY name, dt/
Essentially apply model to this data set
Conjure dates
Model ExplainExecution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=5 Card=8 Bytes=232 1 0 SORT (ORDER BY) (Cost=5 Card=8 Bytes=232) 2 1 SQL MODEL (ORDERED) (Cost=5 Card=8 Bytes=232) 3 2 SORT (GROUP BY) (Cost=5 Card=8 Bytes=232) 4 3 TABLE ACCESS (FULL) OF 'CUSTOMER' (TABLE) (Cost=3 Ca 5 2 WINDOW (IN SQL MODEL) (SORT)
Statistics---------------------------------------------------- 5 recursive calls 15 consistent gets 4 sorts (memory)
…and no joins
Also noteworthy cum_amt[any] = sum(amt)[dt <= cv(dt)] -- Sum dates before iteration date
Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=5 Card=8 Bytes=232 1 0 SORT (ORDER BY) (Cost=5 Card=8 Bytes=232) 2 1 SQL MODEL (ORDERED) (Cost=5 Card=8 Bytes=232) 3 2 SORT (GROUP BY) (Cost=5 Card=8 Bytes=232) 4 3 TABLE ACCESS (FULL) OF 'CUSTOMER' (TABLE) (Cost=3 Ca 5 2 WINDOW (IN SQL MODEL) (SORT)
cum_amt[any] = sum(amt)[any] -- Aggregate all dates, no window sort
Performance
• Replaces multiple joins/unions• Scalable in size & parallelism• Explain plan
216 + 1SELECT COUNT(*) FROM ( SELECT * FROM sales_mth_view MODEL RETURN ALL ROWS PARTITION BY (country) DIMENSION BY (product, year, month) MEASURES (sales) RULES UPSERT SEQUENTIAL ORDER (sales[ANY,ANY,ANY] = sales[CV(), CV(), CV()]*1.2 -- Plus forecast sales... ,sales[FOR product IN (SELECT prod_name FROM sh.products) ,FOR year FROM 2008 TO 2015 INCREMENT 1 ,FOR month FROM 1 TO 12 INCREMENT 1 ] = sales[CV(), CV()-8, CV()]*1.5))
sw10g> /
COUNT(*)---------- 156157
1 row selected.
Try that in Excel…
Performance
• Replaces multiple joins/unions• Scalable in size & parallelism• Explain plan
Sequential Automatic
Cyclic
yn
Ordered
Fast Acyclic Cyclic
Fast
Order
Sequential Automatic
Cyclic
yn
Ordered
Fast Acyclic Cyclic
Fast
Order
SELECT country, product, year, salesFROM sales_viewWHERE country IN ('Australia','Japan')MODEL UNIQUE DIMENSIONPARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)RULES UPSERT SEQUENTIAL ORDER(sales['Bounce',2003] = AVG(sales)[ANY,2002] * 1.5,sales['Y Box', 2003] = sales['Bounce',2003] * .25);
Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=ALL_ROWS 1 0 SQL MODEL (ORDERED FAST) 2 1 SORT (GROUP BY) 3 2 HASH JOIN 4 3 TABLE ACCESS (FULL) OF 'TIMES' (TABL 5 3 HASH JOIN 6 5 TABLE ACCESS (FULL) OF 'PRODUCTS' 7 5 HASH JOIN 8 7 HASH JOIN 9 8 TABLE ACCESS (FULL) OF 'COUNTR 10 8 TABLE ACCESS (FULL) OF 'CUSTOM 11 7 PARTITION RANGE (ALL) 12 11 TABLE ACCESS (FULL) OF 'SALES'
Cyclic
y
Cyclic
Order
SELECT country, product, year, salesFROM sales_viewWHERE country in ('Australia','Japan')MODEL UNIQUE DIMENSIONPARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales)IGNORE NAV RULES UPSERT AUTOMATIC ORDER(sales['Y Box',2003] = 0.25 * sales['Bounce', 2003],sales['Bounce',2003] = sales['Y Box',2003] + sales['Bounce',2002]);
Execution Plan---------------------------------------------------------- 0 SELECT STATEMENT Optimizer=ALL_ROWS 1 0 SQL MODEL (CYCLIC) 2 1 SORT (GROUP BY) 3 2 HASH JOIN 4 3 TABLE ACCESS (FULL) OF 'TIMES' (TABLE) 5 3 HASH JOIN 6 5 TABLE ACCESS (FULL) OF 'PRODUCTS' (TABL 7 5 HASH JOIN 8 7 HASH JOIN 9 8 TABLE ACCESS (FULL) OF 'COUNTRIES' ...
Automatic
Why?When?
1. New language?2. Array/Sets3. Recursion4. Flexibility & Power5. Can it be done in SQL?
sales[cv()-1]
• DECODE• Analytics• Model
– CV()– Reference Model
DECODESELECT country ,SUM(DECODE(year,2007,sales,NULL)) current_year ,SUM(DECODE(year,2006,sales,NULL)) previous_year ,SUM(DECODE(year,2005,sales,NULL)) before_thatFROM sales_viewWHERE year BETWEEN 2005 AND 2007GROUP BY country;
COUNTRY CURRENT_YEAR PREVIOUS_YEAR BEFORE_THAT-------- ------------ ------------- -----------Greece 27540 25067 20835Italy 23738 20831 22867
DECODESELECT country, year ,SUM(DECODE(year,2007,sales,NULL)) current_year ,SUM(DECODE(year,2006,sales,NULL)) previous_year ,SUM(DECODE(year,2005,sales,NULL)) before_thatFROM sales_viewWHERE year BETWEEN 2005 AND 2007GROUP BY country, year;
COUNTRY YEAR CURRENT_YEAR PREVIOUS_YEAR BEFORE_THAT-------- ----- ------------ ------------- -----------Italy 2005 22867Italy 2006 20831 Italy 2007 23738 Greece 2005 20835Greece 2006 25067 Greece 2007 27540
AnalyticsSELECT country, year, current_year, previous_year, before_thatFROM ( SELECT year, country, tot_sales current_year ,LAG(tot_sales) OVER (PARTITION BY country ORDER BY year) previous_year ,LAG(tot_sales,2) OVER (PARTITION BY country ORDER BY year) before_that FROM ( SELECT country ,year ,SUM(sales) tot_sales FROM sales_view WHERE year BETWEEN 2005 AND 2007 GROUP BY year, country)) WHERE year = 2007;
COUNTRY YEAR CURRENT_YEAR PREVIOUS_YEAR BEFORE_THAT---------- ----- ------------ ------------- -----------Greece 2007 27540 25067 20835Italy 2007 23738 20831 22867
AnalyticsSELECT country, year, current_year, previous_year, before_thatFROM ( SELECT year, country, tot_sales current_year ,LAG(tot_sales) OVER (PARTITION BY country ORDER BY year) previous_year ,LAG(tot_sales,2) OVER (PARTITION BY country ORDER BY year) before_that FROM ( SELECT country ,year ,SUM(sales) tot_sales FROM sales_view WHERE year BETWEEN 2005 AND 2007 GROUP BY year, country)) WHERE year >= 2005;
COUNTRY YEAR CURRENT_YEAR PREVIOUS_YEAR BEFORE_THAT---------- ----- ------------ ------------- -----------Greece 2005 20835Greece 2006 25067 20835Greece 2007 27540 25067 20835Italy 2005 22867Italy 2006 20831 22867Italy 2007 23738 20831 22867
Model – CV()SELECT country, year, sales, previous_year, before_thatFROM sales_viewGROUP BY year, countryMODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (year) MEASURES (SUM(sales) AS sales ,CAST(NULL AS NUMBER) previous_year ,CAST(NULL AS NUMBER) before_that) RULES ( previous_year[2007] = sales[CV()-1] ,before_that [2007] = sales[CV()-2] );
COUNTRY YEAR SALES PREVIOUS_YEAR BEFORE_THAT-------- ----- ------------ ------------- -----------Greece 2007 27,540 25067 20835Italy 2007 23,738 20831 22867
Model – CV()SELECT country, year, sales, previous_year, before_thatFROM sales_viewGROUP BY year, countryMODEL RETURN UPDATED ROWS PARTITION BY (country) DIMENSION BY (year) MEASURES (SUM(sales) AS sales ,CAST(NULL AS NUMBER) previous_year ,CAST(NULL AS NUMBER) before_that) RULES ( previous_year[ANY] = sales[CV()-1] ,before_that [ANY] = sales[CV()-2] );
COUNTRY YEAR SALES PREVIOUS_YEAR BEFORE_THAT-------- ----- ------------ ------------- -----------Greece 2005 20,835Greece 2006 25,067 20835Greece 2007 27,540 25067 20835Italy 2005 22,867Italy 2006 20,831 22867Italy 2007 23,738 20831 22867
Model - ReferenceSELECT country, year, sales, prev, befFROM sales_viewGROUP BY country, yearMODEL REFERENCE r ON (WITH years AS (SELECT TO_CHAR(SYSDATE,'YYYY')-LEVEL+1 y FROM DUAL CONNECT BY LEVEL <=10) SELECT y, y-1 prev, y-2 bef FROM years) DIMENSION BY (y) MEASURES (prev, bef)MAIN calcPARTITION BY (country)DIMENSION BY (year)MEASURES (SUM(sales) AS sales ,CAST(NULL AS NUMBER) prev, 0 bef)RULES ( prev[ANY] = sales[r.prev[CV(year)]] ,bef [ANY] = sales[r.bef [CV(year)]]);
Give it time, it will grow on you…