baking or cooking – learn the difference between sas ...€¦ · learn the difference between sas...

Copyr i g ht © 2013, SAS Ins t i tu t e Inc . A l l r ights reser ve d .

BAKING OR COOKING –LEARN THE DIFFERENCE BETWEEN

SAS ARRAYS AND HASH OBJECTS

WISCONSIN ILLINOIS SAS USERS GROUPMILWAUKEE

29 JUNE 2016

CHARU SHANKARSAS INSTITUTE INC.

2

Teaching Experience• Computer languages• English language• Business communication skills

SAS experience• SAS programming• SQL language• DS2• Hadoop• Business Intelligence classes

Hobbies• Writing-SAS training blog• Fitness-Yoga instructor• Singer• Working with children• Cooking blog & catering

About CharuSome fun facts• 9 years at SAS Canada July 4th

• Born & raised in India with sunshine 365 days/year

• Never dreamed I would live in Canada with cold 365 days of the year

• This Is my 4th time presenting at WIILSU.• I find coding & teaching creative processes

like singing & cooking • I like helping others. If you are looking for

work, join the linkedin in group I’ve created 21-day free SAS challenge

Thanks LeRoy and all of you for continuing to find value in my teaching & invite me back. And my manager at SAS Canada & all the SAS folks that make this happen

3

Baking or Cooking – learn the difference between SAS arrays and hash objects

1. Bake - Using Arrays

2. Cook - Using Hash Objects

3. Compare and contrast

Copyright © 2012, SAS Institute Inc. All rights reserved.

Is this oven baked or cooked on the stovetop?

cooked


oven baked or cooked stovetop?

cooked


baked




cooked

8

Overview of Arrays (Review)An array is similar to a row of numbered buckets.

SAS puts a value in a bucket based on the bucket number.

A value is retrieved from a bucket based on the bucket number.

1 2 3 4

9

Baking or Cooking –learn the difference between SAS arrays and hash objects



3. Compare and contrast

10

Defining Arrays (Review)An array is a temporary grouping of SAS variables that are arranged in a particular order and identified by an array name.The following tasks can be accomplished using an array: performing repetitive calculations on a group

of variables creating many variables with the same attributes restructuring data performing a table lookup with one or more numeric factors

An array exists only for the duration of the currentDATA step.

11

Using One-Dimensional Arrays (Review)To use an array, declare the array by using an ARRAY statement.General form of the one-dimensional ARRAY statement:

ARRAY array-name {number-of-elements} <$> <length><list-of-variables> <(initial-values)>;

12

Objectives Load an array from a SAS data set. Use the array as a look up table to match records.

Chef wants to create a food theme with an international menu for the summer

You have been asked to provide food names with country origin(match country codes to country names)

• Country has country names, codes & food names• CodeFoodDesc has food name, description & country code• Using country dataset as input to the array, we want to

populate the new dataset by using the array as a lookup table

13

Business Scenario – WiilsuThe dataset CodeFoodDesc contains food name, description & country code

partial listing of CodeFoodDesc

14

Business ScenarioTo help the chef, you need to combine CodeFoodDesc with the Country dataset which contains country names

partial listing of Country

15

Using a One-Dimensional Array

data baking(keep=code country food description);array C{&num} $255 _temporary_ (&maxnum*' ');if _n_=1 then do code =1 to &maxnum;

set bakecook.country;C{code}=country;

end;/* array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;country=c{code};run;

4

16

Execution

Partial PDVC1 C2 C3 C4 C5 . . .

. . . . .

...

C32 C33 . . . C860 Code Country Food Description _N_

. . . . . 1

D D D D

D D D D D

data baking(keep=code country food description);array C{&num} $255 (&maxnum*' ');if _n_=1 then do code =1 to &maxnum;



D

17

Execution

Partial PDVC1 C2 C3 C4 .. C32

...

.. C36 . . . C860 Code Country Food Description _N_

32 Argentina 1

D D D

D D

D

data baking(keep=code country food description);array C{&num} $255 &maxnum*' ');if _n_=1 then do code=1 to &maxnum;



D

18

Execution


Argentina

...


32 Argentina 1

D D D

D D

D




D

19

Execution – fully loaded array


...


Australia Uzbekistan 860 Uzbekistan 1

D D D

D D

D




D

20

Execution


Argentina

...


AustraliaUzbekista

n. 1

D D D

D D

D

data baking(keep=code country food description);array C{&num} $255 (&maxnum*' ');if _n_=1 then do code=1 to &maxnum;



D

21

Execution


Argentina

...

.. C36 . . . C860 Code Country Food Description_N_

AustraliaUzbekista

n32 Asado Cuts of meat .. 1

D D D

D D

D




D

22

Execution


Argentina

...


AustraliaUzbekista

n32 Asado Cuts of meat .. 1

D D D

D D

D



end;/* array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;

run;

D

Country = C(code);

23

Execution


Argentina

...


AustraliaUzbekista

n32 Argentina Asado Cuts of meat .. 1

D D D

D D

D



end;/* array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;

run;

D

Country = C(code);

24

Execution


Argentina

...


AustraliaUzbekista


D D D

D D

D



end;/* array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;Country =C(code);run;

D

Implicit OUTPUT;Implicit RETURN;

25

Execution


Argentina

...


AustraliaUzbekista


D D D

D D

D



end;/* array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;Country =C(code);run;

D

26

Execution


Argentina

...


AustraliaUzbekista

n860

Uzbekistan

Plov A grain such as.. 1

D D D

D D

D



end;/*array retains values automatically if initialized in array statement*/set foods.'codefooddesc$'n;Country =C(code);run;

D

Continue until EOF.

27

Resulting Dataproc print data=baking noobs;run;

Using One Dimensional Arrays

Year_Obs Employee_ID Hired Salary Average Salary_Dif1 120101 2003 $163,040.00 $35,082.50 $127,957.502 120102 1989 $108,255.00 $88,588.75 $19,666.253 120103 1974 $87,975.00 $39,243.61 $48,731.394 120104 1981 $46,230.00 $36,436.67 $9,793.335 120105 1999 $27,110.00 $36,533.75 $-9,423.756 120106 1974 $26,960.00 $39,243.61 $-12,283.617 120107 1974 $30,475.00 $39,243.61 $-8,768.618 120108 2006 $27,660.00 $27,883.71 $-223.71

PROC PRINT Output

29

Review of Arrays Array

The subscript value(s) must be numeric.

One data value can be associated with the subscript value(s).

An array uses less memory than other in-memory lookup techniques. The size of the array is determined at compilation time.

Subscript values must be consecutive integers.

An array selects values by direct access based on the subscript value.Arrays can only be used in the DATA step.

30

Baking or Cooking –learn the difference between SAS arrays and hash objects



Compare and contrast

31

PollHave you used hash objects in SAS or other computer languages?

Yes No

32

DATA Step Hash ObjectsThe DATA step hash object has the following attributes: provides in-memory data storage and retrieval has a data component and a key component uses the key for quick data retrieval can store multiple data items per key does not require the data to be sorted is sized dynamically

The hash object is a good choice for lookups usingunordered data that can fit into memory.

33

Overview of a Hash Object (Review)A hash object is similar to rows of buckets that are identified by the value of a key.

SAS puts value(s) in the data bucket(s) based on the value(s) in the key bucket.

Value(s) are retrieved from the data bucket(s) based on the value(s) in the key bucket.

Key Data Data

34

DATA Step Hash Objects The hash object resembles a table with rows and columns. The columns have the following characteristics: can be numeric or character can be loaded from hardcoded values can be loaded from a SAS data set exist for the duration of the DATA step can be output to a SAS data set

35

DATA Step Hash Objects The key component has the following attributes: can consist of numeric and character values maps key values to data rows must be unique before SAS 9.2 can be composite

The data component has the following attributes: can contain multiple data values per key value can consist of numeric and character values

Data components and key components are DATA step variables.

36

Using Hash Objects The DATA step hash object has these characteristics: is created with a DECLARE statement has attributes and methods is manipulated with object dot syntax

An attribute is a property.A method is a function.

37

Objectives Load a hash object from a SAS data set. Use a hash object method to match records.

Chef wants to create a food theme with an international menu for the summer

You have been asked to provide food names with country origin(match country codes to country names)

• Country has country names, codes & food names• CodeFoodDesc has food name, description & country code• Using country dataset as input to hash object, we want to

populate the new dataset by using hash as a lookup table

38

Business Scenario – WiilsuThe dataset CodeFoodDesc contains food name, description & country code

partial listing of CodeFoodDesc

39

Business ScenarioTo help the chef, you need to combine CodeFoodDesc with the Country dataset which contains country names

partial listing of Country

40

Loading Data from a SAS Data Set into a hash object

data cooking;’Drop rc;length country $255;if _n_=1 then do;dcl hash C(dataset:'bakecook.country');

C.definekey('code');C.definedata('country');C.definedone();call missing(country);

end;set foods.'codefooddesc$'n;rc=C.find();run;

41 ...

rc _N_

. 1. . .D

Execution

DCountry Food Description Code

Partial PDV

data cooking;’Drop rc;length country $255;if _n_=1 then do;

set foods.'codefooddesc$'n;rc=C.find();run;

dcl hash C(dataset:'bakecook.country');C.definekey('code');C.definedata('country');C.definedone();call missing(country);

end;

42 ...

rc _N_

. 1. . .D

Execution

DCountry Food Description Code

Asado Cuts of Meat…

Partial PDV


rc=C.find();run;


end;set foods.'codefooddesc$'n;

43 ...

rc _N_

0 1. . .D

Execution

D


set foods.'codefooddesc$'n;

run;

Country Food Description Code

Asado Cuts of meat .. 32

Partial PDV


end;

rc=C.find();

44 ...

rc _N_

0 1. . .D

Execution

D



run;


Asado Cuts of meat .. 32

Partial PDV


end;

rc=C.find();

True

45 ...

rc _N_

0 1. . .D

Execution

D



run;


Argentina Asado Cuts of meat .. 32

Partial PDV


end;

rc=C.find();

46 ...

rc _N_

0 1. . .D

Execution

D



run;



Partial PDV


end;

rc=C.find();

Implicit OUTPUT;Implicit RETURN;

47 ...

rc _N_

0 1. . .D

Execution

D



run;



Partial PDV


end;

rc=C.find();

Continue until EOF.

48

Resultstitle 'International food theme for summer with country origins of foods';proc print data=cooking;run;

p306d02

Partial PROC PRINT Output

50

EfficiencyThe program created the variable rc and then dropped it. How can you avoid creating the variable so that you do not have to drop it?

a. Use a WHERE statement or a WHERE= data set option.b. Use a KEEP= or DROP= data set option in foods.’codefooddesc$’n.c. Test the result of the FIND method in the subsetting IF statement.d. Use a KEEP or DROP statement.

51

Multiple Choice Poll – Correct AnswerThe program created the variable rc and then dropped it. How can you avoid creating the variable so that you do not have to drop it?

a. Use a WHERE statement or a WHERE= data set option.b. Use a KEEP= or DROP= data set option in foods.’codefooddesc$’n.c. Test the result of the FIND method in the subsetting IF statement.d. Use a KEEP or DROP statement.

52

Not Creating rcThe program created the variable rc and then dropped it. How can you avoid creating the variable so that you do not have to drop it?

data cooking;’Drop rc;length country $255;if _n_=1 then do;dcl hash C(dataset:'bakecook.country');

C.definekey('code');C.definedata('country');C.definedone();call missing(country);

end;set foods.'codefooddesc$'n;

run;If C.find() = 0;

54

QuizHow do you know the length of the character variable Country?

55

Quiz – Correct AnswerHow do you know the length of the character variables Country

You use PROC CONTENTS, PROC DATASETS, or the Explorer window to view the descriptor portion of orion.supplier.

56

Defining PDV Variables dynamicallyInstead of the LENGTH statement, you can use an IF-THEN statement.

Because the IF condition is false during execution, the SETstatement is compiled, but not executed. The PDV includesall the kept variables from bakecook.country.

data cooking;’Drop rc;length country $255;if _N_=1 then do;

if 0 then set bakecook.country(keep=code Country);

declare hash C(dataset:'bakecook.country ');C.definekey('code');C.definedata('country');C.definedone();

end;set foods.'codefooddesc$'n;If c.find()=0;run;

57

Using DATA Set OptionsIn SAS 9.2, you can use SAS DATA set options to limit the amount of data loaded into a hash object.

data mostfoods;if _N_=1 then do;

if 0 then set bakecook.country(keep=code Country);

declare hash C(dataset:"bakecook.country (where=(code=&ccode))");C.definekey('code');C.definedata('country');C.definedone();

end;set foods.'codefooddesc$'n;If c.find()=0;run;

58

Advantages and Disadvantages of Hash ObjectsAdvantages Disadvantages

use of character and numeric keys

use of composite keys

faster lookup than formats or merges/joins

memory requirements

ability to be loaded from a SAS data set

fine level of control (flexibility)

Amt of memory your SAS session has available determines how big your hash object can be. Reducing the # of obs & restricting data items loaded into the hash object to only those that the program needs is a way to conserve memory. While it may be, it may be seem counter-intuitive, it may be more efficient to load your larger data into the hash object, esp, if it is your lookup data set. Action of reading your smaller data set sequentially and looking up information in a large hash object is likely to process more quickly than if you read your larger data set sequentially and look up info for each of its obs in a small hash object

59

Comparing Arrays and Hash ObjectsArray Hash Object

The subscript value(s) must be numeric. The keys can be character, numeric, or both.

One data value can be associated with the subscript value(s).

Multiple data items can be associated with the key value.

An array uses less memory than a hash object.

A hash object uses more memory than an array.

flexibility The size of the array is determined at compilation time.When you define an array with the ARRAY statement you must specify the number of elements in your array. If the number of elements changes the next time you use the data step, you must update the ARRAY statement, or possibly maintain additional code like macro programs that could update this for you.

The size of the hash object is determined at execution time.SAS dynamically allocates memory as it needs it. You do not have to determine the size of your hash object even if the next time you use it, you have many more obs to load into it

Subscript values must be consecutive integers. The keys do not have to be consecutive or sorted.An array selects values by direct access based on the subscript value.

A hash object uses a hash function for the lookup process.

Arrays can only be used in the DATA step. Hash objects can only be used in the DATA step.

60

Cool resources

Value of arrays Better hashingArrays in compile timeReading excel using xlsx engineDifferent ways of combining SAS tablesStep by step explanation of SAS arraysI cut my processing time by 90% by using hash objects

https://support.sas.com/resources/papers/97529_Using_Arrays_in_SAS_Programming.pdf

https://support.sas.com/resources/papers/sgf2008/better

https://support.sas.com/resources/papers/97529_Using_Arrays_in_SAS_Programming.pdf

http://blogs.sas.com/content/sasdummy/2015/05/20/using-libname-xlsx-to-read-and-write-excel-files/

http://www2.sas.com/proceedings/sugi22/CODERS/PAPER88.PDF

http://jpsmonline.umd.edu/SASOnlineTutor/sot12/en/60476/m35/m35_27.htm

http://www.lexjansen.com/nesug/nesug07/bb/bb16.pdf

61

C o p y r i g h t © 2 0 1 4 , S A S In s t i t u t e In c . A l l r i g h t s r e s e r ve d .

Questions and Comments


Contact

EMAIL [email protected]

SAS BLOG http://blogs.sas.com/content/sastraining/author/charushankar/

TWITTER CharuYogaCan

LINKEDIN https://ca.linkedin.com/in/charushankar

Thanks for your time Charu ShankarSenior Technical Training SpecialistSAS institute Inc.

http://www.sas.com/

http://www.sas.com/

baking or cooking – learn the difference between sas ...€¦ · learn the difference between sas...

Documents