Palo 1.0

Evaluation Guide

Published Apr 18, 2006

Authors: Kristian Raue, Holger Barczaitis, Torsten Wolff

© Jedox GmbH 2005, 2006

2

Table of Contents TABLE OF CONTENTS............................................................................................................................ 2

PALO BASICS.......................................................................................................................................... 4

How does Palo work?...................................................................................................................................................4 Cell-based 4Multidimensionality 4Hierarchies 8Memory-based 8

WORKING WITH PALO............................................................................................................................ 9

Setting up Palo..............................................................................................................................................................9 Local Setup 9

Before you start (only needed if you have previously installed Palo TP1)...............................................................................9 Setup.......................................................................................................................................................................................9

Settings 9Starting Palo 9

Server Setup................................................................................................................................................................10 Local Server Setup 10 Connect to a Remote Server 10

Creating a Palo-Worksheet ........................................................................................................................................11 Paste View 11 Paste Elements 17

Getting Data from a Cube to the Worksheet ............................................................................................................20 Paste Data Function 20

Creating Data structures............................................................................................................................................21 Modeller 21

Importing data from external sources ......................................................................................................................24 Importing Raw Data 24

Importing cell data.................................................................................................................................................................24 Importing Dimension Elements .............................................................................................................................................29

Importing from ODBC Source 35 Connecting the ODBC source: ..............................................................................................................................................35 Importing into Palo ................................................................................................................................................................35

Importing Data from external Excel Sheet 37 Entering data manually 40

Edit cells................................................................................................................................................................................40 Splashing (Writing into Aggregated Cells).............................................................................................................................42

Techniques..................................................................................................................................................................44 Text Elements 44 Calculated Elements (Push Rules) 46

APPENDIX.............................................................................................................................................. 50

Palo Spreadsheet Formulas ......................................................................................................................................50 PALO.DATA(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) ............................................................50 PALO.DATAC(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;)..........................................................50 PALO.DATAT(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) ..........................................................50 PALO.DATAV(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) ..........................................................50 PALO.EADD(Server/Database;Dimension;Type;Element;Parent Element;Weight;Clear) ....................................................50 PALO.ECHILD(Server/Database;Dimension;Parent Element;Element;Number of Child).....................................................50

3

PALO.ECHILDCOUNT(Server/Database;Dimension;Element).............................................................................................50 PALO.ECOUNT(Server/Database;Dimension) .....................................................................................................................50 PALO.EFIRST(Server/Database;Dimension)........................................................................................................................51 PALO.EINDENT(Server/Database;Dimension;Element).......................................................................................................51 PALO.EINDEX(Server/Database;Dimension;Element) .........................................................................................................51 PALO.EISCHILD(Server/Database;Dimension;Parent Element;Element) ............................................................................51 PALO.ENAME(Server/Database;Dimension;Position) ..........................................................................................................51 PALO.ENEXT(Server/Database;Dimension;Element)...........................................................................................................51 PALO.EPARENT(Server/Database;Dimension;Element;Number of parent elements) .........................................................51 PALO.EPARENTCOUNT(Server/Database;Dimension;Element).........................................................................................51 PALO.EPREV(Server/Database;Dimension;Element) ..........................................................................................................51 PALO.ESIBLING(Server/Database;Dimension;Element;Number of sibling) .........................................................................51 PALO.ETYPE(Server/Database;Dimension;Element)...........................................................................................................51 PALO.SETDATA(Value;Splash; Server/Database;Cube;Coordinate1; Coordinate2;… Coordinate16) ................................51

Acknowledgements ....................................................................................................................................................52

4

Palo Basics

How does Palo work?

Palo is a cell-based database that is multidimensional, hierarchical and memory-based. Now what do those terms mean in particular?

Cell-based If you run a query in a relational database, the smallest result which you will get is one record (usually consisting of several fields). If you run a query in Palo, your result is one single cell value. So Palo thinks in cells, not in records. This is what makes it so compatible with Microsoft Excel, because Excel is also cell-based.

Multidimensionality Multidimensionality is an alternative way of organizing data in a database. In a relational database, you are used to structures like tables, records and fields. In a multidimensional database you are thinking in terms of cubes and dimensions and cells. You know what a cell is, but what is a dimension and a cube? To answer the question it makes sense to take another look at Microsoft Excel. In the screenshot below you find a list of row titles in Excel.

In Palo this list is called a dimension. So basically a dimension is a list of categories that has some contextual relationship like a list of products, regions, customers or months. In Palo the above Excel list would be modelled as a dimension e.g. with the name „Products”.

5

So far in our Excel sample we have one dimension on the sheet. Now let’s add another dimension on the Excel sheet. It will be a horizontal list of months and quarters as you can see on the following screenshot. In Palo this dimension could be called „Months”.

Now there is an interesting point. What do you do in Excel if you want to organize your data in more than two dimensions? Yes, you can start to use the sheet tabs to create a third dimension, but what comes after that? Experience shows, that large organizations usually require five to ten dimensions to store their data according to their needs. An Excel Workbook only supports three of them. This is one of the reasons why Palo was made. Palo supports up to 256 Dimensions. It allows to structure data in a much more detailed manner than pure Excel. In order to do that, another thing is needed apart from the dimensions, and that is called a „cube”. Now let’s learn what a cube is. Let’s look at the two-dimensional model that we created in Excel so far. There is a cell range defined by the two dimensions „Products” and „Months” (Range C3:J11). This range consists of cells and each cell has a cell address (like D4 for example). In Palo this cell range would be called a two-dimensional cube.

6

So basically a cube is a collection of cells that is defined by two or more dimensions. Now the question remains, how do you access and display data from a multidimensional cube if Excel is only capable of displaying two dimensions on a worksheet at a time? To answer the question, we first have to look at the way a cell is referenced in Excel. Excel uses a simple naming scheme for cells; it’s called the A1-Style. So the „Product B” value for „February” is stored in cell D4. In a Palo data cube the value would be stored in a cell that does not use Letters and numbers as cell address, but the real-life description of the data instead. In our example the address of the cell would be „Product B”, „Feb”. To display data from a Palo cube in Excel, a special database function is used. It is called „PALO.DATAC”. The following formula would pull a value from the multidimensional data cube into an Excel cell (ignore Server name and Cube name at this point):

=PALO.DATAC(„Servername”,”Cubename”,” Product B”, „Feb”). It is easy to see that this data formula could hold more than two dimension coordinates, for example it might read like this to access a five-dimensional cube:

=PALO.DATAC(„Servername”,”Cubename”,” Product B”, „Feb”,”2006”,”Europe”, „Units”).

In the following screenshot you see how a 5-dimensional cube is displayed on an Excel spreadsheet using a 5-dimensional data formula.

If you follow the references in the data formula (in the formula bar) you will discover that the data formula references the strings in the row and column titles to get some of the coordinates, but it also references cells which are on the top of the page (cell A3, A4, A5). These cells contain the coordinates for those dimensions that could not be displayed as row or column titles. Using these „page selectors” is one way how you display views on a multi-dimensional database in a two-dimensional spreadsheet grid.

7

Keep in mind that you only have to change the content of cell A3 or A4 to see the data for a different year or a different region. Thus you can view a lot of data with just one Excel sheet. Think of how many sheets you would need if you wanted to manage the same amount of data just using Excel! A final question might be how you enter values into the database. This is actually very easy. You simply overwrite the formula in the spreadsheet (for example cell C9) with a value. The Palo Excel Add-In will then make sure, that the value is automatically saved to the specified cell in the cube. After that has happened, Palo will restore the original formula which then shows the new value.

8

Hierarchies Looking at the last screenshot you might wonder why the value for Productgroup1 and Total Products is also displaying the value 123, although we only entered a value for Product B. The answer to this question is hierarchical aggregation. The dimension Product is actually not a flat list. When we defined this dimension in the Palo database modeller, we added a hierarchy to the list, which looks like this:

The use of hierarchies when defining a dimension adds a lot of power to the Palo data cubes. With hierarchical dimensions, the cube is able to do multidimensional aggregations in the cube. So you don’t have to do these calculations with Excel formulas, instead you directly pull the aggregated values form the cube.

Memory-based Palo is memory-based. This basically means speed. Data retrieval of base cells and also of aggregated cells is done real time. No time consuming queries to run, results should be displayed within milliseconds (or seconds at the longest). There are other multidimensional databases on the market which are based on a relational technology (ROLAP databases). They have their advantages in terms of capacity, but they are usually slower and they usually do not support write-back of data to the cubes.

9

Working with Palo

Setting up Palo

Palo consists of four setup files which you can download according to your requirements:

• Local setup, including the Excel Add-In and a local Palo database setup • Windows Server setup • Linux Server setup • APIs

Local Setup Before you start (only needed if you have previously installed Palo TP1) To make sure all your modifications have been saved in your Palo database, start the existing Palo version and wait a couple of minutes to allow all entries which may be contained in the log files to be written to the database. Then close Palo again. Now please back up your data and uninstall your existing Palo version.

If during setup you indicate the installation subdirectory where the prior version was installed before, the procedure will attempt to convert your existing data into the format required by the new version of Palo. During this conversion, your existing data will be copied to a new directory named Jedox\Palo\old_data. Depending on the size of your .log files and the construction of your cubes, conversion could take extremely long and may not succeed in all cases. To make the conversion run as smoothly as possible, we recommend that you delete all the *.log and *.data files contained in Jedox\Palo\data\<databasename>. Your data will be lost. Note that you will find a change log listing new and modified features under Start-Programs-Palo after setup. Setup

Make sure all programs are closed, particularly Excel. Now start the setup routine. You may have to start your machine after setup, there will be a notice to inform you in that case.

Settings Open Start-Programs-Palo-Settings from the Start-Menu of the Windows-Desktop: Define whether the Palo Add-In will always be started together with Excel, and select your desired language for the Add-In. If you intend to work through this manual, you may find it useful to leave it in English for the time being.

Starting Palo Unless you defined that Palo will start together with Excel, start Palo by either clicking the desktop icon or opening it through the Start – Programs menu.

10

Server Setup

Local Server Setup With the installation of Palo a local server is installed on your computer. Within a network you can log into the local servers of other computers (which makes every local server factually a public server). You can set users and passwords to manage the rights of other users to access your server, but you must not add another server on the same computer.

Connect to a Remote Server To make a Server accessible for external clients you have to edit the \Jedox\Palo\data\config.xml.In the config.xml you will find a line reading: <Server hostname="localhost" service="1234" protocol="TCP"></Server> Under this line you have to add another line with the Server’s computer name:

<Server hostname="<COMPUTERNAME>" service="1234" protocol="TCP"></Server>

If you need to modify the authorization of your server, edit it in the \Jedox\Palo\data\auth.xml.

To connect to a remote Server you will have to register this Server one-time and will then have this server in your server list unless you unregister it again. To register a remote server start Palo and open Palo Wizard from the menu Palo:

In the next selection key in the server name. In the default settings Port is 1234, user is ‚user’ and password ‚pass’. ‚Connection name’ is simply a comment, type in some helpful description:

11

Creating a Palo-Worksheet

Palo comes with a demo database so you will not have to worry about database modelling at this point. Simply go through this instruction to understand the ways of working, using the demo database. Unless you defined that Palo will start together with Excel, start Palo by either clicking the desktop icon or opening it through the Start–Programs menu. Excel is opened, and you will see a new menu item Palo:

Paste View

The Paste View dialog offers a quick way of generating database views on a worksheet. Here you can select and arrange the dimensions and elements in the way you want to see them. If you are familiar with pivoting, you will recognize some of the functionality: Dimensions can be moved around to be either row or column titles or to serve as selection criteria.

Move for example the ‘Datatype’ dimension icon upward to the ‘Page Selector’ section. Move ‘Products’ to the ‘Row titles’ section. The result should look like this:

12

Once you moved a dimension to row or column titles, you can now click on the three dots (…) next to the dimension name (e.g. ‘Products’). A new dialog appears which lets you select the elements that you want to see in the resulting view. Click “Expand all” on top of the element list to display all levels of the dimension:

You see that the structure of the selected dimension as we defined it in the Demo-database is articulately mapped. (Note that you can display the elements either as a tree structure like in the example above or as a flat list in the order in which they were defined by unselecting the Show Tree checkbox) You are allowed to pick base elements and / or aggregated elements in any combination you like. There are several ways how you can pick the elements which you want to display:

• highlight the desired item in the left window (you can hold SHIFT or CTRL while highlighting to select multiple items) and move it to the list using the blue arrow button

• select the wanted items individually by double-clicking them • Use the selection tools in the middle of this window to select a whole group of elements: For

example click on ‘1’ to select all visible elements from the first level, ‘2’ for all elements from the second level and so on, or ‘B’ for all base elements.

A click on ‘Show all selection tools’ will display two more options:

• ‘Invert Selection’ means, if for example you selected all base elements with a click on ‘B’, ‘Invert Selection’ will deselect them and select all non-base elements instead.

• ‘Search & Select’ is a string search. You can either enter the complete name of the element or use

wildcards in front and/or behind the string. For example, ‘*TT’ finds ‘Server Power TT’ as well as ‘Notebook TT’.

13

To change the order in which the selected items will appear choose an item from the right window and move it up or down in the list with the “Arrow Up” or ”Arrow Down”. After you selected all items you wanted to be displayed finish the selection process with a click on the OK-Button. If there is nothing in the right window, a click on OK will paste any elements that are currently highlighted in the left window. In case there are elements in the right window, they will be pasted no matter what is highlighted in the left window.

Now select all the elements from the second level by clicking ‘2’. Move them to the right window:

(Note that you can sort the elements in your selection in ascending/descending alphabetical order) Click OK to apply your selection. Proceed accordingly to have ‘Units’ as a column. After you clicked OK and Paste, this is what you should have (some of the numbers displayed here are negative since the selected data type is ‘Variance’):

14

Yellow fields in the upper left corner represent all the other dimensions of the current cube which are neither rows nor columns but can serve as selection criteria. You can double click them to select a different element, like for example a different year, a group of countries or a single country in Europe:

With the Paste View command we could easily create another view on our data: Click on the Paste View command in the Palo menu and change the setup of the page. Move Measure to Page selector and move the ‘Months’ and ‘Datatypes’ dimensions to Column titles:

15

Now click on the ‘…’ button of Months and select Jan, Feb and Mar. Also select ‘Budget’ and ‘Actual’ from dimension Datatype by clicking ‘…’. After you pasted your selection, this is what you will see:

The row and column headers of the table do not just indicate the name of the rows and columns but they provide additional functions if you double click them:

• The free area on the top left (here A7 and A8) will open the Paste View dialog where you can adjust your table settings

• A consolidated element will switch between showing and hiding all subordinate elements • If you select Show element selector on doubleclick:

The first element in a column or row will directly open the Paste View menu for the selected dimension where you can edit your table settings

16

Notice We would like to draw your attention to a very important concept in Palo. To make it visible, set the cursor to cell C9 and press F2 to see the formula and its references:

The cell C9 holds the function PALO.DATAC(). Its parameters are: Server/Database and Cube name as found in cell A1 and A2 plus the cell coordinates in the data cube, in our example as follows:

• $A9 = Stationary PC’s • $A$3 = Europe • C$7 = Jan • $A$4 = 2002 • C$8 = Actual • $A$5 = Units

With these arguments the PALO.DATAC function is able to pull a value from the very cell in the cube that is exactly identified through the supplied arguments. The PALO.DATAC function is the most important function in Palo because it links Excel cells to Palo database cells. Note how the $ character is applied in the suitable way to make sure copying and pasting of formulas will produce the expected result. Among other cells the formula refers to cell C7 which appears to be empty. This is only for design reasons, when you click on C7 you will see that it holds the string ‘Jan’ just like the one left to it. Nothing is displayed however due to its custom cell format ‘;;;’. These three semicolons format a field to display nothing no matter what it contains, so it can be used in formulas even though there is nothing to see.

17

Paste Elements Instead of using the Paste View function, you can also insert element names on your spreadsheet using the Paste Elements function. Create a new worksheet and make sure the cursor is in the upper left corner (A1). Click on the „Paste Elements” command in the Palo Menu:

Selection tools and logic are similar to the Paste Elements dialog in ‘Paste View’ apart from the fact that here you see all the dimensions. Note that you can only select items out of one dimension at a time. In case you selected only one single item, you will only have a ‘Paste’ button, in case there are more than one there will be a ‘Paste horizontally’ and a ‘Paste vertically’ button.

18

Let us start with the insertion of page selector elements: Open dimension ‘Months’, click on ‘Year’ and click ‘Paste’. The window closes and in Excel you will see that cell A1 now displays ‘Years’ and at the same time contains a formula:

Double click A1. A drop down menu opens, allowing you to choose a different element from the same dimension:

Set the cursor to A2 and open the ‘Paste Elements’ dialog again. Select ‘Europe’ from the ‘Regions’ dimension, pasting it into cell A2. Now we have two elements in the page selector section.

19

The next thing will be to create rows: Move the cursor to cell B5 and open Paste Elements again. This time, open the ‘Products’ dimension and select all items from the second level by clicking ‘2’:

Paste them vertically. This is what you should have now:

In order to have column titles, move the cursor to cell C4. Open the Paste Elements dialog again and select ‘Budget’ and ‘Actual’ from the ‘Datatypes’ dimension and paste them horizontally. Our table should now look like this:

Last step: We need to insert the adequate formulas to retrieve the data from the Palo database. For this we will use the Paste Data function.

20

Getting Data from a Cube to the Worksheet

Paste Data Function Mark the entire data section in your table and select ‘Paste Data Function’ from the Palo menu: You will have to indicate the database and cube where the data is stored, there is only one of each at the moment. Be sure the ‘Guess Arguments’ checkbox is checked, and then click ‘Paste’.

Palo will now analyze the surrounding cells and compose a formula that seems to make sense in this context:

If you select one of the newly inserted datacells (for instance C5) you will see that again the PALO.DATAC-Function is fetching the right data and inserts it into the worksheet:

=PALO.DATAC("localhost/Demo";"Sales";$B5;$A$2;$A$1;"2002";C$4;"Units")

21

Creating Data structures

Modeller Data structures will be composed from cubes, dimensions and elements (= single cells). Now we will see how to set up new cubes, dimensions and elements using the modeller. This is what you will see when opening the modeller:

You may expect that you would have to create a cube first and then define its dimensions; however you will see that it is logical to create dimensions first and then define one or more cubes which can hold different combinations of dimensions.

Let us add a new dimension ‘Customers’: Click the ‘add’ icon in the lower left corner of the screen. A new item is inserted in the left window and you can type in the name of the new dimension. After you pressed return, the new dimension will appear immediately as an additional tab at the top of the window:

22

Either double clicking the new entry in the left window or a click on the tab at the top will open a new window where you can add elements to the new dimension. Press ‘Return’ to create the first element. Create four elements: ‘Smith Corp’, ‘Meyer Ltd’, ‘Miller LLC’ and ‘Total’:

Next thing we will assign the customers we created to ‘Total’ as a consolidated element: Double click Total. Total is now bold, and in the right window you will see the message ‘Add Source Elements by double clicking in the list of elements…’. This means, you are about to define which sub elements belong to the element Total. You can select items either by a double click or by highlighting them and moving them to the right with the blue arrow button. Select those three customer names which you created before and click the green OK icon. You will see that the symbols alongside every element have now changed: Total is shown as a consolidated element, the others as numeric elements. Check the ‘Show Tree’ checkbox to see the structure like this:

Note that you can use the mouse to move the elements within the dimension’s structure.

Of course you can set up multiple levels here, but we will leave that for now and, as a last step, create a new cube. Return to the database tab by a click on ‘Database’. Now click on the ‘New Cube’ icon:

23

This action takes you to the Cube Creation Wizard. Enter a name for the new cube; let us call it ‘Market’. Add the dimensions Datatypes, Months, Measures, Products, Regions and Years to the new cube by a double click or using the blue arrow button. A click on ‘Finish’ will create the new cube.

Now we have created another cube. We could use it the same way we did with the prefabricated cube “Sales” which we used in the examples before. When you try this (for example using the Paste Data Function) you will see that the new cube suffers a big drawback - as a newly created data structure our new cube contains no data yet:

Of course you could now start to fill the cube by entering data into the single cells. But most likely you already have the data to be existent at an external data source. So now we have to think of how to get this data into the cube.

24

Importing data from external sources

When you take a closer look, you will notice that our new cube ‘Market’ was built using the same dimensions that were already used in ‘Sales’. So regarding the data structure ‘Market’ is a copy of ‘Sales’, but it contains no data yet. In this chapter we will learn how to import external data into our new cube.

Importing Raw Data Importing cell data Together with the Palo setup comes a flat file named ‚testimport.txt’ which you will find in „My Documents\Jedox”. It contains some 30.000 records which look like this:

This could be a typical data export from any database system, reflecting budgeted and actually sold numbers of units from various countries and periods. The contents of the ‘Sales’ cube, the structure of which you can see in the Palo Modeller, is based on these records. We will now demonstrate how data import is accomplished. In this case, where we work with the demo database, we can start the Import Wizard right away, but please bear in mind that in any other case you will have to define a cube with its dimensions in the Palo Modeller before you can import data. Clear your Excel sheet and click on Data Import in the Palo menu. This is what you will see:

25

We will explain the ODBC interface later on; let us start with the Flat Text file option: Click ‚Next’.

In the next screen you will be asked for the source path and filename so click on the ‘Browse’ Button and select the Palo test import data which is in

My Documents\Jedox\testimport.txt Now indicate the field separator. In our example fields are tab separated. There are no decimals in our test data so you can leave the decimal point option unchanged. Click ‚Next’ to proceed to this screen:

Now click ‚Next’ once and then ‚Close’. This is what you will see in your Excel worksheet:

This is the first record of our raw data. Once we will click ‚Finish’ instead of ‚Next’ or ‚Cancel’ in the Data Import Wizard, all the records will be inserted into row 1 one after the other and the Excel worksheet will be recalculated after each record. The actual import will then be brought about through a formula in the worksheet which is executed again and again, with the values of each record. What we will have to do now therefore is to create this formula to import the contents of row 1 into Palo.

If you already went through the chapter on the Paste Data Function, you will remember that it can ‚guess’ the parameters of formulas. This is what we will now make use of: Move the cursor to a cell beneath the last value in row 1, for example F4. It is important that the cell resides in the same column as the last field of the import record, or in a column to the right of it. Now click ‚Palo – Paste Data Function’. Select the database and cube, activate „Guess Arguments“ and select the formula ‚PALO.DATA’, then click ‚Paste’:

26

This is what you will see after you pressed F2:

Most of the components of the target address have been identified correctly: Server/Database, cube and, in this case, six dimension elements to define a cell in the ‚Sales’ cube. But „Variance“ is not quite correct: „Variance“ is a consolidated element, the difference between Budget and Actual. And now we understand why the formula parameters are not all complete: The current field value in this place is ‚Planned’, the respective element name however is ‚Budget’. (You might wonder why there is an element ‘Planned’ in the importfile. We arranged this to show you how to fit the DATA-function in case of mismatches between the old data structure and the new one that you have defined in your Palo cube. Just imagine the situation that we don’t want to keep the old name ‘Planned’ from the external data source but with the import to Palo we want it to be ‘Budget’ now.) To deal with this, we will define an import rule: If E1 reads ‚Planned’, select element ‚Budget’. Insert the following formula into E2:

=IF(E1="Planned";"Budget";E1) Then change to DATA-function in F4 by replacing the wrong reference “Variance” by a reference to $E$2:

Now select ‚Data Import’ from the Palo menu again in order to check if the formula is also correct for the other records. The options which you indicated before are still there, so you can simply click ‚Next’ twice to see the import window again. Now click ‚Next’ a number of times and check if the formula in F4 produces the expected result. After a few records, you will encounter this:

27

PALO.DATA does not retrieve a cube cell with the coordinates it has. Click ‚Cancel’ to close the wizard. A right click on cell F4 displays the menu shown at the lefthandside.

Select ‚PALO’ to see the error message which shows what the problem is:

(until messages will come in clear text please use: http://forum.palo.net/thread.php?threadid=107)

Checking the elements in dimension ‚Months’ shows that the proper name should read ‚Mar’ instead of the German abbreviation ‚Mrz’. So what we will have to do is create a translation table like in the screenshot below:

Now update the DATA Function again to implement the recent changes:

Start the Data Import Wizard again and check if the formula now works correctly with all of the records.

28

Once you can confirm that it is now time to actually write into the cube. The PALO.DATA function has just been a test whether the imported data structure will match the data structure of our cube. To write the external data to the cube we use the PALO.SETDATA function. It uses the same parameters we just have found for the PALO.DATA function. But as it will be writing into the cube PALO.SETDATA has two additional parameters: - the value which should be set of course

- the SPLASH parameter (TRUE or FALSE) by which you indicate whether or not you want to write values into consolidated elements and have them broken down to their respective elements. These two parameters are the first ones after the bracket:

In some installations the value in F1 is not recognized as a number even if the field is formatted properly. In that case Palo will produce an error message (‚cannot write string to a number cell’). To overcome this, add to G1 the formula ‘=VALUE(F1)’ and refer to it in the PALO.SETDATA function:

=PALO.SETDATA($G$1;FALSE;"localhost/Demo";"Market";$E$2;$D$2;"Units";$B$1;$A$1;$C$1)

Save the worksheet and start Data Import again. To be sure everything is correct, step through a number of record imports by clicking ‚next’. If the write is performed properly, PALO.SETDATA will return the value it wrote to the cube:

Once you are sure no problems remain, restart the import and click ‚Finish’. After the data import has finished clear the Excel sheet and use the Paste Data function to satisfy yourself that all the data is now stored in our previously empty cube ‘Market’.

In case you need to know if an import of data is currently in progress you can use the _paloimportactive variable. You find it in the menu Insert – Names of Excel after the first import was made. It will have the value TRUE while data is imported and will be reset to FALSE immediately after the import finished.

29

Importing Dimension Elements The Palo function PALO.EADD allows the import of dimension elements from a flat file. Again we prepared an example to show how it works. In this example, we will import the elements of three dimensions in one go: Time, Costcenter and Region. In order to avoid conflicts with other demo data, create a new database which contains these three dimensions. Open the Palo Wizard to create the new database:

After clicking ‘Next’ enter ‘Demo1’ as our new databases name:

30

Now open the modeller, select the new database and create three new dimensions ‘Time’, ‘Costcenter’ and ‘Region’ as in the screenshot below:

Close the modeller. In the folder „My Documents\Jedox” you will find an Excel file named metadataimport.xls, please open it:

31

You will notice that row 1 is empty. Like in the previous chapter where we demonstrated how to import data, the Palo Data import function will populate this row line by line with the contents of the sample flat file metadata.txt (in the same directory). The records which it contains look like this:

You will find the column headers again in the spreadsheet in row 2. This is mainly to make more transparent what happens in the worksheet. Only cell A2 is referred to in a formula (from B5): If A1 equals A2, we assume that the currently imported row is a header row and it will be skipped. Cell B4 actually contains the name of the database which you created before. ( Please make sure that it is: localhost/Demo1 )

To understand in detail what the formulas in this worksheet do, let us now start with the import of the first record: click Palo – Data Import to open the Import Wizard, select ‘Flat Textfile’ and click ‘Next’: You will find the source file ‘metadata.txt’ under „My Documents\Jedox. Its fields are tab separated so make sure that the option ‘Tab’ is selected and click ‘Next’: The first line will be read with another click of ‘Next’ - It contains the column headers:

32

Now click ‘Next’ once more to import the first actual record:

Now we will take a closer look at the three sections where the element names are calculated and written to the cube. Let us start with the fields under ‘Time dimension’. As you will see, month, quarter and year are calculated from the date field in cell A1:

To put it in OLAP terms, base entities of the time dimension are months. Months will be consolidated to quarters and quarters again will be consolidated to years. Now let us look at the PALO.EADD function in cell B13 (which returns FALSE while the import loop is not executed):

33

The ‘IF’ argument in the formula makes the function skip the header row.

The PALO.EADD function requires the following parameters:

Server / Database String Dimension String Type You can always use element type „n” (base element). The element type

will be converted to „c” anyway in case the import loop assigns sub elements to it later on.

Element Name String Parent Name String Weight Number Clear Erase all elements in dimension when starting (TRUE / FALSE).

Note: Make sure all occurrences of the EADD function in the worksheet read the same (TRUE or FALSE) since otherwise the result is arbitrary.

The formula in B13 therefore says: „Jan” is an element in dimension „Time”. Its parent element is „Qtr. 1” and its weight factor in „Qtr. 1” is 1. Provided this is the very first command in the loop (which we cannot predict since the sequence of Excel formula execution in the worksheet cannot be predicted), all elements of the ‘Time’ dimension will be deleted and two elements will be created, „Jan” and „Qtr. 1”. If other formulas describing one (or both) of these two elements have executed before and therefore it already exists, it will remain unchanged except for the „n” or „c” flag being changed if it has to. Now we understand what the formula underneath in cell B14 says in everyday speech:

„Qtr. 1” is element in dimension „Time”. Its parent element is „2005”, and its weight factor in „2005” is 1.

Since a PALO.EADD formula always describes two elements and their relationship, only two formulas are required to describe three elements of a dimension.

34

The procedure is similar for the next dimension ‘Costcenter’: The .txt file contains cost center numbers which are base elements. The next level is cost center group, its description being picked from the table in D18:D26. Last but not least there is a ‘Total’ level on top:

Again we have two formulas which describe three elements, always two of them at a time.

The setup of the third dimension ‘Region’ is a somewhat advanced example since it is asymmetric: The dimension ‘Region’ contains three element types: Countries, sub regions and regions. You will see that in this model European countries will be consolidated to sub regions South, East and West before these sub regions again are summed up to region ‘Europe’. American and Asian countries however will be consolidated to regions, leaving out the sub regions level. The result in Palo will look like this (some sub regions collapsed here for readability):

Now check how the formulas are set up to reflect this:

The formula in B34 only executes in case a sub region is indicated. It defines the sub region as a parent element to the country. In any case the formula in B35 will run, defining the region as a parent of either the sub region (if there is one) or the country.

35

Now start the Data import again and click through a number of records to see if everything works properly, then click ‘Finish’ to have the entire structure imported. The next step would then be to import the values in column G. See the previous chapter ‘Importing Raw Data’ on how to do that. Note: Do not use PALO.SETDATA together with PALO.EADD in the same workbook. It might otherwise occur that a dimension element has not yet been created while the system attempts to write its value to the cube because you cannot predict the order in which formulas in a worksheet are executed.

Importing from ODBC Source The import techniques we used above for the raw data import could similarly be used for other data sources using ODBC. Together with Palo comes an Access-Database which contains the same data as the testimport.txt. You find it as well in „My Documents\Jedox”. The file is named testimport.mdb.

Connecting the ODBC source: Before you can import the data from your ODBC database you will need to setup the DSN. In Windows XP this is easily achieved within four steps:

• open "control panel/administrative tools/data sources(ODBC)" from the Start-Menu

• goto "SYSTEM-DSN" and select "add"

• select the Microsoft-Access-Driver from the list and click "OK"

• in the next menu enter "palotest" as the "data source name" and select the "testimport.mdb" as the

"database".

Importing into Palo Open Palo - Data Import and select ODBC Query as the data source:

36

After you clicked ‘Next’ in the following screen you have to enter the following data:

As the data source name (DSN) you select ‘palotest’ which we had just created. As we had defined no username or password for our data source leave the accordant fields blank. To fetch the data from the database a Query statement has to be formulated in the textbox named ‘Enter SQL Statement’. In our example this is SELECT * FROM Sheet1.(This will get all data from a table named ‘Sheet1’. ‘Sheet1’ is the only table in the demo database and contains all the data we are about to import.) Note that as an additional feature you can save all the SQL statements you have made and reload them later. This way you do not have to formulate your SQL-Staements again and again. We will ignore this in this example. Once you clicked on next you will be on the same track you went to import the textfile:

Go ahead using the ‘Next’ and/or ‘Finish’ button to accomplish the data import.

37

Importing Data from external Excel Sheet In many cases there is only a small amount of data which is existent on one or just a few Excel sheets to be imported. In this case it seems not worth while to export the data into a raw textfile first to import it from there to Palo. We could as well transfer the data from the Excel data source into the cube creating a transfering Excel sheet to do so. Let’s assume you want to import the data you find in My Documents\Jedox\excelimport.xls into the cube ‘Sales’ of the ‘Demo’-database that we used in the beginning. Please load the excelimport.xls that we can start to modify it to become an import tool for Palo:

First of all we have to remember that a cube is a multidimensional hierarchical structure. So we must decide at which cube and where in the cube the data shall be inserted. We will set that information the same way the Paste View function uses to define the actual element to display:

The meaning of the added cells is to specify the target destination of the import data: B1: into Database ‘Demo’ on server ‘localhost’ B2: into the cube ‘Sales’ B3-B6: The target elements of the other dimensions of the cube apart from the dimensions ‘Products’

(because this is represented by the range B9:B13) and ‘Month’ (represented by the range C8:N8)

38

Now we have specified the destination of the import we need to add a conversion routine which reads the data from range C9:N13. To create this we just copy the original data table ( the line in row 15 has no function it is just added to visually divide the original data from what becomes our conversion routine):

Like we did when importing raw data, we will use the benefit of the PALO.DATA function to help us to assign the right coordinates to the import function and to test the match of the given data and the data structure of the cube. So select cell C18 and use the Paste Data function from the ‘Palo’ menu:

The result will be the following formula in B18 which has assigned the references for the selected cell:

39

After you made sure the PALO.DATA function is refers to the right cells we could expand the formula from B18 to all cells in row 18 (you will notice that the references use the $-sign at the right place to enable right referrers after dragging) : To drag the PALO.DATA function to the adjacent cells mark cell B18 click with the mouse at the little square at the right bottom and hold it while dragging the mouse to the right.

Then mark the range C18:N18 and expand it downwards to fill the whole data range C18:N22 with the PALO.DATA formula. The result will look like that:

Obviously there is a problem with the months ‘Mrz’, ‘Mai’, ‘Okt’ and ‘Dez’. We had the same problem already when importing the raw data so we know that the problem is that the cube ‘Sales’ uses the English abbreviations for the elements of the dimension ‘month’. As we are just importing little data we could easily solve the problem by replacing the wrong abbreviations by their english versions:

(You will notice that the second table now shows different data than the first one. The first table contains the data you are about to import while the PALO.DATA function reads the data which is currently stored in the cube.)

Now having no more problems we can replace the PALO.DATA formula by the PALO.SETDATA formula which actually writes the data to the database. The SETDATA formula uses the same coodinates as the DATA formula. What we have to add is the values which we want to import (the data of the range C9:N13) and a flag whether to splash the imported data (FALSE). Select cell C18 and change the formula =PALO.DATA($B$1;$B$2;$B19;$B$4;C$17;$B$5;$B$3;$B$6) to =PALO.SETDATA(C9;FALSE;$B$1;$B$2;$B18;$B$4;C$17;$B$5;$B$3;$B$6) The data in the cube is immediately changed and SETDATA gives the new value as result. Proceed by expanding the SETDATA function FROM C18 to the whole row 18 and then to the complete range C18:N22. After doing so the data import is accomplished and both tables will show the same values.

40

Entering data manually Edit cells Clear the document and open ‘Paste View’ and select the cube ‘Market’ which we created before: Move the ‘Months’ dimension to Column titles, ‘Products’ to Row titles and the others to the Page selector section:

Click on the dots next to Months and select the first three months to be pasted as column titles. Likewise, select the five desktop computers from Products ( you find them in subdirectory Stationary PC’s) customer names to display as row titles. Click ‘OK’ and ‘Paste’ to see the result:

What you see are the values we have imported to ‘Market’ before. Let us now clear the values of the cube and then add some data manually.

41

To clear the cube open the modeller and click onto ‘Market’ with the right mouse button and select ‘Clear cube’ (NOT ‘Delete cube’ !)from the context menu:

Afterwards all values are deleted from the cube while it’s structure is left intact.

To select the actual location where we enter the data to doubleclick the page selector Variance and set it to Actual.

Afterwards set the page selector Europe to Germany the same way.

42

When you enter values, they will be written to the database. Note that although you entered a value, the formula is still there. The value which you keyed in before is written to the database, and the formula retrieves it again for display:

Thus you can easily enter data in order to simulate scenarios for bottom up planning. Of course you will want to have top down planning as well, and you will now see how this is done.

Splashing (Writing into Aggregated Cells) Double click the region name which you selected before and then set the field to ‘All products’ again:

First of all, you will again see the value you have entered before since this consolidation includes the item you had chosen. Now go to another field and try to enter something. The following error message will be displayed.

You can however indicate that you do not type something by accident but you intentionally want to ‘splash’ values. To do this, start with a ‘pipe’ character and then key in the value:

On german (QWERTZ) keyboards the pipe character is created by simultaneously pressing AltGr and <, on international standard (QWERTY) keyboards press Shift and \.

43

The value you have entered will be broken down to all levels below the current one. You can check that when you browse through all the products. In case at least one value was already there, the total will be broken down following the previous pattern of distribution. That means if for example you increase the total by 100, all the existing values will be modified accordingly, leaving the others to be zero. If no values were there before, the total you keyed in will be distributed in equal parts to all the elements below the current level. Apart from the ‘pipe’ character, there are four more parameters with which you can write data into aggregated elements:

#

This parameter does exactly the same as the ‘pipe’ character.

# … %

Use this structure to add a percentage. #10% will add 10 % to the current value.

!! Two exclamation marks followed by a value will write this value into all the base elements of the aggregated element.

44

Techniques

Text Elements Let us assume we want to add comments to our ‘Market’ cube. Open the Palo Modeler and create the element ‘Comment’ in dimension ‘Months’. Make it a string element; this can be done in the context menu which opens on a right mouse click:

Add this column to the view we created above so that it looks for example like this:

45

You will know that an Excel cell can not hold more than 255 characters; however you may want to add a comment longer than that. This is possible: A double click on the cell will open this window in which you can enter as many characters as you need:

However, still only the first 255 characters will be displayed in Excel. To overcome this, use the PALO.DATAT formula. Since PALO.DATAT returns an array of elements which are not longer than 255 characters each, you can proceed as follows: Insert another row in order to have two cells to hold the entire comment. Mark these two cells and change the PALO.DATAC formula to a PALO.DATAT formula by replacing the “C” by a “T”. Enter this change as an array formula by pressing CTRL+Shift+Enter instead of the normal Enter. The result could look like this:

46

Calculated Elements (Push Rules) One way of working with calculated dimension elements is to set up Server Rules. However, in complex models with a large amount of data this may result in a performance issue since literally millions of operations could be necessary to recalculate the entire cube. A good alternative is to use Push Rules; that means rules (or formulas) which define the way how elements are calculated. The difference is, calculation is done outside the database in Excel, the result however is written to the cube. Calculation is a lot faster since only the subset of data which is currently viewed is recalculated. Here is an example. Let us assume that in addition to what we have so far in our test database, we want to create three elements:

• Item price • Turnover EUR • Turnover USD

The item price element will be populated simply by keying in the values, the turnover elements will be calculated using push rules. Open the Modeller and add these three new elements to the ‘Measures’ dimension:

47

Now create a view of the ‘Sales’ cube which contains the new elements plus some of the product names. Please bear in mind that when working with push rules it usually only makes sense to select base elements:

When thinking of currencies we need exchange rates of course. One could imagine having them in a cube as well, but we will keep it simple and assume fixed exchange rates which we key in:

Columns D and E currently contain PALO.DATAC formulas. To write the result of our calculation to the database, we will convert them into PALO.SETDATA functions. The PALO.DATAC formula in D9 for example looks like this:

These parameters are the coordinates which identify the cell in the database. We can use them exactly the way they are for PALO.SETDATA: Simply replace the string ‘PALO.DATAC’ with ‘PALO.SETDATA’.

48

PALO.SETDATA however requires two additional parameters which we will now add before the existing ones:

• The most crucial parameter which we need to insert of course is the value which SETDATA is supposed to write. It is the first one in the sequence of parameters.

• The second additional parameter required by PALO.SETDATA is a flag to indicate whether we want to splash, i.e. break down the values we are entering to the levels below the current one. Since we work with base elements, this flag will read FALSE in our example.

The ‘value’ parameter can be a number, a cell reference or a formula. In this parameter we will define how the two Turnover elements are calculated. This is where the push rule is set up. You have the entire scope of Excel functions at your fingertips to calculate what you would like to have as the contents of the new element. In our example, the push rule is not all too complex; we will simply multiply the number of units with the item price and the exchange rate: Note that ‘$’ signs are applied in the suitable way so that the formula can be copied to the entire table. After copying key in some item prices and see what happens:

The price is multiplied by the number of units and by the conversion rate. The result is written to the database, the return value of the PALO.SETDATA function is the value which it wrote. It is displayed in the cell.

49

You will remember from the previous chapters that although the cells in column C contain PALO.DATAC formulas, you can type in numbers which will then be written to the database. You can therefore modify three dimension elements in one operation: When entering the item price you automatically calculate the expected turnover in EUR and USD at the same time.

Note: Bear in mind that the calculated elements (Turnover EUR and Turnover USD) are based on the number of units, the item price and the exchange rate, so you must not change any of these (e.g. on a different worksheet) while no push rules for the two calculated elements are present since your cube would become inconsistent.

50

Appendix

Palo Spreadsheet Formulas

Note: You may have to replace the semicolon (;) with a comma (,) in the following formulas depending on your country settings. PALO.DATA(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;)

Retrieves the value of the specified element from the cube, e.g.: PALO.DATA("localhost/Demo";"Sales";"Desktop L";"Germany";"Oct";2002;"Actual";"Units")

PALO.DATAC(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) Retrieves the value of the specified element from the cube, e.g.: PALO.DATAC("localhost/Demo";"Sales";"Desktop L";"Germany";"Oct";2002;"Actual";"Units") The PALO.DATAC collects all formulas in a worksheet and executes them in one go. Compared to PALO.DATA it is faster but you can not use PALO.DATAC results as parameters in other PALO formulas. Simplified you can say: use PALO.DATA if you want to use the result in further calculations and use PALO.DATAC if you want to use the result for output.

PALO.DATAT(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) Retrieves the value of the specified string element, e.g.: PALO.DATAT("localhost/Demo";"Market";"Comment";"Year";"Units";"All Products";"2006";"Total") This function allows displaying more than 256 characters when entered as an array function. In that case the contents exceeding 256 characters will be displayed in the next cell of the defined array.

PALO.DATAV(Server/Database;Cube; Coordinate1; Coordinate2;… Coordinate16;) Retrieves the value of the specified element from the cube, e.g.: PALO.DATAV("localhost/Demo";"Sales";"Desktop L";"Germany";"Oct";2002;"Actual";"Units") To PALO.DATAV apply the same restrictions as to PALO.DATAC. It is even a bit faster than PALO.DATAC but therefore it only works if all PALO.DATAV formulas sit in a coherent array ( it is not possible to have cells with other functions in between ).

PALO.EADD(Server/Database;Dimension;Type;Element;Parent Element;Weight;Clear) Adds the specified dimension element to the database Sample: PALO.EADD(„localhost/Demo”;”Years”;”n”;”2010”;””;1;FALSE) Sample2: PALO.EADD(„localhost/Demo”;”Years”;”c”;”2011”;”2002”;1;FALSE) The ‘Clear’ parameter defines whether to erase all elements in dimension when starting (TRUE / FALSE). In case you use EADD more than once on a worksheet, make sure the ‘Clear’ parameter reads the same in all the occurrences since otherwise the result is arbitrary. Note: This function only executes when the Import Wizard recalculates the sheet.

PALO.ECHILD(Server/Database;Dimension;Parent Element;Element;Number of Child)

Retrieves the name of the specified child element Sample: PALO.ECHILD(„localhost/Demo”;”Regions”;”South”;3) retrieves „Spain”

PALO.ECHILDCOUNT(Server/Database;Dimension;Element) Retrieves the number of children in the specified consolidated element Sample: PALO.ECHILDCOUNT(„localhost/Demo”;”Regions”;”South”) retrieves „3”

PALO.ECOUNT(Server/Database;Dimension) Retrieves the amount of dimension elements in the specified dimension Sample: PALO.ECOUNT(„localhost/Demo”;”Years”) retrieves „8”

51

PALO.EFIRST(Server/Database;Dimension) Retrieves the first element in the specified child dimension Sample: PALO.EFIRST(„localhost/Demo”;”Regions”) retrieves „Germany”

PALO.EINDENT(Server/Database;Dimension;Element) Retrieves the indention level of the specified element Sample: PALO.EINDENT(„localhost/Demo”;”Regions”;”Italy”) retrieves „3”

PALO.EINDEX(Server/Database;Dimension;Element) Retrieves the position of the specified dimension element Sample: PALO.EINDEX(„localhost/Demo”;”Regions”;”France”) retrieves 2

PALO.EISCHILD(Server/Database;Dimension;Parent Element;Element) Checks if a consolidated element contains the specified element Sample: PALO.EISCHILD(„localhost/Demo”;”Regions”;”West”;”Germany”) retrieves TRUE

PALO.ENAME(Server/Database;Dimension;Position)

Retrieves the name of the element at the specified position (First Position is 1) Sample: PALO.ENAME(„localhost/Demo”;”Regions”;2) retrieves „France”

PALO.ENEXT(Server/Database;Dimension;Element)

Retrieves the name of the succeeding element of a dimension element Sample: PALO.ENEXT(„localhost/Demo”;”Regions”;”Italy”) retrieves „Portugal”

PALO.EPARENT(Server/Database;Dimension;Element;Number of parent elements)

Retrieves the name of the n’th parent of the specified element Sample: PALO.EPARENT(„localhost/Demo”;”Regions”;”Italy”;1) retrieves „South”

PALO.EPARENTCOUNT(Server/Database;Dimension;Element) Retrieves the number of consolidated elements which contain the specified element Sample: PALO.EPARENTCOUNT(„localhost/Demo”;”Regions”;”Italy”) retrieves „1”

PALO.EPREV(Server/Database;Dimension;Element)

Retrieves the name of the preceding element of a dimension element Sample: PALO.EPREV(“localhost/Demo”;”Regions”;”Italy”) returns „Austria”

PALO.ESIBLING(Server/Database;Dimension;Element;Number of sibling)

Retrieves the name of the specified sibling Sample: PALO.ESIBLING(„localhost/Demo”;”Regions”;”Italy”;3) Retrieves „Denmark”

PALO.ETYPE(Server/Database;Dimension;Element) Retrieves the type of the specified element (numeric, string or consolidated) Sample: PALO.ETYPE(„localhost/Demo”;”Regions”;”Italy”) Retrieves “numeric”

PALO.SETDATA(Value;Splash; Server/Database;Cube;Coordinate1; Coordinate2;… Coordinate16)

Sets the value of the element specified by the coordinates. Set Splash to TRUE if you need to write to an aggregated value, otherwise set to FALSE

52

Acknowledgements

The translations of Palo to other languages were made by:

Dmytro Neustroyev Russian

Louis Jaouan French louis.jaouan@coopanet.com *Claude-Henri Meledo

Bent Madsen Danish

Bálint Varga Hungarian Gaojun Pan Chinese pgj_27@163.com

( * you can contact him when you have suggestions for a better translation )

We are thankful for their help to make it possible to use Palo in your favourite language…

The Jedox Team