heavy duty primavera usage
TRANSCRIPT
Heavy duty Oracle Primavera usage in
enterprise environments: maximising
ROI
A paper presented at the InSync10 Conference, Melbourne, Australia, 16 ‐17 August 2010.
Anthony Spierings
Strategy and Information Manager, ENERGEX
Heavy duty Oracle Primavera usage in enterprise environments
Page | 2
1. Contents
1. Contents .......................................................................................................................................... 2
2. Table of Figures ............................................................................................................................... 3
3. Executive Summary ......................................................................................................................... 4
4. Purpose ........................................................................................................................................... 5
5. Oracle Primavera @ ENERGEX in 2010 ........................................................................................... 5
5.1. About ENERGEX ...................................................................................................................... 5
5.2. How Oracle Primavera is used ................................................................................................ 5
5.3. Oracle Primavera Configuration ............................................................................................. 8
6. ENERGEX’s journey with Oracle Primavera .................................................................................... 9
7. Statistics about how Oracle Primavera is used ............................................................................... 9
8. Interfaces to other applications ...................................................................................................... 9
9. The Limits of 32‐bit computing ..................................................................................................... 10
9.1. How much memory are you using ........................................................................................ 10
9.2. Issues for ENERGEX due to 32‐bit limitations (desktop client) ............................................. 12
9.3. Options for Five and Ten year PoW ...................................................................................... 13
10. A day in the life of XBNEWAS18 ................................................................................................ 14
10.1. Job Services 32‐bit limits ................................................................................................... 15
10.2. Using Job Services 32‐bit platforms .................................................................................. 15
10.2.1. Get the required data directly from the database ........................................................ 16
10.2.2. Oracle Primavera Enterprise Reporting Database ........................................................ 16
10.2.3. Split the jobs into smaller chunks and recombine ........................................................ 17
10.2.4. Go to a 64‐bit solution .................................................................................................. 18
10.3. Other Issues with Job Services .......................................................................................... 18
11. Database Performance ............................................................................................................. 20
12. Conclusions ............................................................................................................................... 21
13. Appendix 1 – How to Split Job Services .................................................................................... 22
Heavy duty Oracle Primavera usage in enterprise environments
Page | 3
2. TableofFigures
Figure 1 ‐ High level data flows around EPPM in ENERGEX .................................................................... 7
Figure 2 ‐ Desktop Application Out of Memory Warning ..................................................................... 10
Figure 3 ‐ Primavera Memory Usage at project screen (no projects open) ......................................... 11
Figure 4 ‐ Primavera Memory Usage with 100 projects opened (display at activity level). ................. 11
Figure 5 ‐ Primavera EPPM memory usage per project inside ENERGEX ............................................. 12
Figure 6 ‐ Primavera resource pool view when preparing 18‐month PoW .......................................... 13
Figure 7 ‐ XBNEWAS18 CPU Load 23 February 2009 ............................................................................ 14
Figure 8 ‐ XBNEWAS18 free memory 23 February 2009 ....................................................................... 14
Figure 9 ‐ XBNEWAS18 free memory 7 March 2010 ............................................................................. 15
Figure 10 – XBNEWAS18 free memory 1 July 2009 .............................................................................. 15
Figure 11 ‐ Screen capture of Job Services running the same report multiple times ........................... 22
Figure 12 ‐ ENERGEX EPS that Job # 440 is being run against............................................................... 23
Figure 13 Job Services CPU Load from 11 July 2010 to 18 July 2010 .................................................... 23
Figure 14 ‐ Server Free Memory from 11 July 2010 to 18 July 2010 .................................................... 24
Figure 15 ‐ Screen capture of the P6 report list .................................................................................... 25
Figure 16 – Report editor, reports have a header row ......................................................................... 26
Figure 17 ‐ Report editor, reports have header row removed ............................................................. 26
Heavy duty Oracle Primavera usage in enterprise environments
Page | 4
3. ExecutiveSummary
ENERGEX uses Oracle Primavera as integral part of its Enterprise Portfolio Project Management
methodologies. ENERGEX works under EPPM are valued at AUD$8 billion FY2010 to FY2015.
ENERGEX has two rolling time horizons of interest; now to +18 months (Tactical Program of Work)
and now to +5 years (Regulatory Program of Work). An emerging requirement is for a ten year view
(Strategic Program of Work). Presently, Primavera is the core system for preparing time‐series
forecasts for; cash consumption, internal resource consumption, material delivery, and desired
external resource capability.
At any one point in time, there are approximately 1 000 projects under active management in the
Tactical Program of Work, with a further 4 000 projects in the Regulatory Program of Work. All up,
the database contains over 35 000 projects (including baselines), with over 1.4M activities.
Enterprise users make heavy demands on the information systems infrastructure. When using
Oracle Primavera on top Windows 32‐bit Operation Systems users may reach the limits of 32‐bit
computing. By observing the application’s memory consumption, End Users can develop simple
“rules of thumbs” and operational techniques to maximise their ROI in computing resources.
There are a number of options available to heavy duty enterprise users when conducting corporate
reporting. Option one; extract the data directly from the back end database. However, due to the
sophisticated use of logical data we cannot recommend this approach. Option two; use the Oracle
Primavera Enterprise Reporting Database. Option three; split large reporting jobs into smaller
chunks. Then recombine the output files. Option four; upgrade to P6V7 version that is packaged for
64‐bit environments. Make provision in the budget for a full release 64‐bit version.
We also note in passing that the ‘out of the box” settings for the back end database are optimised
for standard installations. ENERGEX strongly recommends that large Oracle Primavera EPPM
installations engage the services of an Oracle database application tuning expert.
The Oracle Primavera EPPM product suite has been integral part in assisting ENERGEX efficiently and
effectively plan, deploy, manage, and deliver a $1B annual Program of Work.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 5
4. Purpose
Enterprise Portfolio Project Management (PPM) is a rapidly evolving methodology for many
organisations. Oracle Primavera provides a suite of information system tools to assist organisations
optimise their program management. EPPM makes heavy demands on the information technology,
particularly in large enterprise environments.
This paper discusses how ENERGEX uses Oracle Primavera EPPM to develop Tactical, Regulatory, and
Strategic Program of Work. It then provides a brief tutorial on 32‐bit computing to assist End Users
understand how Oracle Primavera uses computer memory, particularly how the product response to
their projects in their environment.
5. OraclePrimavera@ENERGEXin2010
5.1. AboutENERGEX
ENERGEX is one of the Australia's largest and fastest‐growing organisations with more than 3500
staff working in a range of roles to supply electricity to a population of around 2.8 million people in
South East Queensland.
A top 100 Australian company with more than $7 billion in total assets, ENERGEX has more than 80
years industry experience and has a strong focus on customer communications, positive
environmental outcomes and community safety.
The company’s South East Queensland distribution area of 25,000 square kilometres includes more
than 50,000km of underground and overhead electricity lines and cables, over 600,000 power poles,
some 43,000 transformers and almost 300,000 street lights.
5.2. HowOraclePrimaveraisused
ENERGEX uses Oracle Primavera in a number of different ways.
First, it is the core application for forecasting across a Program of Works that has a $1B annual
expenditure. Oracle Primavera is used to provide time‐series (temporal) data on expected cash
consumption, internal labour requirements, major material delivery dates, and contractor capability.
There are number of time horizons that interests ENERGEX;
Time Horizon 1. now to eighteen months (Tactical Program of Work)
Time Horizon 2. now to five years (Regulatory Program of Work)
Time Horizon 3. now to ten years (Strategic Program of Work)
The now to eighteen time horizon is used to prepare the rolling Tactical Program of Work. To ensure
that ENERGEX’s “internal production line” is running in an optimal manner. Cash consumption
forecasts, internal resource usage, and synchronising the program with the delivery of major
electrical plant items are front of mind.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 6
The now to five year time horizon (Regulatory Program of Work) is partly driven by the regulatory
environment that ENERGEX operates within. Indeed, much of the core financial modelling for
project related work that ENERGEX prepared for its recent submission to the Australian Energy
Regulator was conducted in the Oracle Primavera environment.
The ENERGEX Executive is now challenging the Program Management Office to explore “over the
horizon”, the now to ten year time frame (Strategic Program of Work).
Oracle Primavera is a link in ENERGEX’s sophisticated end‐to‐end methodology for prudently
managing a growing network. Oracle Primavera accepts upstream data from the electricity planners
(e.g. project scope, estimates, delivery dates) and turns this into forecasting outputs for downstream
service providers (e.g. when are major materials required). This is achieved by using the Primavera
API to; import data from the Network Planner’s database, cross‐match the project to an existing
template project, and then build the project in Primavera1. We believe that our methodologies for
preparing Programs of Work place ENERGEX ahead of the pack. It is a significant step ahead of the
plethora of spreadsheet that most other organisations still resort to.
Second, project managers use Oracle Primavera in a more traditional manner on a subset of projects
within the overall Program of Work. That is, to use Oracle Primavera’s capability to generate Gantt
and Pert Charts to develop the project delivery plan for important projects. These projects, like
building a new electricity substation, use Prince 2 methodologies. There are around 1 000 active
projects with approximately $0.6B in annual spend that are managed using traditional project
management techniques.
An overview of data flows into and out of Oracle Primavera EPPM is shown in Figure 1. Note that
while this schematic has EPPM at the “centre of the universe”, in practice, the EPPM is just one link
in a rather long value chain. Also note that most of the links are not automatic data flows, but a
representation of how data flows between systems.
1 Refer to Spierings’ presentation at Primavera Conference, Perth, 2008 for more information
Heavy duty Oracle Primavera usage in enterprise environments
Page | 7
Oracle Primaveradatabase
Build 18-month POW
Ellipse Estimates
ResourceAvailability
SIFT(Network Needs)
OrganisationalConstraints(e.g. budget, community obligations,
acceptable risk profile, ROI, etc)
Day to day project
management
Current project statusSkill sets
Unconstrained dates
Forecast; resource profile
Qld GovtState Budget
Project completion dates
Constrained dates
Controlled plantDelivery dates(Transformers,
modules, switchgear, relays,
etc)
Balance mix of internal and
external resources
Manufacturers source raw materials
Manufacturers stay or seek work
elsewhere
QLD Govt Infrastructure
plans
HR Human Capital Plans
AER Submission
ENERGEX Funding
Build 5 year POW
Regulatory Outcomes
Ellipse Works Orders
(dates, budget ..)
Future resource capability
Network StandardBuilding Blocks
DO Work
Real Time Network
Equipment Ratings
Economic forecasting
engine
Controlled PlantContracted Deliveries
Figure 1 ‐ High level data flows around EPPM in ENERGEX
Heavy duty Oracle Primavera usage in enterprise environments
Page | 8
5.3. OraclePrimaveraConfiguration
Primavera runs on an Oracle database. Power Users use the Primavera desktop client to connect
directly to the database. Regular users have their Primavera desktop client running on top of Citrix.
Table 1 lists the current hardware configuration.
Primary Data Centre Oracle Database Server. Running three applications: Primavera, plus two other document management systems. myPrimavera Web: Xeon Quad Core Virtualised (one production, one development, one UAT). Also configured as warm backup for job services. Citrix Farm: Four virtual servers for Primavera running on Citrix.
Backup Data Centre Oracle database server. Cold backup, normal duties as development server. Job Services: Xeon Quad Core Virtualised (one production, one development, one UAT). Also configured as warm backup for myPrimavera Web. Runs schedule task for API in command line mode. Citrix Farm: Two virtual servers for Primavera running on Citrix.
Desktop Client P6.1 (on the desktop): 10 P6.1 (via Citrix): 70 users
Primavera database disk size
55GB
Projects in database (including baselines)
35 000
Activities in database 1 400 000
Time span of projects 2007 – 2020
User numbers 87 named, 14 concurrent, 40 web, 53 portfolio
Table 1 ‐ Oracle Primavera configuration within ENERGEX
Heavy duty Oracle Primavera usage in enterprise environments
Page | 9
6. ENERGEX’sjourneywithOraclePrimavera
At some point in the seemingly distant past (03/03/2003 actually), the Primavera install disks were
first placed in the optical drive. Initially there were only a few users, connecting to a back end Oracle
database via the desktop client running on top of Windows XP (in a Standardised Operating
Environment).
Oracle Primavera’s usage inside ENERGEX was modest for the first few years. By early 2005 a
Program Office was established, with ENERGEX making more use of Oracle Primavera as an EPPM
tool to manage a $0.5B Program of Work. As the application’s usage increased, there was a gradual
decline, imperceptible at first, in the database’s performance. In many respects, poor database
performance (or our lack of ability to tune the application) remains one of the Oracle Primavera
ENERGEX custodian’s primary concerns.
The year 2008 was a watershed year for Oracle Primavera within ENERGEX. The Program
Management Office was heavily involved in the organisation’s preparation of an $8B regulatory
submission. Our methodologies were evolving and maturing at a rapid rate. We figured out ways of
using the Primavera API to shuffle extraordinary amounts of project data into and out of the product
suite.
The process was stressful in more ways than one. It was a period of intense creativity. It was also a
period when we discovered significant improvement opportunities in our organisational processes.
While we always envisioned that the process would be iterative, we never fully appreciated how the
wealth of larger and improved data sets would precipitate even more demands on people and
systems.
7. StatisticsabouthowOraclePrimaveraisused
The ENERGEX database contains over 35 000 projects with approximately 1.4M activities. Around
10 000 projects are visible at any one point in time, covering approximately 445 000 activities. The
value of works being modelled is approximately $30B over a time span from two years in the past
(concluding projects) to approximately 20 years into the future. Of this, there are 1 000 projects
under intensive project management. ENERGEX Project Managers are responsible for maintaining
their projects in P6.
Of the 35 000 projects in the database, 25 000 are baselines. Project managers are encouraged to
create baselines. On top of this, ENERGEX runs an API over the Program of Work to establish fixed
baselines at known points in the annual budget cycle. These baselines become fixed reference
points for determining changes to the PoW.
8. Interfacestootherapplications
ENERGEX runs P6.1 (the version before web services became available). Most of the upstream data
for future works is imported into Primavera using an API to connect to an in‐house database. This
allows us the ability to automatically build the outlying years automatically and with little effort.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 10
Data is also imported from the ERP (Mincom Ellipse). Data is imported at the project and activity
level into UDF (meta‐data, actual expenditure, approval status, etc). The Primavera material and
contractor resource pool is also synchronised with the ERP on a weekly basis. Further, we do have
the ability to import actual from the ERP into the Primavera past period actual field.
9. TheLimitsof32‐bitcomputing
It would be reasonable to suggest that most Oracle Primavera EPPM installations that are in service
today are installed on top of a 32‐bit version of a Microsoft operation system (e.g. Windows XP,
Windows Vista, or Windows 7). While there are 64‐bit versions available of Windows XP (2005) and
Vista, it was not until Windows 7 64‐bit was released that organisations have generally started to
consider moving off 32‐bit operating systems.
32‐bit applications have a maximum address 2^32 bytes, which is approximately 4GB of memory.
Under the typical Windows install, each application is given virtual 4GB of address space. However,
the operating system reserves 2GB of this memory for the kernel and only gives 2GB for applications
(the default settings). Most 32‐bit applications written to run on Microsoft 32‐bit operating systems
(e.g. Oracle Primavera) have a maximum memory space of 2GB. Until recently, Oracle Primavera
EPPM2 was only available as a 32‐bit application.
Often, the first warning to the End User that they have reached the limits of 32‐bit computing is a
message like the one shown in Figure 2
Figure 2 ‐ Desktop Application Out of Memory Warning
9.1. Howmuchmemoryareyouusing
Below is a simple test that End Users can run on the desktop client. Open windows task manager
and select the processes tab. Identify the image name “PM.exe”. This is the Primavera application.
On the day this screen capture was taken in the ENERGEX environment, the application had a
memory footprint of 115MB (see Figure 1)
Next, open up a number of projects (at least 100). Note the memory footprint the application now
requires. In the example provided in Figure 4, the memory footprint had risen to 432MB.
2 See Knowledge article ID 910591.1
Heavy duty Oracle Primavera usage in enterprise environments
Page | 11
Figure 3 ‐ Primavera Memory Usage at project screen (no projects open)
Figure 4 ‐ Primavera Memory Usage with 100 projects opened (display at activity level).
As an exercise, using Windows Task Manager, I have plotted opened projects (displayed at the
activity view) verses reported memory consumed Figure 5. In the ENERGEX environment, Primavera
uses approximately 120MB of memory while having zero projects open. On average, each additional
project opened uses around 0.7MB of memory. In this particular test, the Primavera EPPM
application exhausted its available memory when 1 950 projects were opened.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 12
Figure 5 ‐ Primavera EPPM memory usage per project inside ENERGEX
The graph in Figure 5 has a trend line superimposed on the data points to provide indicative average
memory consumption per opened project. We believe that the grouping of data points, first below
then above, the average trend line is nothing more than an artefact of ENERGEX’s Enterprise Project
Structure (EPS).
9.2. IssuesforENERGEXdueto32‐bitlimitations(desktopclient)
ENERGEX has around 600 large capital works projects with expenditure in the 2010/11 fiscal year3
currently being modelled in Oracle Primavera. In addition, there is another 600 sub‐programs
(power pole replacement programs, load break switch replacement programs, etc) modelled in
Oracle Primavera using “bucket” or “volumetric” approaches.
3 ENERGEX PoW Year is the same as the government’s fiscal year, starting 1 July.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 13
The 32‐bit memory limit means that, given the way ENERGEX presently uses Oracle Primavera, we
can only open two years worth of work. This means that we can use the desktop client when
preparing the Program of Work for Time Horizon 1. However, we can no longer fully use the desktop
client when preparing longer terms Programs of Work for Time Horizon 2 (five year) and Time
Horizon 3 (ten year) as shown in Figure 6.
Rolling 18-month PoW
Five Year PoW
This area filtered out to fit within 32-bit memory limit
Projects in this area are hidden from analysis in the desktop client
Rolling 18-month PoW
Five Year PoW
This area filtered out to fit within 32-bit memory limit
Projects in this area are hidden from analysis in the desktop client
Figure 6 ‐ Primavera resource pool view when preparing 18‐month PoW
9.3. OptionsforFiveandTenyearPoW
There are approximately 4 000 projects in the five year time span4. This means that the limits of 32‐
bit computing (2GB application limit under Windows XP) are preventing us from using Oracle
Primavera for five year program management in our preferred manner. This paper will discuss our
workarounds later on.
4 Recall that ENERGEX’s larger projects can span up to four years. Hence, it is not unusual to see a long “tail” of
resources into future years for projects that are active in the current year.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 14
10. AdayinthelifeofXBNEWAS18
For many years, a single server (designated XBNEWAS18) was used to provide Web and Job Service
functionality. This provided cost effective use of computing resources. During the day, the server
was primarily serving Web Clients. During the night, the server was busy running Job Services
activities.
By late 2008 we started to notice that the myPrimavera application suite was becoming more and
more erratic. By early‐2009, we had a string of Monday morning’s in a row where we would arrive
at work to find both Primavera Web and Job Services had stalled. Sometimes, the server itself was
completely locked up and unresponsive. The server’s log file was not exactly full of clues either.
The performance graphs provided the best hints as to what the problem could be. Figure 7 is the
CPU load on Monday, 23 February 2009 (a normal day in the office for XBNEWAS18). The CPU graph
is unremarkable.
Figure 7 ‐ XBNEWAS18 CPU Load 23 February 2009
Figure 8 is a graph of free memory on the server. This offers more clues as to what the problem was.
At one point, free memory was down to 44MB. This was coinciding with Job Service activities that
we had scheduled to run during the day.
Figure 8 ‐ XBNEWAS18 free memory 23 February 2009
By this stage, it was slowly dawning on us that we were up against the memory limitations of 32‐bit
computing. While we were aware of these low memory events and understood that it was Job
Services causing the issue, we could not understand exactly what we could do to prevent this from
Heavy duty Oracle Primavera usage in enterprise environments
Page | 15
reoccurring (see Figure 9 & Figure 10); particularly, if we wanted to keep running critical financial
reports out of the system.
Figure 9 ‐ XBNEWAS18 free memory 7 March 2010
Figure 10 – XBNEWAS18 free memory 1 July 20095
Concluding that Job Services was the culprit, we stole borrowed acquired another virtualized server
and divided myPrimavera’s functionality. One server (XBNEWAS18) would be primarily responsible
for P6 Web access, with the other (XBNEWAS06) primarily responsible for P6 Job Services.
Immediately, the server for web clients became stable and has remained so ever since.
10.1. JobServices32‐bitlimits
Job Services is a 32‐bit application. As such, it has the same technical restrictions as the 32‐bit
desktop client. There is a subtle difference in that Job Services has a smaller memory footprint than
the desktop client and appears to be more memory efficient. (We surmise that this is because Job
Services does not require a G.UI.)
10.2. UsingJobServices32‐bitplatforms
This section discusses some of the issues and workarounds with Job Services.
5 This graph has an error in its title. The graph is titled CPU load, however, it is actually a graph of free
memory.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 16
10.2.1. Gettherequireddatadirectlyfromthedatabase
One solution is to bypass Job Services and to get the required information directly from the
database. There is, however, a small problem; particularly when dealing with time‐series data. The
Primavera application makes clever use of the concept of logical6 data. We define logical data as
information that can be reconstructed logically. This reconstruction is often done on the fly.
Consider the following equation;
321 nnn
If we wanted to store this equation inside the database, we only need to store n1 and n2 inside the
database. When we retrieve these two values, we can logically reconstruct n3 (as n1 + n2) on the fly.
As mentioned before, Primavera makes extensive use of logical data. Some of the time‐series7 data
that we were after simply does not exist inside the database. It is reconstructed by the desktop
client, web client, or Job Services when the project is loaded into memory. It is theoretically possible
to reverse engineer the calculations using the information inside the database, but this was beyond
our technical capability and resources available at the time.
Key Learning’s for ENERGEX
Important that data extractors understand the difference between persistent and non-persistent data.
Only extract persistent data directly from the database. And only then if you must.
10.2.2. OraclePrimaveraEnterpriseReportingDatabase
The obvious solution to the memory limitations is to install the Oracle Primavera Enterprise
Reporting Database (ERD) and extract the data required from the star‐schema. While we were (and
still are) keen to install the ERD, there are some issues.
First issue is the price. Our budget estimate to install the ERD was $500,000 AUD. While this may
appear somewhat extravagant for an ERD, consider the following. There is the ERD licence itself,
new database server (which requires both hardware and licensing), disk space to suit, and new
virtual servers for the ETL processing. Because this would form part of our core financial system,
times everything by three for the; Development, User Acceptance Testing, and Production
environments. On top of this, would be the report writing expenses. These costs are traded off
against an existing investment (DEV, UAT, PROD) that is working satisfactorily.
The next part of the problem is cost justifying this expense. As a general comment, Business
Intelligence solutions are a hard product to sell to senior management. Part of the reason is that
6 Logical data is sometimes called “virtual data” or “non‐persistent” data.
7 We define time‐series data as information that has temporal ordering. For example a monthly cash flow
forecast.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 17
Business Intelligence is a paradigm shift for organisations. They provide answers to questions that
have not even been thought of yet. It is only after a BI solution is installed that organisations look
back with wonder at how they could have possibly survived with antiquated reporting solutions.
Another part of the reason for ENERGEX not proceeding with the ERD solution was a perceived clash
between the Oracle Primavera Enterprise Reporting Database functionality and an existing corporate
project to install an organisational wide Business Intelligence solution. This author failed in his
attempt to show that the Oracle Primavera ERD is a complementary product solution, not a
competitor.
That said, the Oracle Primavera ERD is a valid solution and is worthy of consideration when
developing integrated reporting solutions. It is presently on the internal development roadmap. The
circumstances of the time forced us to find another alternative to installing the Oracle Primavera
ERD.
Key Learning’s for ENERGEX
ERD turns logical data into persistent data for integration into Business Intelligence solutions ERD will remove a lot of stress from the daily working life ERD is immune to 32-bit computing limitations ERD is a complementary product
10.2.3. Splitthejobsintosmallerchunksandrecombine
The Primavera knowledge base offers the following advice for circumstances when the 32‐bit
memory limit is exceeded.
“Workarounds for using Project Management in a non-LAA environment:
Using the PM client, open smaller EPS nodes/download less data”
Primavera Knowledge Article ID 910591.1
Key Learning’s for ENERGEX
Existing Job Services tool suite is a cost effective use of existing functionality Splitting jobs across the Enterprise Project Structure is an effective technique to manage
memory requirements Job Services outputs are available in time-series data Tip: consider use TAB delimiters in the output
Heavy duty Oracle Primavera usage in enterprise environments
Page | 18
10.2.4. Gotoa64‐bitsolution
Note: At this time of writing, while it has been discussed at various conferences, Oracle Primavera
has yet to release a true 64‐bit client.
With the release of P6V7, the Primavera client is now Large Address Aware. When the /3GB switch
is set in the Boot.ini file on Windows XP Professional, the application can now access up to 3GB of
memory. When P6V7 is installed on a 64‐bit operating system, the application can address up to
4GB of memory. Referring back to Figure 5, memory testing in the ENERGEX environment provided
the formula;
This roughly equates to 5000 projects that can be opened simultaneously in the ENERGEX
environment. This would probably allow ENERGEX to simultaneously open all of the projects in the
five‐year Program of Work. However, the ten‐year Program of Work remains out of reach.
If it were available, a true 64‐bit solution running on top of Windows 7 64‐bit Enterprise, maxed out
with 192GB of RAM, would allow ENERGEX to simultaneously open somewhere around 250 000
projects. Such a package would certainly alleviate the current issues.
ENERGEX Position
P6 V7 64-bit packaged version offers immediate relief for memory limitations Eagerly await the release of Oracle Primavera EPPM full 64-bit desktop client
10.3. OtherIssueswithJobServices
As mentioned above, cash flow forecasting is a core function of the ENERGEX Program Management
Office. This requires nightly reports that are reliable and consistent.
As memory management is an issue for large Programs of Work, we recommend that particular care
is exercised to sequence the jobs to occur one after the other, lest two jobs overlap and exhaust
available memory. Remember to also coordinate daily jobs (e.g. cash forecasts) with weekly jobs
(e.g. XER backups).
Be aware that completion times for certain jobs are highly variable. For example, the summarisation
process will skip projects that it ascertains has not had any changes since last summarisation.
Conducting a project schedule (F9) is a flag to Summarisation that it must re‐summarise this project.
If a schedule as occurred across the large sections of the database (for example, we have completely
rebuilt the five year Strategic Program of Work) this can literately add hours to the daily
summarisation process. As there are no interfaces available to flag that Job Services is complete, we
rely on having buffer dead time between when we expect Job Services to be complete in the worst
case and the start time for other process to uplift the data into the ETL for insertion into the data
warehouse.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 19
There are two tricks that we have learnt when dealing with Job Services;
a. Restart job services (from inside the Windows Server’s control panel) whenever a
change has occurred.
b. Regularly check that the outputs have a current date stamp.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 20
11. DatabasePerformance
The performance of the database has been an ongoing concern. Over the last few years, we have
undertaken some analysis of the database. The author is not fully convinced that all of the ENERGEX
instigated changes have been for the best. So, the comments below are rather general in nature.
Our suggestions to improve database performance are as follows;
a. Strongly recommend that large Primavera installations engage a database application
tuning consultant as a starting point.
b. In the ENERGEX environment, the database appears I/O bound. Increasing cache sizes and
disk speed improves performance. Pay particular attention to how the database server
conducts I/O operations and the UNDO table during summarisation.
c. Check that network issues are not constraining performance. (This is not an issue with
ENERGEX and the geographic locality is small and the Primavera End Users are located on
fat pipes.)
d. Including User Defined Fields and user created Project Codes in the layouts slows down
initial opening of the application and projects. Consider providing a layout to uses that
sticks to the default Primavera fields to improve response times when opening the
product.
e. The administrator guide recommends turning off the loading of resources when opening to
improve performance. This is good advice when there are large resource pools.
f. The ENERGEX methodology of deleting and rebuilding (recall how the outer years of the
Program of Work are built automatically) knocks around the various indexes. Talk to the
DBA about the best methods to manage the internal indexes.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 21
12. Conclusions
ENERGEX uses Oracle Primavera as an integral part of its EPPM methodologies. ENERGEX eagerly
awaits the release of a true 64‐bit Oracle Primavera desktop client (and job services).
AboutAnthonySpierings
Anthony has over 28 years experience in the energy industry. Anthony’s current work interests are
in the preparation and management of large, multi‐billion, multi‐year, Programs of Work.
Anthony is a qualified electrician. He also holds; an Associate Diploma in Electrical Engineering
(Power), Bachelor of Business (Economics, Banking and Finance), and a Masters in Project
Management (twice making the Dean's List) all from the Queensland University of Technology.
Anthony is presently a part‐time PhD Student at the University of the Sunshine Coast, researching
why End Users develop Feral Information Systems (like spreadsheets) in preference to the
sanctioned Enterprise System.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 22
13. Appendix1–HowtoSplitJobServices
This section describes a technique to split large jobs to fit within the memory limits of 32‐bit.
It is relatively simple, if tedious, to replicate the reports so that each report only runs against subsets
of the Enterprise Project Structure. Figure 11 is a screen capture of this technique. Job # 440, 441,
and 442 run the same report against various parts of the Enterprise Project Structure.
Figure 11 ‐ Screen capture of Job Services running the same report multiple times
Figure 12 is a screen capture of part of the ENERGEX EPS that Job 440 et al is being run against.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 23
Figure 12 ‐ ENERGEX EPS that Job # 440 is being run against
The impact of splitting up the jobs into smaller chunks is a reduction in CPU and memory usage.
Figure 13 shows the CPU load on the Job Server XBNEWAS06. This snapshot was taken during the
week from Sunday, 11 July 2010 to Sunday, 18 July 2010. The higher CPU usage is attributable to
improved database I/O performance (a separate exercise that is discussed later). Improving
database performance meant that CPU performance is now a constraint on performance.
Figure 13 Job Services CPU Load from 11 July 2010 to 18 July 2010
What is perhaps more instructive is the graph of available memory on the server8 as shown in Figure
14. This virtual server has 4GB of memory and 4GB page file. The assumption is that Primavera’s
memory usage is now down to 0.79G of memory (3.45G – 2.66G).
8 Recall that XBNEWAS18 has been replaced by XBNEWAS06 for Job Services.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 24
Figure 14 ‐ Server Free Memory from 11 July 2010 to 18 July 2010
The problem with this solution is that output data is split across a number of files. Each file has its
own header, which is an annoyance when trying to merge the data. A simple workaround is to have
two sets of reports. The first report in the series has the column headers included. The second and
subsequent reports are identical, except the headers have been removed. See Figure 15 for an
example.
Heavy duty Oracle Primavera usage in enterprise environments
Page | 25
Figure 15 ‐ Screen capture of the P6 report list
Removing the report headers is a simple exercise. Copy the original report. Open the report up with
the editor. Figure 16 shows the report with a header row included (above the page header area). To
delete, click on each header and delete in turn. The end result should look like the screen capture in
Figure 17
Heavy duty Oracle Primavera usage in enterprise environments
Page | 26
Figure 16 – Report editor, reports have a header row
Figure 17 ‐ Report editor, reports have header row removed
Finally, write a small script to join the output files together. For example;
copy/b /y %INPATH%%INFILE1% + %INPATH%%INFILE2% + %INPATH%%INFILE3%
+ %INPATH%%INFILE4% %OUTPATH%%OUTFILE1%
Heavy duty Oracle Primavera usage in enterprise environments
Page | 27
Yes, it is a kludge. But it works surprisingly well. ENERGEX then imports this file into the corporate
ETL for further analysis. The only remaining issue is whether the data in the file requires a further
“group by” summarisation. If the text file is based on a unique ID (e.g. Project ID) in the Primavera
database, only one row will appear in the output. If the output is based on a non‐unique ID (e.g.
resource usage), then this information may appear multiple times in the report and require further
processing before it can be used.