IBM Spectrum Storage Professional Certification Program

Study Guide Series

Exam C9060-800 Fundamentals of Applying IBM Spectrum Computing V1

Purpose of Exam Objectives .......................................................... 3

High-level Exam Objectives ............................................................ 4

Detailed Exam Objectives ............................................................... 6

Section 1 - Discovery ....................................................................................... 6

Section 2 - Solution Proposal ......................................................................... 8

Section 3 - Conductor .................................................................................... 14

Section 4 - Conductor with Spark ................................................................. 18

Section 5 - LSF ............................................................................................... 19

Section 6 - Symphony ................................................................................... 24

Next Steps...................................................................................... 29

Purpose of Exam Objectives

When an exam is being developed, the Subject Matter Experts work together to define the role the certified individual will fill. They define all of the tasks and knowledge that an individual would need to have in order to successfully implement the product. This creates the foundation for the objectives and measurement criteria, which are the basis for the certification exam. The Spectrum Storage Certification item writers use these objectives to develop the questions that they write and which will appear on the exam. It is recommended that you review these objectives. Do you know how to complete the task in the objective? Do you know why that task needs to be done? Do you know what will happen if you do it incorrectly? If you are not familiar with a task, then go through the objective and perform that task in your own environment. Read more information on the task. If there is an objective on a task there is about a 95% chance that you WILL see a question about it on the actual exam. After you have reviewed the objectives and completed your own research, then take the assessment exam. While the assessment exam will not tell you which question you answered incorrectly, it will tell you how you did by section. This will give you a good indication as to whether you are ready to take the actual exam or if you need to further review the materials. Note: This is the high-level list of objectives. As you review these objectives, click for a more detailed level of how to perform the task.

High-level Exam Objectives

Section 1 - Discovery

1.1 Determine customer’s existing applications and computing needs.

1.2 Determine the size and details of customers physical computing environment.

1.3 Determine the appropriate management framework: Symphony, LSF, Conductor, Conductor with Spark, Spectrum Cluster Foundation.

1.4 Assess client application onboarding requirements.

Section 2 - Solution Proposal

2.1 Describe the value proposition for Spectrum Computing solutions.

2.2 Respond to RFIs and RFPs.

2.3 Describe how Spectrum Computing can help consolidate existing compute infrastructures.

2.4 Describe the competitive landsacpe for Spectrum Computing offerings.

2.5 Recommend best practice for compute environments.

2.6 Describe High Performance Services offerings on SoftLayer (HPC Cloud).

2.7 Identify if a customer requires professional services for an IBM Spectrum Computing solution.

2.8 Identify Spectrum Computing products to increase resource utilization.

2.9 Explain difference between open source and Spectrum Compute solutions.

2.10 Describe and differentiate: LSF, Symphony and Conductor.

2.11 Describe Host Grouping.

2.12 Explain Spectrum Compute capabilities for managing application data.

Section 3 - Conductor

3.1 Describe benefits of Conductor.

3.2 Describe the Conductor architecture.

3.3 Define and deploy an application template.

3.4 Describe user management and security in Conductor.

3.5 Describe reporting features in Conductor.

3.6 Describe Conductor API.

3.7 Describe licensing models for Conductor.

Section 4 - Conductor with Spark

4.1 Describe the benefits and value of Conductor with Spark.

4.2 Describe Conductor with Spark instance groups.

4.3 Describe Conductor with Spark notebooks.

Section 5 - LSF

5.1 Describe the benefits of IBM Spectrum LSF.

5.2 Explain workload management in LSF.

5.3 Describe IBM Spectrum LSF architecture.

5.4 Describe user management and security in IBM Spectrum LSF.

5.5 Describe licensing models for IBM Spectrum LSF.

5.6 Describe IBM Spectrum LSF product add-ons.

5.7 Describe LSF use cases.

5.8 Describe reporting features in LSF.

Section 6 - Symphony

6.1 Describe the benefits of Spectrum Symphony.

6.2 Describe the Spectrum Symphony Architecture.

6.3 Describe user management and security in Symphony.

6.4 Describe how to onboard an application with Spectrum Symphony.

6.5 Describe resource management in Spectrum Symphony.

6.6 Describe reporting features in Symphony.

6.7 Describe licensing models for Symphony.

Detailed Exam Objectives

Section 1 - Discovery

1.1. Determine customer’s existing applications and computing needs. SUBTASK(S): 1.1.1. Identify eligible applications (high performance computing, analytics, big

data) for Spectrum Computing products 1.1.1.1.High performance computing: Ansys, Matlab, SAS, LS-Dyna,

Openfoam, DataStage, homegrown and others 1.1.1.2.Analytics: Murex, Calypso, Algorithmics, homegrown and others 1.1.1.3.Big Data: Hadoop, NoSQL, Spark, Kafka and others 1.1.1.4.Docker containers, Micro-services

1.1.2. Identify deployment model and size of the environment 1.1.2.1.Installation on single or multiple datacenters 1.1.2.2.Disaster Recovery strategy 1.1.2.3.Number of servers/cores for each application 1.1.2.4.Usage of private or public cloud

1.1.3. Determine customer pain points 1.1.3.1.Performance issues 1.1.3.2.No resources sharing between applications or instances of the

same application 1.1.3.3.Lack of management and monitoring 1.1.3.4.Resiliency/Stability issues

1.1.4. Determine customer’s licensing environment 1.1.4.1.Identify applications with licenses needs 1.1.4.2.Identify licensing model (type of licenses, number of licenses,

features per license, tokens, etc) and need for each application (number of projects, mapping of projects and licenses/features/tokens required, etc)

1.1.5. Identify business owners, program managers and/or application development leads of existing distributed applications

1.2. Determine the size and details of customer’s physical computing

environment. SUBTASK(S): 1.2.1. Determine the number and types of compute clients (operating systems,

amount of memory, processor types, special software, selinux usage). 1.2.2. Determine networking environment

1.2.2.1.Public, outside access networks and connectivity 1.2.2.2.Administration networks 1.2.2.3.High-speed networks (10+ GbE, Infiniband QDR/FD/EDR, etc.) 1.2.2.4.Firewalls (on the OS and on networks)

1.2.3. Determine the amount and types of storage: 1.2.3.1.Determine storage hardware, file system profile and file system

types 1.2.3.2.Determine storage access protocols in use, and for what purpose

(native file system mounts, NFS, SMB, SFTP, FTPS, HTTP, etc.) 1.2.3.3.Determine I/O workload volume, usage patterns, and performance

requirements 1.2.4. Determine user and access environment

1.2.4.1.Remote access capability and requirements 1.2.4.2.User authentication method (flat file, LDAP, etc.) 1.2.4.3.User home, data, and “scratch” directories

1.2.5. Determine the presence and use of any co-processors such as GPGPUs or Xeon Phi

1.2.6. Determine if the customer uses or is interested in using Docker containers to run their workloads.

1.3. Determine the appropriate management framework: Symphony, LSF,

Conductor, Conductor with Spark, Spectrum Cluster Foundation. SUBTASK(S): 1.3.1. List the applications and find the appropriate framework for applications

already integrated with one of the product (Symphony, LSF, Conductor) 1.3.2. Identify workload pattern of the applications to match with the appropriate

framework 1.3.2.1.Short jobs/tasks: Symphony 1.3.2.2.Long jobs: LSF 1.3.2.3.Long running services: Conductor 1.3.2.4.Spark and notebooks: Conductor with Spark

1.3.3. For homegrown applications, identify the appropriate integration model based on application type and customer wish 1.3.2.5.SOA API: Symphony 1.3.2.6.Process execution: LSF or Symphony (symexec) 1.3.2.7.Services: Conductor

1.3.4. Identify if customer needs a solution to provision the operating system => Spectrum cluster foundation

1.4. Assess client application onboarding requirements.

SUBTASK(S):

1.4.1. Review SLAs 1.4.2. Determine memory footprint 1.4.3. Determine block IO requirements 1.4.4. Recommend optimizations

Section 2 - Solution Proposal

2.1. Describe the value proposition for Spectrum Computing solutions. SUBTASK(S): 2.1.1. Describe the extreme performance, scalability, and richness of scheduling

options that Spectrum Computing solutions have through long experience with a diverse set of customers in industry, and academic/applied research centers

2.1.2. Explain how Spectrum Computing solutions can help increase resource utilization.

2.1.3. Describe the job throughput advantages that LSF and Symphony provide over alternative solutions

2.1.4. Describe the multi-cluster capabilities and benefits in LSF and Symphony 2.1.4.1.Simplify management and monitoring of multiple LSF and

Symphony clusters 2.1.4.2.Use for Disaster Recovery 2.1.4.3.Use for cloud bursting Reference: https://www.ibm.com/support/knowledgecenter/SSZUMP_7.1.2/sym_kc/sym_kc_multicluster.html

2.1.5. Describe the extended capabilities in using, monitoring and managing a batch or analytics compute environment that Spectrum Computing add-ons provide (IBM Spectrum LSF, but also IBM Spectrum Symphony and Conductor)

2.2. Respond to RFIs and RFPs. SUBTASK(S): 2.2.1. Identify the appropriate Spectrum Computing product(s) to match RFI/RFP

document, and consulting/services required to deliver and install the solution.

2.2.2. Provide details on Spectrum Computing product(s) proposed. 2.2.3. Explain how Spectrum Computing product(s) will be deployed and

integrated in customer environment. 2.2.4. Describe benefits of Spectrum Computing product(s) in customer’s context 2.2.5. Describe additional capabilities that Spectrum Computing solutions can

provide in extension of product(s) proposed to respond RFI/RFP, which might be required by the customer in the future

2.3. Describe how Spectrum Computing can help consolidate existing

compute infrastructures. SUBTASK(S): 2.3.1. Understand the value of consolidating existing compute infrastructures

Reference:

http://www-03.ibm.com/systems/data/flash/platformcomputing/tco/ Reference: https://www.youtube.com/watch?v=JgOS-gTFsuE 2.3.1.1.Describe possible disadvantages with a number of siloed

applications instead of consolidated infrastructures. 2.3.2. Understand existing client applications and workflows 2.3.3. Map existing and future workloads to logical groups / consumers 2.4. Describe the competitive landscape for Spectrum Computing

offerings. SUBTASK(S): 2.4.1. Understand the key competition for IBM Spectrum LSF

2.4.1.1.Commercial batch workload managers 2.4.1.1.1. Moab 2.4.1.1.2. PBS Pro 2.4.1.1.3. Univa Grid Engine 2.4.1.1.4. RTDA

2.4.1.2.Open source batch workload managers 2.4.1.2.1. SLURM 2.4.1.2.2. Sun Grid Ending derivatives: Open Grid Scheduler,

“Son of Grid Engine”, etc. 2.4.1.2.3. Open PBS 2.4.1.2.4. Lava (open source LSF derivative)

2.4.2. Understand the IBM Spectrum Symphony competition 2.4.2.1.TIBCO DataSynapse 2.4.2.2.Hadoop Distributions: Apache, Cloudera, Hortonworks, MapR,

and others 2.4.2.3.Microsoft HPC and others

2.4.3. Understand the IBM Spectrum Conductor competition 2.4.3.1.Mesosphere DC/OS (commercial) 2.4.3.2.Mesos (open source) 2.4.3.3.Kubernetes and others

2.4.4. Understand the IBM Spectrum Conductor for Spark 2.4.4.1.Apache Spark (and other Spark distributions) 2.4.4.2.Hortonworks 2.4.4.3.Cloudera and others

2.4.5. Understand the IBM Spectrum Cluster Foundation (SCF) competition 2.4.5.1.Bright Cluster Manager 2.4.5.2.Clusterware (Penguin Computing) 2.4.5.3.xCAT (open source CLI-driven provisioning engine in SCF) 2.4.5.4.VMware 2.4.5.5.Apache OpenStack and commercial derivatives 2.4.5.6.ROCKS and others

2.5. Recommend best practice for compute environments.

SUBTASK(S):

2.5.1. Review SLAs 2.5.2. Review hardware architecture noting bottlenecks in IO 2.5.3. Review findings and suggest alternatives to current architecture 2.5.4. Review software architecture (OS, workload management, site/company

patch process, etc) 2.5.5. Review on-going administrative personnel training levels 2.5.6. Review growth projections 2.5.7. Determine optimal workload management 2.5.8. Suggest monthly review of critical applications (all the above) 2.6. Describe High Performance Services offerings on SoftLayer (HPC

Cloud). SUBTASK(S): 2.6.1. Describe IBM High Performance Services for HPC 2.6.2. Describe IBM High Performance Services for Analytics 2.6.3. Describe IBM High Performance Services for Data 2.6.4. Describe hybrid cloud use case (bursting workload to the cloud) using High

Performance Services and on-premise LSF, Symphony, and/or Spectrum Scale

Reference: http://www-03.ibm.com/systems/spectrum-computing/solutions/hpccloud.html 2.6.5. Describe advantages of cloud-based high performance services

2.6.5.1.HPC Cloud Services allows for the adding of High Performance Computing Capacity as needed

2.6.5.2.Can be integrated with an on-premise LSF cluster as a hybrid cloud

2.6.5.3.Can be integrated with an on-premise big data cluster for both public and hybrid cloud engagements

2.6.6. Describe customer use cases that can benefit most from IBM Spectrum Computing High Performance Services 2.6.6.1.Organizations that have a variable workload demand inherent in

their business 2.6.6.2.Companies that need clustered resources but do not have the

budget for a high performance infrastructure 2.6.6.3.Application providers who need the infrastructure for applications

to be available on a pay-per-use basis 2.6.7. Describe advantages of IBM Spectrum Computing High Performance

services over other cloud-based alternatives 2.6.7.1.Improved time to results and faster time to market with access to

ready-to-run high performance clusters on demand 2.6.7.2.Enhanced user productivity with workload management software

that delivers the same user experience for cloud resources as it does for local resources

2.6.7.3.High performance software defined storage provides end-to-end data availability, reliability and scalability

Reference: http://www-03.ibm.com/systems/spectrum-computing/products/high-performance-services/index.html 2.7. Identify if a customer requires professional services for an IBM

Spectrum Computing solution. SUBTASKS: 2.7.1. Understand the specific use cases the customer has (e.g. they want to run

certain applications), or specific configurations or capabilities they need in the workload manager (e.g. a customer ELIM script to report on a customer-specific resource)

2.7.2. Determine whether sample scripts, configurations or applications exist that can provide what the customer needs (Reference (for Conductor): https://hub.jazz.net/user/ibmasc; Check with the IBM technical sales team for other available examples for LSF and Symphony)

2.7.3. Assess whether the customer needs assistance in developing the customization (or wants IBM to perform the work)

2.8. Identify Spectrum Computing products to increase resource

utilization. SUBTASK(S): 2.8.1. Use RTM and Analytics to identify resources utilization and patterns of the

different applications and workloads 2.8.2. Identify LSF scheduling algorithm, queue definitions and resource mapping

that increase utilization in the customer environment. 2.8.3. For Symphony/Conductor

2.8.3.1.Use a resource plan to identify priorities and SLAs required for the different applications and workloads

2.8.3.2.Create an allocation plan accordingly to workload patterns and SLAs required

2.8.4. Simulate potential benefits for the applications (applications that will be able to use more resources, number of resources that could be freed without impact on workloads)

2.9. Explain difference between open source and Spectrum Compute

solutions.

SUBTASK(S): 2.9.1. Explain performance and architectural advantages of Symphony vs. open

source solutions Reference: http://www.ibm.com/support/knowledgecenter/SSZUMP_7.1.2/foundations_sym/failover_sym_admin_perspective.html Reference:

http://www.ibm.com/support/knowledgecenter/SSZUMP_7.1.2/mapreduce_user/map_reduce_workflow.html Reference: https://stacresearch.com/asset/comparison-ibm-infosphere-biginsights-enterprise-edition-apache-hadoop-using-swim

2.9.2. Explain performance and architectural advantages of Conductor with Spark vs. OSS Reference: http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?htmlfid=DCL12416USEN

2.9.3. Explain performance and architectural advantages of LSF vs. OSS 2.9.4. Understand advantages of vendor software and support 2.10. Describe and differentiate: LSF, Symphony and Conductor.

SUBTASK(S): 2.10.1. LSF: a workload management platform for distributed HPC

environments. It provides a comprehensive set of policy-driven scheduling features that enable infrastructure resource utilization and ensure optimal application performance with complex workloads. Typical Use Case: Batch workloads Reference: http://www-03.ibm.com/systems/spectrum-computing/products/lsf/

2.10.2. Symphony: an enterprise-class grid manager for running distributed

application services on a scalable, shared, heterogeneous grid. Typical Use Case: High volume, real-time workloads Reference: http://www-03.ibm.com/systems/spectrum-computing/products/symphony/

2.10.3. Conductor: a generalized service controller for complex long-running

application services. It enables a wide variety of applications to share resources and coexist on the same infrastructure, providing higher resiliency and performance. Typical Use Case: Distributed cloud application workloads Reference: http://www-03.ibm.com/systems/spectrum-computing/products/conductor/

2.10.4. Conductor with Spark: enables organizations to deploy Apache Spark

efficiently and effectively, supporting multiple versions and instances of Spark, increasing performance and scale, maximizing resource usage, and eliminating silos of resources that would otherwise each be tied to separate Spark implementations. Typical User Case: Distributed cloud application workloads (with Spark) Reference: http://www-03.ibm.com/systems/spectrum-computing/products/conductor/

2.11. Describe Host Grouping. SUBTASK(S): 2.11.1. In IBM Spectrum Symphony, resource groups are logical groups of hosts

that help organize a heterogeneous resource pool 2.11.1.1.Host groups provide a way of organizing and grouping hosts 2.11.1.2.Groups can be made of resources that are listed by names or that

specify a specific requirement in terms of operating system, memory, swap space, CPU factor, and so on.

2.11.1.3.Resource groups are matched with a consumer to run workload 2.11.2. In LSF, host resources can be grouped together for identification

purposes, resource allocation, or administrating and monitoring purposes. Reference: http://www.ibm.com/support/knowledgecenter/SSZUMP_7.1.2/help/admin/resource_groups_home.html http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_foundations/resources_overview.html

2.12. Explain Spectrum Compute capabilities for managing application data.

SUBTASK(S):

2.12.1. Explain the impact on workload processing of application data. 2.12.2. Explain the benefits of data aware scheduling.

Reference: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/development_sym/data_aware_app_dev.html

2.12.3. Describe how data aware scheduling can be configured within Spectrum Symphony.

2.12.4. Describe the benefits and operation of LSF Data Manager References: http://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_welcome/lsf_kc_data_manager.html

2.12.5. Describe the benefits of GridSync in Spectrum Symphony References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/management_sym/gridsync.html

Section 3 - Conductor

3.1. Describe benefits of Conductor.

SUBTASK(S): 3.1.1. Describe how Conductor can simplify deployment and management of

distributed applications 3.1.2. Describe how Conductor can share and optimize resources between

applications based on their demands and resource plan 3.1.3. Explain how Conductor can automate the deployment of distributed

applications 3.1.4. Explain how different lines of business and teams can use Conductor on a

multi-tenant environment with fine-grained user management 3.2. Describe the Conductor architecture. SUBTASK(S): 3.2.1. Understand the options available to install Conductor, with Spectrum Scale

and/or Spectrum Cluster Foundation References:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/install/integrations.html

3.2.2. Understand workflow to manage lifecycle of applications References:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/get_started/service_lifecycle_mgmt.html

3.2.3. Understand the components and concepts of Conductor References:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/get_started/overview_components.html http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/get_started/asc_basic_concepts.html

3.2.4. Understand support of containers (cgroups, Job Object and Docker) for applications’ services deployed on Conductor

References: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/manage_services/container_support.html http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/manage_services/docker_integration.html

3.2.5. Understand reporting components and capabilities of Conductor References:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/manage_reports/reports.html

3.3. Define and deploy an application template.

SUBTASK(S):

3.3.1. Consult the “Application Template Reference” Reference: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/reference/app_template_reference.html

3.3.2. Create an application template from scratch or modify an existing application template. 3.3.2.1.Add or remove application template parameters as necessary. 3.3.2.2.Refine resource activity parameters, such as log, path, working

directory, and environment variables. 3.3.2.3.Refine resource service parameters, such as executionuser,

numinstances, and needcredential. 3.3.2.4.Refine outputs if you need to output more or less information from

the application instance. 3.3.3. Create any scripts that are required by the application template.

3.3.3.1.If applicable, modify the Application Service Controller scripts that are included with the application template samples for service initialization, starting, stopping, decommission, and cleanup as needed. Details are included in the applicationtemplatename_scripts folder, which is located in the corresponding sample application template folder.

3.3.3.2.Debug the scripts before you package and run them with IBM Spectrum Conductor.

3.3.4. Download the required binaries for the system, which is based on network accessibility of the target environment. Or if applicable, run the scripts to download the binaries.

3.3.5. Save the application template and scripts in a local directory. 3.3.6. Register the application instance and verify that it was registered correctly. Reference: http://www.ibm.com/support/knowledgecenter/SS4H63_2.1.0/develop_app_instances/app_template_create.html 3.4. Describe user management and security in Conductor. SUBTASK(S): 3.4.1. Understand authentication and authorization mechanisms in Conductor

Reference: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/get_started/security_overview.html

3.4.2. Understand Role-based access control model and the different roles and permissions that can be assigned to users Reference: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/install/rbac_home.html

3.4.3. Describe how to create a user account Reference:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/shared_files/user_account_create.html

3.4.4. Describe how to delete a user account Reference:

http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/shared_files/user_account_delete.html

3.4.5. Describe how to assign roles to users or user groups Reference: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/shared_files/rbac_build_user_role_relationship.html

3.4.6. Describe how to enable external authentication (PAM) Reference: http://www.ibm.com/support/knowledgecenter/en/SS4H63_2.1.0/management_sym/user_auth.html

3.5. Describe reporting features in Conductor.

SUBTASK(S):

Reporting Framework: 3.5.1. Database: the Derby database for use with the reporting feature. Derby is

a small-footprint database, and is only appropriate for demo clusters. If you want to produce regular reports for a production cluster, you must use a supported commercial database.

3.5.2. Data sources: files that store cluster operation and workload information such as host status changes, session, and task status, and so on. IBM Spectrum Conductor uses several files as data sources. These include daemon status files, and event files.

3.5.3. Data loaders: collect the operational data from the data sources and load the data into tables in a relational database. The data loaders connect to the database using a JDBC driver. The data loaders handle daylight savings automatically by using GMT time when collecting data.

3.5.4. Loader controller: a service (plc) controls the data loaders that collect data from the system and writes the data into the database.

3.5.5. Data purger: a service (purger) maintains the size of the database by purging old records from the database and archiving them. By default, the data purger purges all data that is older than 14 days, and purges data every day at 12:30 AM.

Reports: 3.5.6. Standard reports 3.5.7. Custom reports 3.5.8. Reports directory: the `perf` directory, which is a subdirectory of the top-

level EGO directory. 3.5.9. Log files: for the reporting services and data loaders are available in the

logs subdirectory in the reporting framework directory (EGO_TOP/perf/logs).

3.5.10. Event data files:

The events logger stores event data in event data files. Reference: http://www.ibm.com/support/knowledgecenter/SS4H63_2.1.0/foundations_sym/perf_inside.html 3.6. Describe Conductor API.

SUBTASK(S): 3.6.1. List the different components that can be managed using Conductor REST

API 3.6.2. Describe consumers and user management capabilities of REST API 3.6.3. Describe how API can be used to provision a new application and scale its

resources allocation References: https://www.ibm.com/support/knowledgecenter/SS4H63_2.1.0/reference/api_reference.html https://www.ibm.com/support/knowledgecenter/SSZU2E_2.1.0/reference_s/api_references.html 3.7. Describe licensing models for Conductor.

SUBTASK(S): 3.7.1. IBM Platform Conductor

3.7.1.1. Per managed socket 3.7.1.2. Volume (Per TB Pricing)

3.7.2. IBM Platform Conductor with Spark

3.7.2.1. Per managed socket 3.7.2.2. Volume (Per TB Pricing)

3.7.3. IBM Platform Conductor Combined 3.7.3.1. Per managed socket 3.7.3.2. Volume (Per TB Pricing)

Reference: http://www.redbooks.ibm.com/redpapers/pdfs/redp5379.pdf

Section 4 - Conductor with Spark

4.1. Describe the benefits and value of Conductor with Spark. SUBTASK(S): 4.1.1. Describe how Conductor with Spark simplify deployment and management

of Spark clusters from end-to-end (operating system to Spark framework) 4.1.2. Describe how Conductor with Spark supports multiple versions of Spark

and notebooks on the same shared infrastructure 4.1.3. Describe resource sharing capabilities of Conductor with Spark 4.1.4. Describe the integration of notebooks 4.1.5. Explain how different lines of business and teams can use Conductor with

Spark on a multi-tenant environment with fine-grained user management 4.2. Describe Conductor with Spark instance groups.

SUBTASK(S):

4.2.1. IBM Conductor Spark Instance Group: a concept for the notion of a Spark tenant. Each Spark instance group is an installation of Apache Spark that can run Spark core services (Master, Shuffle, and History) and notebooks as configured.

4.2.2. A Spark instance group contains the following elements: 4.2.2.1.Spark core services 4.2.2.2.Spark tools and notebook 4.2.2.3.User and applications 4.2.2.4.Basic isolation

4.2.3. A Spark instance group can be created to serve a line of business or a group member of a business organization.

Reference: http://www.redbooks.ibm.com/redpapers/pdfs/redp5379.pdf 4.3. Describe Conductor with Spark notebooks. SUBTASK(S): 4.3.1. Describe notebooks concepts and how they are used by data scientist to

quickly develop and execute code and visualize results 4.3.2. List supported notebooks (Zeppelin and IPython/Jupyter) 4.3.3. Describe how customer can integrate other notebooks in Conductor with

Spark 4.3.4. Describe how multiple users can each have a different instance of

notebook

Section 5 - LSF

5.1. Describe the benefits of IBM Spectrum LSF. SUBTASK(S): 5.1.1. Manages and presents a heterogeneous set of compute, resource, and

network resources as a single shared resource pool 5.1.2. Provides superior job throughput, pending workload, and scheduler

efficiency compared to alternative workload managers 5.1.3. Multi-cluster capabilities to pool compute clusters together within the data

center or across geography, and enables bursting peak workload into the cloud

5.1.4. A diverse set of scheduling algorithms and capabilities to enable the most efficient execution of workload given competing resource requirements

5.1.5. A full complement of add-on offerings that integrate seamlessly with LSF that provide enhanced job submission and management; operational dashboard and historical analysis tools for administering, monitoring, and optimizing LSF cluster resources; and enhance job scheduling and throughput capabilities

5.2. Explain workload management in LSF. SUBTASK(S): 5.2.1. Describe LSF queues

Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_resource_sharing/share_queue_rsrc.html

5.2.2. Describe LSF resources and resource strings Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_resource_sharing/chap_rep_lsf_resources.html

5.2.3. Describe LSF scheduling options (FCFS, fair share, backfill, etc.) Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_welcome/lsf_kc_short_jobs.html

5.2.4. Describe LSF application profiles Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/chap_app_profiles_lsf_admin.html

5.2.5. Describe job preemption Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/chap_preemptive_lsf_admin.html

5.3. Describe IBM Spectrum LSF architecture.

SUBTASK(S): 5.3.1. Describe the daemons and processes that IBM Platform LSF runs on the

master and compute hosts Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_foundations/lsf_host_processes_overview.html

5.3.2. Describe IBM Spectrum LSF cluster communication paths used between its processes References: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_foundations/communication_paths_lsf.html

5.3.3. Describe how IBM Platform LSF runs parallel jobs Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/chap_parallel_lsf_admin.html

5.4. Describe user management and security in IBM Spectrum LSF. SUBTASK(S): 5.4.1. Describe the LSF Admin user account and role 5.4.2. Describe LSF process owner and starting and stopping LSF by a non-root

user 5.4.3. Describe user groups in LSF 5.4.4. Describe the overall LSF security model

Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_foundations/security_lsf_overview.html

5.4.5. Describe enablement and restriction of resource access by user or user group

5.4.6. Describe capabilities to restrict job information access by users other than the job owner Reference: https://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_security/lsf_security_jobs.htmlhttps://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_security/lsf_security_jobs.html

5.5. Describe licensing models for IBM Spectrum LSF. SUBTASK(S): 5.5.1. Describe differences between IBM Spectrum LSF editions

5.5.1.1.Express Edition 5.5.1.2.Standard Edition 5.5.1.3.Advanced Edition

5.5.2. Describe the available IBM Spectrum LSF bundled offerings 5.5.2.1.IBM Spectrum LSF Community Edition 5.5.2.2.IBM Spectrum LSF Suite for Workgroups

5.5.2.3.IBM Spectrum LSF Suite for HPC Reference Reference: http://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_offering/lsfce10.1_quick_start.html?view=kc

5.5.3. Describe licensing of IBM Spectrum LSF editions and offerings

5.5.3.1.IBM Spectrum LSF and add-on entitlement measured by RVU (by core count adjusted at high-core-count clip levels) and entitled as perpetual license, with annual support renewal

5.5.3.2.IBM Spectrum LSF RTM server licensed per installed server 5.5.3.3.IBM Spectrum LSF bundled offerings (except for Community

Edition) are priced per socket for a fixed term 5.5.4. Describe IBM Spectrum LSF support offerings

5.5.4.1.Purchased support for IBM Spectrum LSF Community Edition 5.5.4.2.One year of support with perpetual license, plus support renewal

charge for subsequent years

5.6. Describe IBM Spectrum LSF product add-ons. SUBTASK(S): 5.6.1. IBM Spectrum LSF Application Center:

5.6.1.1.GUI-based or RESTful client job submission and LSF job monitoring

5.6.1.2.standardized form-based job submission through job templates 5.6.2. IBM Spectrum LSF Process Manager:

5.6.2.1.run and manage complicated, and/or automated workflows with many job dependencies

5.6.2.2.a graphical workflow design environment 5.6.3. IBM Spectrum LSF License Scheduler:

5.6.3.1.token-based licensing (using FlexLM or Reprise License Manager) and has license contention

5.6.4. IBM Spectrum LSF Session Scheduler: 5.6.4.1.schedule an extremely large number of short jobs simultaneously

5.6.5. LSF Resource Connector: 5.6.5.1.Dynamic resource connector

5.6.6. Docker Connector 5.6.7. Spark Hadoop Connector 5.6.8. IBM Spectrum MPI:

5.6.8.1.high-performance MPI solution for distributed computing 5.6.9. IBM Spectrum LSF RTM:

5.6.9.1.an operational analytics dashboard from which they can drill down on cluster and job performance issues and historical data

5.6.10. IBM Spectrum LSF Analytics: 5.6.10.1.generate analyses and reports of compute resource consumption

over periods of time to gain insight into usage patterns and undertake capacity planning

5.6.11. IBM Spectrum LSF Data Manager: 5.6.11.1.intelligent, workload-driven data staging within or between

computing clusters 5.6.12. IBM Spectrum Cluster Foundation

5.6.12.1.Automate initial provisioning 5.6.12.2.maintain system image 5.6.12.3.support remote administration

5.7. Describe LSF use cases. SUBTASK(S): 5.7.1. Be able to run sample scripts in LSF to submit jobs 5.7.2. Describe appropriate use cases for LSF:

5.7.2.1.High number of batch-oriented jobs 5.7.2.2.Product Design Simulation 5.7.2.3.Modeling 5.7.2.4.Any case that requires high job throughput, maximizing of

resources and 5.7.2.5.optimal application performance 5.7.2.6.Preemption 5.7.2.7.Goal-Oriented Scheduling SLAs

Reference: http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=WH&infotype=SA&appname=STGE_DC_ZQ_USEN&htmlfid=DCW03031USEN&att achment=DCW03031USEN.PDF 5.8. Describe reporting features in LSF. SUBTASK(S): 5.8.1. Explain the role of bacct in reporting finished jobs 5.8.2. Explain the role of bjobs in filtering and reporting information about LSF

jobs 5.8.3. Explain the role of bhist in reporting historical information about jobs 5.8.4. Explain how RTM can provide extensive reporting capabilities as an

operational dashboard 5.8.5. Explain how LSF Analytics can provide extensive capacity planning and

business intelligence reporting 5.8.6. Explain how to turn on job energy usage 5.8.7. Explain the features of the IBM Spectrum LSF Application Center Reference: http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_admin/EAS_CPU_job_ene rgy_usage_reporting.html http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/dm_using/dm_bjobs_top.html http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_command_ref/bacct.1.html

http://www.ibm.com/support/knowledgecenter/SSWRJV_10.1.0/lsf_command_ref/bhist.1.html http://www-03.ibm.com/systems/spectrum-computing/products/lsf/ http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=WH&infotype=SA&appname=STGE_DC_ZQ_USEN&htmlfid=DCW03031USEN&att achment=DCW03031USEN.PDFlfid=DCD12362USEN&attachme nt=DCD12362USEN.PDF#loaded

Section 6 - Symphony

6.1. Describe the benefits of Spectrum Symphony. SUBTASK(S): 6.1.1. Review the resources available on the public IBM website

References: http://www-03.ibm.com/systems/spectrum-computing/products/symphony/resources.html

6.1.2. List the key benefits of Spectrum Symphony – 6.1.2.1.Faster time to results (faster throughput, application efficiency,

access additional resources through resource sharing) 6.1.2.2.Reduced infrastructure costs (consolidation through resource

sharing) 6.1.2.3.Reduced operational costs (management tools and consolidation

allows operational teams to manage more resources) 6.1.2.4.Speed time to market (development edition speed development) 6.1.2.5.Enterprise ready (enterprise and mission critical client examples)

6.1.3. Detail key Symphony performance and scalability metrics (latency, maximum cluster size, scheduling interval, number of concurrent application services, cluster efficiency) 6.1.3.1.40,000 concurrent services 6.1.3.2.17,000 tasks submitted per second 6.1.3.3.Reallocates 1000 services per second 6.1.3.4.Sub second latency

6.1.4. Relate impact of scheduling performance (overhead) to impact on business processing and time to results – ability to parallelize workload more fine grained and therefore reduce end to end processing time.

References: http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=BR&infotype=PM&htmlfid=DCB03023USEN&attachment=DCB03023USEN.PDF 6.2. Describe the Spectrum Symphony Architecture.

SUBTASK(S):

6.2.1. Describe the differences between workload management and resource management or orchestration. And outline the responsibilities of EGO and SOAM within Spectrum Symphony. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/symphony_cluster_components.html

6.2.2. Describe the key components of SOAM. 6.2.3. Describe the key components of EGO. References:

http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/ego_component_overview.html

6.2.4. Explain the concept of the consumer and the consumer hierarchy References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/consumers_overview.html

6.2.5. Describe the relationship between consumers, applications, service packages and service binaries. Explain how service binaries are deployed to the cluster nodes.

6.2.6. Outline the differences between management and compute nodes within a cluster. List the key processes that run on each node and their responsibilities.

6.2.7. Describe how high availability is implemented. 6.3. Describe user management and security in Symphony. SUBTASK(S): 6.3.1. Understand different authentication and authorization options in Symphony

References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/management_sym/authentication_authorization_symphony.jpg

6.3.2. Understand different types of user accounts References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/users_home.html

6.3.3. Create a user account 6.3.4. Delete a user account 6.3.5. Assign roles to users or user groups

References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/rbac_build_user_role_relationship.html

6.3.6. Grant users control and view permissions for application workload placement

6.3.7. Enable external authentication (PAM) References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/management_sym/user_auth.html

6.4. Describe how to onboard an application with Spectrum Symphony.

SUBTASK(S):

6.4.1. Describe SOA (service oriented application) processing/runtime model and application objects. Provide high-level description of Symphony client and Symphony services. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/symphony_developer_objects.html

6.4.2. Outline the role of the application profile for a Symphony application. Describe how the application profile is responsible for defining the characteristics of an application and the environment in which it runs. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/application_profile_overview html

6.4.3. List available programming languages for client and service API 6.4.4. Outline a skeleton client and its required functions – connection to grid,

create session, create tasks, submit tasks and fetch completed tasks. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/development_sym/API_classes.html

6.4.5. Discuss different options available to developer in when building client – synchronous or asynchronous task submit/fetch, management of Symphony sessions, common data, session attributes, direct data transfer, data compression References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/development_sym/chap_clients_developing.html

6.4.6. Outline Service container interface and describe purpose of each method. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/services_soam.html

6.4.7. Describe how application binaries are deployed to cluster nodes. 6.4.8. Describe the advanced features available when onboarding an application,

these should include recursive programming, multi task service, data aware scheduling and mechanisms for reducing service startup time. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/development_sym/chap_recursive_workload_apps.html http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/development_sym/multiple_task_service_featureref.html

6.4.9. Describe how to run an existing executable on Spectrum Symphony through the symexec API. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/execution_application_featureref.html

6.5. Describe resource management in Spectrum Symphony.

SUBTASK(S): 6.5.1. Outline how Symphony allows multiple applications / Line of Business to

share the resources managed within a single grid or cluster. Compare and contract with server virtualization – specifically rather than make one server look like many make multiple servers look like one.

6.5.2. List the advantages that can be gained from consolidating multiple silo clusters into a single cluster that supports multiple applications.

Understand why it is important to have a policy or policies that control sharing of resources across applications so that application owners can guarantee that they can meet application processing times

6.5.3. Describe how resources can be arranged into resource groups and why it is useful. Explain what the slot definition is.

6.5.4. Explain how resources are allocated when clients submit workload to the cluster. Ensure to outline the relationship between application managers/schedulers (SSM) and the resource manager (EGO) References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/foundations_sym/resources_how_ego_gives_to_soam_overview.html

6.5.5. Explain the core parts of the resource allocation plan. This should include the following: 6.5.5.1.Ownership Policy 6.5.5.2.Sharing Policy 6.5.5.3.Borrowing / Lending Policy 6.5.5.4.Reclaim Policy 6.5.5.5.Rank

6.5.6. Also describe how a resource plan relates to resource groups. Explain how the consumer hierarchy effects resource allocation and resource reclaim. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/resource_plan_default.html

6.5.7. Compare and contrast the slot allocation policy and multi-dimensional scheduling allocation policy. References: http://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/chap_mds.html

6.6. Describe reporting features in Symphony. SUBTASK(S): 6.6.1. Understand the value of reporting for a distributed compute environment 6.6.2. Describe the standard reports available in Spectrum Symphony

References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/standard_reports.html

6.6.3. Describe custom reports 6.6.4. Export reports for offline viewing 6.6.5. Enable reporting database

References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/management_sym/reports_database_derby_enable.html

6.6.6. Demonstrate the reporting feature References:

https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/management_sym/reports_test.html

6.6.7. Use an external database to store reports References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/reports_database_move_sym.html

6.6.8. Disable reporting References: https://www.ibm.com/support/knowledgecenter/en/SSZUMP_7.1.2/shared_files/reports_disable.html

6.6.9. List the different components that are used for reporting 6.7. Describe licensing models for Symphony. SUBTASK(S): 6.7.1. Describe the four versions of Symphony available

6.7.1.1. Developer Edition 6.7.1.2. Express Edition 6.7.1.3. Standard Edition 6.7.1.4. Advanced Edition

6.7.2. Describe the add-ons available 6.7.2.1. Co-processor harvesting 6.7.2.2. Desktop harvesting 6.7.2.3. Server and VM harvesting 6.7.2.4. GPU harvesting

6.7.3. Describe how Symphony is controlled by an encrypted key in an entitlement file

6.7.4. Explain that add-ons also require an encrypted key entitlement file 6.7.5. Explain the per-core licensing model 6.7.6. Describe how upgrading to a different version of Symphony will require the

right entitlement file Reference: http://www.ibm.com/support/knowledgecenter/SSZUMP_7.1.2/management_sym/licensing_sym.html