REMINDER

Check in on the

COLLABORATE mobile app

How to Turn New Recruits Into Oracle EPM Infrastructure Gurus

Prepared by:

Nicholas King

Session ID#: 13888

Topics

■ Level 1 – Beginner

▪ What is Oracle EPM?

▪ What’s so Tough about EPM?

▪ EPM Modules

■ Level 2 - Intermediate

▪ Monitoring

▪ Patching

▪ SSL

■ Level 3 – Guru

▪ EPM Technology Stack

▪ Troubleshooting

-What is Oracle EPM Infrastructure?

-What’s so Tough about EPM?

------------------------------------------------------------------ Level 1 - Beginner

Why is Infrastructure Support Critical?

■ Financial close is so critical that companies will spend millions of dollars to streamline the process

■ Software issues during close can be catastrophic

■ Oracle EPM is a “best-of-breed” product representing some of the most advanced tools in Finance

How to Become Best-of-Breed

■ Key to Understanding EPM Architecture

■ Acquisitions:

▪ 1998 - Hyperion Software merges with Arbor and the combined

company is renamed Hyperion Solutions (Essbase)

▪ 1999 - Hyperion acquires Sapling Corporation (Strategic

Finance)

▪ 2003 - BI Consolidation continues as Hyperion acquiers Brio

(Reporting)

▪ 2006 - Hyperion acquires UpStream (FDM)

▪ 2007 - Oracle buys Hyperion

How to Become Best-of-Breed

Implications of Acquisitions

■ How do you integrate millions of lines of code from many different products?

■ When will … work on Linux?

■ What else?

What’s so Tough?

What’s so Tough?

Helpful Background

■ Coding – Java, .NET, Scripting

■ OS Administration

■ SSL

■ Web / Application Server

■ High Availability

■ Troubleshooting

Oracle EPM is: - a conglomeration of many custom modules - proprietary and learned only from experience

Absolutely Required

■ Time

■ Patience

■ Dedication to Learn Proactively

Beginner’s Homework

■ Become familiar with EPM modules

■ Execute basic validation procedures

■ Install and configure EPM test environments

■ Internalize the Oracle documentation – Read up!

▪ http://docs.oracle.com/cd/E17236_01/index.htm

▪ EPM Module Admin Guides

▪ EPM Module User Guides

▪ Infrastructure:

— Support Matrix

— Installation and Configuration

— High Availability and Disaster Recovery

— Security and SSL

-Monitoring

-Patching

-SSL

------------------------------------------------------------------ Level 2 - Intermediate

Monitoring and Alerting

■ Keeping track of the modules

■ Alert when modules are down

■ Good monitoring identifies problems without having to perform intensive manual effort

■ Dashboard – Known “good state”

Types of Monitoring

■ Passive Monitoring

▪ Servers up and running

▪ EPM modules up and running

▪ Tip: Use OHS access_log to find specific URLs

■ Active Monitoring

▪ Load Runner – simulate user activity

https://server/hfm/GlobalWorkspaceNav/bpm/conf/HfmConfig.xml https://server:8543/calcmgr/conf/CalcMgrConfig.xml https://server/hyperion-bpma-server/Sessions.asmx https://server/awb/conf/AwbConfig.xml https://server:28443/interop/index.jsp https://server:10083/easconsole/console.html

Patching – Why Patch?

■ Patches are NOT included with the installation media

■ Business needs and stability

■ Patches come out frequently

■ Proactively Patch

■ A typical install has about 20 separate patches to apply after installation

Patching Process

■ Search for new patches for each module

■ What other patches are required dependencies?

■ Detailed Implementation plan – repeatable, concise

Search Dependencies Plan

Searching for Patches

http://support.oracle.com

Search Dependencies Plan

Myriad of HFM Patches

PSU 300

PSU 301

PSU 302

Search Dependencies Plan

Patch Plan – Review Readme

■ Cumulative or need specific base version, like .300 first?

■ Prerequisite patches for other Modules

Search Dependencies Plan

Patch Plan – Review Readme

For Each Patch Identified

■ Full Opatch command

■ Post-install steps

■ When to redeploy webapp? If patch has EAR/WAR

■ What nodes should the patch be applied

■ Are there any client patches or adapters? Fr studio.. FDM..

■ If redeploying – reset java heap sizes

Search Dependencies Plan

Sample Plan

--- Requirement: ADF Patch (hosts: Web1, Web2, Plan1, Plan2): opatch.bat apply D:\Oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\16763038 -oh D:\Oracle\Middleware\oracle_common -jre D:\Oracle\Middleware\jdk160_29 --- Requirement: CalcMgr Patch .303 (hosts: Web1, Web2) requires - Oracle Hyperion Workspace 11.1.2.2.300 (EPM_PSU13936087_IE9) (ok) opatch.bat apply D:\oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\16384766 -oh D:\oracle\Middleware\EPMSystem11R1 -jdk D:\oracle\Middleware\jdk160_29 delete <MIDDLEWARE_HOME>/user_projects/domains/<DOMAIN_NAME>/servers/<CalcMgr0>/tmp/ Redeploy CalcMgr --- Planning (hosts: Web1, Web2, Plan1, Plan2) opatch.bat apply D:\oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112 -oh D:\oracle\Middleware\EPMSystem11R1 -jdk D:\oracle\Middleware\jdk160_29 del <MIDDLEWARE_HOME>/user_projects/domains/EPMSystem/servers/<server> Web Hosts Only: copy /Y \oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112\files\common\ADM\Planning\11.1.2.0\lib\HspJS.jar \oracle\Middleware\EPMSystem11R1\common\ADM\Planning\11.1.2.0\lib copy /Y \oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112\files\common\ADM\Planning\11.1.2.0\lib\HspJS.jar \oracle\Middleware\EPMSystem11R1\common\ADM-64\Planning\11.1.2.0\lib Redeploy Planning Reset Heap sizes for CalcMgr and Planning

Search Dependencies Plan

SSL Keystore / Wallet

Identity/Private What cert the server will serve up when URL is requested - https://mylbname.com/workspace

You must know when your SSL certificates expire and how to rotate them. Understanding SSL is critical if it is enabled in the environment.

Key

sto

re /

Wal

let

Trusted What certificates from other servers are accepted - mydomain.com - serv1.mydomain.com - serv2.mydomain.com

SSL Wallet Example

Wal

let

Each Layer Uses a Different SSL Management Tool

OHS

Oracle Wallet Manager

.sso WebLogic

Admin Server

Console .jks

IIS

MMC – Certificates Inetmgr Site

Bindings .p12 Java

Keytool

.jks

WebLogic

OHS

Java

IIS

Trusted SSL

SSL Implementation

■ Installing and Configuring Oracle® Hyperion Enterprise Performance Management System 11.1.2 with SSL Enabled on All Layers http://www.oracle.com/technetwork/middleware/performance-

management/tutorials/index-087654.html

■ Review each technology stack’s SSL documentation

▪ IIS

▪ WebLogic

▪ Java Keytool

▪ OHS

■ Tip: WLST can be used to script WebLogic SSL components

Intermediate Homework

■ Become familiar with operational tasks

▪ Patching

▪ Perform SSL certificate rotation

▪ Jobs - LCM exports, FDM loads, etc.

▪ Backup and Restore procedure

▪ Password changes – admin and database accounts

▪ Oracle Service Requests (SR) - follow tickets from beginning to

resolution

-EPM Technology Stack

-EPM Troubleshooting

- Approach

- Examples

------------------------------------------------------------------ Level 3 – EPM Support Guru

Understanding The Technology Stack

OHS

WebLogic IIS

Java .Net VB/C++

Web Proxy

Application

Server

Core Code

EPM Modules and the Technology Stack

WebLogic

Workspace Financial Reporting

Planning Most other Modules

DRM HFM Web*

EPMA Web* FDM Web

Java

Framework Services

.Net VB/C++ Essbase

HFM & FDM App

Web

Application

Server

Core Code

WebLogic

Java

IIS

.NET VB/C++

* Some components also WebLogic

Understanding WebLogic

WebLogic

Workspace Financial Reporting

Planning Most other Modules

Web Application

Server

WebLogic

A WebLogic Application is…

▪ Framework for Running Java Web Applications

▪ Java Web Applications are bundled into packages and deployed

as part of the WebLogic configuration

▪ Applications are exposed via web pages to end users

■

First Build the Application 1. Core Code (Java)

2. Java Libraries

3. WAR File

4. EAR File

Next Deploy App in WebLogic

Main Menu

Deployment Details

WebLogic Admin Server: http://wlserver:7001/console

WebLogic in Oracle EPM

■ Deploy to Application Server

■ Deployed using WLST – WebLogic Scripting Language

■ Each product has an EAR file on the file system EPMSystem11R1\products\Planning\AppServer\

InstallableApps\Common\HyperionPlanning.ear

■ Patching modifies the ear files and requires redeploying the application server

Distributed Env – WebLogic Domain

WebLogic Domain server1

Planning1 server2

Planning2 server3

HFMWeb server4

WebLogic Domain Server Pushes Deployment Files to Each Node HyperionPlanning.ear HFMWeb1.ear

On WebLogic Domain node: - must install all web products - must be patched along with all products

What is OHS?

■ Essentially an Oracle branded Apache Web Server

■ Three main configuration files

▪ httpd.conf – proxy to IIS

▪ ssl.conf

▪ mod_wl_ohs.conf – proxy to WebLogic modules

■ Files are auto-generated in Configure Web Server task in the EPM Configuration tool.

OHS is a proxy and load balancer to all EPM modules

Pictorial OHS & Load Balancing

Textual OHS & Load Balancing

ProxyPass /hfm balancer://hfmCluster stickysession=ORA_EPM_hfm_ROUTE_ID nofailover=Off timeout=7200 <Proxy balancer://hfmCluster> Header add Set-Cookie "ORA_EPM_hfm_ROUTE_ID=hfm.%{BALANCER_WORKER_ROUTE}e; path=/hfm;" env=BALANCER_ROUTE_CHANGED BalancerMember https://hfm1.com:443/hfm loadfactor=1 route=0 BalancerMember https://hfm2.com:443/hfm loadfactor=1 route=1 ProxySet lbmethod=bytraffic </Proxy>

<LocationMatch ^/HyperionPlanning> SetHandler weblogic-handler WeblogicCluster plan2.com:8343,plan1.com:8343 WLIOTimeoutSecs 14400 Idempotent OFF WLSocketTimeoutSecs 600

■ httpd.conf - IIS

■ mod_wl_ohs.conf - WebLogic

Troubleshooting Basics

■ Check monitoring dashboard – everything up?

■ Reproduce the problem

■ Narrow down the products involved

■ Look through the logs

■ Do the messages make sense?

■ External Resources

▪ Google

▪ Oracle Knowledgebase

▪ Oracle Forums

▪ Oracle Support

Sifting Through Logs

■ 100+ log files in EPM deployment

■ Types of logs

▪ WebLogic Application Logs

user_projects\domains\EPMSystem\servers

▪ EPM Application Logs

Oracle ODL logging framework

user_projects\epmsystem1\diagnostics\logs

▪ Windows Event Viewer

▪ HFM Error Log Viewer

▪ FDM Logs in FDMData Folder

Troubleshooting EPM - Example

Show Details

End user messages are too generic and misleading

Notice the framework module is having issues

Troubleshooting EPM - Example

Check WebLogic Logs for Framework Module

■ Primary log user_projects\domains\EPMSystem\servers\RaFramework0\logs\

RaFramework0.log

■ Other logs in this dir, sort by date

▪ RaFramework_CommonClient.log

▪ Framework.log

▪ RaFramework_stdout_console_servlets.log

GsmNotFound com.brio.one.services.globalservicemanager.GSMException:

GsmNotFound

Troubleshooting EPM - Example

Check Application Tier

■ Quick Port Test telnet localhost 6800

Connecting To localhost...Could not open connection to the host, on port

6800: Connect failed

■ Primary EPM Application Log user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\

stdout_console_RAF_AGENT_MODULE.log

[oracle.EPMRAF.stdout] Invoking Graceful shutdown...

■ agent.log

Caused by: com.oracle.cmc.agent.RTAgentException: Error While Loading LSM Info

Caused by: com.sqribe.transformer.DBConnectionException: Database Server error:

DBConnectionPool::get: ORA-28000: the account is locked

Guru Homework

■ Install and configure a new SSL, load balanced environment

■ Take responsibility for infrastructure issues

■ Troubleshoot EPM

▪ Quickly understand interactions between products and evaluate

error messages from logs to quickly resolve issues

■ Become Part of the Oracle Community

▪ Blog

▪ Forums

■ Begin to train others in the art of EPM infrastructure

Questions? kingn@google.com Hyperion Error of the Day epm-errors.blogspot.com

------------------------------------------------------------------ Thank you Gurus!

Please complete the session evaluation We appreciate your feedback and insight

You may complete the session evaluation either

on paper or online via the mobile app