Exadata dbnode update

X2-2 quarter rack image version from 11.2.3.2.1 to

11.2.3.3.0 from 11.2.3.3.1 to

12.1.2.1.0

ver.1.5

Infrastructure & Governance

Date : 2015. 1.30 작성자 : 김 종 인

■ DB node update [ 문서 ID 1553103.1] dbnodeupdate.sh: Exadata Database Server Patching using the DB Node Update Utility

-DB node update utility 란 ? (= dbnodeupdate.sh)exadata 의 dbnode 의 new release 를 적용하기 위한 모든 수동작업을 대체하고 작업전 check 및 적용과정을 자동으로 수행해주는 기능을 한다 .

-dbupdate.sh support (4 월 23 일 기준 최신버전 Release 3.23)

• Oracle Linux 5.5 and later (/etc/enterprise-release)• Upgrades starting at release 11.2.2.4.2  (running on OL 5.5 or later)• Rollbacks starting at release 11.2.2.4.2  (running on OL 5.5 or later)• 'Known issues' for upgrades to release 11.2.3.2.1 and later. Example: 112320 => 112321 supported but has no ‘known issues’ dbnodeupdate.sh is aware of

※ dbupdate.sh 를 사용하는 4 가지 Case

① One-Time Setup (updating procedure for Exadata Database Servers running on release 11.2.2.4.2 on Oracle Linux 5.5 or later - dbnodeupdate.sh will detect and execute the One-Time Setup when required)② Updating database servers running Exadata releases later than 11.2.2.4.2 on Oracle Linux 5.5 or later③ Rolling back updates④ Post-Patching (or Post-Rollback steps) (relinking the Oracle homes, enabling Grid Infrastructure to start)

※ dbupdate.sh 로 제공되는 2 가지 방법

① Upgrading from 11.2.3.2.1 to 11.2.3.3.0 via http (yum 이용 )[root@dm01db01 u01]# ./dbnodeupdate.sh -u -l http://my-repo.us.oracle.com/yum/unknown/EXADATA/dbserver/11.2.3.3.0/base/x86_64/ -s –n② Upgrading from 11.2.3.2.1 to 11.2.3.3.0 via iso

■ DB node update- dbnodeupdate.sh 가 제공하는 기능들

• Creates log file to track script execution and changes• Creates diag file with 'before patching' situation• Provides 'next steps'• Includes checks and workarounds for known issues• Creates and runs the backup utility • Checks space requirements of /boot filesystem• Checks for left-over snapshots• Checks for required files and utilities• Includes 'check-only' option• Includes 'quiet mode'• Relinks all Database and Grid Infrastructure (GI) Homes• Enables / disables GI to stop/start• Checks for nfs/smbfs mounts and unmounts when possible• Validates provided media (zip/ISO, http)• Validates user• Provides rollback option• Generates 'yum repo'-file• Validates yum.conf-file• Logs console history• Stops OSwatcher before patching• Unpacks media and helper zip-files• Stops db console and em agent• Checks for failed 'pvmove' actions• Checks for left over snapshots and failed backups• Supports multiple em agents• Checks for 3GB free space in "/”

■ DB node update

1. Download p16486998_121110_Linux-x86-64.zip ( 이 zip 파일안에 dbnodeupdate.sh and dbupdate-helpers.zip 파일이 포함되어 있다 .) * dbupdate-helpers.zip 파일은 압축해제하지 말것 (update 설치에 사용되는 dbnode ISO 이미지는 버전별로 888828.1 의 가이드에 따라 down 받는다 )2. unzip p16486998_121110_Linux-x86-64.zip

inflating: dbnodeupdate.sh inflating: dbupdate-helpers.zip inflating: README.txt

3. dbnodeupdate.sh 실행 (3 가지 main mode 가 존재 )

-u for updating and/or OneTime : For updates for release 11.2.2.4.2 and later (including OneTime Setup) -r for rolling back : Rolling back to a previous release -c for completing post upgrade or rollback steps. : Validations and relinking steps for post upgrade, post roll -back steps

추가적인 flags

- (-l) <baseurl/zip file> : For the url of the repository, can be http:// or zip file (containing the ISO) for the reposi-tory. - (-s) : To shutdown the stack in case still up- (-q) : Quiet mode, no prompting- (-n) : Disable backup- (-p) : Bootstrap phase (1 or 2) only to be used when instructed by dbnodeupdate.sh- (-v) : Verify prereqs only, no patch is applied.- (-t) <to release> : 'to release' - used when in quiet mode or used when updating to one-offs/releases via 'lat-est' channel (requires 11.2.3.2.1)- (-b) : Perform backup only (to be used in combination with upgrade flags)- (-a) <alert.sh> : Full path to shell script used for alert trapping- (-V) : Print version

■ DB node update (11.2.3.2.1 - > 11.2.3.3.0)root user 로 수행

[root@exa1 WorkDir]# ./dbnodeupdate.sh -u -l /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/p17809253_112330_Linux-x86-64.zip –n ( 백업하지 않겠다 )

Default 로 수행시 백업을 진행하기 때문에 미리 Backup 을 받고 (-b Backup only) Update 를 수행할것 권장함 .########################################################################################################################### ## Guidelines for using dbnodeupdate.sh (rel. 3.21): ## ## - Prerequisites for usage: ## 1. Refer to dbnodeupdate.sh options. See MOS 1553103.1 ## 2. Use the latest release of dbnodeupdate.sh. See patch 16486998 ## 3. Run the prereq check with the '-v' option. ## ## I.e.: ./dbnodeupdate.sh -u -l /u01/my-iso-repo.zip -v ## ./dbnodeupdate.sh -u -l http://my-yum-repo -v ## ## - Failures due to dependency issues occur because of customization: ## - The prereq checks detect these dependency issues that need to be addressed prior to running a successful update. ## - Customized rpm packages may fail the built-in dependency check and system updates cannot proceed until resolved. ## ## When upgrading from releases later than 11.2.2.4.2 to releases before 11.2.3.3.0: ## - Conflicting packages should be removed before proceeding the update. ## ## When upgrading to releases 11.2.3.3.0 or later: ## - When the 'exact' package dependency check fails 'minimum' package dependency check will be tried. ## - When the 'minimum' package dependency check also fails, ## the conflicting packages should be removed before proceeding. ## ## - As part of the prereq checks and as part of the update, a number of rpms will be removed.

## This removal is required to preserve Exadata functioning. This should not be confused with obsolete packages. ## - See /var/log/cellos/packages_to_be_removed.txt for details on what packages will be removed. ## ## - In case of any problem when filing an SR, upload the following: ## - /var/log/cellos/dbnodeupdate.log ## - /var/log/cellos/dbnodeupdate.<runid>.diag ## - where <runid> is the is of the failing run. ## ###########################################################################################################################Continue ? [Y/n] Y

■ DB node update(*) 2014-04-15 13:24:24: Unzipping helpers (/opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/dbupdate-helpers.zip) to /opt/oracle.SupportTools/dbnodeupdate_helpers (*) 2014-04-15 13:24:24: Initializing logfile /var/log/cellos/dbnodeupdate.log (*) 2014-04-15 13:24:25: Collecting system configuration details. This may take a while...

Warning: File /etc/oratab not in place. dbnodeupdate.sh can continue without /etc/oratab but will then assume no Oracle homes are installed.

Continue ? [Y/n] Y

(*) 2014-04-15 13:24:55: Validating system details for known issues and best practices. This may take a while... (*) 2014-04-15 13:24:55: Checking free space in /u01 (*) 2014-04-15 13:24:55: Unzipping /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/p17809253_112330_Linux-x86-64.zip to /u01/app/oracle/stage.150414132410, this may take a while (*) 2014-04-15 13:25:04: Original /etc/yum.conf moved to /etc/yum.conf.150414132410, generating new yum.conf (*) 2014-04-15 13:25:04: Generating Exadata repository file /etc/yum.repos.d/Exadata-computenode.repo (*) 2014-04-15 13:25:12: Validating the specified source location. (*) 2014-04-15 13:25:13: Cleaning up the yum cache. (*) 2014-04-15 13:25:14: Preparing update for releases 11.2.3.3.0 and later (*) 2014-04-15 13:25:18: Drive /dev/sda has more than two partitions. This is non-standard. If hotspare still exists it will not be reclaimed (*) 2014-04-15 13:25:20: Performing yum package dependency check for 'exact' dependencies. This may take a while... (*) 2014-04-15 13:25:24: 'Exact'package dependency check succeeded. (*) 2014-04-15 13:25:24: 'Minimum' package dependency check succeeded. Active Image version : 11.2.3.2.1.130109Active Kernel version : 2.6.32-400.11.1.el5uekActive LVM Name : /dev/mapper/VGExaDb-LVDbSys1Inactive Image version : n/aInactive LVM Name : /dev/mapper/VGExaDb-LVDbSys2Current user id : rootAction : upgrade Upgrading to : 11.2.3.3.0.131014.1 (to exadata-sun-computenode-exact)Baseurl : file:///var/www/html/yum/unknown/EXADATA/dbserver/150414132410/x86_64/ (iso)Iso file : /u01/app/oracle/stage.150414132410/repoimage.isoCreate a backup : No Shutdown stack : No (Currently stack is down - /etc/oratab not found)RPM exclusion list : Not in use (add rpms to /etc/exadata/yum/exclusion.lst and restart dbnodeupdate.sh)RPM obsolete list : /etc/exadata/yum/obsolete.lst (lists rpms to be removed by the update) : RPM obsolete list is extracted from exadata-sun-computenode-11.2.3.3.0.131014.1-1.x86_64.rpm

■ DB node updateExact dependencies : No conflictsMinimum dependencies : No conflictsLogfile : /var/log/cellos/dbnodeupdate.log (runid: 150414132410)Diagfile : /var/log/cellos/dbnodeupdate.150414132410.diagServer model : SUN FIRE X4170 M2 SERVERdbnodeupdate.sh rel. : 3.21 (always check MOS 1553103.1 for the latest release of dbnodeupdate)Note : After upgrading and rebooting run './dbnodeupdate.sh -c' to finish post steps.Automatic checks incl. : Issue - Database Server upgrades may hit network routing issues after the upgrade : Issue - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12 : Issue 1.7 - Updating database servers with customized partitions may remove partitions already in use : Issue - 11.2.3.3.0 and 12.1.1.1.0 require disabling SDP APM settings. See MOS 1623834.1 : Issue - Incorrect validation name for ExaWatcher in /etc/cron.daily/cellos stops ExaWatcherManual checks todo : Updates from releases earlier than 11.2.3.3.0 may hang during reboot. See MOS 1620826.1 for more details

Continue ? [Y/n] Y (*) 2014-04-15 13:26:55: Collecting console history for diag purposes (*) 2014-04-15 13:27:17: Unmount of /boot successful (*) 2014-04-15 13:27:17: Check for /dev/sda1 successful (*) 2014-04-15 13:27:17: Mount of /boot successful (*) 2014-04-15 13:27:17: Disabling stack from starting (*) 2014-04-15 13:27:17: Unable to disable GI, no /etc/oratab (*) 2014-04-15 13:27:17: OSWatcher stopped successful (*) 2014-04-15 13:27:17: Validating the specified source location. (*) 2014-04-15 13:27:18: Cleaning up the yum cache. (*) 2014-04-15 13:27:19: Preparing update for releases 11.2.3.3.0 and later (*) 2014-04-15 13:27:20: Drive /dev/sda has more than two partitions. This is non-standard. If hotspare still exists it will not be reclaimed (*) 2014-04-15 13:27:22: Performing yum update. Node is expected to reboot when finished. (*) 2014-04-15 13:30:15: Waiting for post rpm script to finish. Sleeping another 60 seconds (60 / 900)

Remote broadcast message (Tue Apr 15 13:30:21 2014):Exadata post install steps started.It may take up to 5 minutes.Remote broadcast message (Tue Apr 15 13:31:03 2014):Exadata post install steps completed with success (*) 2014-04-15 13:31:15: Waiting for post rpm script to finish. Sleeping another 60 seconds (120 / 900) (*) 2014-04-15 13:32:15: All post steps are finished. (*) 2014-04-15 13:32:15: System will reboot automatically for changes to take effect (*) 2014-04-15 13:32:15: After reboot run "./dbnodeupdate.sh -c" to complete the upgrade (*) 2014-04-15 13:32:19: Rebooting now...

Broadcast message from root (pts/0) (Tue Apr 15 13:32:19 2014):The system is going down for reboot NOW!

■ DB node update3. [root@exa1 WorkDir]# ./dbnodeupdate.sh –c ( 업그레이드 or 롤백후 후속작업 완료 )########################################################################################################################### ## Guidelines for using dbnodeupdate.sh (rel. 3.21): # # ## ## - Prerequisites for usage: ## 1. Refer to dbnodeupdate.sh options. See MOS 1553103.1 ## 2. Use the latest release of dbnodeupdate.sh. See patch 16486998 ## 3. Run the prereq check with the '-v' option. ## ## I.e.: ./dbnodeupdate.sh -u -l /u01/my-iso-repo.zip -v ## ./dbnodeupdate.sh -u -l http://my-yum-repo -v ## ## - Failures due to dependency issues occur because of customization: ## - The prereq checks detect these dependency issues that need to be addressed prior to running a successful update. ## - Customized rpm packages may fail the built-in dependency check and system updates cannot proceed until resolved. ## ## When upgrading from releases later than 11.2.2.4.2 to releases before 11.2.3.3.0: ## - Conflicting packages should be removed before proceeding the update. ## ## When upgrading to releases 11.2.3.3.0 or later: ## - When the 'exact' package dependency check fails 'minimum' package dependency check will be tried. ## - When the 'minimum' package dependency check also fails, ## the conflicting packages should be removed before proceeding. ## ## - As part of the prereq checks and as part of the update, a number of rpms will be removed.

## This removal is required to preserve Exadata functioning. This should not be confused with obsolete packages.

## - See /var/log/cellos/packages_to_be_removed.txt for details on what packages will be removed.

## ## - In case of any problem when filing an SR, upload the following: ## - /var/log/cellos/dbnodeupdate.log ## - /var/log/cellos/dbnodeupdate.<runid>.diag ## - where <runid> is the is of the failing run. ## ###########################################################################################################################Continue ? [Y/n] Y

■ DB node update(*) 2014-04-15 13:40:06: Unzipping helpers (/opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/dbupdate-helpers.zip) to /opt/oracle.SupportTools/dbnodeupdate_helpers (*) 2014-04-15 13:40:06: Initializing logfile /var/log/cellos/dbnodeupdate.log (*) 2014-04-15 13:40:07: Collecting system configuration details. This may take a while...

Warning: File /etc/oratab not in place. dbnodeupdate.sh can continue without /etc/oratab but will then assume no Oracle homes are installed.

Continue ? [Y/n] Y Active Image version : 11.2.3.3.0.131014.1Active Kernel version : 2.6.39-400.126.1.el5uekActive LVM Name : /dev/mapper/VGExaDb-LVDbSys1Inactive Image version : n/aInactive LVM Name : /dev/mapper/VGExaDb-LVDbSys2Current user id : rootAction : finish-post (perform post steps, relink enable/disable crs)Shutdown stack : No (Currently stack is down - /etc/oratab not found)Logfile : /var/log/cellos/dbnodeupdate.log (runid: 150414133958)Diagfile : /var/log/cellos/dbnodeupdate.150414133958.diagServer model : SUN FIRE X4170 M2 SERVERdbnodeupdate.sh rel. : 3.21 (always check MOS 1553103.1 for the latest release of dbnodeupdate) : Issue - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12 : Issue 1.7 - Updating database servers with customized partitions may remove partitions already in use : Issue - 11.2.3.3.0 and 12.1.1.1.0 require disabling SDP APM settings. See MOS 1623834.1 : Issue - Incorrect validation name for ExaWatcher in /etc/cron.daily/cellos stops ExaWatcher

Continue ? [Y/n] Y

(*) 2014-04-15 13:41:07: Verifying firmware updates/validations. Maximum wait time: 30 minutes. (*) 2014-04-15 13:41:07: If the node reboots during this execution, re-run './dbnodeupdate.sh -c' after the node restarts.. (*) 2014-04-15 13:41:07: Collecting console history for diag purposes (*) 2014-04-15 13:41:21: No rpms to remove (*) 2014-04-15 13:41:23: ExaWatcher stopped successful (*) 2014-04-15 13:41:23: Fix for bug 17885493 applied successful (*) 2014-04-15 13:41:23: Starting ExaWatcher (*) 2014-04-15 13:41:23: All post steps are finished.

It is recommended run the following commands to remove old file(s) from previous patching runs : rm -i /u01/app/oracle/stage.150414132410/repoimage.iso

■ DB node update (11.2.3.3.1 -> 12.1.2.1.0 = OEL 5.9 -> OEL 6.6)[EXA1]root@exa1:/root/patch# ./dbnodeupdate.sh -u -l /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/p20170913_121210_Linux-x86-64.zip –n ########################################################################################################################### ## Guidelines for using dbnodeupdate.sh (rel. 4.11): ## ## - Prerequisites for usage: ## 1. Refer to dbnodeupdate.sh options. See MOS 1553103.1 ## 2. Use the latest release of dbnodeupdate.sh. See patch 16486998 ## 3. Run the prereq check with the '-v' option. ## ## I.e.: ./dbnodeupdate.sh -u -l /u01/my-iso-repo.zip -v ## ./dbnodeupdate.sh -u -l http://my-yum-repo -v ## ## - Prerequisite dependency check failures can happen due to customization: ## - The prereq check detects dependency issues that need to be addressed prior to running a successful update. ## - Customized rpm packages may fail the built-in dependency check and system updates cannot proceed until resolved. ## ## When upgrading from releases later than 11.2.2.4.2 to releases before 11.2.3.3.0: ## - Conflicting packages should be removed before proceeding the update. ## ## When upgrading to releases 11.2.3.3.0 or later: ## - When the 'exact' package dependency check fails 'minimum' package dependency check will be tried. ## - When the 'minimum' package dependency check also fails, ## the conflicting packages should be removed before proceeding. ## ## - As part of the prereq checks and as part of the update, a number of rpms will be removed. ## This removal is required to preserve Exadata functioning. This should not be confused with obsolete packages. ## - See /var/log/cellos/packages_to_be_removed.txt for details on what packages will be removed. ## ## - In case of any problem when filing an SR, upload the following: ## - /var/log/cellos/dbnodeupdate.log ## - /var/log/cellos/dbnodeupdate.<runid>.diag ## - where <runid> is the unique number of the failing run. ## ###########################################################################################################################Continue ? [y/n] y

■ DB node update(*) 2015-01-30 09:23:34: Unzipping helpers (./dbupdate-helpers.zip) to /opt/oracle.SupportTools/dbnodeupdate_helpers (*) 2015-01-30 09:23:34: Initializing logfile /var/log/cellos/dbnodeupdate.log

Warning: Active network mounts found on this DB node. Before taking a backup or performing the actual update these need to be unmounted. For the actual update dbnodeupdate.sh will try unmounting them silently. During collection of system configuration (prereq) stale network mounts may cause long waits and dbnodeupdate.sh to stall It is therefore recommended (not required) to unmount any active network mount before proceeding the actual update.

Continue ? [y/n] y

(*) 2015-01-30 09:23:37: Collecting system configuration settings. This may take a while... (*) 2015-01-30 09:24:04: Validating system settings for known issues and best practices. This may take a while... (*) 2015-01-30 09:24:04: Checking free space in /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/iso.stage.300115092333 (*) 2015-01-30 09:24:04: Unzipping /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/p20170913_121210_Linux-x86-64.zip to /opt/oracle.Sup -portTools/onecommand/linux-x64/WorkDir/iso.stage.300115092333, this may take a while (*) 2015-01-30 09:24:16: Original /etc/yum.conf moved to /etc/yum.conf.300115092333, generating new yum.conf (*) 2015-01-30 09:24:16: Generating Exadata repository file /etc/yum.repos.d/Exadata-computenode.repo (*) 2015-01-30 09:24:25: Validating the specified source location. (*) 2015-01-30 09:24:26: Cleaning up the yum cache. Active Image version : 11.2.3.3.0.131014.1Active Kernel version : 2.6.39-400.126.1.el5uekActive LVM Name : /dev/mapper/VGExaDb-LVDbSys1Inactive Image version : n/aInactive LVM Name : /dev/mapper/VGExaDb-LVDbSys2Current user id : rootAction : upgrade Upgrading to : 12.1.2.1.0.141206.1 - Oracle Linux 5->6 upgradeBaseurl : file:///var/www/html/yum/unknown/EXADATA/dbserver/300115092333/x86_64/ (iso)Iso file : /opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/iso.stage.300115092333/exadata_ol6_base_repo_141206.1.isoCreate a backup : Yes (backup at update mandatory when updating from OL5 to OL6)Shutdown stack : No (Currently stack is down)RPM exclusion list : Function not available for OL5->OL6 upgradesRPM obsolete list : Function not available for OL5->OL6 upgradesExact dependencies : Function not available for OL5->OL6 upgradesMinimum dependencies : Function not available for OL5->OL6 upgradesLogfile : /var/log/cellos/dbnodeupdate.log (runid: 300115092333)Diagfile : /var/log/cellos/dbnodeupdate.300115092333.diagServer model : SUN FIRE X4170 M2 SERVERRemote mounts exist : Yes (dbnodeupdate.sh will try unmounting)dbnodeupdate.sh rel. : 4.11 (always check MOS 1553103.1 for the latest release of dbnodeupdate.sh)Note : After upgrading and rebooting run './dbnodeupdate.sh -c' to finish post steps.

■ DB node updateThe following known issues will be checked for but require manual follow-up: (*) - Issue - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12 (*) - Issue - Exafusion silently enabled for database 12.1.0.2.0 with kernel 2.6.39-400.200 and later. See MOS 1947476.1 for more details.

Continue ? [y/n] y (*) 2015-01-30 09:24:53: Verifying GI and DB's are shutdown (*) 2015-01-30 09:24:55: Collecting console history for diag purposes (*) 2015-01-30 09:25:09: Successfully unmounted network mount /NFS (*) 2015-01-30 09:25:09: Unmount of /boot successful (*) 2015-01-30 09:25:09: Check for /dev/sda1 successful (*) 2015-01-30 09:25:09: Mount of /boot successful (*) 2015-01-30 09:25:09: Disabling stack from starting (*) 2015-01-30 09:25:09: Performing filesystem backup to /dev/mapper/VGExaDb-LVDbSys2. Avg. 30 minutes (maximum 120) depends per environment........ (*) 2015-01-30 09:32:21: Backup successful (*) 2015-01-30 09:32:27: ExaWatcher stopped successful (*) 2015-01-30 09:32:28: Capturing service status and file attributes. This may take a while... (*) 2015-01-30 09:32:33: Service status and file attribute report in: /etc/exadata/reports (*) 2015-01-30 09:32:33: Validating the specified source location. (*) 2015-01-30 09:32:34: Cleaning up the yum cache. (*) 2015-01-30 09:32:35: Executing OL5->OL6 upgrade steps, system is expected to reboot multiple times. (*) 2015-01-30 09:34:56: Initialize of Oracle Linux 6 Upgrade successful. Rebooting now...

Broadcast message from root (pts/0) (Fri Jan 30 09:34:57 2015):

The system is going down for reboot NOW!

■ DB node updateThe following known issues will be checked for but require manual follow-up: (*) - Issue - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12 (*) - Issue - Exafusion silently enabled for database 12.1.0.2.0 with kernel 2.6.39-400.200 and later. See MOS 1947476.1 for more details.

Continue ? [y/n] y (*) 2015-01-30 09:24:53: Verifying GI and DB's are shutdown (*) 2015-01-30 09:24:55: Collecting console history for diag purposes (*) 2015-01-30 09:25:09: Successfully unmounted network mount /NFS (*) 2015-01-30 09:25:09: Unmount of /boot successful (*) 2015-01-30 09:25:09: Check for /dev/sda1 successful (*) 2015-01-30 09:25:09: Mount of /boot successful (*) 2015-01-30 09:25:09: Disabling stack from starting (*) 2015-01-30 09:25:09: Performing filesystem backup to /dev/mapper/VGExaDb-LVDbSys2. Avg. 30 minutes (maximum 120) depends per environment........ (*) 2015-01-30 09:32:21: Backup successful (*) 2015-01-30 09:32:27: ExaWatcher stopped successful (*) 2015-01-30 09:32:28: Capturing service status and file attributes. This may take a while... (*) 2015-01-30 09:32:33: Service status and file attribute report in: /etc/exadata/reports (*) 2015-01-30 09:32:33: Validating the specified source location. (*) 2015-01-30 09:32:34: Cleaning up the yum cache. (*) 2015-01-30 09:32:35: Executing OL5->OL6 upgrade steps, system is expected to reboot multiple times. (*) 2015-01-30 09:34:56: Initialize of Oracle Linux 6 Upgrade successful. Rebooting now...

Broadcast message from root (pts/0) (Fri Jan 30 09:34:57 2015):

The system is going down for reboot NOW!

■ DB node update[EXA1]root@exa1:/root/patch# ./dbnodeupdate.sh -c########################################################################################################################### ## Guidelines for using dbnodeupdate.sh (rel. 4.11): ## ## - Prerequisites for usage: ## 1. Refer to dbnodeupdate.sh options. See MOS 1553103.1 ## 2. Use the latest release of dbnodeupdate.sh. See patch 16486998 ## 3. Run the prereq check with the '-v' option. ## ## I.e.: ./dbnodeupdate.sh -u -l /u01/my-iso-repo.zip -v ## ./dbnodeupdate.sh -u -l http://my-yum-repo -v ## ## - Prerequisite dependency check failures can happen due to customization: ## - The prereq check detects dependency issues that need to be addressed prior to running a successful update. ## - Customized rpm packages may fail the built-in dependency check and system updates cannot proceed until resolved. ## ## When upgrading from releases later than 11.2.2.4.2 to releases before 11.2.3.3.0: ## - Conflicting packages should be removed before proceeding the update. ## ## When upgrading to releases 11.2.3.3.0 or later: ## - When the 'exact' package dependency check fails 'minimum' package dependency check will be tried. ## - When the 'minimum' package dependency check also fails, ## the conflicting packages should be removed before proceeding. ## ## - As part of the prereq checks and as part of the update, a number of rpms will be removed.

## This removal is required to preserve Exadata functioning. This should not be confused with obsolete packages. ## - See /var/log/cellos/packages_to_be_removed.txt for details on what packages will be removed.

## ## - In case of any problem when filing an SR, upload the following: ## - /var/log/cellos/dbnodeupdate.log ## - /var/log/cellos/dbnodeupdate.<runid>.diag ## - where <runid> is the unique number of the failing run. ## ###########################################################################################################################Continue ? [y/n] y

■ DB node update(*) 2015-01-30 10:28:17: Unzipping helpers (./dbupdate-helpers.zip) to /opt/oracle.SupportTools/dbnodeupdate_helpers (*) 2015-01-30 10:28:18: Initializing logfile /var/log/cellos/dbnodeupdate.log (*) 2015-01-30 10:28:23: Collecting system configuration settings. This may take a while... Active Image version : 12.1.2.1.0.141206.1Active Kernel version : 2.6.39-400.243.1.el6uekActive LVM Name : /dev/mapper/VGExaDb-LVDbSys1Inactive Image version : 11.2.3.3.0.131014.1Inactive LVM Name : /dev/mapper/VGExaDb-LVDbSys2Current user id : rootAction : finish-post (validate image status, fix known issues, cleanup, relink and enable crs to auto-start)Shutdown stack : No (Currently stack is down)Logfile : /var/log/cellos/dbnodeupdate.log (runid: 300115102813)Diagfile : /var/log/cellos/dbnodeupdate.300115102813.diagServer model : SUN FIRE X4170 M2 SERVERdbnodeupdate.sh rel. : 4.11 (always check MOS 1553103.1 for the latest release of dbnodeupdate.sh)

The following known issues will be checked for but require manual follow-up: (*) - Issue - Yum rolling update requires fix for 11768055 when Grid Infrastructure is below 11.2.0.2 BP12 (*) - Issue - Exafusion silently enabled for database 12.1.0.2.0 with kernel 2.6.39-400.200 and later. See MOS 1947476.1 for more details.

Continue ? [y/n] y (*) 2015-01-30 10:31:52: Verifying GI and DB's are shutdown (*) 2015-01-30 10:31:55: Verifying firmware updates/validations. Maximum wait time: 60 minutes. (*) 2015-01-30 10:31:55: If the node reboots during this firmware update/validation, re-run './dbnodeupdate.sh -c' after the node restarts.. (*) 2015-01-30 10:31:55: Collecting console history for diag purposes (*) 2015-01-30 10:32:12: No rpms to remove (*) 2015-01-30 10:32:13: Relinking all homes (*) 2015-01-30 10:32:13: Unlocking /u01/app/11.2.0.4/grid (*) 2015-01-30 10:32:19: Relinking /u01/app/11.2.0.4/grid as oracle (with rds option) (*) 2015-01-30 10:32:31: Relinking /u01/app/oracle/product/11.2.0.4/dbhome_1 as oracle (with rds option) (*) 2015-01-30 10:32:45: Locking and starting Grid Infrastructure (/u01/app/11.2.0.4/grid) (*) 2015-01-30 10:34:45: Sleeping another 60 seconds while stack is starting (1/5) (*) 2015-01-30 10:34:45: Stack started (*) 2015-01-30 10:34:45: Enabling stack to start at reboot. Disable this when the stack should not be starting on a next boot (*) 2015-01-30 10:34:46: Capturing service status and file attributes. This may take a while... (*) 2015-01-30 10:34:52: Service status and file attribute report in: /etc/exadata/reports (*) 2015-01-30 10:34:52: All post steps are finished.

■ DB node update (11.2.3.2.1 - > 11.2.3.3.0)[root@exa1 ~]# imageinfo

Kernel version: 2.6.39-400.126.1.el5uek #1 SMP Fri Sep 20 10:54:38 PDT 2013 x86_64Image version: 11.2.3.3.0.131014.1Image activated: 2014-04-15 13:31:00 +0900Image status: successSystem partition on device: /dev/mapper/VGExaDb-LVDbSys1

[root@exa1 ~]# imagehistoryVersion : 11.2.3.2.1.130109Image activation date : 2014-04-11 10:04:05 +0900Imaging mode : freshImaging status : success

Version : 11.2.3.3.0.131014.1Image activation date : 2014-04-15 13:31:00 +0900Imaging mode : patchImaging status : success

■ DB node update (11.2.3.3.1 - > 12.1.2.1.0)[EXA1]root@exa1:/root# imageinfo

Kernel version: 2.6.39-400.243.1.el6uek.x86_64 #1 SMP Wed Nov 26 09:15:35 PST 2014 x86_64Image version: 12.1.2.1.0.141206.1Image activated: 2015-01-30 10:24:43 +0900Image status: successSystem partition on device: /dev/mapper/VGExaDb-LVDbSys1

[EXA1]root@exa1:/root# imagehistoryVersion : 11.2.3.3.0.131014.1Image activation date : 2015-01-06 13:11:03 +0900Imaging mode : freshImaging status : success

Version : 12.1.2.1.0.141206.1Image activation date : 2015-01-30 10:24:43 +0900Imaging mode : patchImaging status : success

■ DB node update (Rollback)[root@exa1 WorkDir]# ./dbnodeupdate.sh -r######################################################################################################## Guidelines for using dbnodeupdate.sh (rel. 3.21): ## - Prerequisites for usage: ## 1. Refer to dbnodeupdate.sh options. See MOS 1553103.1 ## 2. Use the latest release of dbnodeupdate.sh. See patch 16486998 ## 3. Run the prereq check with the '-v' option. ## ## I.e.: ./dbnodeupdate.sh -u -l /u01/my-iso-repo.zip -v ## ./dbnodeupdate.sh -u -l http://my-yum-repo -v ## ## - Failures due to dependency issues occur because of customization: ## - The prereq checks detect these dependency issues that need to be addressed prior to running a successful update. ## - Customized rpm packages may fail the built-in dependency check and system updates cannot proceed until resolved. ## ## When upgrading from releases later than 11.2.2.4.2 to releases before 11.2.3.3.0:

## - Conflicting packages should be removed before proceeding the update. ## ## When upgrading to releases 11.2.3.3.0 or later: ## - When the 'exact' package dependency check fails 'minimum' package dependency check will be tried. ## - When the 'minimum' package dependency check also fails, ## the conflicting packages should be removed before proceeding. ## ## - As part of the prereq checks and as part of the update, a number of rpms will be removed. ## This removal is required to preserve Exadata functioning. This should not be confused with obsolete packages. ## - See /var/log/cellos/packages_to_be_removed.txt for details on what packages will be removed. ## ## - In case of any problem when filing an SR, upload the following: ## - /var/log/cellos/dbnodeupdate.log ## - /var/log/cellos/dbnodeupdate.<runid>.diag ## - where <runid> is the is of the failing run. # #######################################################################################################Continue ? [Y/n] Y

■ DB node update (Rollback)(*) 2014-04-15 14:16:58: Unzipping helpers (/opt/oracle.SupportTools/onecommand/linux-x64/WorkDir/dbupdate-helpers.zip) to /opt/oracle.SupportTools/dbnodeupdate_helpers (*) 2014-04-15 14:16:58: Initializing logfile /var/log/cellos/dbnodeupdate.log (*) 2014-04-15 14:16:58: Collecting system configuration details. This may take a while...

Warning: File /etc/oratab not in place. dbnodeupdate.sh can continue without /etc/oratab but will then assume no Oracle homes are installed.

Continue ? [Y/n] y Active Image version : 11.2.3.3.0.131014.1Active Kernel version : 2.6.39-400.126.1.el5uekActive LVM Name : /dev/mapper/VGExaDb-LVDbSys1Inactive Image version : n/aInactive LVM Name : /dev/mapper/VGExaDb-LVDbSys2Current user id : rootAction : rollback Rolling back to : n/aCreate a backup : No (backups cannot be made for rollbacks)Shutdown stack : No (Currently stack is down - /etc/oratab not found)Logfile : /var/log/cellos/dbnodeupdate.log (runid: 150414141636)Diagfile : /var/log/cellos/dbnodeupdate.150414141636.diagServer model : SUN FIRE X4170 M2 SERVERdbnodeupdate.sh rel. : 3.21 (always check MOS 1553103.1 for the latest release of dbnodeupdate)

Continue ? [Y/n] y

(*) 2014-04-15 14:18:26: ExaWatcher stopped successful (*) 2014-04-15 14:18:26: Creating backup of /boot (*) 2014-04-15 14:18:31: Switching root partition (*) 2014-04-15 14:18:32: Restoring /boot

ERROR: Unable to find boot_backup.tbz file, exiting ( 백업본이 없으므로 roll-back 실패 )

Infrastructure & Governance