合 在RHEL 7.6上安装12.2.0.1 rac时碰到的几个坑
👉 本文共约1340个字，系统预计阅读时间或需6分钟。
目录
这几天在客户这安装Oracle rac，环境为华为云的ECS，RHEL 7.6上安装12.2.0.1的rac，结果碰到了之前都没见过的几个坑，分享给大家。
错误1、CLSRSC-400: A system reboot is required to continue installing
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | [root@dbr710a ~]# /u01/app/12.2.0/grid/root.sh Check /u01/app/12.2.0/grid/install/root_dbr710a_2021-11-09_17-10-00-226097250.log for the output of root script You have new mail in /var/spool/mail/root [root@dbr710a ~]# tailf /u01/app/12.2.0/grid/install/root_dbr710a_2021-11-09_17-10-00-226097250.log /u01/app/grid/crsdata/dbr710a/crsconfig/rootcrs_dbr710a_2021-11-09_05-10-00PM.log 2021/11/09 17:10:03 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'. 2021/11/09 17:10:03 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2021/11/09 17:10:03 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2021/11/09 17:10:03 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'. 2021/11/09 17:10:04 CLSRSC-363: User ignored prerequisites during installation 2021/11/09 17:10:04 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'. 2021/11/09 17:10:05 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'. 2021/11/09 17:10:06 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'. 2021/11/09 17:10:07 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'. 2021/11/09 17:10:24 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'. 2021/11/09 17:10:24 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'. 2021/11/09 17:10:26 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'. 2021/11/09 17:10:27 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'. 2021/11/09 17:11:13 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'. 2021/11/09 17:11:13 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'. 2021/11/09 17:11:28 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' 2021/11/09 17:11:49 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'. 2021/11/09 17:11:52 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'dbr710a' CRS-2673: Attempting to stop 'ora.evmd' on 'dbr710a' CRS-2673: Attempting to stop 'ora.mdnsd' on 'dbr710a' CRS-2673: Attempting to stop 'ora.gpnpd' on 'dbr710a' CRS-2673: Attempting to stop 'ora.gipcd' on 'dbr710a' CRS-2677: Stop of 'ora.gipcd' on 'dbr710a' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'dbr710a' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'dbr710a' succeeded CRS-2677: Stop of 'ora.evmd' on 'dbr710a' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dbr710a' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. 2021/11/09 17:12:55 CLSRSC-400: A system reboot is required to continue installing. The command '/u01/app/12.2.0/grid/perl/bin/perl -I/u01/app/12.2.0/grid/perl/lib -I/u01/app/12.2.0/grid/crs/install /u01/app/12.2.0/grid/crs/install/rootcrs.pl ' execution failed |
详细日志：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | 2021-11-09 17:12:09: Oracle High Availability Services is online 2021-11-09 17:12:09: Executing cmd: /u01/app/12.2.0/grid/bin/acfsdriverstate supported 2021-11-09 17:12:25: Command output: > ACFS-9200: Supported >End Command output 2021-11-09 17:12:25: acfs is supported 2021-11-09 17:12:25: Stopping ora.drivers.acfs if it exists, so that it doesn't race. 2021-11-09 17:12:25: isACFSSupported: 1 2021-11-09 17:12:25: Executing cmd: /u01/app/12.2.0/grid/bin/crsctl stat res ora.drivers.acfs -init 2021-11-09 17:12:25: Command output: > CRS-2613: Could not find resource 'ora.drivers.acfs'. >End Command output 2021-11-09 17:12:25: Executing '/u01/app/12.2.0/grid/bin/acfsroot install -t2' 2021-11-09 17:12:25: Executing cmd: /u01/app/12.2.0/grid/bin/acfsroot install -t2 2021-11-09 17:12:55: Command output: > ACFS-9176: Entering 'get ora home' > ACFS-9500: Location of Oracle Home is '/u01/app/12.2.0/grid' as determined from the internal configuration data > ACFS-9182: Variable 'ORACLE_HOME' has value '/u01/app/12.2.0/grid' > ACFS-9177: Return from 'get ora home' > ACFS-9176: Entering 'ga admin name' > ACFS-9176: Entering 'va admin group' > ACFS-9178: Return code = 0 > ACFS-9177: Return from 'va admin group' > ACFS-9178: Return code = asmadmin > ACFS-9177: Return from 'ga admin name' > ACFS-9180: Sub-command is 'install' > ACFS-9505: Using acfsutil executable from location: '/u01/app/12.2.0/grid/usm/install/cmds/bin/acfsutil' > ACFS-9176: Entering 'install' > ACFS-9300: ADVM/ACFS distribution files found. > ACFS-9176: Entering 'uld usm drvs' > ACFS-9178: Return code = USM_SUCCESS > ACFS-9177: Return from 'uld usm drvs' > ACFS-9176: Entering 'fail mode' > ACFS-9178: Return code = 0 > ACFS-9177: Return from 'fail mode' > ACFS-9176: Entering 'uninstall' > ACFS-9176: Entering 'uld usm drvs' > ACFS-9178: Return code = USM_SUCCESS > ACFS-9177: Return from 'uld usm drvs' > ACFS-9314: Removing previous ADVM/ACFS installation. > ACFS-9315: Previous ADVM/ACFS components successfully removed. > ACFS-9178: Return code = USM_SUCCESS > ACFS-9177: Return from 'uninstall' > ACFS-9307: Installing requested ADVM/ACFS software. > ACFS-9503: ADVM and ACFS driver media location is '/u01/app/12.2.0/grid/usm/install/Oracle/EL7/x86_64/3.10.0-229/3.10.0-229-x86_64/bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/Oracle/EL7/x86_64/3.10.0-229/3.10.0-229-x86_64/bin/oracleadvm.ko' to the path '/lib/modules/3.10.0-229.1.2.el7.x86_64/extra/usm/oracleadvm.ko' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/Oracle/EL7/x86_64/3.10.0-229/3.10.0-229-x86_64/bin/oracleoks.ko' to the path '/lib/modules/3.10.0-229.1.2.el7.x86_64/extra/usm/oracleoks.ko' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/Oracle/EL7/x86_64/3.10.0-229/3.10.0-229-x86_64/bin/oracleacfs.ko' to the path '/lib/modules/3.10.0-229.1.2.el7.x86_64/extra/usm/oracleacfs.ko' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/advmutil' to the path '/sbin/advmutil' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/advmutil.bin' to the path '/sbin/advmutil.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/fsck.acfs' to the path '/sbin/fsck.acfs' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/fsck.acfs.bin' to the path '/sbin/fsck.acfs.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/mkfs.acfs' to the path '/sbin/mkfs.acfs' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/mkfs.acfs.bin' to the path '/sbin/mkfs.acfs.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/mount.acfs' to the path '/sbin/mount.acfs' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/mount.acfs.bin' to the path '/sbin/mount.acfs.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/acfsdbg' to the path '/sbin/acfsdbg' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/acfsdbg.bin' to the path '/sbin/acfsdbg.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/acfsutil' to the path '/sbin/acfsutil' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/acfsutil.bin' to the path '/sbin/acfsutil.bin' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/umount.acfs' to the path '/sbin/umount.acfs' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/usm/install/cmds/bin/umount.acfs.bin' to the path '/sbin/umount.acfs.bin' > ACFS-9176: Entering 'ga admin name' > ACFS-9178: Return code = CACHED_ASMADMIN > ACFS-9177: Return from 'ga admin name' > ACFS-9176: Entering 'getparm' > ACFS-9178: Return code = grid > ACFS-9177: Return from 'getparm' > ACFS-9504: Copying file '/u01/app/12.2.0/grid/lib/libacfs12.so' to the path '/opt/oracle/extapi/64/acfs/orcl/1/' > ACFS-9308: Loading installed ADVM/ACFS drivers. > ACFS-9321: Creating udev for ADVM/ACFS. > ACFS-9323: Creating module dependencies - this may take some time. > ACFS-9176: Entering 'ld usm drvs' > ACFS-9154: Loading 'oracleoks.ko' driver. > modprobe: FATAL: Module oracleoks not found. > ACFS-9109: oracleoks.ko driver failed to load. > ACFS-9178: Return code = USM_FAIL > ACFS-9177: Return from 'ld usm drvs' > ACFS-9428: Failed to load ADVM/ACFS drivers. A system reboot is recommended. > ACFS-9310: ADVM/ACFS installation failed. > ACFS-9178: Return code = USM_REBOOT_RECOMMENDED > ACFS-9177: Return from 'install' > ACFS-9176: Entering 'acroot ex' > ACFS-9178: Return code = 3 > ACFS-9177: Return from 'acroot ex' >End Command output 2021-11-09 17:12:55: /u01/app/12.2.0/grid/bin/crsctl disable crs ... disabling CRS in preparation for reboot. 2021-11-09 17:12:55: Executing cmd: /u01/app/12.2.0/grid/bin/crsctl disable crs 2021-11-09 17:12:55: Command output: > CRS-4621: Oracle High Availability Services autostart is disabled. >End Command output 2021-11-09 17:12:55: /u01/app/12.2.0/grid/bin/crsctl disable crs ... CRS disabled, ready for reboot. 2021-11-09 17:12:55: USM driver install status is 3 2021-11-09 17:12:55: ACFS drivers unable to be installed. 2021-11-09 17:12:55: Executing cmd: /u01/app/12.2.0/grid/bin/clsecho -p has -f clsrsc -m 400 2021-11-09 17:12:55: Command output: > CLSRSC-400: A system reboot is required to continue installing. >End Command output 2021-11-09 17:12:55: CLSRSC-400: A system reboot is required to continue installing. 2021-11-09 17:12:55: Invoking "/u01/app/12.2.0/grid/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_ACFSINST -state FAIL" 2021-11-09 17:12:55: trace file=/u01/app/grid/crsdata/dbr710a/crsconfig/cluutil3.log 2021-11-09 17:12:55: Running as user grid: /u01/app/12.2.0/grid/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_ACFSINST -state FAIL 2021-11-09 17:12:55: s_run_as_user2: Running /bin/su grid -c ' echo CLSRSC_START; /u01/app/12.2.0/grid/bin/cluutil -ckpt -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_ACFSINST -state FAIL ' 2021-11-09 17:12:56: Removing file /tmp/nwyE03yAD7 2021-11-09 17:12:56: Successfully removed file: /tmp/nwyE03yAD7 2021-11-09 17:12:56: pipe exit code: 0 2021-11-09 17:12:56: /bin/su successfully executed 2021-11-09 17:12:56: Succeeded in writing the checkpoint:'ROOTCRS_ACFSINST' with status:FAIL 2021-11-09 17:12:56: Install node: dbr710a 2021-11-09 17:12:56: ckpt: -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE 2021-11-09 17:12:56: Invoking "/u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE" 2021-11-09 17:12:56: trace file=/u01/app/grid/crsdata/dbr710a/crsconfig/cluutil4.log 2021-11-09 17:12:56: Running as user grid: /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE 2021-11-09 17:12:56: s_run_as_user2: Running /bin/su grid -c ' echo CLSRSC_START; /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE ' 2021-11-09 17:12:56: Removing file /tmp/YZjKbGtTXy 2021-11-09 17:12:56: Successfully removed file: /tmp/YZjKbGtTXy 2021-11-09 17:12:56: pipe exit code: 0 2021-11-09 17:12:56: /bin/su successfully executed 2021-11-09 17:12:56: Invoking "/u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE -status" 2021-11-09 17:12:56: trace file=/u01/app/grid/crsdata/dbr710a/crsconfig/cluutil5.log 2021-11-09 17:12:56: Running as user grid: /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE -status 2021-11-09 17:12:56: s_run_as_user2: Running /bin/su grid -c ' echo CLSRSC_START; /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -chkckpt -name ROOTCRS_FIRSTNODE -status ' 2021-11-09 17:12:56: Removing file /tmp/8vuYgOz6Rs 2021-11-09 17:12:56: Successfully removed file: /tmp/8vuYgOz6Rs 2021-11-09 17:12:56: pipe exit code: 0 2021-11-09 17:12:56: /bin/su successfully executed 2021-11-09 17:12:56: The 'ROOTCRS_FIRSTNODE' status is START 2021-11-09 17:12:56: Global ckpt 'ROOTCRS_FIRSTNODE' state: START 2021-11-09 17:12:56: First node operations have not been done, and local node is installer node. 2021-11-09 17:12:56: Local node: dbr710a is the first node. 2021-11-09 17:12:56: ORACLE_BASE is shared: 0 2021-11-09 17:12:56: Invoking "/u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_FIRSTNODE -state FAIL -nodelist dbr710a,dbr710b -transferfile" 2021-11-09 17:12:56: trace file=/u01/app/grid/crsdata/dbr710a/crsconfig/cluutil6.log 2021-11-09 17:12:56: Running as user grid: /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_FIRSTNODE -state FAIL -nodelist dbr710a,dbr710b -transferfile 2021-11-09 17:12:56: s_run_as_user2: Running /bin/su grid -c ' echo CLSRSC_START; /u01/app/12.2.0/grid/bin/cluutil -ckpt -global -oraclebase /u01/app/grid -writeckpt -name ROOTCRS_FIRSTNODE -state FAIL -nodelist dbr710a,dbr710b -transferfile ' 2021-11-09 17:12:57: Removing file /tmp/AgeOkpqZE8 2021-11-09 17:12:57: Successfully removed file: /tmp/AgeOkpqZE8 2021-11-09 17:12:57: pipe exit code: 0 2021-11-09 17:12:57: /bin/su successfully executed 2021-11-09 17:12:57: succeeded to write global ckpt 'ROOTCRS_FIRSTNODE' with status 'FAIL' |
解决办法：打最新的RU:33290750 ，可以参考：https://www.dbaup.com/oracle-database-or-gi-update-revision-psu-spucpu-bundle-patches-patchsets-and-base-releases.html 下载最新的RU补丁。
从12.2开始，在安装grid之前就可以先打RU补丁。
最新RU（11G叫做PSU）,如果18C，可以完美避过此BUG。
升级OPATCH并打补丁
You must use the OPatch utility version 12.2.0.1.27 or later to apply this patch. Oracle recommends that you use the latest released OPatch version for 12.2, which is available for download from My Oracle Support patch 6880880 by selecting ARU link for the 12.2.0.1.0 release. It is recommended that you download the OPatch utility and the patch in a shared location to be able to access them from any node in the cluster for the patch application on each node.
1 2 3 4 5 6 | su - grid unzip -d $ORACLE_HOME /soft/Linux-x86-12201/p6880880_210000_Linux-x86-64.zip unzip p33290750_122010_Linux-x86-64.zip /u01/app/12.2.0/grid/gridSetup.sh -applyPSU /soft/Linux-x86-12201/33290750/ |
执行过程：
1 2 3 4 5 6 7 8 9 | [grid@dbr710a Linux-x86-12201]$ /u01/app/12.2.0/grid/gridSetup.sh -applyPSU /soft/Linux-x86-12201/33290750/ ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable. Preparing the home to patch... Applying the patch /soft/Linux-x86-12201/33290750/... Successfully applied the patch. The log can be found at: /tmp/GridSetupActions2021-11-09_05-41-50PM/installerPatchActions_2021-11-09_05-41-50PM.log No X11 DISPLAY variable was set, but this program performed an operation which requires it. [grid@dbr710a Linux-x86-12201]$ |
DB打最新的补丁
建议DB也打最新的PSU，在安装完集群和DB软件之后进行操作，如下：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | -- 节点1 srvctl stop db -d jde91 su - oracle unzip -d $ORACLE_HOME /soft/Linux-x86-12201/p6880880_210000_Linux-x86-64.zip unzip p33261817_122010_Linux-x86-64.zip cd 33261817 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ $ORACLE_HOME/OPatch/opatch apply chown oracle.oinstall /u01/app/oracle/product/12.2.0/dbhome_1/bin/extjobO $ORACLE_HOME/OPatch/opatch lspatches scp p33261817_122010_Linux-x86-64.zip oracle@dbr710b:/soft/ scp p6880880_210000_Linux-x86-64.zip oracle@dbr710b:/soft/ -- 节点2 su - oracle unzip -d $ORACLE_HOME /soft/Linux-x86-12201/p6880880_210000_Linux-x86-64.zip unzip p33261817_122010_Linux-x86-64.zip cd 33261817 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ $ORACLE_HOME/OPatch/opatch apply chown oracle.oinstall /u01/app/oracle/product/12.2.0/dbhome_1/bin/extjobO $ORACLE_HOME/OPatch/opatch lspatches -- 更新数据库(只在一个节点) srvctl start db -d jde91 $ORACLE_HOME/OPatch/datapatch -verbose select PATCH_ID from dba_registry_sqlpatch; cd $ORACLE_HOME/rdbms/admin @utlrp.sql |
注意：在grid的RU安装包中，其实也包含了33261817的DB补丁包，可以直接使用的。
日志：
本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！
