环境如下:
Linux操作系统:Centos 6.5 64bit (这个版本的redhat 6内核等OS在安装grid最后执行root.sh时会出现crs-4124,是oracle11.2.0.1的bug)
VMware version:Workstation 8.0.3 build-703057
Oracle版本:11G R2 grid/oracle 64位 11.2.0.1.0 for linux
步骤规划:
1,配置虚拟机、安装Linux操作系统
2,配置OS
3,配置ASM
4,安装Oracle grid
5,安装Oracle database
6,创建ASM磁盘组
7,建立数据库实例
1,配置虚拟机、安装Linux操作系统
1.1 新建虚拟机:RAC1和RAC2,并安装OS
下载Centos的iso后新建虚拟机后安装即可,这里略。注意内存分配,每个虚拟机至少1.5G内存比较好,我的机器8G内存,给每个虚拟机分了2G,给每个虚拟机分配50GB硬盘容量,配置2个桥接网卡。
1.2 配置Share Disks
新建共享硬盘,在DOS VM安装目录下执行:
cd D F:/Software/vm8/
vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskocr.vmdk vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskocr2.vmdk vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskvotingdisk.vmdk vmware-vdiskmanager.exe -c -s 20000Mb -a lsilogic -t 2 K:VMDiskSharediskdata.vmdk vmware-vdiskmanager.exe -c -s 10000Mb -a lsilogic -t 2 K:VMDiskSharediskbackup.vmdk
配置虚拟机RAC1 RAC2目录下的vmx虚拟机配置文件,在行最后添加如下:
scsi1.present = "TRUE" scsi1.virtualDev = "lsilogic" scsi1.sharedBus = "virtual"
scsi1:1.present = "TRUE" scsi1:1.mode = "independent-persistent" scsi1:1.filename = "K:VMDiskSharediskocr.vmdk" scsi1:1.deviceType = "plainDisk" scsi1:2.present = "TRUE" scsi1:2.mode = "independent-persistent" scsi1:2.filename = "K:VMDiskSharediskvotingdisk.vmdk" scsi1:2.deviceType = "plainDisk" scsi1:3.present = "TRUE" scsi1:3.mode = "independent-persistent" scsi1:3.filename = "K:VMDiskSharediskdata.vmdk" scsi1:3.deviceType = "plainDisk" scsi1:4.present = "TRUE" scsi1:4.mode = "independent-persistent" scsi1:4.filename = "K:VMDiskSharediskbackup.vmdk" scsi1:4.deviceType = "plainDisk" scsi1:5.present = "TRUE" scsi1:5.mode = "independent-persistent" scsi1:5.filename = "K:VMDiskSharediskocr2.vmdk" scsi1:5.deviceType = "plainDisk" disk.locking = "false" diskLib.dataCacheMaxSize = "0" diskLib.dataCacheMaxReadAheadSize = "0" diskLib.DataCacheMinReadAheadSize = "0" diskLib.dataCachePageSize = "4096" diskLib.maxUnsyncedWrites = "0"
保存退出。
最后虚拟机图大致如下:
再次提醒注意内存、硬盘和网卡配置。
配置虚拟机、安装Linux操作系统完成。
2,配置OS
以下没有特别说明都需要同时在两个节点执行。
2.1 关闭防火墙、selinux(考虑到后面安装时会碰到一些情况,所以需要执行这步骤) 关闭selinux
setenforce 0
[root@rac1 ~]# vim /etc/selinux/config
SELINUX=disabled
关闭防火墙
service iptables stop
chkconfig iptables off
2.2 配置网络
修改节点主机名为rac1,rac2
hostname rac1
hostname rac2
或修改/etc/sysconfig/network
RAC1
eth0: 192.168.1.130
eth1: 10.10.17.221
RAC2
eth0: 192.168.1.140
eth1: 10.10.17.222 host文件,两个节点都配置
Vim /etc/hosts
192.168.1.130 rac1 192.168.1.140 rac2 192.168.1.131 rac1-vip 192.168.1.141 rac2-vip 10.10.17.221 rac1-priv 10.10.17.222 rac2-priv 192.168.1.200 scan-ip
保存退出。
配置完之后互相ping看看是否连通。
2.3 配置CTSS服务、节点SSH等效
CTSS
需要把ntp服务停止和删除配置文件及pid文件
service ntpd stop
chkconfig ntpd off
rm -rf /etc/ntpd.conf
rm -rf /var/run/ntpd.pid
查看ctss服务资源
crsctl check ctss
SSH
在grid安装时会提供配置步骤,所以不需要手工配置。
ssh具体配置
rac1 rac2 mkdir ~/.ssh chmod 700 ~/.ssh ssh-keygen -t rsa ssh-keygen -t dsa rac1 rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
测试SSH
ssh rac2
ssh rac1
2.4 配置Oracle软件安装用户、组、环境变量、安装目录
#组 groupadd -g 1300 dba groupadd -g 1301 oper groupadd -g 1000 oinstall groupadd -g 1200 asmadmin groupadd -g 1201 asmdba groupadd -g 1202 asmoper
#用户 useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -s /bin/bash grid
useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -s /bin/bash oracle #用户环境变量修改home下的 .bash_profile
ORACLE
export TMP=/tmp export TMPDIR=$TMP export ORACLE_SID=RAC1 # RAC1 export ORACLE_SID=RAC2 # RAC2 export ORACLE_BASE=/u01/app/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1 export TNS_ADMIN=$ORACLE_HOME/network/admin export PATH=/usr/sbin:$PATH export PATH=$ORACLE_HOME/bin:$PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib umask 022 GRID export TMP=/tmp export TMPDIR=$TMP export ORACLE_SID=+ASM1 # RAC1
export ORACLE_SID=+ASM2 # RAC2
export ORACLE_BASE=/u01/app/grid export ORACLE_HOME=/u01/app/11.2.0/grid export PATH=/usr/sbin:$PATH export PATH=$ORACLE_HOME/bin:$PATH export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib umask 022
配置目录 mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/grid mkdir -p /u01/app/oracle chown grid:oinstall /u01/app/11.2.0/grid chown grid:oinstall /u01/app/grid chown oracle:oinstall /u01/app/oracle chmod -R 775 /u01/ chown -R grid:oinstall /u01
2.5 配置系统文件
配置oracle、grid用户的shell限制
vim /etc/security/limits.conf
添加
grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 修改/etc/pam.d/login配置文件
Vim /etc/pam.d/login 添加 session required pam_limits.so
修改内核配置文件/etc/sysctl.conf
fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmall = 2097152 kernel.shmmax = 1054472192 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048586 net.ipv4.tcp_wmem = 262144 262144 262144 net.ipv4.tcp_rmem = 4194304 4194304 4194304 保存退出后执行 sysctl -p 立即生效。
2.6 配置共享磁盘 (在一个节点执行即可)
查看分区
ls /dev/sd*
使用fdisk进行分区,其他区按照自己的规划划分就可以了,例:
fdisk /dev/sdb
[root@rac1 ~]# fdisk /dev/sdb Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1000,default 1): Using default value 1 Last cylinder,+cylinders or +size{K,M,G} (1-1000,default 1000): +500M Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (502-1000,default 502): Using default value 502 Last cylinder,G} (502-1000,default 1000): Using default value 1000
Command (m for help): w The partition table has been altered!
最后配置完后查看分区
fidks -l
[root@rac1 ~]# ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1 /dev/sdc /dev/sdc1 /dev/sdd /dev/sdd1 /dev/sdd2 /dev/sde /dev/sde1 /dev/sde2 /dev/sdf /dev/sdf1
2.7 安装Oracle grid等需要的包 binutils-2.20.51.0.2-5.11.el6 (x86_64)
compat-libcap1-1.10-1 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6 (x86_64)
gcc-c++-4.4.4-13.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
glibc-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6 (i686)
libgcc-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6 (x86_64)
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6 (x86_64)
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6
sysstat-9.0.4-11.el6 (x86_64)
3,配置ASM
3.1 查看操作系统内核版本对应下载
uname -a
下载oracleasm http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html
安装
yum -y install kmod-oracleasm.x86_64
rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpm rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm
3.2 配置ASM磁盘
在两个节点上执行 [root@rac1 oracle]# /etc/init.d/oracleasm configure Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: grid Default group to own the driver interface []: asmdba Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done Initializing the Oracle ASMLib driver: [ OK ] Scanning the system for Oracle ASMLib disks: [ OK ] [root@rac1 oracle]# /etc/init.d/oracleasm scandisks Scanning the system for Oracle ASMLib disks: [ OK ]
[root@rac1 ~]# /etc/init.d/oracleasm listdisks VOL1 VOL2 VOL3 VOL4 VOL5 VOL6 VOL7
注:如果没有查看到asm磁盘组需要查看之前的selinux是否有禁用。
4,安装Oracle grid
安装前可以执行Oracle提供的检查环境工具
./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose
4.1 在安装目录下执行./runinstaller,如果出现安装grid提示must be configured to display at least 256 colors Failed,就执行如下 root 下先执行#xhost + su - grid export DISPLAY=:0.0
注意在此建立SSH等效。执行setup后点test测试。如果不成功就手工配置SSH再进行测试。
这个地方忘记截图了,就是类似这个图。定义磁盘组名字,这里我选择normal冗余,需要选择3个VOL。
这里是把OCR和votingdisk安装在所选磁盘。后面还需要新建asm磁盘组来安装其他数据文件。
这里的software location应该是 /u01/app/11.2.0/grid
通过Oracle的系统环境检查,安装缺失的包和修改系统环境参数等。
安装65%的时候可能会出现卡顿的情况,这个时候需要检查防火墙和selinux是否关闭,详细情况可以点击detail查看安装进度,如出现拷贝文件到RAC2节点的信息则正常。
根据提示执行orainstRoot.sh和root.sh脚本,先执行rac1后再执行rac2。
在执行root .sh时可能在的时候会提示出现crs-4124,这个是oracle的bug,当在redhat 6等OS下安装11.2.0.1版本Oracle的时候会出现此错误。
解决如下:
在执行root.sh时,在出现Adding daemon to inittab时,需要不停地执行命令:
/bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1
[root@rac1 grid]# ./root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2015-01-06 23:46:35: Parsing the host name
2015-01-06 23:46:35: Checking for super user privileges
2015-01-06 23:46:35: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root',privgrp 'root'..
Operation successful.
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started. ohasd is starting ADVM/ACFS is not supported on centos-release-6-6.el6.centos.12.2.x86_64 CRS-2672: Attempting to start 'ora.gipcd' on 'rac1' CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1' CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1' CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac1' CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'rac1' CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded ASM created and started successfully. DiskGroup DATA created successfully. clscfg: -install mode specified Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root',privgrp 'root'.. Operation successful. CRS-2672: Attempting to start 'ora.crsd' on 'rac1' CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded CRS-4256: Updating the profile Successful addition of voting disk 60b5c5168a574f3fbfabf7d93a17c18f. Successful addition of voting disk a59fab87e8634f00bf5b44656d6caaf1. Successful addition of voting disk 228f0986e7c84f20bf66ed6905b69e43. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 60b5c5168a574f3fbfabf7d93a17c18f (ORCL:VOL1) [DATA] 2. ONLINE a59fab87e8634f00bf5b44656d6caaf1 (ORCL:VOL2) [DATA] 3. ONLINE 228f0986e7c84f20bf66ed6905b69e43 (ORCL:VOL4) [DATA] Located 3 voting disk(s). CRS-2673: Attempting to stop 'ora.crsd' on 'rac1' CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac1' CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1' CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1' CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac1' CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1' CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1' CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1' CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1' CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac1' CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1' CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac1' CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.ctssd' on 'rac1' CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac1' CRS-2676: Start of 'ora.asm' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac1' CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.evmd' on 'rac1' CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac1' CRS-2676: Start of 'ora.asm' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1' CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded rac1 2015/01/06 23:52:15 /u01/app/11.2.0/grid/cdata/rac1/backup_20150106_235215.olr Preparing packages for installation... cvuqdisk-1.0.7-1 Configure Oracle Grid Infrastructure for a Cluster ... succeeded Updating inventory properties for clusterware Starting Oracle Universal Installer... Checking swap space: must be greater than 500 MB. Actual 3906 MB Passed The inventory pointer is located at /etc/oraInst.loc The inventory is located at /u01/app/oraInventory 'UpdateNodeList' was successful.
root.sh执行完毕。在rac2节点执行也需要此步骤。
检查grid组件启动情况 [grid@rac1 ~]$ crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [grid@rac1 ~]$ crs_stat -v -t Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1 ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1 ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1 ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE rac1 ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1 ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1 ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1 ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1 ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1 ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1 ora....SM2.asm application 0/5 0/0 ONLINE ONLINE rac2 ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE rac2 ora.rac2.gsd application 0/5 0/0 OFFLINE OFFLINE ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2 ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac2 ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
备注:
在重启节点后出现Oracle has不能通信的情况
1,[root@rac1 ~]# crsctl check crs CRS-4639: Could not contact Oracle High Availability Services
需要在启动时执行 /bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1。
然后再执行 crsctl check crs查看服务启动情况。
2,如果出现以下情况,需要查看日志提示,有一次是因为内核参数配置值的问题导致了这个错误。
[root@rac1 bin]# ./crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online
5,安装Oracle database软件
需要用oracle用户安装
su - oracle
./runInstaller
仅安装数据库软件即可,这里就根据OUI的提示安装就可以了,最后按照提示执行相关脚本。
找时间补上截图
6,创建ASM磁盘组
需要新建其他磁盘组来安装数据文件等
su - grid
asmca
根据界面提示创建磁盘组。
找时间补上截图
7,建立数据库实例 su - oracle
./runInstaller
根据OUI步骤提示选择rac选项新建数据库实例即可。 (编辑:李大同)
【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容!
|