Setting up an Oracle 11G R2 Real Application Cluster (RAC) using Single Client Access Name (scan IP)
<<Setup of Oracle Linux Server Setup of Node2>>
Setup of Node1
Once the setup is finished and the system has been restarted, open the system as a root user and prepare for the RAC setup step by step.
Step 1
In VMWare workstation, right-click on the RAC1 virtual machine and select install vmware tools.
The VM tool CD will get automatically mounted. Open the CD’s directory, within the /media directory.
|
1 |
sh vmware-config-tools.pl |
VMWare gets loaded after hitting enter on all the questions.
Step 2
Type vmware-toolbox in the terminal, press enter and tick the ntp checkbox in the window.
Copy the grid and database setup in the /setup folder.
|
1 2 3 |
chown -R oracle:oinstall /setup chmod -R +x /setup |
Step 3
Loading the necessary RPMs
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
rpm -Uvh binutils-2.* rpm -Uvh compat-libstdc++-33* rpm -Uvh elfutils-libelf-0.* rpm -Uvh elfutils-libelf-devel-* rpm -Uvh gcc-4.* rpm -Uvh gcc-c++-4.* rpm -Uvh glibc-2.* rpm -Uvh glibc-common-2.* rpm -Uvh glibc-devel-2.* rpm -Uvh glibc-headers-2.* rpm -Uvh ksh-2* rpm -Uvh libaio-0.* rpm -Uvh libaio-devel-0.* rpm -Uvh libgcc-4.* rpm -Uvh libstdc++-4.* rpm -Uvh libstdc++-devel-4.* rpm -Uvh make-3.* rpm -Uvh sysstat-7.* rpm -Uvh unixODBC-2.* rpm -Uvh unixODBC-devel-2.* |
Loading the RPMs needed for the ASM these are available within the first 2 file CDs. The 3rd CD can be downloaded from this link: http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el5.x86_64.rpm
|
1 2 3 4 5 |
rpm -Uvh oracleasm-2.6.18-238.el5-2.0.5-1.el5.x86_64.rpm rpm -Uvh oracleasm-support-2.1.4-1.el5.x86_64.rpm rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm |
In the /grid/rpm directory
|
1 |
rpm -Uvh cvuqdisk-1.0.7-1.rpm |
Step 4
Procedures for users
|
1 2 3 4 5 6 7 |
groupadd -g 1000 oinstall groupadd -g 1200 dba useradd -u 1100 -g oinstall -G dba oracle passwd oracle |
|
1 2 3 4 5 6 7 8 9 |
mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/oracle/product/11.2.0/db_1 chown -R oracle:oinstall /u01 chmod -R 775 /u01/ |
Step 5
Network Settings
The contents of /etc/hosts
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
127.0.0.1 localhost.localdomain localhost # Public 192.168.137.20 rac1.sysdba.org rac1 192.168.137.21 rac2.sysdba.org rac2 # Virtual 192.168.137.40 rac1-vip.sysdba.org rac1-vip 192.168.137.41 rac2-vip.sysdba.org rac2-vip # Private 10.0.0.20 rac1-priv.sysdba.org rac1-priv 10.0.0.21 rac2-priv.sysdba.org rac2-priv 10.0.0.10 nas.sysdba.org nas |
Contents of /etc/resolve.conf
|
1 |
nameserver 192.168.137.100 |
Step 6
System settings:
Add the following lines to /etc/sysctl.conf, the same line numbers automatically get updated.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
fs.aio-max-nr = 1048576 fs.file-max = 6815744 kernel.shmall = 2097152 kernel.shmmax = 1054504960 kernel.shmmni = 4096 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default=262144 10 net.core.rmem_max=4194304 11 net.core.wmem_default=262144 12 net.core.wmem_max=1048586 |
|
1 |
/sbin/sysctl -p |
/etc/security/limits.conf
|
1 2 3 4 5 6 7 |
oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 |
/etc/pam.d/login
|
1 |
session required pam_limits.so |
As the Oracle Cluster Time Synchronization Service (ctssd) wşll be used, Linux’s ntp service will be stopped.
|
1 2 3 4 5 6 7 |
# service ntpd stop # chkconfig ntpd off # mv /etc/ntp.conf /etc/ntp.conf.org # rm /var/run/ntpd.pid |
Step 7
Add the following lines to Oracle’s user profile (/home/oracle/.bash_profile)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
# Oracle Settings TMP=/tmp; export TMP TMPDIR=$TMP; export TMPDIR ORACLE_HOSTNAME=rac1.sysdba.org; export ORACLE_HOSTNAME ORACLE_UNQNAME=rac; export ORACLE_UNQNAME ORACLE_BASE=/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/db_1; export ORACLE_HOME ORACLE_SID=RAC1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM PATH=$ORACLE_HOME/bin:$PATH; export PATH PATH=/usr/sbin:$PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH if [ $USER = "oracle" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi fi |
Note: If Oracle’s grid has been setup in /oragrid within /oracle, add the following lines into the oracle user and the root user profiles.
For the oracle user
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
ORACLE_HOSTNAME=rac1.sysdba.org; export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME ORACLE_BASE=/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db; export ORACLE_HOME ORACLE_SID=RAC1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM PATH=/usr/sbin:$PATH; export PATH PATH=$ORACLE_HOME/bin:$PATH; export PATH |
For the root user (who’s automatically also a grid owner)
|
1 2 3 4 5 |
ORAGRID_BASE=/oragrid ORAGRID_HOME=$ORAGRID_BASE/product/11.2.0/grid; export ORAGRID_HOME PATH=$ORAGRID_HOME/bin:$PATH; export PATH |
Step 8
Checking whether iscsi (Internet SCSI) has been loaded
|
1 2 3 4 5 6 7 8 9 |
rpm -qa | grep iscsi-initiator-utils if it hasn't been loaded mount the CD mount -r /dev/cdrom /media/cdrom cd /media/cdrom/Server.... rpm -Uvh iscsi-initiator-utils.... iscsi is loaded and started |
It’s necessary to add LUN to the openfiler’s iscsi and map the volumes.
Following this, in the “iscsi target” “Network ACL” tab give permission to “internal_network”.
|
1 2 3 |
iscsiadm -m discovery -t sendtargets -p 10.0.0.10 iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.de0b812f5012 -p 10.0.0.10 -l |
The following lines should appear during the operating system’s restart sequence:
The disks should automatically be visible to the os after restarting (before it has been formatted).
Step 9
(The procedures shown here can also be accomplished using the asmca gui tool (ASM configuration assistant)
Configuring the disks (using the sequence of “n”, “p”, “1”, “w”)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 |
[root@rac1 Server] $ fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 12446. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-12446, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-12446, default 12446): Using default value 12446 Command (m for help): p Disk /dev/sdb: 102.3 GB, 102374572032 bytes 255 heads, 63 sectors/track, 12446 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 12446 99972463+ 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. |
Carrying out the same procedures in both the sdc and sdd disks
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
[root@rac1 Server]# fdisk -l çıktısının aşağıdaki gibi olması lazım. Disk /dev/sda: 64.4 GB, 64424509440 bytes 255 heads, 63 sectors/track, 7832 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 7832 62806117+ 8e Linux LVM Disk /dev/dm-0: 60.0 GB, 60095987712 bytes 255 heads, 63 sectors/track, 7306 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/dm-0 doesn't contain a valid partition table Disk /dev/dm-1: 4194 MB, 4194304000 bytes 255 heads, 63 sectors/track, 509 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/dm-1 doesn't contain a valid partition table Disk /dev/sdb: 102.3 GB, 102374572032 bytes 255 heads, 63 sectors/track, 12446 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 12446 99972463+ 83 Linux Disk /dev/sdc: 102.3 GB, 102374572032 bytes 255 heads, 63 sectors/track, 12446 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 12446 99972463+ 83 Linux Disk /dev/sdd: 102.3 GB, 102374572032 bytes 255 heads, 63 sectors/track, 12446 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdd1 1 12446 99972463+ 83 Linux Disk /dev/sde: 51.1 GB, 51170508800 bytes 64 heads, 32 sectors/track, 48800 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk /dev/sde doesn't contain a valid partition table |
Step 10
Setting the shared disks of the ASM environment and loading the kernel modules
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
oracleasm configure -i Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting <ENTER> without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: Writing Oracle ASM library driver configuration: done |
Loading the kernel modules
|
1 2 3 4 5 6 7 8 9 |
oracleasm init Loading module "oracleasm": oracleasm Mounting ASMlib driver filesystem: /dev/oracleasm |
To create the ASM disks configure the paths of the disks which were created using iSCSI
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#oracleasm createdisk DISK1 /dev/sdb1 Writing disk header: done Instantiating disk: done #oracleasm createdisk DISK2 /dev/sdc1 Writing disk header: done Instantiating disk: done #oracleasm createdisk DISK3 /dev/sdd1 Writing disk header: done Instantiating disk: done |
Scanning the disks
#oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks…
Scanning system for ASM disks…
Listing the disks
|
1 2 3 4 5 6 7 |
#oracleasm listdisks DISK1 DISK2 DISK3 |
RAC1 has now been setup, moving on to RAC2.
