Using ASM with ASMlib or raw devices


Another post to fix some things better in memory.

Some typical tasks when using ASM, possibly with Oracle VM

If virtual, create the disk first
Either through the Oracle VM Manager interface (Virtual Machine Configure -> Storage -> Create New Virtual Disk), or by stopping the virtual machine and editing the vm.cfg file (adding lines to the disk clause like for example: ‘file:/OVS/running_pool/21_db01/asm01.img,xvde,w’,). In the latter case, these new virtual disks must exist before the vm is started again, typically by creating zero-filled files with dd (place in the directory specified in the vm.cfg):
dd if=/dev/zero of=asm01.img bs=1M count=50000

I don’t know of a way to make the virtual machine pick up changes like new disks in the vm.cfg without restarting, but there should be one, since doing the same thing through the OVM Manager interface does not cause any rebooting. I’ll make a note to figure this out, but any feedback from someone who might know is appreciated.

Partition each storage disk with one primary partition per disk, using fdisk.
fdisk /dev/sdX

Configuring Oracle Cluster Synchronization Services (CSS)

Run localconfig as root (which probably also means you need to exchange the value for the ORACLE_HOME manually):
$ORACLE_HOME/bin/localconfig all

Setup for ASMlib
Installation: If not already present in OS; download oracleasm packages from OTN, install and configure (also as root)
/etc/init.d/oracleasm configure
Stamp disks for use with ASMlib:
/etc/init.d/oracleasm createdisk VOLn /dev/sdX1
scandisks if using with additional RAC nodes, or listdisks for verification.
/etc/init.d/oracleasm listdisks

Setup for raw devices
Edit mapping file /etc/sysconfig/rawdevices (rawpath devicepath)
Restart rawdevices service
service rawdevices restart
Set ownership (oracle:oinstall) and mode 0600 for newly defined raw devices, and add to /etc/rc.local as well.

ASM instance creation
Use DBCA, “Configure Automatic Storage Management”, follow the instructions (about CSS and passwords), create new disk group.

Database creation
Verify the listener is running and that +ASM is registered. If not, register with alter system register, and check lsnrctl status again.
Choose ASM when creating the database, using the ASM disk group (typically +DATA) for Oracle-managed files, flash recovery area and archiving.

Switching  from raw/ASMlib and back

  1. Shutdown other instances, connect to +ASM.
    Install ASMlib, without the disk stamping.
    Set ASM_DISKSTRING to the preferred label for the disks, ex. ‘ORCL:VOL*’, shutdown +ASM.
    Stamp the disks, but use renamedisk, since createdisk would fail when the disks have already been used raw.
    /etc/init.d/oracleasm renamedisk /dev/sdX1 VOL1
    and so on.
    Startup +ASM and other instances.
  2. Shutdown other instances, connect to +ASM.
    Set ASM_DISKSTRING to wherever the raw devices can be found; ex. ‘/dev/raw/raw*’, shutdown +ASM.
    Perform raw device setup (edit mapping, start service, set file ownership).
    Startup +ASM and other instances.