Using ASM with ASMlib or raw devices

ASM

Another post to fix some things better in memory.

Some typical tasks when using ASM, possibly with Oracle VM

If virtual, create the disk first
Either through the Oracle VM Manager interface (Virtual Machine Configure -> Storage -> Create New Virtual Disk), or by stopping the virtual machine and editing the vm.cfg file (adding lines to the disk clause like for example: ‘file:/OVS/running_pool/21_db01/asm01.img,xvde,w’,). In the latter case, these new virtual disks must exist before the vm is started again, typically by creating zero-filled files with dd (place in the directory specified in the vm.cfg):
dd if=/dev/zero of=asm01.img bs=1M count=50000

I don’t know of a way to make the virtual machine pick up changes like new disks in the vm.cfg without restarting, but there should be one, since doing the same thing through the OVM Manager interface does not cause any rebooting. I’ll make a note to figure this out, but any feedback from someone who might know is appreciated.

Partitioning
Partition each storage disk with one primary partition per disk, using fdisk.
fdisk /dev/sdX

Configuring Oracle Cluster Synchronization Services (CSS)

Run localconfig as root (which probably also means you need to exchange the value for the ORACLE_HOME manually):
$ORACLE_HOME/bin/localconfig all

Setup for ASMlib
Installation: If not already present in OS; download oracleasm packages from OTN, install and configure (also as root)
/etc/init.d/oracleasm configure
Stamp disks for use with ASMlib:
/etc/init.d/oracleasm createdisk VOLn /dev/sdX1
scandisks if using with additional RAC nodes, or listdisks for verification.
/etc/init.d/oracleasm listdisks

Setup for raw devices
Edit mapping file /etc/sysconfig/rawdevices (rawpath devicepath)
Restart rawdevices service
service rawdevices restart
Set ownership (oracle:oinstall) and mode 0600 for newly defined raw devices, and add to /etc/rc.local as well.

ASM instance creation
Use DBCA, “Configure Automatic Storage Management”, follow the instructions (about CSS and passwords), create new disk group.

Database creation
Verify the listener is running and that +ASM is registered. If not, register with alter system register, and check lsnrctl status again.
Choose ASM when creating the database, using the ASM disk group (typically +DATA) for Oracle-managed files, flash recovery area and archiving.

Switching  from raw/ASMlib and back

  1. Shutdown other instances, connect to +ASM.
    Install ASMlib, without the disk stamping.
    Set ASM_DISKSTRING to the preferred label for the disks, ex. ‘ORCL:VOL*’, shutdown +ASM.
    Stamp the disks, but use renamedisk, since createdisk would fail when the disks have already been used raw.
    /etc/init.d/oracleasm renamedisk /dev/sdX1 VOL1
    and so on.
    Startup +ASM and other instances.
  2. Shutdown other instances, connect to +ASM.
    Set ASM_DISKSTRING to wherever the raw devices can be found; ex. ‘/dev/raw/raw*’, shutdown +ASM.
    Perform raw device setup (edit mapping, start service, set file ownership).
    Startup +ASM and other instances.

4 thoughts on “Using ASM with ASMlib or raw devices

  1. Please mind raw devices are deprecated both by linux/redhat as by oracle.
    Since (database version) 10.2.0.2 it’s possible to use blockdevices directly, because they are opened using the ‘O_DIRECT’ flag (so nothing is buffered at the o/s level)

    asmlib adds a kernel module dependency

    1. Good point. I haven’t used raw devices for anything but testing in a long while, so I wasn’t aware of this.

      Actually, in production settings, I’ve only used raw devices on certain HP-UX (for RAC) systems, so I additionally wasn’t aware until I checked now, that the raw device driver in Linux was never an actual driver, but just a kind of alias to using an (unmounted) block device (with O_DIRECT). And that this is the main (?) reason why it was deprecated. Did I understand that correctly?

  2. That’s correct, it was a workaround for not having a rdsk alias with the disk devices (as solaris or HPUX (if my memory serves me correct) has)

    especially for testing purposes it’s very usefull to use LVM for the ASM disk devices.

    another funny thing: the disk label on the asmlib level is a label that is not used by the other oracle software. even funnier: you can move an entire disk to another device or give it another name, and ASM will be able to use it without any message, as long as the disk is found by ASM. ASM reads the asm header on the device and knows what disk it is.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s