Veritas ODM Async and Oracle

简单来说对于Veritas Vxfs和ODM FILE而言filesystemio_options不生效(This parameter is not applicable to VxFS files, ODM files, or Quick I/O files.),必要检查ODM SO是否正确LINK。


ls -l $ORACLE_HOME/lib/libodm*




(Since the version 10g, Oracle will open data files located on the JFS2 file system with the O_CIO option if the filesystemio_options initialization parameter is set to either directIO or setall.)




Disk utility output shows 100% usage for disk continously:

ProcList CPU Rpt Mem Rpt Disk Rpt NextKeys SlctProc Help Exit GlancePlus C.04.50.00 19:15:39 bplita3 ia64 Current Avg High ———————————————————————- CPU Util S SN NU U | 90% 90% 90% Disk Util F F |100% 100% 100% <=== Disk Too heavily loaded
In Statspack or AWR report, high ‘free buffer waits’ seen even after with considerably increasing db_cache_size and db writers.

Statspack Report Instance Efficiency Percentages (Target 100%) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Buffer Nowait %: 99.99 Redo NoWait %: 99.99 Buffer Hit %: 58.92 In-memory Sort %: 99.82 Library Hit %: 89.19 Soft Parse %: 83.51 Execute to Parse %: 50.76 Latch Hit %: 100.00 Parse CPU to Parse Elapsd %: 81.48 % Non-Parse CPU: 93.21 …. Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Ela Time ——————————————– ———— ———– ——– free buffer waits 76,390 74,807 55.82 enqueue 4,071 11,625 8.67


The problem is related to Disk I/O.



1. Use the O/S striping software

Try to use the O/S striping software to distribute database files over as many disks as you can.

2. Use Direct IO

Mount the filesystem with direct IO option.
For example:

% Mount –F vxfs –o remount,nodatainlog,mincache=direct,convosync=direct /dev/vg00/lv_ora /soft/oracle

mincache and convosync

“mincache=direct” => bypass buffer cache on read
“convosync=direct” => force direct I/O for DB writers

Mincache=direct and convosync=direct allow data to be transferred directly from Oracle buffer cache to disk and disk to Oracle buffer cache. This avoids double buffering by bypassing the file system buffer cache and can improve physical read/write performance. However, cases where the disk read could have been avoided because a required block was in file system buffer cache may be negatively impacted.

If your filesystem is mounted with this option, then FILESYSTEMIO_OPTIONS default setting of ASYNCH can be used in order to use DIO.

Parameters in Oracle influencing the use of Direct IO

FILESYSTEMIO_OPTIONS defines the IO operations on filesystem files .This parameter should not normally be set by the user.
The value may be any of the following:
asynch – Set by default on HP. This allows asynchronous IO to be used where supported by the OS.
directIO – This allows directIO to be used where supported by the OS. Direct IO bypasses any Unix buffer cache.
setall – Enables both ASYNC and DIRECT IO.
none – This disables ASYNC IO and DIRECT IO so that Oracle uses normal synchronous writes, without any direct io options.

Document 120697.1 Init.ora Parameter “FILESYSTEMIO_OPTIONS” Reference Note
DISK_ASYNCH_IO controls whether I/O to datafiles, control files, and logfiles is asynchronous (that is, whether parallel server processes can overlap I/O requests with CPU processing during table scans). If your platform supports asynchronous I/O to disk, Oracle recommends that you leave this parameter set to its default value. However, if the asynchronous I/O implementation is not stable, you can set this parameter to false to disable asynchronous I/O. If your platform does not support asynchronous I/O to disk, this parameter has no effect.

If you set DISK_ASYNCH_IO to false, then you should also set DBWR_IO_SLAVES or DB_WRITER_PROCESSES to a value other than its default of zero in order to simulate asynchronous I/O.
see comments in DISK_ASYNCH_IO

Again the default setting of ASYNCH can be used when implementing direct I/O on HP-UX / Veritas.

3. Concurrent I/O

An alternative solution to Direct I/O is to use Concurrent I/O. Concurrent I/O is available in OnlineJFS 5.0.1.

To enable Concurrent I/O, the filesystem must be mounted with “-o cio”.

mount -F vxfs -o nodatainlog,cio /soevxfs/redos /oracle/mnt/redos

Please note that remount should not be used to enable Concurrent I/O on mounted filesystems.

“-o cio”

Concurrent I/O allows multiple processes to read from or write to the same file without blocking other read(2) or write(2) calls.With Concurrent I/O, the read and write operations are not serialized. This advisory is generally used by applications that require high performance for accessing data and do not perform overlapping writes to the same file. It is the responsibility of the application or the running threads to coordinate the write activities to the same file. It also avoids double buffering by bypassing the filesystem buffer cache and thus improves physical read/write performance significantly. Concurrent I/O performs very close to that of raw logical volumes.


The value may be any of the following:
“asynch” – This allows asynchronous IO to be used where
supported by the OS.
“directIO” – This allows directIO to be used where
supported by the OS. Direct IO bypasses any
Unix buffer cache. As of 10.2 most platforms
will try to use “directio” option for NFS
mounted disks (and will also check NFS
attributes are sensible).
“setall” – Enables both ASYNC and DIRECT IO.
“none” – This disables ASYNC IO and DIRECT IO so that
Oracle uses normal synchronous writes, without
any direct io options.


We assume Veritas ODM driver is installed, mounted and available.

The following steps to enable/disable ODM for Oracle database (note that with different versions of the Solaris OS, the path may change to the ODM files).

Enable ODM

Log in as oracle user.

i) Shutdown the database

ii) Change directories:
$ cd $ORACLE_HOME/lib

iii) Take a backup of existing original ODM library

PA systems

$ mv $ORACLE_HOME/lib/ $ORACLE_HOME/lib/

IA systems

$ mv $ORACLE_HOME/lib/ $ORACLE_HOME/lib/

(* Note 9i library name ie,10g is and 11g is

iv) create a soft link to veritas ODM library

PA systems

ln -s /opt/VRTSodm/lib/ $ORACLE_HOME/lib/

IA systems

ln -s /opt/VRTSodm/lib/ $ORACLE_HOME/lib/

iv) Start the database and check

Once the database instance is enabled with ODM, the following message is displayed in the Oracle alert log:

Example :-

???¢????????Oracle instance running with ODM: VERITAS 4.1 ODM Library, Version 1.1???¢????????
Disable ODM

Log in as oracle user.

i) Shutdown the database

ii) Change directories:
$ cd $ORACLE_HOME/lib
iii) Remove the softlink

$ cd $ORACLE_HOME/lib

PA systems
$ rm

IA systems
$ rm
iv) Copy the orginal $ORACLE_HOME/lib/ or file back to disable the ODM library.

PA systems
$ cp $ORACLE_HOME/lib/ $ORACLE_HOME/lib/

IA systems
$ cp $ORACLE_HOME/lib/ $ORACLE_HOME/lib/

v) Start the database



Applies to:

Oracle Server – Enterprise Edition – Version: to – Release: 10.2 to 11.1
Information in this document applies to any platform.
Database installations using Veritas Storage Foundation for Oracle / Oracle RAC (SFRAC)
and making use of the Veritas ODM (Oracle Disk Manager) library

The Veritas shared library used by Oracle Disk Manager could be lost upon RDBMS patchset
install such as patchset versions,, or
If this problem is not identified then poor IO performance or a change in IO profile of the
database can result such as cached IO being done to a Veritas filesystem rather than the expected
direct IO as was the case before applying the patchset.

Applying one of the currently available RDBMS patchsets:

on platforms AIX, Linux, HP-UX, Solaris.

Further RDBMS patchset versions which are not yet released as this Note is written can cause the same problem.

Upon patchset install in the $ORACLE_HOME/lib directory the library libodm<Version>.so is replaced by the the patchset’s own libodm<Version>.so which is created as a soft link to the dummy library libodmd<Version>.so.

In addition, by relinking the oracle binaries during patchset install on Linux, HP-UX or Solaris the dummy library $ORACLE_HOME/rdbms/lib/libodm<Veritas>.a will be statically linked in.
On AIX the make command would link the libodm<version>.so
rather than the static library.
If the previous installed library in $ORACLE_HOME/lib was a soft link or a copy of the Veritas libodm and the library was linked to the oracle binary then the ODM functionality would be lost after patchset install.

If the database is affected by this problem it would be visible from the alert.log file i.e. by the disappearance of a message like below:

“Oracle instance running with ODM: VERITAS ODM Library, Version 1.1 ”




If the Veritas libodm has been lost following steps need to be executed to get it back in place:

Example: Oracle RDBMS after patchset install on Solaris

su – <oracle install user>
ldd $ORACLE_HOME/bin/oracle shows no

Create soft link of Veritas ODM library
(below examples are valid for Oracle 10g and Veritas 5.0 installs)
AIX, HP-UX IA64, Linux, Sparc-Solaris:

And then
ln -s /opt/VRTSodm/ $ORACLE_HOME/lib/
ln -s /opt/VRTSodm/lib/ $ORACLE_HOME/lib/
ln -s /opt/VRTSodm/lib/ $ORACLE_HOME/lib/
ln -s /opt/VRTSodm/lib/ $ORACLE_HOME/lib/
ln -s /opt/VRTSodm/lib/sparcv9/ $ORACLE_HOME/lib/
Avoid linking the rdbms/lib static dummy library

mv $ORACLE_HOME/rdbms/lib/libodm10.a $ORACLE_HOME/rdbms/lib/libodm10.a.10204
make -f ioracle

Confirm is linked to the oracle binary:

ldd $ORACLE_HOME/bin/oracle shows
and starting up the Oracle instance would show a message like:
“Oracle instance running with ODM: VERITAS … ”





In Oracle9 Release 2 (9.2), you can use the filesystemio_optionsinit.ora
parameter to enable or disable asynchronous I/O, direct I/O, or Concurrent I/O
on file system files. This parameter is used on files that reside in non-VxFS
filesystems only. This parameter is not applicable to VxFS files, ODM files, or
Quick I/O files.
See your Oracle documentation for more details.






沪公网安备 31010802001379号