Using Oracle dNFS Multi-path

Oracle Direct NFS (dNFS) Multi-path

If you have multiple interfaces on your database server and your NFS server also supports multiple interfaces you should consider using Oracle dNFS multi-path.

Oracle dNFS multi-path removes the need to distribute and manage datafiles over multiple NFS mount points, Oracle dNFS multi-path can also improve throughput if you were not previously using all your available network interfaces and bandwidth.

In Oracle dNFS (Direct NFS) is disabled by default but can be easily enabled or disabled by following the steps below.

However, before we go through the Oracle dNFS configuration I will describe my lab set-up. My Database Server is configured with 4 x 10GbE all all on different sub-nets, and my  Pure Storage FlashBlade is configured with 4 NFS interfaces exporting 3 NFS mounts for Oracle data, Oracle FRA and shared configuration for my Oracle 12cR2 RAC cluster.

For further details see MOS Note: How to configure DNFS to use multiple IPs (Doc ID 1552831.1). It’s also possible to use the same subnet if required, checkout MOS Note: How to Setup Direct NFS client multipaths in same subnet (Doc ID 822481.1)

dNFS

The oranfstab file

To use Oracle dNFS MultiPath you will need an oranfstab file, Oracle will look in the following order and locations for the configuration file.

  • /etc/oranfstab – Server wide
  • $ORACLE_HOME/dbs – Oracle Home specific

If no oranfstab file is found Oracle will use the /etc/mtab file to enable dNFS but you will have no MultiPath

Below is a copy of my $ORACLE_HOME/dbs/oranfstab

server: flashblade
local: 192.168.4.2 path: 192.168.4.100
local: 192.168.5.2 path: 192.168.5.100
local: 192.168.6.2 path: 192.168.6.100
local: 192.168.7.2 path: 192.168.7.100
nfs_version: nfsv3
export: /z-fbhosts_oradata  mount: /u01/oradata
export: /z-fbhosts_orafra   mount: /u01/app/oracle/fast_recovery_area
export: /z-fbhosts_oraconfig mount: /u01/shared_config

The parameters used in my oranfstab are described below:

server
The NFS server name, this can be anything and is used for information rather than any network lookups.

local
Up to four paths on the database host, specified by IP address or by name.

path
Up to four network paths to the NFS server, specified either by IP address, or by name.

export
The exported path from the NFS server.

mount
The corresponding local mount point for the exported volume.

nfs_version
Specifies the NFS protocol version used by Direct NFS Client. Possible values are NFSv3, NFSv4, NFSv4.1, and pNFS. The default version is NFSv3. If you select NFSv4.x, then you must configure the value in oranfstab for nfs_version. Specify nfs_version as pNFS, if you want to use Direct NFS with Parallel NFS.

Enabling dNFS

Change dir to $ORACLE_HOME/rdbms/lib and use the make command e.g
 cd $ORACLE_HOME/rdbms/lib
  make -f ins_rdbms.mk dnfs_on

You should see something similar to this response:

rm -f /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so; \
  cp /u01/app/oracle/product/12.2.0.1/db_1/lib/libnfsodm12.so /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so

Verify Oracle dNFS Usage

After applying changes restart the database and confirm your configuration change have had the desired impact.

The Oracle Alert file should now report the Oracle dNFS library is in use as it’s version.

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 4.0

We can also see that Oracle is using multiple paths e.g.

Direct NFS: channel id [0] path [192.168.4.100] to filer [flashblade] via local [192.168.4.2] is UP
Direct NFS: channel id [1] path [192.168.5.100] to filer [flashblade] via local [192.168.5.2] is UP
Direct NFS: channel id [2] path [192.168.6.100] to filer [flashblade] via local [192.168.6.2] is UP
Direct NFS: channel id [3] path [192.168.7.100] to filer [flashblade] via local [192.168.7.2] is UP

have had the desired impact by querying the v$dnfs_servers view.

Check to see if Enabled

SQL> COLUMN svrname FORMAT A20
SQL> COLUMN dirname FORMAT A50
SQL> SELECT svrname, dirname, nfsversion FROM v$dnfs_servers;

SVRNAME     DIRNAME                                     NFSVERSION
-------------------- -----------------------------------------------
flashblade     /z-fbhosts_oraconfig                       NFSv3.0
flashblade     /z-fbhosts_oradata                         NFSv3.0
flashblade     /z-fbhosts_orafra                          NFSv3.0

dNFS views

  • v$dnfs_stats          – shows dNFS performance statistics
  • v$dnfs_channels  – shows network channels is use by dNFS
  • v$dnfs_servers     – shows dNFS servers
  • v$dnfs_files           – shows files open for dNFS

Disabling dNFS

To disable dNFS change directory to $ORACLE_HOME/rdbms/lib and enter the following commands:

cd $ORACLE_HOME/rdbms/lib
  make -f ins_rdbms.mk dnfs_off

You should see something similar to this response:

rm -f /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so

You can also now remove the previously created oranfstab file

Check to see if Disabled

SQL> COLUMN svrname FORMAT A20
SQL> COLUMN dirname FORMAT A50
SQL> SELECT svrname, dirname, nfsversion FROM v$dnfs_servers;
 
no rows selected

In my next post I will share some before and after test results.


3 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s