Using Oracle dNFS Multi-path

Oracle Direct NFS (dNFS) Multi-path

If you have multiple interfaces on your database server and your NFS server also supports multiple interfaces you should consider using Oracle dNFS multi-path.

Oracle dNFS multi-path removes the need to distribute and manage datafiles over multiple NFS mount points, Oracle dNFS multi-path can also improve throughput if you were not previously using all your available network interfaces and bandwidth.

In Oracle dNFS (Direct NFS) is disabled by default but can be easily enabled or disabled by following the steps below.

However, before we go through the Oracle dNFS configuration I will describe my lab set-up. My Database Server is configured with 4 x 10GbE all all on different sub-nets, and my  Pure Storage FlashBlade is configured with 4 NFS interfaces exporting 3 NFS mounts for Oracle data, Oracle FRA and shared configuration for my Oracle 12cR2 RAC cluster.

For further details see MOS Note: How to configure DNFS to use multiple IPs (Doc ID 1552831.1). It’s also possible to use the same subnet if required, checkout MOS Note: How to Setup Direct NFS client multipaths in same subnet (Doc ID 822481.1)

dNFS

The oranfstab file

To use Oracle dNFS MultiPath you will need an oranfstab file, Oracle will look in the following order and locations for the configuration file.

  • /etc/oranfstab – Server wide
  • $ORACLE_HOME/dbs – Oracle Home specific

If no oranfstab file is found Oracle will use the /etc/mtab file to enable dNFS but you will have no MultiPath

Below is a copy of my $ORACLE_HOME/dbs/oranfstab

server: flashblade
local: 192.168.4.2 path: 192.168.4.100
local: 192.168.5.2 path: 192.168.5.100
local: 192.168.6.2 path: 192.168.6.100
local: 192.168.7.2 path: 192.168.7.100
nfs_version: nfsv3
export: /z-fbhosts_oradata  mount: /u01/oradata
export: /z-fbhosts_orafra   mount: /u01/app/oracle/fast_recovery_area
export: /z-fbhosts_oraconfig mount: /u01/shared_config

The parameters used in my oranfstab are described below:

server
The NFS server name, this can be anything and is used for information rather than any network lookups.

local
Up to four paths on the database host, specified by IP address or by name.

path
Up to four network paths to the NFS server, specified either by IP address, or by name.

export
The exported path from the NFS server.

mount
The corresponding local mount point for the exported volume.

nfs_version
Specifies the NFS protocol version used by Direct NFS Client. Possible values are NFSv3, NFSv4, NFSv4.1, and pNFS. The default version is NFSv3. If you select NFSv4.x, then you must configure the value in oranfstab for nfs_version. Specify nfs_version as pNFS, if you want to use Direct NFS with Parallel NFS.

Enabling dNFS

Change dir to $ORACLE_HOME/rdbms/lib and use the make command e.g
 cd $ORACLE_HOME/rdbms/lib
  make -f ins_rdbms.mk dnfs_on

You should see something similar to this response:

rm -f /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so; \
  cp /u01/app/oracle/product/12.2.0.1/db_1/lib/libnfsodm12.so /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so

Verify Oracle dNFS Usage

After applying changes restart the database and confirm your configuration change have had the desired impact.

The Oracle Alert file should now report the Oracle dNFS library is in use as it’s version.

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 4.0

We can also see that Oracle is using multiple paths e.g.

Direct NFS: channel id [0] path [192.168.4.100] to filer [flashblade] via local [192.168.4.2] is UP
Direct NFS: channel id [1] path [192.168.5.100] to filer [flashblade] via local [192.168.5.2] is UP
Direct NFS: channel id [2] path [192.168.6.100] to filer [flashblade] via local [192.168.6.2] is UP
Direct NFS: channel id [3] path [192.168.7.100] to filer [flashblade] via local [192.168.7.2] is UP

have had the desired impact by querying the v$dnfs_servers view.

Check to see if Enabled

SQL> COLUMN svrname FORMAT A20
SQL> COLUMN dirname FORMAT A50
SQL> SELECT svrname, dirname, nfsversion FROM v$dnfs_servers;

SVRNAME     DIRNAME                                     NFSVERSION
-------------------- -----------------------------------------------
flashblade     /z-fbhosts_oraconfig                       NFSv3.0
flashblade     /z-fbhosts_oradata                         NFSv3.0
flashblade     /z-fbhosts_orafra                          NFSv3.0

dNFS views

  • v$dnfs_stats          – shows dNFS performance statistics
  • v$dnfs_channels  – shows network channels is use by dNFS
  • v$dnfs_servers     – shows dNFS servers
  • v$dnfs_files           – shows files open for dNFS

Disabling dNFS

To disable dNFS change directory to $ORACLE_HOME/rdbms/lib and enter the following commands:

cd $ORACLE_HOME/rdbms/lib
  make -f ins_rdbms.mk dnfs_off

You should see something similar to this response:

rm -f /u01/app/oracle/product/12.2.0.1/db_1/rdbms/lib/odm/libnfsodm12.so

You can also now remove the previously created oranfstab file

Check to see if Disabled

SQL> COLUMN svrname FORMAT A20
SQL> COLUMN dirname FORMAT A50
SQL> SELECT svrname, dirname, nfsversion FROM v$dnfs_servers;
 
no rows selected

In my next post I will share some before and after test results.


Advertisements

Oracle Linux Container Registry

You may have read one of my previous posts Oracle 12c on Docker where I described  how you can get hold of official Oracle images from the Docker Store.

In this post I will share how you can now use the Oracle Container Registry to obtain Docker images for all licensable Oracle products.

To access the Oracle Registry Server you must have an Oracle Single Sign-On (SSO) account, this is the same account you use for support.oracle.com and opn.oracle.com etc..

Navigate to https://container-registry.oracle.com/ and login using your SSO credentials, once authenticated you will be presented with the ‘Explore’ webpage.
Oracle_Container_Registry

You can then need to select the required Business Area e.g. ‘Database’ and click the red ‘Continue’ button, you will then be presented with the ‘Oracle Standard Terms and Restrictions’ agreement, review and if acceptable click ‘Accept’ and the bottom of the page.

You will then be returned to the ‘Explorer’ and notice that the Oracle Standard Terms and Restrictions shows you acceptance, this is valid for 8 hours only.

Standard_Terms

Note: you will not be able to pull and image until you have accepted Oracle Standard Terms and Restrictions.

You are now ready to pull the required image by clicking the repository  e.g. ‘enterprise’ and copying & pasting the provided pull command e.g.

pull_command
The Repository Detail page provides set-up and usage information for the selected image.

Ok, now I have outlined the process let me walk you through it.

1) Login to Registry

ronsmac:~ ronekins$ docker login container-registry.oracle.com
Username: ron.ekins@icloud.com
Password:
Login Succeeded

2a) Pull Oracle Linux

Select ‘OS’ from the ‘Explore Official Business Areas’  and click the red ‘Continue’ button for oraclelinux. As before, accept Oracle Terms.

Repo_Linux

Now copy the docker pull command and paste in in your Mac or Linux shell.

OC_Reg

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/os/oraclelinux
Using default tag: latest
latest: Pulling from os/oraclelinux
80d2e45a33d8: Pull complete
Digest: sha256:d31c2987a6c427eeca70fb28ccefd6e788e96b3202dc715aa3c80bcb23673f6d
Status: Downloaded newer image for container-registry.oracle.com/os/oraclelinux:latest

2b) Pull Oracle Database 12c Enterprise Edition

Register_Database

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/database/enterprise
Using default tag: latest
latest: Pulling from database/enterprise
cbb9821ba51c: Pull complete
9bd4d110366e: Pull complete
af8b29651e27: Pull complete
4c242ab1add4: Pull complete
7bda1e55bd08: Pull complete
Digest: sha256:42809e491491d7f07a2aa76903bb5feabe3a0d23abcb7e680264074f043a604c
Status: Downloaded newer image for container-registry.oracle.com/database/enterprise:latest

3) Check Image size

Your Oracle two images are now available for use locally, you can check this with the docker image ls command. e.g.
ronsmac:~ ronekins$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
container-registry.oracle.com/os/oraclelinux latest 6c33a25f4a29 5 weeks ago 229MB
container-registry.oracle.com/database/enterprise latest 12a359cd0528 2 months ago 3.44GB

Follow-up ‘Registry’ teaser

Within the ‘OS’ area Oracle provides a registry image which you can use to deploy a local registry so you can pull images and customise as required, the local registry also avoids the need the to authenticate every 8 hours, I will detail all of this in a follow-up post.

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/os/registry
Using default tag: latest
latest: Pulling from os/registry
a3ed95caeb02: Pull complete
89937cfc6593: Pull complete
bd07ebf08156: Pull complete
Digest: sha256:13d190c8838ebeb1e9cbf87b3edcc1fc6b6948d1b5d2200ec4dc64c638a56402
Status: Downloaded newer image for container-registry.oracle.com/os/registry:latest

 

ODC Appreciation Day: Online Database Move Datafile

ODC Appreciation Day – 10th October 2017

Last year Tim aka ‘ORACLE-BASE’ launched the the first OTN (Oracle Technology Network) Appreciation Day, after the success of last year this is being repeated again this year under the new moniker of ODC.

As today is Tuesday 10th October 2017 expect some great content from fellow Oracle ACE’s and Oracle community folk coming out throughout the day as the different timezones come online.

It’s my pleasure to contribute to this year’s ODC Appreciation Day (Oracle Developer Community) and my favourite reason feature is the Oracle 12c Online database move datafile feature.

This feature is great for fixing naming or directory structure errors, re-organising your database layout, adopting new standards or migrating between storage systems. It supports Oracle ASM and filesystems with the only restriction being you need to be on Oracle 12c, so upgrade it your still on Oracle 11gR2 and embrace online database move datafile.

You can read more about it at my Blog  Online Database Move Datafile

One-Click Oracle 12c Database Clone

Background

Earlier in the year at the excellent OUgf Finnish Oracle User Group Harmony 17 conference, I performed a Live demo where I refreshed an Oracle 12c database using an Ansible Playbook in just over a minute, 1min 16sec to be more precise.

Timings

Once I had completed my presentation Frits Hoogland mentioned that he had recently used and Blogged on Ansible-Sempahore , an Open Source alternative to Ansible Tower which provides a Web UI and API’s for launching Ansible Tasks.

On my return to the UK I read Frits’ Blog – How to install semaphore UI for running Ansible and the installation instructions on the GitHub site, and was soon up and running.

Why this Blog

In this Blog post I am going to share how I used Ansible-Semaphore to launch my Ansible playbook to perform an Oracle 12c database refresh with a single click.

Once I had Semaphore installed, I configure the Inventory, Environment, Key Store and Playbook Repositories and added my Task Templates all through the Semaphore UI, all pretty straight forward, so no need to screenshot it.

semaphore Task Templates
semaphore Task Templates

A powerful feature of Semaphore is it’s integration with GitHub, this ensures every time a Playbook is run the last version is used as the code is pulled back from the repository.

Running my ‘Oracle Database Clone’ Ansible playbook is as now as simple as clicking ‘run!’ (no Playbook or Environment Overrides are required for this playbook)

Create_Task

At the end of each run semaphore provides a Task Log which is retained and accessible from the dashboard for historic review.

Task_Log

There we have it a One-Click Oracle 12c Database refresh in just under 1 minute 15 seconds.

If you want to see more, check out my YouTube channel to watch a demo of the above.