Docker, Oracle 12c and Persistent Storage

Introduction

If you have read my previous posts on Docker you may have already installed Docker and have an Oracle 12c database running in a Docker container, if not you may want to read my first post on Docker – Running Oracle 12.2.01 on Docker

However, since I wrote the above post Oracle have published Docker images for many of their products, these can be found at the Docker Store. The Oracle images on the Docker Store are not just a great resource and help accelerate adoption, but are also certified and supported by Oracle.

Before continuing with this post you need to check-out my posts on using the new Oracle 12c Docker images and the Pure Storage Docker plugin as this post can be considered the ‘Return of the Jedi’  in the series.

Oracle 12c with Persistent Storage

In this post I will create a Docker container running an Oracle 12c database called PSTG using 4 mount points /u01, /u02, /u03 & /u04 mounted from a Pure Storage FlashArray.

Database environment Layout

/u01 – OraInventory and Oracle 12c product directories
/u02 – Oracle Datafile
/u03 – Fast Recovery Area
/u04 – RedoLog files.

The 4 mount points will all be using persistent All Flash Storage providing low latency performance, space efficient crash consistent snapshots and requiring no local storage.

First, confirm that you have installed the Oracle 12c image using the the following:

$ docker images list

docker_imagesYou should be able see the store/oracle/database-enterprise 12.1.0.2 Docker Image, if missing you need to read my previous post.

Ok, copy the db_env.dat file created in the previous post to a new environment file $ORACLE_SID_env.dat and update database name and password e.g.

####################################################################
## Copyright(c) Oracle Corporation 1998,2016. All rights reserved.##
##                                                                ##
##                   Docker OL7 db12c dat file                    ##
##                                                                ##
####################################################################
##------------------------------------------------------------------
## Specify the basic DB parameters
##------------------------------------------------------------------
## db sid (name)
## default : ORCL
## cannot be longer than 8 characters
DB_SID=PSTG
## db passwd
## default : Oracle
DB_PASSWD=Oracle
## db domain
## default : localdomain
DB_DOMAIN=localdomain
## db bundle
## default : basic
## valid : basic / high / extreme 
## (high and extreme are only available for enterprise edition)
DB_BUNDLE=basic
## end

We will now create the 4 volumes using the docker volume create command and Pure Storage Docker plug-in providing name, size and label.

 docker volume create --driver=pure --opt size=20GB --name=PSTG-u01 --label=product
 docker volume create --driver=pure --opt size=100GB --name=PSTG-u02 --label=oradata
 docker volume create --driver=pure --opt size=200GB --name=PSTG-u03 --label=fast_recovery_area
 docker volume create --driver=pure --opt size=10GB --name=PSTG-u04 --label=redo

We can use the docker volume ls command with a filter to list the 4 database volumes we are interested in using the following syntax

$ docker volume ls -f name=environment

docker_vols

The Pure Storage Docker Plugin also supports docker volume inspect, from here we can see the labels which we defined earlier, great way to see where and what the mount is being used for.

$ docker volume inspect volume name

docker_inspect.png

Now that we have installed the Oracle 12c Docker image and created the 4 volumes we are ready to create our Oracle 12c container using the docker run command with -v to present our newly volumes to our container.

docker run -d –env-file ./PSTG_env.dat -p 1527:1521 -p 5507:5500 -it –name PSTG -v PSTG-u01:/u01 -v PSTG-u02:/u02 -v PSTG-u03:/u03 -v PSTG-u04:/u04 –shm-size=”8g” store/oracle/database-enterprise:12.1.0.2

Progress can be monitored using the docker logs command.

docker logs -f PSTG

Once the container has started we can now connect to it and check out our Oracle 12c environment using the docker exec command and container name

docker exec -it PSTG /bin/bash

From within our Oracle 12c ‘PSTG’ container we can see our Pure Storage volumes (/u01, /u02, /u03 & /u04) and the sizes we specified.

df.png

Now we are in our Docker container we can use SQLPlus to see data file locations, instance name, docker assigned hostname etc…

sqlplus

instance

Docker Volume plugin for Pure Storage

You may have noticed that you can now download a Docker Volume Plugin for Pure Storage from the Docker Store. The Pure Storage Docker Volume plugin provides seamless integration with Docker containers, enabling persistent storage which is a must for many enterprise and database applications

Installation of the Docker Volume is extremely simple and is well documented at the Docker Store.

Once installed the first thing you must do is to let the Volume plugin know about your Pure FlashArray, update /etc/pure-docker-plugin/pure.json to use your FlashArray’s management (not iSCSI) IP address and API Token.

Starting pure-docker-plugin

You can now start the pure-docker-plugin using systemctl start e.g.

[root@z-oracle ~]#  systemctl start pure-docker-plugin

You can also check the log file using systemctl status e.g.

[root@z-oracle ~]#  systemctl status pure-docker-plugin -l

Create a Volume

You can create a Pure Storage FlashArray volume from your Linux server using the docker volume create command.

[root@z-oracle ~]# docker volume create –driver=pure -o size=100GB –name=ron101 –label=ron
ron101

You should now be able to see the newly created volume within the Pure Storage FlashArray.docke_FA

List a Volume

You can also use the docker volume ls command and filters to show available volumes.

[root@z-oracle ~]# docker volume ls -f name=ron
DRIVER              VOLUME NAME
pure                ron100
pure                ron101

Inspect a Volume

You can use the docker volume inspect command to get a bit more info and to also see where it’s being used.

[root@z-oracle ~]# docker volume inspect ron101
[
{
“Name”: “ron101”,
“Driver”: “pure”,
“Mountpoint”: “”,
“Labels”: {
“ron”: “”
},
“Scope”: “global”
}
]

When the Docker Volume is used the Mountpoint will be visible e.g.

[root@z-oracle ~]# docker volume inspect ron100
[
{
“Name”: “ron100”,
“Driver”: “pure”,
“Mountpoint”: “/pure-docker-mounts/z-oracle/ron100”,
“Labels”: {
“ron”: “”
},
“Scope”: “global”
}
]

Delete a Volume

The Pure Storage Volume plugin also supports volume delete.

[root@z-oracle ~]# docker volume rm ron101
ron101
[root@z-oracle ~]# docker volume ls -f name=ron
DRIVER              VOLUME NAME

In this Blog I have shared how we can create, inspect and delete Docker Volumes using the Pure Storage Docker Volume plug-in, in my next post I will share you you can use these volumes for persistent database storage.

Check-out the Pure Storage Developer Community for more examples.

Automate to avoid database cloning disasters.

“Accidentally destroyed production database on first day of a job”

Wow, that headline grabbed my attention.

Earlier this week you may have seen an article reported by The Register about a post in  reddit from a junior software developer going by the name of  “cscareerthrowaway567”, who on his/her first day destroyed a Production database and lost their job.

I was basically given a document detailing how to setup my local development environment. Which involves run a small script to create my own personal DB instance from some test data. After running the command i was supposed to copy the database url/password/username outputted by the command and configure my dev environment to point to that database. Unfortunately instead of copying the values outputted by the tool, i instead for whatever reason used the values the document had.

Unfortunately apparently those values were actually for the production database (why they are documented in the dev setup guide i have no idea). Then from my understanding that the tests add fake data, and clear existing data between test runs which basically cleared all the data from the production database. Honestly i had no idea what i did and it wasn’t about 30 or so minutes after did someone actually figure out/realize what i did.

Now, we can not be sure if the story by “cscareerthrowaway567” is true or not but it does provide an great example of the potential dangers of manual database cloning.

The story raises many issues around process, security, change management, training and not least why someone thought it was a good idea to include Production account details in a database cloning document ?

However, this could have easily been avoided through the use of Automation or a CDM (Copy Data Management) tools e.g. Catalogic, Actifio, Delphix etc..

Many modern All-Flash Arrays (AFA) provide powerful REST API’s which can be called from various programming and scripting languages e.g. Python, Java, Perl, PowerShell etc… to create custom solutions.

AFA Storage API’s can also be consumed by DevOPS Automation and Provisioning software e.g. Ansible, Chef, Puppet etc.. to provide full-stack or end-to-end Automation of Database Cloning, improving security and also removing the chance of human error.

Check out the Pure Storage Developer Community area for code examples.

Oracle 12c on Docker

Getting the Image

Great news, during DockerCon 2017 Oracle announced support for Oracle 12.1 Database Docker images. Press release: Oracle Brings Oracle’s Flagship Databases and Developer Tools to the Docker Store

Previously to run an Oracle Database in a Docker container you needed to create your own Docker image, I actually Blogged on this earlier in the year, if you interested you can read about it here.

However, you can now obtain an official Oracle 12.1 Database Docker images from the Docker Store

Before you can ‘pull’ the new Docker image you will need to login to the Docker Store and click on the ‘Get Content’ button and then complete the form, providing your name, company, phone number and then accepting the Oracle license agreement.

Docker_Store

The ‘Setup’ button provides details on how ‘pull’ the image and also how to set-up the your Oracle database within Docker.

docker pull store/oracle/database-enterprise:12.1.0.2

pull
The Docker image includes all the required Oracle files so is pretty large coming in at ~4.87GB, so it may take a while to download.

Running the Container

The environment file allows for customisation of the Database name, password, domain and feature bundle.

The command for creating a database container is:

$ docker run -d –env-file -p :1521 -p :5500 -it –name –shm-size=”4g”

Parameters:
<path_to_env_file> is the path to the environment file you created using above example.
<listener_port> is the port on host machine to map the container’s 1521 port (listener port).
<http_port> is the port on host machine to map the container’s 5500 port (http service port).
<container_name> is the container name you want to create.
<shared_memory_size> is the memory size for the container to run. The minimum requirement is 4GB (–shm-size=”4g”).
<image_name> is the image that you use to create a container.

In this example our environment file is called db_env.dat and our container is going to be called dockerDB

docker run -d –env-file ./db_env.dat -p 1527:1521 -p 5507:5500 -it –name dockerDB –shm-size=”8g” store/oracle/database-enterprise:12.1.0.2

We should now be able to see our new Oracle 12.1 image and running container using:

docker image ls
docker container ls

images

From my MacBook Pro we check the progress of the build using the Docker logs option e.g.

docker logs -f dockerDB

ready

We can also connect to the running container using the Docker exec command e.g.

docker exec -it dockerDB /bin/bash

oracle.png
From here we can see the Oracle Docker is using Oracle Linux 7.3 and the database is called ‘OraDoc’ as per our environment file.

As you can see from the above we are connected as root, if you want to use the oracle account then type ‘su – oracle’ and from there try out sqlplus inside your container.

database

Oracle 12c on an MBP Done!