Adding new devices (VMware) to Linux without re-booting server

 

Background

Today, I found myself needing to add 3 new disks to one of my Oracle database server VMware VM’s, unfortunately I was unable to re-boot the server so I followed the steps below to add the new devices.

Create LUN’s using your storage platform and add add them to your Virtual Machine using your preferred  vSphere Client.

newDevices
VMware vSphere Web Client – 3 new devices highlighted

Use yum to install sg3_utils

[root@z-oracle1 ~]# yum install sg3_utils

Use the Linux ‘fdisk’ command to list current devices

[root@z-oracle1 dev]# fdisk -l

Rescan SCSI bus using the rescan-sci-bus.sh script.

[root@z-oracle1 dev]# rescan-scsi-bus.sh
Scanning SCSI subsystem for new devices
Scanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 0 0 0 0 …
OLD: Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: NECVMWar Model: VMware IDE CDR00 Rev: 1.00
Type: CD-ROM ANSI SCSI revision: 05
Scanning host 1 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning host 2 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNs
Scanning for device 2 0 0 0 …
OLD: Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: VMware Model: Virtual disk Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 02
Scanning for device 2 0 1 0 …

Use ‘fdisk’ to identify new device names

[root@z-oracle1 dev]# fdisk -l

Disk /dev/sdi: 1099.5 GB, 1099511627776 bytes, 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes

Disk /dev/sdj: 3298.5 GB, 3298534883328 bytes, 6442450944 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes

Disk /dev/sdk: 1099.5 GB, 1099511627776 bytes, 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes

We should now be able to see the device in /dev.

[root@z-oracle1 dev]# ls -l /dev/sd*
brw-rw—-. 1 root disk 8, 0 May 14 14:14 /dev/sda
brw-rw—-. 1 root disk 8, 1 May 14 14:14 /dev/sda1
brw-rw—-. 1 root disk 8, 2 May 14 14:14 /dev/sda2
brw-rw—-. 1 root disk 8, 16 May 14 14:14 /dev/sdb
brw-rw—-. 1 root disk 8, 32 Jul 23 14:31 /dev/sdc
brw-rw—-. 1 root disk 8, 48 Jul 23 14:30 /dev/sdd
brw-rw—-. 1 root disk 8, 64 Jul 23 14:31 /dev/sde
brw-rw—-. 1 root disk 8, 80 Jul 9 21:02 /dev/sdf
brw-rw—-. 1 root disk 8, 96 Jul 9 21:02 /dev/sdg
brw-rw—-. 1 root disk 8, 112 Jul 9 21:02 /dev/sdh
brw-rw—-. 1 root disk 8, 128 Jul 24 10:46 /dev/sdi
brw-rw—-. 1 root disk 8, 144 Jul 24 10:46 /dev/sdj
brw-rw—-. 1 root disk 8, 160 Jul 24 10:46 /dev/sdk

We are now ready to partition our new disks using ‘fdisk’ (n = new p = print w = write).

[root@z-oracle1 dev]# fdisk /dev/sdi
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x1fea2f04.

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (8192-2147483647, default 8192):
Using default value 8192
Last sector, +sectors or +size{K,M,G} (8192-2147483647, default 2147483647):
Using default value 2147483647
Partition 1 of type Linux and of size 1024 GiB is set

Command (m for help): p

Disk /dev/sdi: 1099.5 GB, 1099511627776 bytes, 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes
Disk label type: dos
Disk identifier: 0x1fea2f04

Device Boot Start End Blocks Id System
/dev/sdi1 8192 2147483647 1073737728 83 Linux
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

I then repeated for ‘/dev/sdj’ & ‘/dev/sdk’ and now ready to format the devices e.g. mkfs.ext4 /dev/sdi

Advertisements

Oracle Linux Container Registry

You may have read one of my previous posts Oracle 12c on Docker where I described  how you can get hold of official Oracle images from the Docker Store.

In this post I will share how you can now use the Oracle Container Registry to obtain Docker images for all licensable Oracle products.

To access the Oracle Registry Server you must have an Oracle Single Sign-On (SSO) account, this is the same account you use for support.oracle.com and opn.oracle.com etc..

Navigate to https://container-registry.oracle.com/ and login using your SSO credentials, once authenticated you will be presented with the ‘Explore’ webpage.
Oracle_Container_Registry

You can then need to select the required Business Area e.g. ‘Database’ and click the red ‘Continue’ button, you will then be presented with the ‘Oracle Standard Terms and Restrictions’ agreement, review and if acceptable click ‘Accept’ and the bottom of the page.

You will then be returned to the ‘Explorer’ and notice that the Oracle Standard Terms and Restrictions shows you acceptance, this is valid for 8 hours only.

Standard_Terms

Note: you will not be able to pull and image until you have accepted Oracle Standard Terms and Restrictions.

You are now ready to pull the required image by clicking the repository  e.g. ‘enterprise’ and copying & pasting the provided pull command e.g.

pull_command
The Repository Detail page provides set-up and usage information for the selected image.

Ok, now I have outlined the process let me walk you through it.

1) Login to Registry

ronsmac:~ ronekins$ docker login container-registry.oracle.com
Username: ron.ekins@icloud.com
Password:
Login Succeeded

2a) Pull Oracle Linux

Select ‘OS’ from the ‘Explore Official Business Areas’  and click the red ‘Continue’ button for oraclelinux. As before, accept Oracle Terms.

Repo_Linux

Now copy the docker pull command and paste in in your Mac or Linux shell.

OC_Reg

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/os/oraclelinux
Using default tag: latest
latest: Pulling from os/oraclelinux
80d2e45a33d8: Pull complete
Digest: sha256:d31c2987a6c427eeca70fb28ccefd6e788e96b3202dc715aa3c80bcb23673f6d
Status: Downloaded newer image for container-registry.oracle.com/os/oraclelinux:latest

2b) Pull Oracle Database 12c Enterprise Edition

Register_Database

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/database/enterprise
Using default tag: latest
latest: Pulling from database/enterprise
cbb9821ba51c: Pull complete
9bd4d110366e: Pull complete
af8b29651e27: Pull complete
4c242ab1add4: Pull complete
7bda1e55bd08: Pull complete
Digest: sha256:42809e491491d7f07a2aa76903bb5feabe3a0d23abcb7e680264074f043a604c
Status: Downloaded newer image for container-registry.oracle.com/database/enterprise:latest

3) Check Image size

Your Oracle two images are now available for use locally, you can check this with the docker image ls command. e.g.
ronsmac:~ ronekins$ docker image ls

REPOSITORY TAG IMAGE ID CREATED SIZE
container-registry.oracle.com/os/oraclelinux latest 6c33a25f4a29 5 weeks ago 229MB
container-registry.oracle.com/database/enterprise latest 12a359cd0528 2 months ago 3.44GB

Follow-up ‘Registry’ teaser

Within the ‘OS’ area Oracle provides a registry image which you can use to deploy a local registry so you can pull images and customise as required, the local registry also avoids the need the to authenticate every 8 hours, I will detail all of this in a follow-up post.

ronsmac:~ ronekins$ docker pull container-registry.oracle.com/os/registry
Using default tag: latest
latest: Pulling from os/registry
a3ed95caeb02: Pull complete
89937cfc6593: Pull complete
bd07ebf08156: Pull complete
Digest: sha256:13d190c8838ebeb1e9cbf87b3edcc1fc6b6948d1b5d2200ec4dc64c638a56402
Status: Downloaded newer image for container-registry.oracle.com/os/registry:latest

 

How to resize an XFS filesystem

A question which I frequently get asked is, how do I a resize my Oracle XFS file system ?

As I needed to resize an Oracle FRA area today, I thought this would make a great topic for a Blog post.

Ok, lets start be checking the current size and geometry using the Linux df -h and xfs_growfs -n commands.

[root@z-oracle ~]# df -h
Filesystem                Size  Used Avail Use% Mounted on
..
/dev/mapper/psta-orafra   1.0T   33M  1.0T   1% /u04/app/oracle/fast_recovery_area
..

[root@z-oracle ~]# xfs_growfs /dev/mapper/psta-orafra -n
meta-data=/dev/mapper/psta-orafra isize=256 agcount=4, agsize=67108864 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0 spinodes=0
data = bsize=4096 blocks=268435456, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal bsize=4096 blocks=131072, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

Now run the multipath command to check the current size of the LUN,  look for the friendly device name within the list of devices.

[root@z-oracle ~]# multipath -ll

psta-orafra (3624a937050c939582b0f46c0000a8f84) dm-17 PURE ,FlashArray
size=1.0T features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’queue-length 0′ prio=1 status=active
|- 10:0:3:20 sdgl 132:16 active ready running
|- 10:0:4:20 sdiq 135:160 active ready running
|- 10:0:5:20 sdkq 66:480 active ready running
|- 10:0:6:20 sdmo 70:256 active ready running
|- 10:0:7:20 sdom 129:288 active ready running
|- 1:0:0:20 sdai 66:32 active ready running
|- 1:0:1:20 sdcr 69:240 active ready running
|- 1:0:2:20 sder 129:48 active ready running
|- 1:0:3:20 sdgr 132:112 active ready running
|- 1:0:4:20 sdil 135:80 active ready running
|- 1:0:5:20 sdkk 66:384 active ready running
|- 1:0:6:20 sdmk 69:448 active ready running
|- 1:0:7:20 sdop 129:336 active ready running
|- 10:0:0:20 sdau 66:224 active ready running
|- 10:0:1:20 sdck 69:128 active ready running
`- 10:0:2:20 sdel 128:208 active ready running

Now resize the volume using the Pure FlashArray UI, command line or REST API.

Screen Shot 2017-05-11 at 10.44.03

We now need to perform a rescan of the SCSI devices on our Linux server to identify any LUNS which have been resized.

[root@z-oracle ~]# rescan-scsi-bus.sh -s
Scanning SCSI subsystem for new devices
Searching for resized LUNs

We can now resize the multipath device using the following command:

[root@z-oracle mapper]# multipathd -k’resize map /dev/dm-17
ok

Great, ok now use the Linux command xfs_growfs to extend the file system, note if you do not specify -D xfs_growfs will grow to use all available space.

[root@z-oracle ~]# xfs_growfs /dev/mapper/psta-orafra
meta-data=/dev/mapper/psta-orafra isize=256 agcount=4, agsize=67108864 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0 spinodes=0
data = bsize=4096 blocks=268435456, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal bsize=4096 blocks=131072, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 268435456 to 536870912

Ok, lets check the results

[root@z-oracle ~]# df -h
Filesystem Size Used Avail Use% Mounted on

/dev/mapper/psta-orafra 2.0T 33M 2.0T 1% /u04/app/oracle/fast_recovery_area
..

As you see from above, within a few minutes I have been able to increase my Oracle fast_recovery_area from 1TB to 2TB.

Resizing Oracle ASM disks

Today I though I would share with how easy it is to resize Oracle ASM volumes with Pure Storage.

Ok, lets first check the Oracle ASM disk sizes using ‘asmcmd -p’

asmcli_pre.png

As you can see from the above, I have 3 volumes each of 100GB, for this test let’s increase them all to 1TB using the purevol command from with the CLI.

Pure_resize.png

Great, my volumes have now al been resized, I could have achieved the same results with the Pure UI or Web Services, but that’s something for another day.

Linux device rescan

Ok, we now need to let Linux and Oracle know about our resized volumes.

As root rescan the SCSI devices to identify which volumes which have been resized using:

rescan-scsi-bus.sh -s

Use ‘multipathd -k ‘resize map ‘ to resize the multipath devices e.g.

multipathd -k'resize map slob-data'

Before you moving onto resizing the Oracle ASM disk groups check your updated multi path configuration with ‘multipath -ll’, look for your device name and size e.g.

slob-data (3624a937050c939582b0f46c000059779) dm-5 PURE,FlashArray      size=1.0T features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=active
...

Oracle ASM resize

As your ‘grid’ user connect as sysasm from sqlplus e.g. sqlplus / as sysasm and perform ‘alter disk group <dg_name> resize all’

sqlplus_resize

Great, job done, but before we more on let’s check out work first using sqlplus as sysasm

SQL>  select name, total_mb/(1024) "Total GiB" from v$asm_diskgroup;
NAME                           Total GiB
------------------------------ ----------
CONTROL_REDO                   1024
FRA                            1024
DATA                           1024

And, now with the ASM command line utility ‘asmcmd’

asmcli_post.png

Or if you prefer the ASM UI ‘asmca’.

asmca.png

Installing Linux Package Groups

Installing Package Groups

I came across an Oracle Linux 7.2 server this week that was missing some required prerequisite packages for my Oracle Database install.

This was easily fixed with yum and package groups, package groups allows you to install a group of dependent packages in one go rather than individually.

A package group has a name a groupid which you can determine with the yum grouplist -v command which lists all package groups.

~]# yum -v grouplist Compatibility\*
Not loading "rhnplugin" plugin, as it is disabled
Loading "ulninfo" plugin
Loading "langpacks" plugin
Adding en_GB.UTF-8 to language list
Config time: 0.023
Yum version: 3.4.3
Setting up Package Sacks
pkgsack time: 0.006
group time: 0.042
Available Groups:
   Compatibility Libraries (compat-libraries)
Done

You can now use the yum groupinstall command to install the package group.

~]# yum groupinstall compat-libraries

Alternatively, you can use the package group name in quotes.

~]# yum groupinstall "Compatibility Libraries"

Or if you prefer, you can use the @ which tells yum you want to use groupinstall 

~]# yum group @compat-libraries

Using udev rules with OL / RH7 on VMware Fusion

Preparing to use udev rules on VMware Fusion

Before you can use udev rules with your new VMware Hard Drives you need to update the Virtual Machines .vmx file and set the following.

disk.EnableUUID = “TRUE”

Without this setting the ‘/usr/lib/udev/scsci_id’ command will no values.

As with all configuration files it’s best practice to create a backup before editing, however this is especially true for the VMware Fusion .vmx file as any mistake can result in a Virtual Machine which will no longer boot.

Shutdown your Virtual Machine before you attempt any edits of the .vmx file.

Creating the udev rules

Once your edit is complete you can reboot your VM and confirm is working ok with ‘usr/lib/udev/scsci_id -g -u -d /dev/’ e.g.

[root@ol72 ~]# /usr/lib/udev/scsi_id –verbose -g -u -d /dev/sdc
36000c298d0c4d6c7ace0c7b990069f48
[root@ol72 ~]# /usr/lib/udev/scsi_id –verbose -g -u -d /dev/sdd
36000c290045c8d9e8660057e191ec523
[root@ol72 ~]# /usr/lib/udev/scsi_id –verbose -g -u -d /dev/sde
36000c29b6dc77c32663c5b8354fffd37
[root@ol72 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sde
36000c29b6dc77c32663c5b8354fffd37

Now you know the Disks unique ID’s you can add them to a new udev rule. e.g.

/etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL==”sd?1″, SUBSYSTEM==”block”, PROGRAM==”/usr/lib/udev/scsi_id -g -u -d /dev/$parent”, RESULT==”36000c298d0c4d6c7ace0c7b990069f48″, SYMLINK+=”dg_prod_oradata1″, OWNER=”grid”, GROUP=”asmadmin”, MODE=”0660″

KERNEL==”sd?1″, SUBSYSTEM==”block”, PROGRAM==”/usr/lib/udev/scsi_id -g -u -d /dev/$parent”, RESULT==”36000c290045c8d9e8660057e191ec523″, SYMLINK+=”dg_prod_oraredo1″, OWNER=”grid”, GROUP=”asmadmin”, MODE=”0660″

KERNEL==”sd?1″, SUBSYSTEM==”block”, PROGRAM==”/usr/lib/udev/scsi_id -g -u -d /dev/$parent”, RESULT==”36000c29b6dc77c32663c5b8354fffd37″, SYMLINK+=”dg_prod_orafra1″, OWNER=”grid”, GROUP=”asmadmin”, MODE=”0660″

Unlike RedHat 5 & 6 there is no need to restart the udev service on RedHat 7

Testing the udev rules

You can now test the udev rules using udevadm test e.g.

udevadm test /block/sdb/sdb1

And check the results with ls -l /dev/

[root@ol72 dev]# ls -l /dev/sd*
brw-rw—- 1 root disk 8, 32 Oct 18 11:29 sdc
brw-rw—- 1 grid asmadmin 8, 33 Oct 18 11:49 sdc1
brw-rw—- 1 root disk 8, 48 Oct 18 11:29 sdd
brw-rw—- 1 grid asmadmin 8, 49 Oct 18 11:55 sdd1
brw-rw—- 1 root disk 8, 64 Oct 18 11:29 sde
brw-rw—- 1 grid asmadmin 8, 65 Oct 18 11:55 sde1

[root@ol72 dev]# ls -l /dev/dg*
lrwxrwxrwx 1 root root 4 Oct 18 11:49 /dev/dg_prod_oradata1 -> sdc1
lrwxrwxrwx 1 root root 4 Oct 18 11:55 /dev/dg_prod_orafra1 -> sde1
lrwxrwxrwx 1 root root 4 Oct 18 11:55 /dev/dg_prod_oraredo1 -> sdd1

Oracle Linux 7.2 install on VMware Fusion 8

Background

I have seen several great Linux install guides for Oracle’s free VirtualBox  but have never come across a version specifically for VMware Fusion, so here goes lets’s put that right.

What is VMware Fusion ?

VMware Fusion is a sofware hypervisor for Mac OS X , it’s VMware’s Mac equivalent of VMware Workstation for Windows and Linux.

Please note these are licenceable products unlike VirtualBox, so you will need to either start a free trial of buy a licence before you use VMware Fusion.

VMware Fusion provides support for big VM’s and some useful integration between your Mac and Windows VM’s. If your a PC user you may have also come across VMware Workstation Player , this is free for personal use and is avaible for  both Windows and Linux,  Workstation Player is able to run VM’s created with either VMware Workstation Pro or VMware Fusion Pro and is a great way of sharing restricted or expiring VM’s, unfortantely there is no Mac OS X version as of yet.

Parallels Desktop is another widley used virtualisation product on the Mac, Parallels Desktop Pro now includes direct support for Docker, Vagrant, Chef and Jenkins so that may provide a source of intresting future Blog.

VMware Fusion

Ok, lets kick this off by starting VMware Fusion.

images-2

Navigate to ‘File’ and select ‘New’

Click ‘Continue’ to use a previoulsy downloaded image.

Install Method

Select the required ISO, if you have previoulsy used an image Fusion will show it otherwise you will need to navigate to it.

Create new VM

Click ‘Customize Settings’ to allow you to change the defaults.

Finish

Give your Virtual Machine a file name ‘Save As’

Navigate

You can now change the initial VM configuration e.g. add additional Network Interfaces, Hard Disks and chnage memory and processor sizes.

Select ‘Processors & Memory’

Settings

In this example I have increased the Memory to 4GB from 1GB and changed the Processor core count. Once complet click ‘Show All’ to return to the main configuration screen.

CPU

Select ‘Hard Disk (SCSI’), here you can add additional devices these will be presented as /dev/sdb, /dev/sdc, /dev/sdd etc… Here I have upped the /dev/sda from 20GB to 40GB.

hard disk

Use ‘Show All’ to return and click the large play button to launch the Oracle Linux installer.

Oracle Linux 7.2 Installer

Click hit ‘I’ to start the install ant the Oracle Linux 7.2 splash screen.

Install Splash

Select language, here I have selected ‘English (United Kingdom)’ this automatically updates the keyboard, timezone and other regional settings.

Language

Installation Summary

The ‘Installation Summary’ guide you through the install requiring the completion of items maked with an exclamation mark before allowing you to proceed.

Summary

Installation Destination

Ok, lets confirm our ‘Installation Destination’

Here we can see out VMware 40GB Virtual Disk we allocated earlier, click ‘Done’ to let the installer use the whole disk.

Install dest

Software Selection

As you can see below my keyboard has changed to ‘gb’, now lets select our packages.

Base Enviroment

  • Server with GUI

Add-Ons for Selected Enviroment

  • Compatibility Libraries
  • Development Tools

Software Selection

 

Create User

We now need to set the ‘Root’ password create out first user account, select ‘Make this user administrator’

Create User

Set Root password

Once the installer is complete click ‘Reboot’ when prompted and accept the licence.

You will now be able to logon using the user account just created, accept language and input sources. You will be offered the chance to configure Cloud storage, click next and your ready to start configuring and using your new Linux 7.2 environment.

Next steps should include, configure network, firewall, yum updates, additional packages and user accounts etc..