Wednesday, October 13, 2010

NexentaStor - iSCSI example

NexentaStor community edition has some limitations on the iSCSI configurations (GUI or CLI interface)

Most of them can be addressed without switching to alternative solution or to the Enterprise edition, especially if the storage is already in use and downtime better be avoided.

Here is an example on the setup of iSCSI volume shared between 2 hosts.
It can be used for clustered file system or as a volume for Oracle database (ready for a RAC implementation).
Obviously, you can apply similar rules for different scenarios and network topologies.

Storage network topology:

In my setup both Host1 and Host2 are running Solaris 10.
Two fully independent switches were used to provide redundancy and additional throughput.

Configure iSCSI initiator on both hosts:
  • Host1
    • Initiator node name:
  • Host2
    • Initiator node name:
On the storage server nexenta-ce ( in GUI or CLI )
  • Create target point group ( if it is still not created)
    • TPG1 (,
  • Create volume
    • zVol ( data/shaedvolume10 )
  • create 2 targets
Nexenta-ce(root shell):

Keep in mind that it is not recommended by Nexenta

nmc@nexenta-ce:/$ option expert_mode =1 -s nmc@nexenta-ce:/$ !bash root@nexenta-ce#
Targets must be offline when you are creating target group:
root@nexenta-ce# stmfadm create-tg tg10 root@nexenta-ce# stmfadm offline-target root@nexenta-ce# stmfadm offline-target root@nexenta-ce# stmfadm add-tg-member -g tg10 root@nexenta-ce# stmfadm add-tg-member -g tg10 root@nexenta-ce# stmfadm online-target root@nexenta-ce# stmfadm online-target
Let's create a host group which includes both host1 and host2:
root@nexenta-ce# stmfadm create-hg host1 root@nexenta-ce# stmfadm create-hg host2 root@nexenta-ce# stmfadm add-hg-member -g host1 root@nexenta-ce# stmfadm add-hg-member -g host2 root@nexenta-ce# sbdadm create-lu /dev/zvol/rdsk/data/tg10 root@nexenta-ce# sbdadm list-lu GUID DATA SIZE SOURCE --------------------------- ------------------- ---------------- 600XXXXXXX0001 214748364800 /dev/zvol/rdsk/data/tg10 root@nexenta-ce# stmfadm add-view -t tg10 -h host1 600XXXXXXX0001 root@nexenta-ce# stmfadm add-view -t tg10 -h host2 600XXXXXXX0001

Both host1 and host2 (root shell)
In current setup I'm using only static discovery.

root@host1# iscsiadm modify discovery -s enable root@host1# iscsiadm add static-config, root@host1# iscsiadm add static-config, root@host1# iscsiadm list target Target: .... Target: .... root@host1# devfsadm -Cv root@host1# echo | format

Result - the created volume is ready for operations on both hosts plus high availability on the network layer.

No comments: