Per apspera ad adstra
Jump directly to

iSCSI - solving laptop woes


The Problem

Previously, I created an iSCSI server, created a small storage device and used that device. Without thinking, I used the IP address of the laptops wired interface (eth0 = 10.0.0.142) to connect. That causes problems, because
  • sometimes I connect the laptop to my wired network and its address may get reassigned to another address in the 10.0.0.xxx range.
  • there are now two interfaces with an 10.0.0.xxx address (wlan0 and eth0) and both may claim to reach my gateway (10.0.0.138). If eth0 wins and I then unplug the cord, networking stops. Even with a connected wlan0.
  • I can solve that by bringing eth0 down, but then the iSCSI-client will fail to connect, filling up log.
The solution is to create a new virtual network on my laptop that handles the iSCSI traffic. Such a network is necessary anyway when I actually start adding virtual machines that use the iSCSI devices.

Create a Virtual Storage Network Segment

Define an extra bridge to be created at startup be editing /etc/rc.d/rc.inet1.conf. Copy and edit the example bridge portion:

# Virtual bridge for internal iSCSI SAN server.
IFNAME[0]="br0"
BRNICS[0]="eth0"
IPADDR[0]="192.168.4.1"
NETMASK[0]="255.255.255.0"
USE_DHCP[0]="no"
DHCP_HOSTNAME[0]=""

Activate it by restarting the inetd process:

$ /etc/rc.d/rc.inet1 restart

Some Virtual Rewiring

Now, the iSCSI server has to be rewired, so it is on the new 192.168.4.0/24 storage network and the iSCSI client has to be rewired so it picks up the extra device there.

Shut down the client and remove the portal from the scsiadm database.

$ iscsiadm -m node -U all
Logging out of session [sid: 1,
                        target: iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc, 
                        portal: 10.0.0.142,3260]
Logout of [sid: 1, 
           target: iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc, 
           portal: 10.0.0.142,3260] successful.

$ iscsiadm -m discoverydb -t st -p 10.0.0.142 -o delete

In the SAN server, redefine the Target Portal Group of the iSCSI volume to expose it on a different IP address.

$ targetcli
targetcli GIT_VERSION (rtslib GIT_VERSION)
Copyright (c) 2011 by RisingTide Systems LLC.
All rights reserved.

/> cd iscsi/iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc/portal
/iscsi/iqn.20...tpgt1/portals> ls
o- portals .............................................. [1 Portal]
  o- 10.0.0.142:3260 ........................... [OK, iser disabled]

/iscsi/iqn.20...tpgt1/portals> delete 10.0.0.142 3260
Deleted network portal 10.0.0.142:3260

/iscsi/iqn.20...tpgt1/portals> create 192.168.4.1 3260
Successfully created network portal 192.168.4.1:3260. 

/iscsi/iqn.20...tpgt1/portals> cd /
/> saveconfig
/> exit

(Re)discover the devices on the new portal address and log in:

$ iscsiadm -m discovery -t st -p 192.168.4.1
192.168.4.1:3260,1 iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc

$ iscsiadm -m node -L all
Logging in to [iface: default,
               target: iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc,
               portal: 192.168.4.1,3260] (multiple)
Login to [iface: default,
          target: iqn.2003-01.org.linux-iscsi.henk.x8664:sn.52a0ccf13ebc,
          portal: 192.168.4.1,3260] successful.

Rewiring done.


The Blooper Reel #1: iscsiadm -o delete

It all looks so easy, doesn't it? Once you understand the details of how stuff interacts (and see the commands), it is. Until then, not so much. Here's me trying to clear the old device from the iscsiadm client (after redefining the iSCSI server portal and discovering the iSCSI device on the new address):

$ iscsiadm -m discovery -S
192.168.4.2:3260 via sendtargets
10.0.0.142:3260 via sendtargets

$ iscsiadm -m discovery -t st -p 10.0.0.142 -o delete
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: cannot make connection to 10.0.0.142: Connection refused
iscsiadm: connection login retries (reopen_max) 5 exceeded
iscsiadm: Could not perform SendTargets discovery: encountered connection failure

The above doesn't work. The iSCSI server no longer has a portal on that address and can't tell the client that the device no longer exists. You have to remove the target from the iscsiadm database using "-m discoverydb -o delete".

The Blooper Reel #2: using virsh to create storage network

Here's me explaining how to create a virtual network to be used when using virtual machines. The solution below works, but during boot, it takes a few seconds for the network to show up after libvirtd is started. But I need the network to start the SAN server and iSCSI client that are started in the same rc.local script.
It is much easier to define a separate bridge. Plus there's no NAT or dhcp server needed for that network (not even sure I need a bridge, to be honest).

At least I got to use virsh a little..

First, create a new virtual network. Create an xml image of the default one, by typing

$ virsh net-dumpxml default > /home/hans/storage.xml

and editing the result to this:

<network>
  <name>storage
  <uuid>07599acf-7617-afee-83eb-992cf682282d</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr1' stp='on' delay='0' />
  <mac address='52:54:00:94:54:75'/>
  <ip address='192.168.4.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.4.10' end='192.168.4.20' />
    </dhcp>
  </ip>
</network>

The storage network doesn't really need a dhcp server: I'll be using fixed addresses everytime. There is also no need for a NAT firewall: there should be no comunication to the outside over this segment.
Both can be solved. Can't be bothered right now.

Now, load the definition to create a persistent virtual network:

$ virsh
virsh # net-define /home/hans/storage.xml
Network storage defined from /home/hans/storage.xml

Then, start it

virsh # net-start storage
Network storage started

And make sure it starts on-boot

virsh # net-autostart storage
Network storage marked as autostarted

virsh # net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 storage              active     yes           yes

Notes

  • You can also create the network using "net-create /home/hans/storage.xml". This creates a transient network. I can't get virsh to change that into a persistent one.
  • You might also use virt-manager to create an extra virtual network.