通过安装Openfiler或者FreeNAS软件，我们可以很方便地将闲置的PC机变废为宝为存储服务器，其中还提供了ISCSI TARGET服务。但很多情况下我们用不到Openfiler的那么多功能(包括NFS,CIFS等等)，同时又希望这台”存储服务器”能扮演更多的角色(例如作为DataGurad的备库或者作为Netbackup备份服务器)，这时Openfiler就无法满足我们了。何不手动来搭建能够基本满足需求的ISCSI服务呢？
/* 首先我们需要一台已安装了RHEL5或者OEL5的服务器，同时已安装了scsi-target-utils包 */
[root@rh3 dsk]# hostname
rh3.oracle.com
[root@rh3 ~]# rpm -qa|grep scsi-target-utils
scsi-target-utils-0.0-6.20091205snap.el5_4.1
/* 创建ISCSI服务映射的磁盘文件 */
[root@rh3 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 48G 3.5G 42G 8% /
/dev/sda2 95G 188M 90G 1% /s01
tmpfs 1.3G 0 1.3G 0% /dev/shm
none 1.3G 104K 1.3G 1% /var/lib/xenstored
/dev/sda4 145G 21G 117G 15% /dsk
[root@rh3 ~]# for i in 1 2 3 4 5 6 7 8 9 10
> do
> dd if=/dev/zero of=/dsk/disk$i.dat bs=1024k count=2048
> done
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 14.9789 seconds, 143 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 28.8623 seconds, 74.4 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 29.6608 seconds, 72.4 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 36.735 seconds, 58.5 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 28.9802 seconds, 74.1 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 28.5747 seconds, 75.2 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 37.6903 seconds, 57.0 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 26.4861 seconds, 81.1 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 38.151 seconds, 56.3 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 31.2119 seconds, 68.8 MB/s
/* 以上在/dsk目录下建立了10个大小为2G的磁盘文件 */
[root@rh3 ~]# service tgtd start
Starting SCSI target daemon: Starting target framework daemon
[root@rh3 dsk]# chkconfig tgtd on
/* 新建iscsi target */
[root@rh3 ~]# tgtadm --lld iscsi --op new --mode target --tid 1 -T iqn.2010-10.com.rh-cluster.nas
/* 启动相关tgtd 服务 */
[root@rh3 dsk]# tgtadm --lld iscsi -o show -m target
Target 1: iqn.2010-010.com.rh-cluster.nas
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
Account information:
ACL information:
/* 进一步验证TARGET添加成功 */
[root@rh3 dsk]# for i in 1 2 3 4 5 6 7 8 9 10
> do
> tgtadm --lld iscsi -o new -m logicalunit --tid 1 --lun $i -b /dsk/disk$i.dat
> done
/* 将10个虚拟磁盘添加到TARGET */
[root@rh3 dsk]# tgtadm --lld iscsi -o show -m target
Target 1: iqn.2010-010.com.rh-cluster.nas
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk1.dat
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk2.dat
LUN: 3
Type: disk
SCSI ID: IET 00010003
SCSI SN: beaf13
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk3.dat
LUN: 4
Type: disk
SCSI ID: IET 00010004
SCSI SN: beaf14
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk4.dat
LUN: 5
Type: disk
SCSI ID: IET 00010005
SCSI SN: beaf15
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk5.dat
LUN: 6
Type: disk
SCSI ID: IET 00010006
SCSI SN: beaf16
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk6.dat
LUN: 7
Type: disk
SCSI ID: IET 00010007
SCSI SN: beaf17
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk7.dat
LUN: 8
Type: disk
SCSI ID: IET 00010008
SCSI SN: beaf18
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk8.dat
LUN: 9
Type: disk
SCSI ID: IET 00010009
SCSI SN: beaf19
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk9.dat
LUN: 10
Type: disk
SCSI ID: IET 0001000a
SCSI SN: beaf110
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /dsk/disk10.dat
Account information:
ACL information:
/* 验证磁盘是否正确添加 */
[root@rh3 dsk]# tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
/* 绑定ip地址为ALL，即所有主机均可访问该ISCSI资源 */
/* 至此iscsi的server端配置完成，我们切换到客户端主机上进行配置 */
[root@rh2 ~]# hostname
rh2
[root@rh2 ~]# rpm -qa|grep iscsi
iscsi-initiator-utils-6.2.0.871-0.16.el5
/* 要求客户端主机上已安装iscsi-initiator-utils rpm包*/
[root@rh2 iscsi]# service iscsi start
iscsid 已死，但 pid 文件仍存
Starting iSCSI daemon: [确定]
[确定]
设置 iSCSI 目标：iscsiadm: No records found!
[确定]
/* 启动iscsi服务 */
[root@rh2 iscsi]# iscsiadm -m discovery -t sendtargets -p rh3.oracle.com
192.168.1.108:3260,1 iqn.2010-010.com.rh-cluster.nas
/* 在客户端尝试发现远程主机上的iscsi target */
[root@rh2 iscsi]# service iscsi restart
Stopping iSCSI daemon:
iscsid 已死，但 pid 文件仍存 [确定]
Starting iSCSI daemon: [确定]
[确定]
设置 iSCSI 目标：Logging in to [iface: default, target: iqn.2010-010.com.rh-cluster.nas, portal: 192.168.1.108,3260]
Login to [iface: default, target: iqn.2010-010.com.rh-cluster.nas, portal: 192.168.1.108,3260]: successful
[确定]
/* 重启本地的iscsi服务，该服务正确登录到了rh3.oracle.com 3260端口上的iscsi target */
[root@rh2 ~]# tail -88 /var/log/messages
Oct 25 20:24:35 rh2 kernel: sde: Write Protect is off
Oct 25 20:24:35 rh2 kernel: SCSI device sde: drive cache: write back
Oct 25 20:24:35 rh2 kernel: SCSI device sde: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:35 rh2 kernel: sde: Write Protect is off
Oct 25 20:24:35 rh2 kernel: SCSI device sde: drive cache: write back
Oct 25 20:24:35 rh2 kernel: sde: unknown partition table
Oct 25 20:24:35 rh2 kernel: sd 5:0:0:3: Attached scsi disk sde
Oct 25 20:24:35 rh2 kernel: sd 5:0:0:3: Attached scsi generic sg5 type 0
Oct 25 20:24:35 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:35 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:35 rh2 kernel: SCSI device sdf: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:35 rh2 kernel: sdf: Write Protect is off
Oct 25 20:24:35 rh2 kernel: SCSI device sdf: drive cache: write back
Oct 25 20:24:35 rh2 kernel: SCSI device sdf: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:35 rh2 kernel: sdf: Write Protect is off
Oct 25 20:24:35 rh2 kernel: SCSI device sdf: drive cache: write back
Oct 25 20:24:35 rh2 kernel: sdf: unknown partition table
Oct 25 20:24:35 rh2 kernel: sd 5:0:0:4: Attached scsi disk sdf
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:4: Attached scsi generic sg6 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdg: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdg: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdg: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdg: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdg: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdg: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdg: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:5: Attached scsi disk sdg
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:5: Attached scsi generic sg7 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdh: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdh: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdh: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdh: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdh: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdh: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdh: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:6: Attached scsi disk sdh
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:6: Attached scsi generic sg8 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdi: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdi: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdi: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdi: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdi: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdi: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdi: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:7: Attached scsi disk sdi
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:7: Attached scsi generic sg9 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdj: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdj: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdj: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdj: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdj: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdj: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdj: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:8: Attached scsi disk sdj
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:8: Attached scsi generic sg10 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdk: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdk: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdk: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdk: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdk: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdk: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdk: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:9: Attached scsi disk sdk
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:9: Attached scsi generic sg11 type 0
Oct 25 20:24:36 rh2 kernel: Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Oct 25 20:24:36 rh2 kernel: Type: Direct-Access ANSI SCSI revision: 05
Oct 25 20:24:36 rh2 kernel: SCSI device sdl: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdl: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdl: drive cache: write back
Oct 25 20:24:36 rh2 kernel: SCSI device sdl: 4194304 512-byte hdwr sectors (2147 MB)
Oct 25 20:24:36 rh2 kernel: sdl: Write Protect is off
Oct 25 20:24:36 rh2 kernel: SCSI device sdl: drive cache: write back
Oct 25 20:24:36 rh2 kernel: sdl: unknown partition table
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:10: Attached scsi disk sdl
Oct 25 20:24:36 rh2 kernel: sd 5:0:0:10: Attached scsi generic sg12 type 0
Oct 25 20:24:36 rh2 iscsid: transport class version 2.0-871. iscsid version 2.0-871
Oct 25 20:24:36 rh2 iscsid: iSCSI daemon with pid=4907 started!
Oct 25 20:24:36 rh2 iscsid: connection1:0 is operational now
/* 从rh2主机上的系统日志可以看到Linux正确识别了iscsi目标附带的虚拟磁盘 */
[root@rh2 ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: WDC WD3200AAJS-0 Rev: 01.0
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: Hitachi HCS72101 Rev: JP4O
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 00
Vendor: IET Model: Controller Rev: 0001
Type: RAID ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 01
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 02
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 03
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 04
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 05
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 06
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 07
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 08
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 09
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi5 Channel: 00 Id: 00 Lun: 10
Vendor: IET Model: VIRTUAL-DISK Rev: 0001
Type: Direct-Access ANSI SCSI revision: 05
/* 共有10个VIRTUAL-DISK */
[root@rh2 ~]# hdparm -tT /dev/sdl
/dev/sdl:
Timing cached reads: 9228 MB in 2.00 seconds = 4620.43 MB/sec
Timing buffered disk reads: 148 MB in 3.03 seconds = 48.77 MB/sec
[root@rh2 ~]# hdparm -tT /dev/sdl
/dev/sdl:
Timing cached reads: 9556 MB in 2.00 seconds = 4782.39 MB/sec
Timing buffered disk reads: 170 MB in 3.01 seconds = 56.47 MB/sec
[root@rh2 ~]# hdparm -tT /dev/sdl
/dev/sdl:
Timing cached reads: 9620 MB in 2.00 seconds = 4814.14 MB/sec
Timing buffered disk reads: 178 MB in 3.02 seconds = 59.00 MB/sec
[root@rh2 ~]# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 9368 MB in 2.00 seconds = 4685.71 MB/sec
Timing buffered disk reads: 306 MB in 3.01 seconds = 101.58 MB/sec
/* 实际测试磁盘速度发现，千兆网络条件下虚拟磁盘的读取速度大约在50-59MB，缓存读取(cached reads)则和普通磁盘大致相仿 */
iscsiadm –m node –op delete –targetname
//在客户端删除iscsi target