2012年10月13日 星期六

NFS Export different LUN with parent directory

今天客戶遇到NFS server的奇怪問題,註記一下以免忘記
情境:
有兩個空間分別mount在/new與/new/2012並將/new nfs分享出去,client端使用指令將server的/new mount起來後,可以看到/new裡面有一個2012資料夾,但資料寫入卻是寫在/new那個空間

測試:
NFS server
1. NFS server 新增兩個硬碟空間,分別為/dev/vdb1,/dev/vdc1

# fdisk -l
….......
Disk /dev/vdb: 2147 MB, 2147483648 bytes
16 heads, 63 sectors/track, 4161 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x8e79b1b7

  Device Boot      Start         End      Blocks   Id  System
/dev/vdb1               1        4161     2097112+  83  Linux

Disk /dev/vdc: 2147 MB, 2147483648 bytes
16 heads, 63 sectors/track, 4161 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7c784fd8

  Device Boot      Start         End      Blocks   Id  System
/dev/vdc1               1        4161     2097112+  83  Linux


2. Server 建立Dir:/new 並將/dev/vdb1 mount 在/new
3. 在/new 目錄下建立2012目錄,修改權限後,mount /dev/vdc1 在/new/2012

   [root@node2 ~]# chmod -R 777 /new
[root@node2 ~]# ll /new
total 20
drwxrwxrwx. 3 777 root  4096 Sep 19 15:28 2012
drwxrwxrwx. 2 777 root 16384 Sep 19 14:29 lost+found
[root@node2 ~]# ll /new/2012/
total 16
drwxrwxrwx. 2 777 root 16384 Sep 19 14:29 lost+found
[root@node2 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root         5.5G  2.9G  2.4G  56% /
/dev/vdb1             2.0G   36M  1.9G   2% /new
/dev/vdc1             2.0G   35M  1.9G   2% /new/2012





4. # vi /etc/exports   加入
  /new     *(sync,rw)


5. # /etc/init.d/nfs restart      重啟nfs service
[root@node2 ~]# /etc/init.d/nfs restart
Shutting down NFS mountd:                                  [  OK  ]
Shutting down NFS daemon:                                  [  OK  ]
Shutting down NFS quotas:                                  [  OK  ]
Shutting down NFS services:                                [  OK  ]
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]



NFS client
1. # showmount -e “NFSserverIP”
2. # mount IP:/new /mnt
[root@solutions ~]# mount 172.16.1.136:/new /mnt
[root@solutions ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_soldtions-LogVol00                   97G   14G   78G  16% /
…...
172.16.1.136:/new     2.0G   35M  1.9G   2% /mnt


3. # ls -al /mnt
[root@solutions ~]# ls /mnt
2012  lost+found
  [root@solutions ~]#


4. # touch /mnt/2012/test    
5. # ls /mnt/2012   
[root@solutions ~]# touch /mnt/2012/test
[root@solutions ~]# ls /mnt/2012/test
/mnt/2012/test
[root@solutions ~]#


6. # ls -al /new/2012    回到NFS server檢查,發現檔案並沒產生
[root@node2 ~]# ls /new/2012/
lost+found
[root@node2 ~]#


7. # /etc/init.d/nfs stop
   # umount /new/2012          停掉NFS並umount /dev/vdc1
   # ls -al /new/2012             發現剛產生的檔案在這邊(dev/vdb1)
[root@node2 ~]# ls /new/2012/
lost+found
[root@node2 ~]# /etc/init.d/nfs stop
Shutting down NFS mountd:                                  [  OK                                     ]
Shutting down NFS daemon:                                  [  OK                                     ]
Shutting down NFS quotas:                                  [  OK                                     ]
Shutting down NFS services:                                [  OK                                     ]
[root@node2 ~]# umount /new/2012/

[root@node2 ~]# ls /new/2012/
test
[root@node2 ~]#


解決方法:(在/etc/exports裡面加上nohide & crossmnt兩個參數)
1. # vim /etc/exports
  /new     *(sync,rw,nohide,crossmnt)

2. # /etc/init.d/nfs start        重啟nfs 服務
3.到Client 將/new mount 起來後
  # echo `date` >  /mnt/2012/test2
  # ls -al /mnt/2012             應該會看到剛產生的檔案
[root@solutions ~]# mount 172.16.1.136:/new /mnt
[root@solutions ~]# echo `date` > /mnt/2012/test2
[root@solutions ~]# ls /mnt/2012/
lost+found  test2
[root@solutions ~]# cat /mnt/2012/test2
Wed Sep 19 15:57:00 CST 2012
[root@solutions ~]#


4. 到NFS server的 /new/2012/ 查看有無看到一樣的檔案
  # cat /new/2012/test2
  # df -h    確認mount point是想要的
[root@node2 ~]# cat /new/2012/
lost+found/ test2
[root@node2 ~]# cat /new/2012/test2
Wed Sep 19 15:57:00 CST 2012
[root@node2 ~]#


5.完工!!