Free Virtual San Appliance on ESXi

I recently had the need to use some NFS storage within my ESXi hosts. I had plenty of data store left on the physical disks and wanted to create a VM with a large amount of storage. Then I ran into the size limitation of VMDK files which derailed my idea for a while.

Then I did some thinking about how to get around this issue and was able to create my own virtual san within a VM. Here is a rough guide of how you can create your own Virtual SAN within VMware ESXi. This guide assumes some Linux knowledge and will create a 1.17 terabyte NFS VM.

First create a VM with a single drive large enough for your distro of choice and with enough ram. I used CentOS and gave the VM a 4gb disk and 512mb ram. I did the minimal install of CentOS for my appliance and this included NFS in the default group of packages. You will also need packages for Linux LVM in your distro.

After Linux is installed edit the VM settings and add 6 200gb disks to the VM and reboot the VM. Once your back up and running You can list them out with fdisk -l. The first thing we need to do is partition each one of our disks. You can do this with fdisk /dev/sdX with X corresponding to the device indicated in the fdisk -l output. You have to do this for each of the 6 disks. Here is the "template" that I used for my disk partitioning.

fdisk /dev/sdX

n - new

1 - partition number

default - start of disk

default - end of disk

t - type

8e - hex for linux LVM

w - write

Now that the drives are partitioned we have to create physical volumes, volume groups and do a number of other steps to get LVM going. Here is part of my template for doing this.

pvcreate /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1

vgcreate vol1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1

lvcreate --name shared --size 1.17T vol1

mkfs.ext3 /dev/vol1/shared

mount /dev/vol1/shared /export/shared

Now that LVM is going we need to mount it to the filesystem. Create the directory /export/shared and mount the filesystem using mount /dev/vol1/shared /export/shared You should be able to see the filesystem mounted in the output of df -h at this point.

Now that we have the filesystem mounted we need to have it mount on boot and export it via NFS to make it useful to us. Here are the steps to do this. In this example vi /filename is followed by the changes needed to the file. You will need to modify the changes to the specific configuration of your network.

vi /etc/fstab

/dev/vol1/shared /export/shared ext3 rw,noatime 0 0

edit /etc/exports

/export/shared 192.168.1.0/255.255.255.0(rw,sync)

Permissions to /export/shared should be changed to fit the needs of your environment. Adjust the permissions on /export/shared to fit the needs of your environment. If this is a well protected test box you can chmod 777 /export/shared for full access. You will also want to make sure NFS is up and running in the VM at this point and that you can mount the filesystems under NFS.

Now you can go in and mount this storage within ESXi. Click on Configuration>Storage>Add Storage and Select Network File System. Put the hostname or IP address of your NFS VM in the Server field and the mount point of /export/shared in the folder field. The Datastore Name field is a label which will be the name that ESXi refers to this NFS storage. This can be set to anything descriptive. I used the hypervisor_hostname-nfs as my name to show which system this NFS storage really lives on. Once you hit next your new data store will come up.

For my needs NFS works fine however one could configure the VM as an iscsi target as well and have an even more usable storage VM.