This week I stumbled upon a technology that was new to me: AoE, or ATA over Ethernet. This protocol was being hyped as a faster, much cheaper alternative to iSCSCI and NFS. In a nutshell, AoE “is a network protocol, designed for simple, high-performance access of SATA devices over Ethernet”.   I found a short description of a typical setup on Martin Glassborow’s site that demonstrated how quickly AoE could be set up.  This sounded very interesting to me and worth investigating further. I was curious to see whether it would be anywhere near local-storage speeds in my set-up.

I hit the Ubuntu documentation for how AoE was set up on a Ubuntu 10.04 virtual machine. I created a small, 10GB test disk with vSphere and attached it to my Ubuntu 10.04 vm. After a reboot, I started up the Disk Utility (System/Administration/Disk Utility),formatted the volume, and then mounted it. I noted the drive letter (sdb), as you will need it in steps 4 and 5. After that I performed the following steps in a command window:

1. sudo apt-get install vblade
2. sudo ip link set eth0 up
3. I then tested to make sure it was working:
   sudo dd if=/dev/zero of=vblade0 coast=1 be=1M
sudo vblade 1 1 eth0 vblade0
4. I exported the storage out:  sudo vbladed 0 1 eth0 /dev/sdb
5. I also edited rc.local to add the above line to startup: sudo nano /etc/rc.local

Next, I began setting up the client side (Windows XP machine).  I downloaded the Starwind Windows AoE Initiator, which required me to register first.  This application was installed on the Windows box, and started up.  I added a new device, chose the adapter, and selected the drive that was listed.  The drive suddenly was recognized by Windows as new and connected.  Very cool.

In XP’s Computer Management, I initialized the new drive and formatted as NTFS.  It took but a few seconds for the 10GB drive to complete.  Everything about this AoE-connected drive made it appear as if it was directly connected to my machine.  The whole process I’ve described to this point took a bit less than twenty minutes, not counting the few minutes to register at Starwind.  Now I wanted to see if it was as fast as local storage.

Testing
I chose a local folder that was 2.62GB in size, with 176 folders within it containing 19,000 files.  This was small enough to not take all night for the tests, but also had enough small files within to make it more challenging that simply a single, large file.  I popped up the system clock (with a second hand), and noted the start time when I pasted the file, and stopped the clock when the transfer completed. I did nothing else on my machine while the tests ran.  The results are below.

Conclusions
While these results are not overly scientific and very specific to my environment, they do give some valuable comparisons.  In my environment, AoE performed no better than an Openfiler or a common Ubuntu (samba) share. It certainly was much slower than local storage but honestly, what isn’t? If AoE had been noticeably faster than OpenFiler, I might have considered finding a use for it. As it stands, it was no faster than any other share on the ESXi box. It did take precious little time and effort to set up and test, so I consider it time well spent. In a large environment, perhaps on a SAN, the results might be different.