NFS vs. Block (FC/iSCSI) Protocol

Hi All,

Protocol seems to be one of those topics you don't discuss over dinner, like politics!  Storage folks tend to be one or the other NFS or Block and I've heard some very heated debates on this topic.  Yep, nerd fights!  I used to be a die hard block guy, especially for enterprise storage.  NFS was cool for file sharing or home directories, but BLOCK was king when it came to databases, mission critical applications, etc.  When I went to work for NetApp my opinion quickly changed, especially when I began working on virtualization products like VMware.

When I'm talking with customers I'm frequently asked, "What is better block or NFS?"  It's a bit loaded question because the customer usually has an idea of what they like better, plus they've probably already spent a ton of money on a new infrastructure.  The last thing I want to do is call their baby ugly!  I usually tell them *I* prefer NFS for these reasons, and I list out some reasons.  But in case they're a block shop, I remind them that NetApp can do both NFS and block concurrently. :-)

I was reading our internal discussion groups this morning and saw a great post by Nick Triantos.  For those of you that don't know Nick, he's a brilliant guy and an avid blogger.  I'm always amazed how quickly and precisely he knows the answers to things that would take me much more time to find out!  He answered a question on NFS vs. iSCSI for VMware this morning and I'd like to share it with you.  Enjoy!


"Datastore resizing is another difference...With NFS you can resize up or down on the fly. with VMFS you can only increase the size of the datastore. That means if you ever need to rebalance a VMFS datastore by Storage VMotioning VMs to another datastore, you now have captive storage you can't reuse, unless you create a new datastore of the required size, move your VMs into it and destroy the old one. Is it a hard thing to do? No. Does it require additional steps on the server and the storage side? Yes.

Deduplication, post process dedup that is, over NFS, allows a vmware admin to immediately realize the space savings on the host without any additional work. The same is not true for block protocols as additional work needs to be done on the storage array.

No limitation on NFS as to the datastore size. The limit is whatever the file server support. The same is not true for block protocols where the datastore size imposed is at 64TB. Although I find hard to believe one will create a 64TB datastore anyway.

Also,  each virtual disk file on NFS has its own I/O queue directly managed by the NFS server. This is not true for block protocols which have per LUN queues and can become a point of IO contention. All that translate to higher fan-in ratios in terms of the number of VM to an NFS datastore vs VMFS datastore. Partners have been telling me for a long time they have customers with 250 and 300 VMs in a single NFS datastore. The only way you can pack that many VMs in a single VMFS datastore without issues is if they are powered off. :-)

The benefit of NFS is day to day operational efficiency and granularity. The architecture does require some thinking upfront and is largely dependent on the switching infrastructure, but once you lay it down, everything else is a breeze.

In the interest of full disclosure...there are some caveats with NFS...No support by Microsoft for Exchange deployments, although at VMworld, last year, I met with customers that ignored the support statement and have been running with it with no issues. You also can't use Microsoft Failover Clusters (you can't use it with Native vSphere iSCSI either). So as long as these are not required, NFS is the right choice, IMO.

Last but not least, considering that Virtual Machines are comprised of a bunch of files, why would use a block protocol to manage them to begin with, if you had a choice?"