I had several issues with NFS speeds on my FreeNAS server. It’s only 3 fairly slow SATA disks with an SSD ZIL drive but it wasn’t really living up to its potential via NFS.
I finally stumbled across some of the issues with the caching issues with NFS and that we basically need to get ZFS to stop dumping cache on every write sync NFS requests. Turns out there is a FreeBSD kernel parameter to fix this:
vfs.zfs.cache_flush_disable=1
You can set this as a tunable in FreeNAS as you can’t really add your own kernel parameters any other way properly. I saw approximately 5-8x improvements in write speeds via NFS depending on the work load. Not too shabby.
NOTE: This can cause serious corruption problems if you lose power unexpectedly so ensure you understand the risks associated with the performance boost.