Tuesday, February 10, 2015

Microsoft Windows File Sharing with Nimble Storage - What About Deduplication?

Hi Friends,

One of my Systems Engineers read my blog and mentioned deduplication.  That's the cool thing about technical people, they're always looking for ways to make something cool even better.  Thanks for the suggestion Kevin!  With Nimble you get thin provisioning and compression built right in, but what about deduplication?  Users are notorious for saving 300 copies of the same document and with Microsoft Windows 2012 R2 we can deduplicate on the file server itself.  So let's go find out how much disk space we can save with native Nimble Storage compression and Microsoft Windows 2012 R2 deduplication.














The first thing we need to do is install the deduplication role on our file server if it's not already there.  It's located under File and Storage Services.



























Now let's head over to Server Manager > File and Storage Services > Volumes.  Here we see our different drives.  Right click and select Configure Data Deduplication.





























Select Configure Data Deduplication.  Note the capacity and the free space.

Here I select General purpose file server, cause well, it's a file server.  :-)  You can also select it as a VDI server.  Choose the age of the files to deduplicate, any file types or folders to exclude.  Now let's set up a schedule.  Click on the Set Deduplication Schedule... button.



Here's I've selected Enable background optimization and Enable throughput optimization.  Basically I've told Windows to throttle the deduplication process so it doesn't impact the system.  Pretty cool eh?  And the second option let's Windows dedupe without any restrictions during certain hours.  This would be good during quiet times when your users are at home asleep.


 




































Now just sit back and enjoy the space savings!  But Brain, what about the effects to the file server and Nimble Storage?  Ugh, okay, here are some graphs....

Here's a deduplication running not being throttled.  Yeah, yeah, I know the screen shot above says 1:45am, I changed it after I took the screen shot.  7% CPU and 28% memory, not bad!  How about on the Nimble?























I kicked off the test around 11:41.  There's a little jump for 3 minutes and then the usage goes down to well, very little.  You're mileage will vary depending on your file server and storage platform.




























So how about those storage savings?!

























Woo Hoo!  111GB of savings!  But wait, that's not all!  How about compression on the Nimble side?































Awesome, 200 more gigs saved!






















Pretty cool huh?!

Until Next Time!
-Brain

4 comments:

  1. Did you see a big increase in the cache consumption as the dedupe process churns through your files?

    ReplyDelete
  2. Hi Jason,

    Thanks for the comment! I decided to answer your comment with a blog. Have a look.
    Dedupe is Great, but What Affect Does it Have on Nimble Storage? http://glicksgraymatter.blogspot.com/2015/02/dedupe-is-great-but-what-affect-does-it.html

    Best Regards,
    Neil

    ReplyDelete
  3. Neil,
    Windows shows 694G used with a space savings of 111G.
    Nimble shows 679G used with compression saving 219G.
    What I get out of that is Nimble did not see any benefit from deduplication. It's native compression was likely already getting it down to 679G. Do you have a before picture of the Nimble stats?
    I'd like to see if dedupe has a positive effect on capacity used, and also how snapshot size plays into this too.

    -Dan

    ReplyDelete