Tuesday, August 26, 2014

Fast Cars & Fast Arrays - OH YEAH!

Hi Friends,

I don't know about you, but I love cars, especially fast ones.  It's something primal, like cooking with fire.  Ever been to a BBQ?  All the guys huddle around the fire and you can almost hear the grunting!  Well that's the way it is with fast cars too.  Guys just go bonkers and will spend gobs of money squeezing every last drop of performance out of their car.  Just go to YouTube and you'll find tons of videos of cars revving their engines, speeding down race tracks or on dynamometers so they can get a graph of just how much horsepower their car is producing at certain RPMs.




(Image borrowed from Google)



(Image borrowed from Google)

So what does this have to do with storage?  Well, I'm glad you asked!  Like watching fast cars I love slamming an array with an Iometer load to see how I can squeeze every bit of a performance out of an array.  Iometer is a really cool tool originally developed by Intel Corporation and later given to Open Source Development Lab.

There's tons of knobs and dials for you to turn and if you're not familiar with the different concepts, it can be a bit overwhelming.  Just like if I was given a tool kit and was told to make a car go faster.  I'd probably take out a wrench, pretend to do something and then proclaim, "I've made it faster!...  No, not really..."

So I thought I'd give you a few tips so you can crank out an Iometer run and show your friends how cool you are!  :-)

**Warning**  I'm a clown and have really no clue what I'm doing.  Before you go following me, read all your documentation so you can tell me how I'm doing things wrong.  Basically, use at your own risk.


(Image borrowed from Google)

First you need the software, I've included the link in the first mention of Iometer.  Install the software on a physical or virtual machine on the same network as your storage.

Does size matter?  When it comes to network bandwidth is sure does!!  Not sure if you guys remember my friend Eddy the elephant.  But in an earlier article he helped me demonstrate a point about trying to shove too much data through a small pipe.




Remember, there's ALWAYS a bottleneck, you just want to make sure you're not the system administrator for it. :-)  So if you've got a smoking fast array, but are only running on 1gig pipes, you're only going to be able to push 1gig.

So we've got our software installed, we're on a fast network.  What's next?  Now it's time to assign some storage to your Window's host.  If you're using Nimble, I recommend using the Nimble Connection Manager.  It's a really cool tool that helps connect your storage, manage your paths, and lots of other cool stuff.



Here's NCM and "vdi-test" is my testing LUN.  I've made it 500GB and it has 4 paths to the array.





Here's a shot of Iometer.  The guys on the left are workers and well, they work.  Think of them as the pit crew gassing up the car, changing the tires, etc.  In the middle of the page we've got our different drives that we can test against.  Since I'm on the manager and not the workers, the drives are grayed out.  Now how about the disk size?  This was a weird concept for me.  Size in sectors....  Well, a sector is 512 bytes, okay great, so what?!  Remember my LUN is 500GB.  I don't want to use up the whole thing because Windows will start to complain that I'm running out of disk space, but I do want to use a good chunk of it.  Why?  Because if I don't use a large enough amount I'll be storing all of the data in RAM and get a false sense of performance because nothing ever left RAM.  So in this case, 200GB.  How does 400,000,000 equal 200GB?

400,000,000 * 512 = 204,800,000,000 bytes or 200GB.

Okay, what about the # of Outstanding I/Os?  That's an interesting one as well.  This is basically the qdepth.  What's qdepth?  Well, in this case it's how many I/Os will stand in line waiting to get processed.  You have to be careful with this number.  My boss came up with a great analogy.  Think of this as how many people you'll let into the night club at once.  If you let too many people in there won't be enough room and people will be forced to line up outside while more people are still trying to stuff themselves into the club.  If you're not careful you can introduce some serious latency issues and your performance will go down.

We've specified how much data, and where to push it, how about how to do it?  We'll find those settings in Access Specification.



LOTS of knobs and dials!!  Let's start with the Transfer Request Size.  This is how big the blocks of data you're asking the array to handle.  Remember, the larger the block, the more work the array has to do.  Think of this as your shovel.  Bigger shovel, you can move more dirt, but will require more work to pick up the shovel and move it.

Percent Random/Sequential Distribution.  This is whether Iometer is sending random or sequential data.  This is a great way to check see how your array behaves with different data types.  Some applications are more random, others are more sequential.

The Burstiness basically is how much I/O will happen in a row.  I like using 2-4.

Percent Read/Write Distribution is how many reads vs. writes you're sending your array.  This will also greatly depend on your application's behavior, but here I'm doing all writes.

And let's not forget Align I/Os on.  This is a REALLY important concept and will greatly depend on your array.  My array aligns on 4K boundaries so that's what I'm using.  If you use something different then what your array does, you could introduce misalignment and performance can go from bad to worse very quickly!

Alright, now that we've set our knobs and dials let's get this thing spinning!!




(Photo borrowed from Google)

YEEEE HAAAA!!  124,000 IOPS, and very little latency!  This is the equivalent of smoking tires for nerds!!



Look at this chart!!  Just LOOK AT IT!!  Now remember, I'm just showing off here.  For VDI, or any other application the chart will look different and I'll probably follow this blog up with a VDI one.

But I wanted to show you how to smoke the tires and grunt like a real man!  :-)

Until Next Time!
-Brain