How to Setup MultiPath (MPIO) with Nimble Storage and Citrix XenServer

Hi Friends,

Here's a follow up to my Citrix XenServer blog that showed you how to attach Nimble Storage to XenServer.  This article will show you how to set up MPIO with Nimble and XenServer.  It was written by a colleague of mine, so a HUGE thanks to Klaus Kupfer!!

 1.  Configure each Data Interface on the Nimble Array to a different subnet.

2. Configure Multipathing on the XEN Host
      a.       Put XEN Server in Maintenance Mode

 b.      Access XEN Server Properties

 

c.       Enable Multipathing
d.      Exit Maintenance Mode

3. Check that the XEN Server NIC’s are in the same subnets as the Nimble Data Interfaces

4.  Edit the /etc/multipath.conf File using console
  
device {

vendor                                    "Nimble"

product                                  "Server"

path_selector                         "round-robin 0"

path_grouping_policy           group_by_serial

rr_weight                                priorities

rr_min_io                                10

path_checker                         tur

failback                                   immediate

no_path_retry                        20

}

5.  Reboot the XEN Server

6. Create New SR 
     a.       Obtain initiator IQN
     b.      Configure Volume on Nimble on Nimble array with all desired settings and LUN initiator groups


c.       Choose New SR...

 

d.  Choose "Software iSCSI“ as Storage Type


 
e.       Configure a name for the SR

  
f.        Enter the iSCSI Discovery Address.  Then click first  "Discover IQNs“ and then "Discover LUNs“ The configured Volume should appear in the list

 g.  Finished

Until Next Time!
-Brain

Nimble Scale-Out - The Curtain is Drawn - Part I

Hi Friends,

In today's blog I wanted to dive a bit deeper into the new features available with Nimble Scale-Out.  When you get your first storage array it's great, you've got your GUI or CLI and you're ready to go!  But then you get your second and it too has a GUI, which is okay.  Then a 3rd with another GUI, which is starting to get confusing.  Let's top is off with a fourth which, it too has it's own GUI.  Did I just delete the volume on Array 1 or Array 4?  Ugh oh....

You get my point, when you have multiple GUI's open, it starts to get a bit confusing.  The cool thing about Scale-Out is you have one GUI, and you don't have to download a separate tool to administer your cluster!  Yep, it's all built in.  Basically you elect one array to be your leader and it's the first one to join the cluster.  As others join they give up their ability to have a separate front end GUI, since you'll be using the "leaders" IP address to administer from 1 to 4 arrays.

What if you want the ability to have a single pane of administration glass, but prefer each array to be their own entity.  You're not ready to start striping volumes or use some of the newer features.  That's okay!  Scale-Out isn't an all or nothing upgrade, you can jump into the deep end or slowly get your feet wet at the shallow end of the pool.

How about some pictures?

Here's we've got a single array called Anooba.  It's group is called t1 (I'll get into groups a little later on) and it's the only array in the group.  We'll click on the Add Array to Group button and....

We get a box that shows us the available 2.x arrays available on the network.  Pretty cool huh?  Click on the one you want to add, here I select Bantha.  Click on the Add button.

What?!  A password dialog!?!  Yep, believe me, this is a GOOD thing, you don't want arrays magically appearing in your cluster.

After you enter your password for the array you're trying to add to the cluster, click Connect.

Check it out!  Both arrays are there.  Was it really that easy?  Well, I cheated a little.  There are some networking considerations that you need to be aware of when adding arrays to your cluster, but that's for another blog.  Hey, I gotta get you coming back!

Until Next Time,
-Brain

Now Introducing Nimble Scale-Out! This IS The Storage You're Looking For!

Hi Friends,

Today I'd like to introduce a super cool new product put out by Nimble Storage.  Scale-Out, Nimble's  next gen version of storage.  Yeah yeah, I'm sure you've heard that a lot and you're thinking, "Blah blah blah, show me the money!"  Ever watch Looney Tunes?  You're probably thinking Scale-Out is like this:

Well my friends in this case you'd be wrong!  Scale-Out is much more like this:

And the great thing about about it?  You can download it today!  It's a release candidate that's available for download through InfoSight.  And in most cases will work on your existing hardware.  AH HA!  You're thinking, "Most cases, meaning not my case."  No, it'll probably work on your hardware too, but there's always a small percentage so it's best to check with your account team.

Okay, okay so Michigan J. Frog aside, let's talk about what's so fab about it.

A large differentiator between Nimble and other storage is it doesn't rely on spindles for performance.  CPUs are cheap, we use them for our performance.  Disks are just a way to store your stuff cheaply!  You need more storage, add a shelf of disks.  Now what if you want the ability to administer your storage from a single GUI?  Stripe a volume across all of your storage controllers for smoking fast performance?  The ability to evacuate an entire controller for maintenance or storage replacement with NO downtime?

Yes, friends, you get this, the car polish AND the set of knives....  Sorry couldn't help myself.

Want to know what else is cool?  The GUI hardly changes too!  When I did my first upgrade I thought I had done something wrong because it looked too much like 1.x Nimble.

Here's a 1.x system:

Here's a 2.x system:

At first glance, not much difference huh?  I kept waiting for the other shoe to drop, but it didn't!  Now of course there are differences in there and some caveats before upgrading which I'll go through in another blog, but on the painful scale I don't think it was even a stubbed toe!

That's all folks!
-Brain

Neil Versus PVS - Part IV - Multiple Sites with KMS

Hi Friends,

Here's an interesting one that I ran into while working with multiple PVS Sites.  After the fact it seems kind of straight forward, but at the time I really had no idea what was going on.

If you take a look at my architecture I have three sites and two PVS servers gong to each site.  Ignore "Site" I was using that for testing.  :-)
 

The thought was with multiple PVS servers, you'll probably start having multiple sites too.  It's also easier with house keeping when you start collecting a lot of servers, volumes, vdisks, etc.  My vDisks are all sitting on a CIFS share so I don't have to have multiple vDisk images, which is VERY handy.  There's only one small problem and I probably wouldn't have discovered it for awhile, but thanks to KMS, my life got very interesting!

I decided to log into UCSPVS3 which was my third PVS server.  I loaded up the Provisioning Services Console and everything looked normal.  I could even manipulate vDisk images mapped to Site 1!  Now that's cool!  Now here's where the problem began.  I went to change the vDisk from Private image mode to Standard image mode and I got this error message:

Unable to map the vDisk.  Mapping was denied by the Server.  Huh?  And when I turned KMS off as an option under vDisk Properties, it switched from Private to Standard image just fine!  So what the heck?

You might be saying, "Just use the Site your vDisk is assigned to Neil!!"  Well, yeah, but isn't the point of having it on a CIFS share is I can share it from anywhere?

Well, not exactly.  So here's something I found out.  When you have the KMS option enabled in the vDisk Properties, the vDisk momentarily gets mounted up as a volume in Microsoft Disk Management on the PVS server you're trying to switch from Private to Standard image.

Here in lies the problem....  I was trying to mount up the volume on a PVS server that didn't have the vDisk assigned to it!  As I said earlier, the only reason I had this problem is with KMS enabled the volume gets momentarily mounted.  So if not for KMS, I probably wouldn't have bumped into this unless I tried to mount up the vDisk.

After the fact it seemed so simple and straight forward, but sometimes when you're lost in the forest, it's hard to see the trees!  :-)

Until Next Time!

-Brain

Happy New Year!

Happy New Year Everyone!

-Brain

Brain & Wendel Show - Best Practices for VDI

Hi Friends,

Have you heard about VDI but aren't sure what really goes into it?  Is it just fancy remote desktop?  Who are the key players in the market right now?  If you're an IT administrator or a CEO, how will it help YOU?  Brain & Wendel de-mystify VDI, and explain how the different VDI technologies can best fit into your environment!

For our iTunes followers:
https://itunes.apple.com/us/podcast/episode-11-virtual-desktop/id688439206?i=212069710&mt=2

For our non-iTunes fans:
Episode 11: Virtual Desktop Infrastructure

-Brain

Videos Won't Play With XenDesktop 7

Hi Friends,

This one was a painful issue!  One of the tests Login VSI runs is video and when I'd get to this part of the test the video would just freeze!

The script calls an HTML file which forwards the webpage to an mp4 file.  To make matters worse when I wasn't running through the Citrix Receiver it would work just fine!  So I started creating policies like crazy in XenDesktop.  I enabled Legacy Graphics, adjusted audio, this, that, and those, but still my video just froze.  Sorry, started channeling Dr. Seuss.  :-)

I was using Internet Explorer 10 and downgraded it to 9 and that didn't work either.  Dennis from Login VSI saved the day!  Under Tools/Internet Options/Advanced, we selected the Use software rendering instead of GPU rendering*.

I'm sure there's a registry setting to fix this or a Policy in XenDesktop to fix this, but I have NO idea which one it is.  So if you're rolling out XenDesktop 7 and your users start complaining their videos won't play, maybe this will work for you!

Until Next Time!
-Brain

Brain & Wendel Episode 10 - Cloud Technology and You!

Hi Friends,

Have you heard buzz around products like OpenStack and CloudStack?  Or heard of Cloud acronyms things like SaaS, IaaS?  Is it all foggy and hazy?  Tune into our latest podcast and hear Wen give an outstanding explanation of what the different "As A Service" acronyms are, what are the key software products for Cloud technologies and some predictions from The Brain.  All this and more in the latest episode of the Brain and Wendel Show!!

iTunes:
https://itunes.apple.com/us/podcast/the-brain-and-wendel-show/id688439206

For our non-iTunes fans:
Episode 10: Cloud Storage

Simple Power Shell Script To Log Off Remote Desktop Users

Hi Friends,

As I've mentioned I'm running automation using Login VSI to test multiple VDI user logins and the problem is clean up.  Isn't clean up ALWAYS a nightmare!?  Well, I've pieced together a small Power Shell script that I've taken from 3 or 4 sources around the Net that I'd like to share with you.  Don't forget to save the file as a .ps1 for Power Shell.

So here's what it does. 

1.  You create a text file called servers.txt, or whatever you'd like and populate it with the names of the machines you want users logged off from.  If your machines are numbered sequentially and easy way to populate the file is using Excel.

2.  For the $cred variable you put in your domain and user who will be logging folks out.

3.  You kick off the script and a dialog box will pop up asking for your password.  This will be the only time it needs the password for that run.

So if you're cleaning up 4 or 40,000, this is a simple script to help you out.  I wish I could remember which sites I pieced the script together from, but I  can't remember.  So I'll send out a blanket thanks for all the sites that helped me!

Until Next Time!
-Brain


$cred = get-credential domain\user

foreach ($_ in get-content servers.txt)

{(gwmi win32_operatingsystem -credential $cred -ComputerName $_).Win32Shutdown(4)}

Neil versus PVS - Part III - More Fun Then Should Be Allowed!

Hi Friends,

Still fighting with PVS, I think I might have it under control now, but it was touch and go for a few days there!  More errors and possible solutions for you if you're tackling a large roll out.  All I can say is working with a large number of desktops on PVS, I have a new respect for people who work with this product every day!  It's a powerful product, but man is it particular!

I've created a CIFS share through Microsoft file sharing and put my vDisk on the share.  Everyone is happy, I'm able to create multiple desktops without a problem.  Now it's time to add a second PVS server to share the load, the plot thickens....  I add the new PVS server using the PVS Provisioning Wizard, it adds great, shows up, the store path validates, but when I look at the vDisk Pool or vDisks in my store it shows up as No Server.  No Server what the heck does that mean?!  You're there, you have the path, people like you, how can you be No Server?  So off to Google!

I look around and find a couple of articles on the Citrix forums, but not a 1:1 condition or concrete answer.  While reading there are a number of folks that talk about permission issues.  Now in the vDisk store you've got essentially three files, the vDisk itself which is the VHD, a LOK file and a PVP file.  Some advice, MAKE A BACKUP of the files before you start messing with or deleting them!!  I decided to start checking the permissions and I noticed that "Users" only had read, read/execute, so I added full control and the vDisk showed up as useable on my second server!!  The only problem is it kept acting weird.  It would pause a lot when I would click on the vDisk, sometimes get hung up, so I knew something else was still wrong.  To top it off when I tried to access the vDisk on boot from a desktop I would get vdisk is locked 0xffff800c.  If I tried using the XenDesktop Setup Wizard I got this error, A required PVS server in the selected site is offline or there is no server assigned to a required store.  I don't know about Denmark, but something was definitely rotten in San Jose!

The weird thing is the first server kept behaving fine, so what's the deal?  I deleted the vDisk and reimported it and now BOTH, yes BOTH showed No Server.  Hey, at least I'm getting somewhere, where that is, I have NO idea!  But I knew it wasn't some place I wanted to stay for long.

Off to the Microsoft share server!  I removed the share and noticed I couldn't add it back!  So I copied over the vDisk contents to a share that was behaving and guess what?  Both servers are now serving the vDisk!

So what's this teach us?  If you're getting weird access errors like this, check your shares.  I think by changing the permissions on the PVP file I was able to sort of fool PVS into sort of working, but the problem was at a higher level.  Why the first server was working, until I removed and re-added the vDisk, NO IDEA!  But if this can help just one person avoid some of the pain I encountered, my job is done!  :-)

Until Next Time
-Brain

Neil versus PVS - Part II - Receive This!

Hi Friends,

So I've been working on a big project and haven't had a ton of time to write, but wanted to share this with anyone who might be struggling with this.  I'm using XenDesktop 7 with PVS to create the desktops and logging into the desktops with Login VSI through a script, which communicates to the desktops through the Citrix Receiver.  Much more on the architecture later, and I have to say Login VSI is a VERY cool tool!

I begin my script run and no matter how many desktops I kick off, about 20% get this error message:

The network connection to your application was interrupted.  Try to access your application later, or contact your help desk.

The frustrating thing is it was different desktops at different times!  I did a bunch of searching around and found a number of people that have had this error and it was fixed by, things like DNS, AD, Firewall, Receiver version, etc.  So the only common thing about it is there's some sort of connection issue from the launcher to the receiver and than to the desktop.

My DNS, AD, Firewall stuff was all sorted out, so I decided to download the latest version of the Citrix Receiver and it seems to have cured the problem!!

So if you're getting this error, give it a try, it might be a quick fix to a VERY frustrating issue.

Until Next Time!
-Brain

Brain & Wendel Show - Live! - Scale Out

Hi Friends,

Some exciting news, Wen and I performed our first live Brain & Wendel Show yesterday.  We discuss our new Scale Out architecture, plus Wen does a great demo showing how efficient and simple the Nimble Connection Manager software works.  This software helps keep IO going to the right place when you start scaling out your arrays.  Sorry I've been a little behind blogs lately, been working on a project that's eating all of my time!  Hopefully I'll have more time to write soon.

Link to the live show:

http://youtu.be/USjELea1x-w

-Brain

Brain & Wendel Show Episode 9: Scale-Out Storage Architecture

Hi Friends,

Yes, the wait is over, this week’s Brain and Wendel show is now available for download!  Too much ego?  :-)  This week we talk about Scale-Out and have Sean Roth and Jeff Feierfeil as our resident experts to help explain what it’s all about, how is it different then 1.x and what’s great about it!

iTunes Link:

https://itunes.apple.com/us/podcast/the-brain-and-wendel-show/id688439206

and for our non-iTunes fans:

http://www.mrkieran.com/Podcasts/Brain_and_Wendel_9.mp3

Until Next Time!

-Brain

Brain & Wendel - Episode 8 - Reference Architectures

Hi Friends,

In this episode Wendel and I discuss Reference Architectures.  What are they?  Why do you need them?  What good are they for?  And how we do reference architectures at Nimble.  Be sure to check out the last 30 seconds for show bloopers!  :-)

-Brain

Non-iTunes:
Episode 8: Reference Architectures

iTunes

Neil versus PVS - Part I - Shadow Copy

Hi Friends,

Oh my goodness, have you ever had one of those days where NOTHING wants to work?  Wow, that's today!  Every time I take one step forward I take two back.  Why is Citrix PVS torturing me so?  I don't know, maybe I pissed it off...  Any who, time for you to enjoy the fruits of my agony.

This is a weirrrrrrrd one.  So I finished installing PVS on my Windows 7 master image and decided to use PvDisk.  So this is before I even got to creating my vDisk.  PvDisk starts to do it's magic and BOOM!  Error, failure, no go, do not collect $200!  I look in the log and I got some weird error that said something like failed to initializebackup.  I searched my buddy Google, but really couldn't find anything.  So I figure maybe it's something to do with PvDisk right?  So I re-create my Windows 7 machine, install XenDesktop without installing PvDisk.

Everything's working cool, I'm creating my vDisk, the machine reboots so it can run the XenConvert and BOOM!

I took a look at the log and the error was surprisingly similar to the PvDisk error.  Hmmm maybe I'm onto something?  The error was something like Failed to initialize backup.  That looks familiar!  So I look around and I find this Citrix article CTX134075.    Well according to this article the Volume Shadow Copy Service (VSS) needs to be running.  So I took a look and sure enough it was disabled.

I go to create a new vDisk, the machine reboots so it can run the XenConvert and BOOM!  What the heck?!  This time the error says something like failed to addtosnapshotset.  UGH!  So I look around on Google and found this thread.  Apparently there's TWO Shadow Copy Services!

I go into Services and sure enough there's a second one called Microsoft Software Shadow Copy Provider.  I turn that service on, create a new vDisk and then the XenConvert started to work!!

I don't know if the PvDisk error is fixed now, but I get a sneaking suspicion it is.  I'll let you know when I re-install it.

So if you're getting weird snapshot errors with PVS/PvDisk, take a look at your Shadow Copy Services, both of them, they might not be running.

Until Next Time!
-Brain

Happy 1st Birthday Glick's Gray Matter!

Hi Friends,

I can't believe a year has already passed and my blog has been viewed over 35,000 times!!  If you would have asked me how many people would want to read what I had to say, I definitely wouldn't say that many!

Thank you to my wonderful Wife for all of her blog ideas and just being great and thank you to all my readers, without you there wouldn't be a Glick's Gray Matter!

If you keep reading, I'll keep writing!

-Neil (The Brain)

Brain & Wendel - Episode 7 - Proactive Customer Support

Hi Friends,

Happy Friday!  This week Brain and Wendel interview Nimble Storage Vice President of Customer Support, Rod Bagg.  Learn about the excellent steps Nimble Storage has taken to help customer's before they even know there is a problem!  Definitely not a show to miss, I learned a ton of information.

iTunes Download

Non-iTunes Link: 
Episode 7: Proactive Customer Support


-Brain

Brain & Wendel - So Much Fun You'll Forget It's Educational!

Hi Friends,

I was out on Holiday, but I'm back!  I have great news, two more Brain & Wendel shows have been posted to iTunes.  You can get them here:

https://itunes.apple.com/us/podcast/the-brain-and-wendel-show/id688439206

If you don't have an Apple product, never fear, you can download the MP3 here:
http://www.mrkieran.com/Podcasts/podcast.xml

In Episode 5 we talk about VMworld, past, present and future.  Iron Tom is our special guest who is an Oracle and Linux guru and was at VMworld for the first time this year.  It's great to get his perspective on the show!  We also discus the software defined datacenter.  What does it mean and what does it mean to you?

In Episode 6 we talk about Big Data.  Larry Lancaster is our special guest, the Chief Data Scientist at Nimble Storage who enlightens us regarding Big Data and products like Hadoop and Vertica.  Big Data is becoming big news, so be sure to check this episode out to learn what all the

Until Next Time!
-Brain

How to Add Nimble Storage to Citrix XenServer

Hi Friends,

When I bring up XenServer a lot of people feel that it's down for the count in the hypervisor world, but I still think there's life in it!  :-)  It recently went open source and can be downloaded from either Citrix or XenServer.org.  I'm still asked from time to time about how to use Nimble Storage with XenServer, so I thought it would be a good idea to create some blogs on the topic.  I'd really be interested in hearing if you use XenServer, in what part of your business and what made you decide to use it.

Well then, on to creating some LUNs!

First let's create our initiator group so our Nimble array knows who our XenServer host is.

1.  Let's grab our initiator IQN from XenServer so we can create our initiator group on the Nimble array.  Click on your XenServer server, click on the General tab and copy down the iSCSI IQN.

2.  Next go to your Nimble Storage array and click on the manage and select initiator groups.  Click on New Initiator Group.

3.  Give our initiator group a name and click on the Add Initiator button.  Enter in the iSCSI iqn that you copied from XenServer and than click on OK and OK again.

Excellent, now the array and XenServer will be able to talk.  Now lets create a volume!

4.  Click on Manage and select Volumes.

5.  Now click on New Volume.

Okay, from here your mileage will vary since there are a bunch of different things you can do to optimize your volume, so I'm going to point out the highlights.

6.  Give your volume a name and select the initiator group you just created.  Click Next.

7.  Enter the size of your volume.  Click Next.

8.  Next create your protection policy.  This is just a test, so I'm not going to create one.

Outstanding, the volume and initiator group is created, we're ready to add storage to XenServer!  

9.  In the Nimble Array, go to Manage, Array.

10.  Here we're looking for the storage network our array and XenServer should be talking over.  Copy the Target Discovery IP Address, we're going to need it soon.

11.  Go to XenCenter, click on your server, then click on the Storage tab and then the New SR button.

12.  Select Software iSCSI as your Virtual disk storage.  Click Next.

13.  Give the new SR or Storage Repository a name.

14.  Remember that Target Discovery IP Address you wrote down from the Nimble Array?  Paste it into the Target Host.  Next click on Discover IQNs and Discover LUNs.  The LUN you just created should be discovered.  Click Finish.

15.  Now since this is a new LUN being attached XenServer, it will most likely want to format the LUN so XenServer can use it.

WARNING:  Make sure you don't have any data on this LUN that you want keep!

That's it!  You've just created a new LUN that can be used by XenServer.

Until Next Time!
-Brain