• Blog
  • About
  • Contact

vSphere 5.5 | Upgrading Gotchas.04 | Backup vCenter First!

4/29/2014

2 Comments

 
This is part four of my ongoing recap of my recent vSphere 5.5 technical deep-dive. To view all the posts in this series, click Geeknicks in the categories list.

Gotcha #4: Backup, Backup, Backup vCenter First! 

Backing up vCenter is a crucial element to all installations. However, few people take the time to do it. Here are a couple of resources to help you. 
  • Backing Up 4.x/5.0 vCenter ADAM instances
  • Backing Up 5.1 vCenter SSO instances
  • Backing Up vCenter SQL Database instances

Why is it so important to backup vCenter? First, because it is the heart of your virtual data center. Take, for example, virtual Distributed Switches (vDS). While traffic will continue to flow in the event of a vCenter outage (when setup using "Static Binding"), you won't be able to make any changes. That alone should cause you to backup, and backup often.

Additionally, if there is a (unlikely) probably during the vCenter installation, that irreparably changes the database or corrupts it...how will you get everything back? The only solution, if you are without backup, is to rebuild, and that is a lot of time and work. 

Save yourself some headache, and back that thing up! 
2 Comments

vSphere 5.5 | Upgrading.Gotchas.03 | File Locking on 8+ Hosts

4/29/2014

0 Comments

 
This is part three of my ongoing recap of the technical deep-dive  into vSphere 5.5 my company, Adcap Network Systems, recently held. To view all the posts in this series, click Geeknicks in the categories list.

Gotcha #3: Don't Open a Single File on More than 8 Hosts (unless it is on NFS)

This is actually not a vSphere 5.5 gotcha in particular, but a vSphere gotcha in general. I first came across it doing large rollout for the DoD, where we would be setting up 20 or 30 Exchange hosts at a time, and as soon as we would mount the ISO to the ninth host, all the hosts would effectively come to a halt during their installation—and a failed Exchange installation is, well, messy to clean up.

This gotcha applies to any file that would be opened by more than 8 hosts, and here are some examples:
  • ISO, custom or otherwise (for example a driver ISO to install a custom VIB file)
  • Golden Master VDMK for a linked clone scenario (this is why you can't have more than 8 hosts accessing a linked clone for Horizon View)
    Note: however, that up to 32 ESXi 5.1+ hosts
     can open a single file on NFS datastores.

The ISO situation actually comes up a LOT, because quite often administrators forgot to disconnect (and disassociate) their ISO images from their VMs after they do their installs (here's how to find which VMs are currently accessing ISO images).

So why does it happen?

In short, the file lock holder runs out of space. Each file has 512 bytes allocated as part of its metadata to maintain a "lock" on the file to prevent other hosts from accessing/modifying it inappropriately (ever had to release a file lock on a failed ESXi host?—see VMware KB 10051).
Picture
vSphere 5.5 | What Happens When You Open a Single File on Nine Hosts
Because the lock space allocated per file is 512 bytes, and the Universally Unique Identifier (UUID) is 64 bytes, well, it is a case of simple math: 512 / 64 = 8. That's how many hosts can fit within this.

So why is NFS unaffected by this? Because there are two types of locking: mandatory, in which the ESXi kernel forces the file lock at all times, and advisory, which is more flexible. The NFS protocol itself does not support mandatory locking, only advisory locking as shown below. 
Picture
vSphere 5.5 | NFS "Advisory Locking"
In these situations, the NFS protocol will interact with the advisory lock and say, "Hey, I need access to this file, can you free up a lock for me?" The locking mechanisms in the kernel will work to free up a lock, commit the write, and and the process will repeat with other hosts, quite rapidly, as you might imagine. 

Note: You must have vSphere 5.1 or higher installed to take advantage of the NFS advisory lock mechanism; this feature is not supported on 4.1 and earlier.

This is, incidentally, also a good use case for why you might choose to make NFS datastore for large-scale ISO content repositories. 
0 Comments

vSphere 5.5 | Upgrading.gotchas.02 | SSO Shows 0 Inventory

4/28/2014

0 Comments

 
Recently, Adcap Network Systems hosted a technical deep-dive called Geeknicks into vSphere 5.5 (yours truly was the presenter). For those unable to attend, and because those who attended, statistically, only retained about 5–10%, I am covering much of the material here in a new series of posts. To view all the posts in this series, click Geeknicks in the categories list.

Gotcha 2: New Users Have 0 Inventory Items When Using SSO

vSphere 5.1 introduced Single Sign-On, or SSO. With that, a user's Active Directory (AD) credentials, when properly configured, can be used to sign on with vCenter and administrator accordingly. 
Picture
vSphere 5.5 Web Client | Single Sign-On Window
The operative phrase here being, "when properly configured." There are a lot of headaches that people have had with SSO. In fact, when I recently did a poll about vSphere 5.5's best new features, one person commented, "I wish you had an entry for SSO that actually worked" (LinkedIn profile and group permissions required to view comment).

A lot of people have waited to upgrade their environment, until a "version 2" of SSO had come out, kind of like the way that many folks will wait to upgrade major version of Microsoft software until the first Service Pack is released—they don't want to be early adopters.

vSphere 5.5 is the "version 2" so-to-speak of SSO, and having used both, I can say it is much better. For instance, if you install vCenter 5.5 while logged in via a domain account, the installer is smart enough to know you want SSO installed and configured for that domain, so it does it for you.

The gotcha comes in two forms: installing vCenter 5.5 for the first time (you have a new environment), and rebuilding vCenter 5.5 from scratch instead of upgrading. Here's what happens. 
  1. You install vCenter, and configure SSO.
  2. You login, and everything looks great.
  3. You add your hosts and such.
  4. You grant access to the rest of your team—doh! They login, but they can't see anything! Everything shows up with 0 inventory! What in the world??!! Yeah. That's about right.
Picture
vCenter 5.5 Web Client | SSO with 0 Inventory Items
The problem is that while SSO enables a "single sign-on" there is still two places you have to set Role-Based Authentication Control (RBAC) permissions: SSO configuration, and Inventory configuration. Most people remember the first, but for some reason forget the second. So in essence, what happens is admins set it up, and grant others the right to login to the system, but they have no rights to actually see anything. 
Picture
vCenter 5.5 Web Client | Configuring Inventory Permissions
The solution is to navigate to the top-level vCenter inventory object, or press the web client shortcut (^ +⌥+2 – vCenter Home View) and select Manage | Permissions. From there, add your groups (preferably, as a best practice) and users according to their business requirements. Wallah! Now you have a fully populated inventory. 
0 Comments

vSphere 5.5 | Upgrading.Gotchas.01 | VM Version 10

4/25/2014

0 Comments

 
Recently, Adcap Network Systems hosted a technical deep-dive called #Geeknicks into vSphere 5.5 (yours truly was the presenter). For those unable to attend, and because those who attended, statistically, only retained about 5–10%, I am covering much of the material here in a new series of posts. To view all the posts in this series, click Geeknicks in the categories list.

Gotcha 1: VM Hardware Version 10

With the release of vSphere 5.5 and ESXi 5.5, there was a corresponding revision upgrade of virtual machine hardware to version 10. Here is a summary of VM hardware versions and their ESXi or ESX compatibility (for the full list, see VMware KB 1003746):

     10     5.5
       9     5.1
       8     5.0
       7     4.x
       4     3.x

So, should you upgrade? Here's the first gotcha. And it's a big enough gotcha that there is VMware KB 2061336 to address it!  If you upgrade, you will incur two potentially problematic issues:
  1. The VM will only be editable in the vSphere Web Client; the legacy (desktop) client will allow you to view but not edit the hardware. 
  2. This is especially problematic if you don't have a good vCenter implementation (which you should) or have some standalone hosts...you won't be able to manage them the way in which you are accustomed.
  3. If you have a standalone host, you will have to use the vCLI or Power CLI to manage the VMs on that host; the only supported GUI is the vSphere 5.5 Web Client.
Picture
vSphere 5.5 | VM Hardware v10 Warning
Tomorrow: Gotcha.02 | SSO and Inventory Permissions
0 Comments

15,360 VMs in Your Pocket | VCSA 5.5 vCSIM

4/24/2014

10 Comments

 
There are so many awesome things about the vCenter Server Appliance (vCSA). Here's a brief list to start:
  • Manageable via SSH
  • Works with SSO and AutoDeploy
  • Scales up to 1,000 hosts and 10,000 virtual machines (with Oracle DB) per instance
  • Up and running in minutes, not hours
And there's more . . . the vCenter Simulator, or vCSim (at least that's how I abbreviate it).

William Lam first put me onto this with his blog, and there are several other great write-ups about it as well. To get yourself started:
  1. Deploy a new instance of vCSA (it will modify the vCenter database, so I recommend a standalone instance just for the simulator.
  2. Log into the vCSA for the first time in your browser and complete the startup wizard:
    https://[ipOrDnsName]:5480
  3. Accept the EULA and choose the default configuration and let the system run for about 15 minutes to configure the defaults.
  4. Remote into the vCSA via SSH and log in with the default credentials root/vmware.
  5. Start the 15k VM configuration by entering this command:
    vmware-vcsim-start /etc/vmware-vpx/vcsim/model/configSamples/vcsim-15KVM.cfg
Boom! You just deployed 15,000 virtual machines on 1000 hosts! 
Picture
vCSA vCSim | 15k VM Config
Now, it took my home lab system—which is not the most powerful, but decent for sure—about 5 hours to fully populate all those VMs, hosts, and resource pools, as you can imagine. But the result is worth it!
10 Comments
<<Previous

    Author

    Husband.Father.Techhead. Lifelong Learner & Teacher. #NetAppATeam. #vExpert.
    Certified in blaa blaa blaa.
    All posts are my own.

    Picture
    Picture
    Tweets by @dancbarber
    Check Out koodzo.com!

    Archives

    March 2017
    June 2016
    December 2015
    July 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014

    Categories

    All
    Best Practices
    Cisco Nexus
    Cisco UCS
    Cloud
    Compute
    Design
    Disaster Recovery
    ESXi
    Flash
    FlexPod
    Geeknicks
    HA/DRS
    HomeLab
    Horizon
    Hyper-Converged
    Management
    Memory
    NetApp
    Networking
    NFS
    Performance Optimization
    Power
    ProTips
    SAN
    Scripts
    Security
    Servers
    SQL
    Storage
    Training/Certification
    Troubleshooting
    VCenter
    VDI
    VMware
    VSOM/vCOPS
    VUM
    Windows

    RSS Feed

✕