VMware datastore volume in VCenter appears as “not consumed” in storage devices

last week I had an issue after Firmware update and restarting one ESXi server. I did some checks and noticed that one datastore appeared as “not consumed” and was inaccessible. I could not reach the VMs on that datastore of course.

Reaching out to VMware support, they immediately understood what was the issue.

Problem

One or more Datastore volume  in vcenter appears as “not consumed” in storage devices. and you cant access the VMs on this datastore.

Datastore volume not consumed
Datastore volume  in vcenter appears as “not consumed” in storage devices

 

 

Cause

Missing partition table on the storage device.


To further investigate this, open SSH to one of your ESxi servers and run this piece of code:

offset="128 2048"; for dev in `esxcfg-scsidevs -l | grep "Console Device:" | awk {'print $3'}`; do disk=$dev; echo $disk; partedUtil getptbl $disk; { for i in `echo $offset`; do echo "Checking offset found at $i:"; hexdump -n4 -s $((0x100000+(512*$i))) $disk; hexdump -n4 -s $((0x1300000+(512*$i))) $disk; hexdump -C -n 128 -s $((0x130001d + (512*$i))) $disk; done; } | grep -B 1 -A 5 d00d; echo "---------------------"; done

Output:

A bad partition table will look like this and will be missing the partition table. —– This Datastore was not consumed in vc

/vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d 

gpt

1336746 255 63 21474836480

Checking offset found at 2048:

0200000 d00d c001

0200004

1400000 f15e 2fab

1400004

0140001d 44 53 5f 50 52 4f 44 5f 30 31 00 00 00 00 00 00 |DS_PROD_01……|

0140002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|

A good partition table will look like this. see the bolded partition table. note the partition table highlighted in yellow:

/vmfs/devices/disks/naa.6742b0f0000004fb00000000001941b7

gpt

1336746 255 63 21474836480

1 2048 21474836446 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

Checking offset found at 2048:

0200000 d00d c001

0200004

1400000 f15e 2fab

1400004

0140001d 44 53 36 5f 50 52 4f 44 5f 31 32 00 00 00 00 00 |DS6_PROD_12…..|

0140002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|

Fix

Disclaimer: Vmware recommends to not perform the fix by yourself and open a support ticket at Vmware

now that you know that VMware advises not to do this on your own. To fix this, we need to recreate the partition table.

Run partedUtil getUsableSectors to find the last usable sector on the volume naa.xxxxxxxxxx

[root@esx006:~] partedUtil getUsableSectors /vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d

34 21474836446

The output we got: 21474836446 – this number is the last usable sector on the volume. We need this for the next command.

Now run partedUtil setptbl using the last usable sector.

Note that 2048 is always the first usable sector and is the same for vmfs5/6

your command should look something like this:

partedUtil setptbl /vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d gpt "1 2048 21474836446 AA31E02A400F11DB9590000C2911D1B8 0"

Thats it. now run the first code again to recheck for missing partition tables on other datastores and on the specific datastore you originally  had the “not consumed”  issue.

How to Create a Shortcut Using Powershell

How to create a desktop shortcut using Powershell

Sometimes I find that simple tasks like creating a shortcut to an Application using a script are not straight forward. Looking for a simple way to place a shortcut on multiple users desktops using a Powershell script lead me to tools like mklink (which creates symbolic links, but not shortcuts).

surprisingly, I did not find a straight forward method to accomplish this.

However, I did find a way to create shortcuts using VBScript. Being more of a Powershell guy, I decided to convert this to a Powershell function in order to be able to reuse it as I see fit.

Continue reading How to Create a Shortcut Using Powershell

Detach LUNs from Multiple ESXi Hosts Using Powercli

Hi there

Migrating an old VMWare Vcenter environment to newer vcsa 6.5 can be challenging. In the process I found myself doing other tasks like converting RDMs (Mapped Raw LUN) to .vmdk files.

Migrating RDMs attached to virtual machines on my Vcenter 5.5 environment to .vmdk disk files appeared like a tedious handy task especially when you have these LUNs attached to many ESXi hosts. With the help of powershell and Powercli the task becomes much easier.

Continue reading Detach LUNs from Multiple ESXi Hosts Using Powercli

Using Powershell to Trace the Source of Account Lockouts in Active Directory

How to: track the source of user account lockout using Powershell

In my last post about how to Find the source of Account Lockouts in Active Directory I showed a way to filter the event viewer security log with a nifty XML query.

In this post I recomposed (Source:Ian Farr) a Powershell script which will ask for the locked user account name and then will scan the active directory DCs security log for relevant events and will present the user lock time and source of the lock out like so:

locked user powershell script

Continue reading Using Powershell to Trace the Source of Account Lockouts in Active Directory

Find the source of Account Lockouts in Active Directory

How to: Find the source of Account Lockouts in Active Directory

It is a very common problem in Active Directory when Users change their password in a domain environment, they might get locked out repeatedly and it can be a frustrating process to identify the source of the lockout.

Try the following steps to track the locked out user and also find the source of AD account lockouts. This procedure assume that you know the username which is locked out.

I recommend doing this procedure on ALL your Domain Controllers, not only your PDC Server, just to be sure you didn’t miss a lockout event. Continue reading Find the source of Account Lockouts in Active Directory

System Administrators Tricks