VMware datastore volume in VCenter appears as “not consumed” in storage devices

last week I had an issue after Firmware update and restarting one ESXi server. I did some checks and noticed that one datastore appeared as “not consumed” and was inaccessible. I could not reach the VMs on that datastore of course.

Reaching out to VMware support, they immediately understood what was the issue.

Problem

One or more Datastore volume  in vcenter appears as “not consumed” in storage devices. and you cant access the VMs on this datastore.

Datastore volume not consumed
Datastore volume  in vcenter appears as “not consumed” in storage devices

 

 

Cause

Missing partition table on the storage device.


To further investigate this, open SSH to one of your ESxi servers and run this piece of code:

offset="128 2048"; for dev in `esxcfg-scsidevs -l | grep "Console Device:" | awk {'print $3'}`; do disk=$dev; echo $disk; partedUtil getptbl $disk; { for i in `echo $offset`; do echo "Checking offset found at $i:"; hexdump -n4 -s $((0x100000+(512*$i))) $disk; hexdump -n4 -s $((0x1300000+(512*$i))) $disk; hexdump -C -n 128 -s $((0x130001d + (512*$i))) $disk; done; } | grep -B 1 -A 5 d00d; echo "---------------------"; done

Output:

A bad partition table will look like this and will be missing the partition table. —– This Datastore was not consumed in vc

/vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d 

gpt

1336746 255 63 21474836480

Checking offset found at 2048:

0200000 d00d c001

0200004

1400000 f15e 2fab

1400004

0140001d 44 53 5f 50 52 4f 44 5f 30 31 00 00 00 00 00 00 |DS_PROD_01……|

0140002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|

A good partition table will look like this. see the bolded partition table. note the partition table highlighted in yellow:

/vmfs/devices/disks/naa.6742b0f0000004fb00000000001941b7

gpt

1336746 255 63 21474836480

1 2048 21474836446 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

Checking offset found at 2048:

0200000 d00d c001

0200004

1400000 f15e 2fab

1400004

0140001d 44 53 36 5f 50 52 4f 44 5f 31 32 00 00 00 00 00 |DS6_PROD_12…..|

0140002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…………….|

Fix

Disclaimer: Vmware recommends to not perform the fix by yourself and open a support ticket at Vmware

now that you know that VMware advises not to do this on your own. To fix this, we need to recreate the partition table.

Run partedUtil getUsableSectors to find the last usable sector on the volume naa.xxxxxxxxxx

[root@esx006:~] partedUtil getUsableSectors /vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d

34 21474836446

The output we got: 21474836446 – this number is the last usable sector on the volume. We need this for the next command.

Now run partedUtil setptbl using the last usable sector.

Note that 2048 is always the first usable sector and is the same for vmfs5/6

your command should look something like this:

partedUtil setptbl /vmfs/devices/disks/naa.6742b0f0000004fb0000000000160b9d gpt "1 2048 21474836446 AA31E02A400F11DB9590000C2911D1B8 0"

Thats it. now run the first code again to recheck for missing partition tables on other datastores and on the specific datastore you originally  had the “not consumed”  issue.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.