Friday, 15 December 2017

Steps to recover the OS Corrupted VM in Azure

  1. Stop the machine – wait until it’s stopped (deallocated)
  2. Save the XML configuration file of the VM
    1. If Azure Powershell is not installed, please install it from Microsoft Azure Powershell.
    2. Open an elevated Azure Powershell session and run the following commands to setup the environment to setup queries to your subscription
    3. Set the variables
$SubscriptionID = "<Suscription ID>"​
$StorageAccount = "<Storage Account>"
$vm = "<VM Name>"
$CloudService = "<Cloud Service>"
$VNET = "<Virtual Network>"
    1. Log in to the subscription​
Add-AzureAccount
Set-AzureSubscription -SubscriptionId $SubscriptionId -CurrentStorageAccountName $StorageAccount
Select-AzureSubscription -SubscriptionId $SubscriptionId 
    1. Stop the VM
Get-AzureVM -Servicename $CloudService -name $vm | stop-azureVM -force
    1. Export the configuration file and once you have it, validate that the file exist and has content.
Export-AzureVM -ServiceName $CloudService -Name $vm -Path C:\Temp\VM.xml
  1. Delete the VM keeping the attached disks.
Remove-AzureVM -ServiceName $CloudService -Name $vm
  1. Waits for around 5mins for Azure to release these disks so you can dispose of them freely
  2. Using Storage Explorer, make a copy of the VM’s OS disk.
  3. Attach the OS disk of AffectedServer to TSVM.
  4. Once the OS disk is attached on a working machine, open up the disk manager and ensure it is ONLINE and take note which is the drive letter assign to the partition holding the \windows folder
  1. If the disk is set to OFFLINE, then set it to ONLINE
  1. Browse up to the location of the binary that was shown on the screenshot and document on this case the version of the file that was found (righ-click\properties\Details tab)
  1. Rename it as <BINARY.SYS>.OLD. For the example, if the binary on the screenshot is \windows\system32\drivers\amdxata.sys this will be renamed as \windows\system32\drivers\amdxata.sys.old
  2. RESTORATION
    1. You can restore this file from its internal repository.
                                          i.    Launch a CMD session and locate the volume holding your Windows directory. Browse to \windows\winsxs and search for the binary displayed on your screenshot.
dir <<binary from the screenshot with extension>> /s
                                         ii.    This command will list all the different versions of that very same file the machine has, pretty much it will give you the path history of that component. You could then choose the latest of the list and proceed to copy that one on the windows\system32 folder path described on the screenshot.
copy <<drive>>:\Windows\WinSxS\<<directory_where_file_is>>\<<binary_with_extension>> <<drive>>:\Windows\System32\Drivers\
Example:
1.      The binary that we need to look for is cmimcext.sys.
2.     On the same way if the latest binary didn't work, then you can always try one version before that one like going back in time for the patch level on that component
3.     On the image below, the query is listed on C but this letter will be instead the one of the faulty drive (the OS disk attached as a data disk on the troubleshooting VM)
    1. You may use as well a working machine from this environment with the same OS and if possible the same patch level you can take the binary from a working machine and replace the corrupt binary on the affected machine however be aware that following this way may ended up on a reboot loop later on if you don't copy the correct file version the machine is waiting for:

  1. Remove the disk from the troubleshooting VM and wait till azure update the disk lease (3 mins tops)
  2. Reassemble the VM 
Proceed to create the VM from the configuration file.
Import-AzureVM -Path C:\Temp\VM.xml | New-AzureVM -ServiceName $CloudService -VNetName $vnet -ReservedIPName <STATIC IP>

Note: Recreating a VM thru its configuration file, will take attach all the disks on a VM plus its configuration on Endpoint Load Balancer, IP configuration and so on. In case of VMs with striping disks, this is the correct way to recreate it.

CLEANUP
Proceed to remove the OS Disk backup we created at the beginning of the case








































































1 comment:

  1. Wonderful post! Thanks for sharing so much of information with us. This will be really helpful for my reference. Keep sharing.
    Microsoft Azure Online Training

    ReplyDelete