As you will be well aware, Intel discovered a vulnerability that affected their chips and almost every vendor is affected by this issue. Modern CPUs perform something called Speculative Execution. This essentially means that CPUs perform tasks that they presume are going to happen, but have not actually been requested by the OS to perform (yet).
Speculative execution often makes a read to system memory that should have otherwise been protected or inaccessible. One possible example of how this can be used is an attacker might be able to read sensitive information in the system’s memory, information like secret keys, passwords and application data like personal information.
Why this becomes really dangerous is that during pen-testing it was shown that an attacker using speculative execution on a VM, created a pathway to that VM’s host and then ping-ponged to other VM’s on that host. This is scary shit, especially if you run a public cloud where one company’s VM is on a host with other company’s VM’s.
Microsoft managed to patch Azure in a few days… like hundreds of thousands of physical servers, that in itself is worth taking a moment to think about. In the blog post-Microsoft let you know that they have patched everything and you will have to reboot your servers for the patch to take effect. Companies were given a grace period, and then mandatory reboots would have to take place (forced by Azure whether you want it or not). While I can understand people feeling that Microsoft is removing your free will, I liken this to forcing all road users to use child seats and safety belts.
So let’s be clear about a few things before we go on;
- Azure was updated in jig time
- You were given some time to reboot on your schedule
- If you didn’t update, you would then be forced to reboot
- However, Microsoft’s SLA commitments of Availability Sets, VM Scale Sets, and Cloud Services were upheld (meaning if you placed your mission-critical workloads under these availability protections you did not see application downtime)
Knowing what you now know about this vulnerability and the possibility of an attacker compromising a VM and “possibly” gain access to other VM’s – Your VM’s! You have to understand why this was a mission-critical patch, end of story.
Now say this reboot was outside your patch cycle and so you are being forced to do an out of band patch… If you feel disappointed with the Azure platform and get onto your Microsoft account team and give them hell that is your choice. If they did nothing or were slow off the mark we would lose faith in the cloud.
In July 2016 Microsoft introduced In-place Migration where a VM is paused for no more than 30 seconds during a host security patch, before this, a host patch required a full restart to VM’s on that host. Microsoft’s goal is to eliminate the need for virtual machine reboots for all updates to host operating systems. In Azure, there are a lot of technologies to help keep VM’s up and going like Update Domains, Fault Domains, Availability Set’s and Affinity Groups.
A single VM in Azure has a guarantee of 99.9% uptime, that’s a total downtime in a given month of 21.6 minutes. You can get that general purpose VM for 45 bucks a month, and a B class for 10 bucks a month or less. I would love to see any enterprise calculation that shows that equivalency on-prem*.
If you are leading an IT org and have chosen not to apply this patch you take a massive risk. The risk can be assessed against these points;
- The world’s leading, highest paid security professionals decided it was mandatory
- The world’s largest and most successful IT companies decided that all the cost of doing this upgrade was justified
- There are already publicly available weaponized hacks, exploiting this vulnerability
Many thanks to Aidan Finn, Susan Bradley, Riis Flemming, and Shamir Charania, the fellow MVP’s that discussed this issue with.
*Reputation health warning – making a claim in the comments that you can provide enterprise-class hosting for a VM for less than 10 bucks a month is more likely going to make people think you are a moran then a genius, but your call.