TheThe Wiz incident response and research teams are ready to assist customers with the SolarWinds attack. Contact us 24x7 at team@wiz.io. If you're not a Wiz customer, you can get a free one-time scan of your cloud infrastructure for SolarWinds threats by following this link.
Latest update: SolarWinds attackers suspected of infiltrating additional 3rd party supply chain vendors, this time a cloud-based vendor providing them immediate access to accounts. More updates to follow.
Fast Facts
The SolarWinds cyber-attack is turning into the most sophisticated and damaging nation state cyber campaign we have seen in recent years.
Initially the attack seemed to be focused on on-prem networks, but it quickly became clear that the attackers were after Office 365 emails systems and other cloud assets. It’s a new form of hybrid cyber-attacks involving on-premises and cloud assets.
The majority of guidance on the web is focused on SOC & IR teams, leaving cloud security teams unclear about immediate steps to assist with detection and response efforts to the SolarWinds attacks.
Wiz has put together guidance, tools, and procedures for cloud security teams in the wake of this attack, and we will continuously update this blog with up-to-date information.
Background & Timeline
What seemed to be a targeted attack against FireEye just weeks ago, quickly turned out to be “one of the worst cyber-espionage campaigns ever suffered by the U.S”.
Jan 13th - SolarWinds attackers connected to Mimecast attack
Infiltrating a cloud service providers represents a new and dangerous type of supply chain attack. By getting access to the vendor secret key, the attackers were able to gain covert and undetectable access to thousands of customers. Unlike on-prem supply chains, the cloud supply chain gives attackers immediate access to data by leveraging the provider’s cloud API permissions. More information on the attack here.
Jan 12th - Additional malware and more links to Russia found
Kaspersky discovers links to a known Russian malware. More info here. CrowdStrike discovers additional malware (SUNSPOT) that injected SUNBURST via CI/CD. More info here.
Jan 6th - JetBrains TeamCity may have been hacked as well and CISA updates guidance
Several news outlets published that the FBI is looking into JetBrains TeamCity as a potential source of the SolarWinds attack. See here.However, JetBrains denies involvement See here.
CISA updates guidance on SolarWinds compromise. More info here.
Jan 5th - US officially names Russia as perpetrator of the attack
In a joint official statement CISA, NSA & FBI claim that “This work indicates that an Advanced Persistent Threat (APT) actor, likely Russian in origin, is responsible for most or all of the recently discovered, ongoing cyber compromises” Link here.
Jan 2nd - The scope of the attack continues to grow
At least 250 federal agencies and organizations have been breached.Reports on cost savings at SolarWinds surface with massive cuts in the security budget in recent years and moving of development to Eastern Europe. Link here.
Dec 25th - Early indications for cloud-only attack vectors
CrowdStrike reports an attempted attack against their Office 365 tenant by the same threat actor. The adversaries used a supply chain attack method that leveraged credentials of the cloud reseller to attempt access to Office 365 emails. Find details here.
Dec 17th - Hybrid attacks: the attackers leveraged initial access to expand into cloud resources
MSRC, the Microsoft research team, reports the adversaries also attacked cloud systems. They describe a new attack method, dubbed “Golden SAML”, targeting Office 365 by leveraging SAML tokens generated by the attackers. Click link for details.
Palo Alto Networks discovers a second attack vector leveraging a zero-day authentication bypass exploit in the SolarWinds product codenamed SUPERNOVA, find details here.
Dec 14th - Reports on a large-scale cyber campaign leveraging the SolarWinds attack vector
CISA issues an alert on a large scale compromise of multiple US agencies with early indications of additional attack vectors leveraging SAML. Link here.
Dec 13th - FireEye discovers the initial attack vector: SUNBURST
First report by FireEye on a suspected breach of their red team tool set, see link.FireEye discovers the initial attack vector, codenamed SUNBRUST, see link.
Cloud attack vectors related to the breach
The attackers leveraged their initial foothold to expand into Azure Active Directory (AD) and Office 365 services, as well as other cloud resources; making this one of the first large-scale cyber campaigns that is truly hybrid and requires on-premises and cloud security teams to work together.
The impact on cloud security teams however, goes far beyond Office 365 email APIs. As depicted in the diagram above, there are multiple cloud attack vectors security teams should be aware of:
Direct exposure to SolarWinds deployed in the cloud – Key questions cloud security teams need to answer first include “Do we have SolarWinds products running in the cloud? What permissions do they have? What versions are they running on?”. This will allow them to get an initial idea of whether there is a direct risk for exposure.
Lateral movement risk – The attackers showed ability to escalate from on-premises to cloud systems and remain dormant while collecting intelligence. The same methods used to escalate into Office 365 can be used to access into Azure (Azure AD is a shared resource) and also AWS (most customers have some degree of on-premises trust into AWS).Long term access – In the past, these attackers have shown a high level of persistency in affected organizations, continuously coming back from different backdoors. Some US security experts went as far as to say that “Ridding so many sprawling federal agencies of the S.V.R. may be futile and that the only way forward may be to shut systems down and start anew.” A worst case scenario for organizations.
Long term access via identity backdoors – Attacks like “Golden SAML” (additional details below), allow attackers to create untraceable backdoors leveraging vulnerabilities in the identity trust chain.
Long term access via backdoors – In parallel to novel techniques, the fact that they went after the FireEye red team tool set shows there is a real and immediate risk that they will continue and move laterally within affected organizations.
As more details of the attack are revealed, we are starting to understand the real scope of the threat. Even organizations that don’t leverage SolarWinds, may still be exposed as a result of attacks against their supply chain vendors like cloud resellers. The attack against CrowdStrike earlier this year is a prime example of this scenario, which is why it’s key for all cloud security teams to run an end-to-end discovery.
Detection & Response Recommendations
It’s important to act now.
Each attack requires time and a skilled team, which is why the attacker’s strategy is to remain dormant in most accounts for extended periods of time. Defenders need to act fast to detect the dormant threat and remove it before it becomes active. Although the SolarWinds breach potentially affected over 18,000 customers, only a small fraction has reported an active cyber campaign so far.
A highly sophisticated threat actor.
It should be assumed that the attackers covered their tracks and have removed evidence of the initial attack. Hence, the goal for defenders is to detect and remove any persistency that attackers might have gained. There are two types of persistency to look for: Backdoor persistency and Identity persistency. Backdoors based on trojanized DLLs are a known threat, but identity persistency is a new threat for most teams due to lack of knowledge and tools to detect this type of threat.
We have compiled a quick check list of activities for security teams to determine if they have been affected. It’s expected that in most organizations these efforts will require numerous tools and the involvement of multiple, internal teams.
1. Identify: Detect the use of SolarWinds in your cloud
The immediate questions security teams need to answer are “Do we have SolarWinds deployed anywhere in our cloud?” and “Which versions are being used?”
Although seemingly simple questions, it’s often difficult to answer them across large scale cloud environments, because existing tools are built for on-premises environments and only provide partial coverage for cloud resources.
| Tools | What to do? | Relevance and limitations for cloud |
|---|---|---|
| Software asset management such as ServiceNow | Search for SolarWinds and affected versions | Irrelevant for the cloud. Most companies don’t have sufficient coverage with these legacy tools in cloud environments. |
| Configuration managers such as Tanium and SCCM | Search for SolarWinds and affected versions, see link here for Tanium. | Irrelevant for the cloud. Most companies don’t have sufficient coverage with these legacy tools in cloud environments. |
| Vulnerability assessment such as Tenable and Qualys | Search for known vulnerabilities and affected versions of SolarWinds, see link here from Tenable. | Partial relevance. Due to complex deployment, most companies have very limited coverage with these tools in the cloud. |
| First party vulnerability assessment solutions such as AWS Inspector and Azure Security Center. | Search for SolarWinds deployments via AWS Inspector, no official guidance published by AWS. | Partial coverage. Depends on VMs that enable the SSM agent, no coverage for containers. No consistent cloud native support across Azure & GCP. |
| Next-gen cloud visibility tools like Wiz | Single scan across the entire cloud footprint. Look for SolarWinds deployments and versions. | Full coverage. The entire environment can be scanned, without agents across VMs, containers and different cloud providers. |
Impact analysis – one of the key reasons that SolarWinds Orion is such a prime target for attackers is that the network monitoring appliance has the “keys to the kingdom”. It holds a massive amount of credentials to critical systems such as routers, network components and cloud API keys and has wide network access to multiple networks. The problem cloud defenders face is that legacy security solutions focus on software & vulnerabilities but fail to provide impact analysis as they lack the context of the environment.
2. Detect compromise: Find known IOCs associated with the attack
There are known indicators of compromise (IOC) for network addresses used by the malware, and signatures for SUNBURST, SUPERNOVA and the stolen FireEye red team tools. Scanning for all of these IOCs across the cloud environment requires a combination of EDR, network scanners, and cloud native tools.
| Tools | What to do? | Relevance and limitations for cloud |
|---|---|---|
| EDR tools such as CrowdStrike and Microsoft Defender ATP | All major EDR vendors released guidance for SolarWinds, here is the CrowdStrike link. | Partial coverage. EDRs depend on agents and suffer from limited coverage in most cloud environments. |
| SIEM solutions such as Splunk, Sumo Logic, Azure Sentinel | All major SIEM solutions released guidance for SolarWinds, here is the Splunk link, Azure Sentinel link. | Complex & expensive. Streaming cloud logs to SIEM solutions to gain visibility into threats is critical. Streaming and storing cloud logs is a complex and expensive task as it may generate large amounts of logs. |
| Cloud native network detection tools such as AWS GuardDuty | AWS GuardDuty may be able to provide network detections. The vendor didn’t provide any guidance. | Not clear. AWS GuardDuty has network detection alerts but unfortunately it doesn’t provide clear explanation on support for SolarWinds detections. |
| Next-gen cloud visibility tools like Wiz | Single scan across the entire cloud footprint. Look for alerts on files with known IOCs. | Full coverage for file scans. Agentless scanning allows for seamless coverage of all compute nodes, VMs and containers for IOCs, however it cannot scan for memory-only threats. |
Scanning backups: Restarting from backups is a good way to revert and start fresh but before you do, it’s important to scan existing backups to ensure they aren’t compromised as well. Agentless scanners like Wiz already support the scanning of these snapshots.
3. Lateral movement: Detect abuse of cloud identities
The SolarWinds campaign is one of the first large-scale cyber-attacks that leverages multiple techniques to move laterally from on-premises to cloud resources. The attackers used novel techniques to gain identity persistency such as Golden SAML to bypass authentication mechanisms and MFA protections.
Here is what CISA shared in their alert:
CISA has observed the threat actor adding authentication tokens and credentials to highly privileged Active Directory domain accounts as a persistence and escalation mechanism. In many instances, the tokens enable access to both on-premise and hosted resources.
The attackers leverage their initial foothold to gain access to dormant highly privileged accounts that can later be used to stage a future attack. There are three aspects of cloud identity backdoors that make this attack vector a gamechanger:
‘Leave no trace”. A skilled attacker can erase all evidence of the initial attack path and leave only an identity backdoor for future use.
Identity backdoors are untraceable. With current tools. Imagine an attacker made a copy of the signing certificate (a.k.a GOLDEN SAML attack) and subsequently removed all evidence of the attack. There is virtually no backdoor that a defender can detect because nothing was left behind.
Identity backdoors are accessible from anywhere – at any point in the future, the attacker can generate a legitimate looking SAML token and use it to access cloud resources.
Even though the attack may have started on-premises, the attacker now has an ultimate stealth backdoor. Due to the nature of ubiquitous access to cloud resources, it can be used from anywhere.
“Breaking the trust” – How SolarWinds attackers exploited our identity trust chain
Each of the methods the attackers use takes advantage of a different component of the cloud identity chain of trust. Let’s break down each of these trust relationships. See this link for a detailed analysis.
| Description | Detection & Mitigation | |
|---|---|---|
| Golden SAML exploiting on-prem to cloud trust | The attacker gains admin access to the server responsible for issuing SAML certificates (usually ADFS) The attacker clones the signing certificateAt any point in time, the attacker can generate a legitimate SAML certificate and login as any user | There are no existing commercial tools in the market that are able to easily detect this attack. Microsoft released a detailed list of recommended events to monitor, here. The best mitigation to prevent attackers from stealing the signing certificate, is to never store it. Using a cloud identity service (AAD or Okta) eliminates the need for on premise signing certificate all together. |
| Account synchronization exploiting on-prem to cloud trust | The threat actor gets admin access to on premise AD and changes passwords of privileged identities. These passwords are later synced to Azure AD. | There are no existing commercial tools in the market that are able to easily detect this attack. Monitoring for suspicious login events might provide some coverage. In terms of mitigation, the recommended approach is to avoid syncing passwords from on-prem. See here. |
| Modify Federation Trusts exploiting Federation-provider-to-cloud trust | The attacker adds a new federation trust or updates an existing federation trust with attacker-owned certificates | Monitor for changes in federation providers. |
| Adding credentials to OAuth apps exploiting the OAuth-to-cloud trust | The attacker adds their own credentials to existing OAuth apps, in some cases the attacker adds permissions to existing apps to create a future-ready backdoor | Monitor for new highly privileged OAuth apps, monitor for changes in OAuth apps credentials. See the section “Identifying unusual addition of credentials to an OAuth app” here. |
The reality is that the security industry has been caught off-guard in regard to protecting cloud identities. Existing tools provide only partial coverage for cloud defenders to protect from complex attacks exploiting our identity chain and it is near impossible to track 3rd parties and correlate them against access credentials and vulnerabilities. A key takeaway is the need to focus on improving visibility and monitoring of privileged identities in the cloud.
Wiz Identity Analysis – Wiz performs an analysis of cloud identities and provides cloud security teams with deep visibility over 3rd party access to their accounts including highly privileged and unused service accounts, exposed or vulnerable resources with high permissions, and more. In future blogs we will provide a more in-depth overview of ways Wiz can assist cloud defenders in taking back control of identities in the cloud.
How can Wiz help you?
As you go through the list of recommendations, it is apparent that fighting these kinds of threats in the cloud with current day tools is a near impossible task. It requires deployment of multiple tools, while most of them have major coverage gaps in the cloud and only provide a partial view. Wiz was built with these fundamental challenges in mind and the goal to change the status quo by giving cloud defenders a cloud visibility tool that works across the entire cloud stack:
Software asset management to detect SolarWinds deployments
Vulnerability assessment to flag any vulnerable versions of SolarWinds
Anti-virus scanning to detect known IOCs related to the SolarWinds attack
Identity analysis to detect suspicious high-privileged identities
Escalation path analysis to determine the potential impact of compromised entities
Customers can deploy Wiz across their cloud environment and within minutes perform a full-stack cloud scan leveraging our built-in SolarWinds detections and dashboard.
We are eager to assist the security community during these challenging times and are here to help cloud security teams with guidance & recommendations. If you’re trying to understand if your cloud environment is affected by the SolarWinds attack, you can get a free scan powered by Wiz and work with our cloud security experts to understand the impact!
Questions? Reach out to our team at research@wiz.io
