WithSecure incident response guidance: Log4j 2 vulnerability

The library is widely used in enterprise Java software and is included in Apache frameworks including:

• Apache Struts2
• Apache Solr
• Apache Druid
• Apache Flink
• Apache Swift

All versions of Log4j 2 are affected, including >=2.0-beta0 and <= 2.15.0-rc1.

NCSC-NL has published a list of related software which may be affected by the vulnerability. The list is maintained here: https://github.com/NCSC-NL/log4shell/tree/main/software. A secondary list of all software that uses Log4j Core library is maintained here: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/usages. Note that custom-built software may also be affected by the vulnerability.

A patched version of the Apache Log4j software was issued in response to CVE-2021-44228, however it was found to be insufficient in addressing the vulnerability in certain non-default configurations. This led to the categorization of a second related vulnerability, CVE-2021-45046. The second vulnerability allows attackers—with control over Thread Context Map (MDC) input data in instances where the logging configuration uses a non-default Pattern Layout with either a Context Lookup (for example, $${ctx:loginId}), or a Thread Context Map pattern (%X, %mdc, or %MDC)—to craft malicious input data using a JNDI Lookup pattern, resulting in a denial of service (DOS) attack.

What's the risk?

The exploitation of these vulnerabilities grants the attacker high privilege access to the compromised server. WithSecure’s threat intelligence (TI) team has observed the following post exploitation actions associated with the vulnerability:

• Deployment of crypto-currency mining software such as xmrig/kinsing
• Deployment of botnets Mirai and Tsunami
• Deployment of Cobalt Strike
• Deployment of Orcus RAT
• Ransomware deployment
• Reverse shells

The TI team has also compiled a list of observed Indicators of Compromise (IoCs) provided in STIX3 format. To request the list, please contact us.

What to do

The following advice has been referenced from the official UK NCSC Log4j 2 alert: https://www.ncsc.gov.uk/news/apache-log4j-vulnerability.

1. Patch your systems

The first priority is to patch systems to ensure that further exploitation cannot occur. This can be achieved by identifying all instances where the Log4j library is present. If the product you use in your organization is not listed, follow step 2 to identify unknown instances of Log4j.

Ensure that vendor-specific advice is followed for every affected software instance.

Start applying the official vendor patch, upgrading to log4j 2.16.0.

Note that this advice supersedes previous guidance which states that 2.15.0 is an effective patch due to the discovery of the secondary vulnerability, CVE 2021-45046.

The upgrade will require Java 8, so any identified legacy versions and related software need to be either segregated and isolated so it cannot be reached (or cannot reach out) or migrated to newer versions.

2. Identify unknown instances

Linux Systems

Java applications can include all the dependent libraries within their installation. A file system search should be undertaken on all suspected systems. This should include EAR, JAR and WAR files. For example:

find / -type f -print0 |xargs -n1 -0 zipgrep -i log4j2 2>/dev/null

If a dependency or package manager is used, this can be searched with the following command:

dpkg -l | grep log4j

Ensure that each copy identified is updated or mitigated, as there may be multiple copies.

Windows Systems

WithSecure recommends referring to Microsoft’s official guidance where service-specific advice has been provided. Microsoft’s guidance can be found here: https://www.microsoft.com/security/blog/2021/12/11/guidance-for-preventing-detecting-and-hunting-for-cve-2021-44228-log4j-2-exploitation/.

Check if any of the identified platforms use Oracle JDK or OpenJDK. Both should have the vulnerable feature disabled by default. To verify that it is disabled, run the following Java code (this command should return "false")

System.getProperty("com.sun.jndi.ldap.object.trustURLCodebasez)

3. Detection: logging, network monitoring, and exploit detection

For organizations with existing logging in a central SIEM, log files containing the string pattern “jndi:ldap” should be highlighted for further investigation. However, it should be noted that string patterns may have also been obfuscated by attackers to evade basic detection. Patterns such as lower or upper commands within the exploitation string jndi:${lower:l}${lower:d}a${lower:p}) and more complicated obfuscation attempts (${${::-j}${::-n}${::-d}${::-i}) that try to bypass string-matching detections have been observed by Microsoft.

Also note that the specific “jndi” string might not be present outside of an HTTP or similar transport protocol log, and thus simply searching for the string pattern may not suffice. WithSecure recommends investigating any confirmed vulnerable instances for signs of exploitation while also increasing wider monitoring around potentially affected systems.

For organizations with network monitoring in place, the first request in the exploitation chain for the vulnerability will look like this:

This will be followed by a request from the logging server to the LDAP server listed in the first request, which can be seen below:

The class file will contain the Java implementation of the actual command (shown in the base64 in the first request).

For any Linux systems identified in the previous steps, the following commands can be used to search for exploitation attempts:

From uncompressed /var/log/ folders and sub folders:

sudo egrep -I -i -r '\$(\{|%7B)jndi:(ldap[s]?|rmi|dns|nis|iiop|corba|nds|http):/[^\n]+' /var/log

From compressed /var/log/ folders and sub folders:

sudo find /var/log -name \*.gz -print0 | xargs -0 zgrep -E -i '\$(\{|%7B)jndi:(ldap[s]?|rmi|dns|nis|iiop|corba|nds|http):/[^\n]+'

A log4j 2 detector script developed by Florian Roth is available to download here: https://github.com/Neo23x0/log4shell-detector. The script will search for any log4shell exploitation attempts.

For Windows Systems, application specific web logs need to be searched for the string pattern (see previous caveat regarding potential obfuscation of the string):

${jndi:ldap}

PowerShell can be used to search through these log files using the command (replace $filename with the appropriate filename or path):

Select-String –Path $filename -Pattern '\$(\{|%7B)jndi:(ldap[s]?|rmi|dns|nis|iiop|corba|nds|http):/[^\n]+'

Any systems where successful exploitation attempts have been identified should be investigated for signs of post-exploitation or lateral movement.

If you have any questions or need further advice, please contact our team.