Forty minutes on PyPI
On March 24, 2026, between 10:39 and 16:00 UTC, two malicious versions of the LiteLLM Python package were live on PyPI. About 40 minutes from upload to detection, then a few more hours before removal. LiteLLM gets pulled roughly 3.4 million times a day. The blast radius from that window is still being mapped.
A week later, AI staffing startup Mercor disclosed a breach involving roughly 4TB of data: video interviews, KYC documents, internal source code, and reportedly records on how multiple frontier AI labs select training data and apply safety labels. The Lapsus$ group claimed credit. Mercor confirmed it was "one of thousands of companies" affected by the LiteLLM supply chain compromise that the security firm Snyk traced back to a threat actor known as TeamPCP.
If you run an AI gateway in production, a model proxy, a fine-tuning pipeline, or an agentic system that calls LLMs through any kind of router, you have new homework. The blast radius of a compromised LLM proxy is not the same as a compromised log shipper. It sits at the most credentialed layer of your stack.
The attack chain in one paragraph
The compromise did not start at LiteLLM. Attackers exploited a pull_request_target workflow misconfiguration in Trivy, the open source vulnerability scanner, to exfiltrate maintainer credentials. They then rewrote Git tags in the trivy-action GitHub Action repository to point a normal-looking version at a malicious release. LiteLLM's CI/CD pipeline ran Trivy as part of its build, installing it from apt without a pinned version, so the next build pulled the malicious tag. That payload extracted LiteLLM's PYPI_PUBLISH token from the GitHub Actions environment. The attackers then published litellm versions 1.82.7 and 1.82.8 to PyPI using a real maintainer credential. See LiteLLM's official advisory for the full timeline.
This is the now-canonical shape of a modern supply chain attack. A security tool compromises a build pipeline. The build pipeline publishes a trusted package. The trusted package roots itself in production. None of the steps look anomalous in isolation.
Why an AI gateway compromise hurts more
Three properties of an LLM proxy make it a higher-value target than most Python dependencies:
- It usually holds API keys for every frontier model your organization uses (Anthropic, OpenAI, Bedrock, Vertex). Those keys are often not scoped per workload.
- It usually holds the encrypted system prompts, tool schemas, and routing logic of every AI feature you have shipped. That is a map of your AI strategy on disk.
- It usually sits inside the same Kubernetes namespace as your other backend services, with cluster-internal traffic that nobody inspects.
Now look at what version 1.82.8 actually did. It dropped a file called litellm_init.pth into the Python site-packages directory. Python automatically executes any .pth file at interpreter startup, regardless of whether litellm is ever imported by the running process. So the malware ran on every Python interpreter on the host, including unrelated workloads sharing the same image base.
The payload then harvested SSH keys, environment variables, Kubernetes service account tokens, AWS IMDSv2 credentials, GCP and Azure tokens, Docker configs, and cryptocurrency wallets, encrypted them with AES-256-CBC, and exfiltrated to models.litellm.cloud. That is a lookalike domain, not a real LiteLLM endpoint. The attackers also installed persistence via systemd services and deployed privileged pods named node-setup-{node_name} in the kube-system namespace. Snyk's full breakdown lists the indicators of compromise.
If you ran 1.82.7 or 1.82.8 anywhere, treat those hosts as compromised regardless of whether the malicious code "executed" in your application sense. It already did.
What you should ship this week
1. Pin model proxy and gateway dependencies, hard
Unpinned dependencies are how the attackers got the PYPI_PUBLISH token in the first place. Apply the same lesson to your own AI infrastructure installs. If you use LiteLLM, pin to a known-clean range:
# pyproject.toml
[project]
dependencies = [
"litellm>=1.83.0,<2.0.0",
]
Better, install with hash verification. pip install --require-hashes will refuse to install if the artifact does not match a recorded SHA-256:
# requirements.txt
litellm==1.83.0 \
--hash=sha256:<published-hash-from-pypi>
Hash pinning would have stopped the malicious 1.82.7 and 1.82.8 from installing even if they had matched a version range, because the artifact hashes did not match a previously observed release.
2. Pin GitHub Actions to commit SHAs, not tags
Tag rewriting is the underlying primitive. A tag in a GitHub Action repository is mutable; an attacker with maintainer access can move it. Commit SHAs are immutable. The GitHub-recommended pattern is:
# .github/workflows/ci.yml
- name: Run Trivy
uses: aquasecurity/trivy-action@76071ef0d7ec797419534a183b498b4d6c643a14 # v0.28.0
Renovate and Dependabot both support SHA pinning with a comment annotation that keeps the version readable. There is no excuse to use a bare tag for a third-party action in 2026.
3. Run model gateway containers, not raw pip installs
The official ghcr.io/berriai/litellm Docker image was not affected because its dependencies are pinned in the image's requirements.txt at build time. Containerizing your AI gateway, with a pinned base layer and a pinned set of dependencies, gives you a single image hash to verify and roll forward atomically. It also means a pip install litellm in some random Jupyter notebook does not become your production attack surface.
4. Scope model API keys per workload and rotate fast
If your gateway holds one shared ANTHROPIC_API_KEY, every workload using the gateway shares the blast radius of any compromise. Use per-workload IAM where the model provider supports it (Bedrock supports IAM, Vertex supports service accounts), and short-lived virtual keys where it does not. LiteLLM's own virtual keys feature issues per-team keys you can revoke without rotating provider credentials. For Bedrock specifically, the same pattern we wrote up in our HIPAA Bedrock deployment guide applies: separate KMS CMKs and tightly scoped bedrock:InvokeModel policies per workload.
5. Add an outbound egress allowlist for model traffic
The attackers exfiltrated to models.litellm.cloud, a lookalike domain. An egress allowlist that restricts the model gateway pod to known model API endpoints (api.anthropic.com, api.openai.com, your Bedrock VPC interface endpoint) would have failed the exfiltration call. This is one VPC config plus a few Kubernetes NetworkPolicy resources. It is one of the highest leverage AI infrastructure controls you can ship this quarter.
6. Scan for the indicators of compromise
If you ran any version of LiteLLM in March 2026, run these checks across your fleet:
# Persistence file dropped by 1.82.8
find / -name 'litellm_init.pth' 2>/dev/null
# Rogue Kubernetes pods
kubectl get pods -n kube-system | grep '^node-setup-'
# Egress in your VPC flow logs or service mesh
# Filter for: models.litellm.cloud OR checkmarx.zone
Treat any hit as a credential rotation event. Cloud keys, SSH keys, Kubernetes service account tokens, model provider keys, and database passwords on the affected hosts. All of them.
A pre-deployment gate for AI infrastructure dependencies
Before you add or upgrade any AI infrastructure library (proxy, eval framework, orchestration, vector store client, MCP server), three checks should be table stakes:
- Hash-pinned in your dependency manifest. No floating versions, no tag-only references, no
latestDocker tag. - Installed inside a versioned container image. No host-level pip into shared environments.
- Egress-restricted at runtime. The pod can only reach model endpoints you have explicitly allowed.
If your environment cannot answer "yes" to all three for any AI library you ship, you are taking on the same shape of risk that the LiteLLM users took on, regardless of which library you actually use. The next compromise will not be LiteLLM. It will be something else with a similar role and a similar blast radius. The same playbook holds for MCP servers, where we covered the broader threat model in our MCP enterprise security guide.
The bigger frame
Open source AI tooling is moving very fast. Vulnerability scanners, LLM proxies, MCP servers, agent frameworks, and eval libraries all ship multiple times a week, often by small teams who are doing real work and do not have a dedicated security organization behind them. That pace is part of what makes the ecosystem useful. It also means each one of these projects is a single supply chain link, and the link is shorter than it used to be.
The right response is not to stop using open source AI infrastructure. The right response is to treat every AI dependency the way you treated your payment processor SDK ten years ago: pinned, audited, scoped, sandboxed, and monitored. The credential surface is the same. The threat actors are the same. The maturity of the controls has to catch up.
About the author
Kai Token leads engineering at Fraktional. We help teams adopt AI without inheriting the security debt of moving fast, and we treat AI infrastructure with the same discipline we apply to the rest of the production stack. Less novelty, more pinning.
