How to Scan LLM Dependencies for Vulnerabilities

Large Language Models (LLMs) have rapidly moved from research prototypes into production applications. While teams spend significant effort evaluating model accuracy and prompt design, the software supply chain behind these applications often receives far less attention. Every LLM-powered application depends on a web of libraries, model weights, datasets, plugins, and API clients — each of which can introduce security vulnerabilities. This tutorial walks through what LLM dependency scanning is, why it matters, and how to implement it in your own projects.

What Is LLM Dependency Scanning?

LLM dependency scanning is the process of identifying, inventorying, and analyzing every component your LLM application relies on — then checking those components against known vulnerability databases and risk indicators. Unlike traditional dependency scanning, which focuses on package manifests like package.json or requirements.txt, LLM scanning must account for a broader surface area:

Python and JavaScript packages such as langchain, llama-index, transformers, openai, and anthropic SDKs.
Model weights and artifacts downloaded from Hugging Face Hub or other registries.
Plugins, tools, and agents that execute arbitrary code or call external services.
Datasets and embeddings that may contain poisoned or sensitive data.
System-level dependencies like CUDA, ONNX runtime, or vector database engines.

A complete scanning strategy combines Software Composition Analysis (SCA), model artifact inspection, and runtime behavior monitoring.

Why It Matters

LLM applications inherit the same supply chain risks as any modern software project, plus several unique threats. A vulnerable version of a popular orchestration library can allow prompt injection, arbitrary code execution, or server-side request forgery. Malicious model files uploaded to public registries have already been caught containing pickle-based deserialization exploits. Because LLM apps frequently run with broad permissions — filesystem access, network egress, database credentials — a single compromised dependency can lead to full system compromise.

Regulatory frameworks such as the EU AI Act and executive orders on AI safety also increasingly require organizations to document and secure their AI supply chains. Scanning dependencies is the foundational step in meeting those obligations.

Setting Up a Scanning Pipeline

Step 1: Inventory Your Dependencies

Before scanning, you need a complete inventory. For Python-based LLM projects, start by exporting your dependency tree. The following command captures both direct and transitive dependencies in a machine-readable format:

pip install pip-audit
pip freeze > requirements-lock.txt
pip-audit -r requirements-lock.txt -f json -o audit-report.json

The pip-audit tool queries the PyPI Advisory Database and OSV.dev to flag known CVEs. For Node.js projects that use the OpenAI or Vercel AI SDKs, use npm audit:

npm install -g better-npm-audit
npm audit --json > npm-audit-report.json

Step 2: Scan Hugging Face Model Artifacts

Model files are a frequently overlooked attack vector. Hugging Face supports a built-in security scanner that checks repositories for malicious code patterns. You can query it programmatically before downloading any model:

import requests

def scan_hf_model(repo_id: str, token: str | None = None) -> dict:
    headers = {"Authorization": f"Bearer {token}"} if token else {}
    url = f"https://huggingface.co/api/models/{repo_id}/scan"
    response = requests.get(url, headers=headers, timeout=30)
    response.raise_for_status()
    return response.json()

report = scan_hf_model("meta-llama/Llama-3.1-8B")
print(report)

If the scan returns findings, do not load the model with transformers or safetensors until you have reviewed the report. Prefer models that use the safetensors format over legacy pickle or torch.save formats, since safetensors prevents arbitrary code execution during deserialization.

Step 3: Integrate SCA Into CI/CD

Dependency scanning should run on every pull request and nightly against the main branch. Below is a GitHub Actions workflow that runs pip-audit, checks Hugging Face artifacts, and fails the build on high-severity findings:

name: LLM Dependency Scan

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    - cron: "0 3 * * *"

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install tools
        run: pip install pip-audit requests

      - name: Audit Python packages
        run: |
          pip install -r requirements.txt
          pip-audit -r requirements.txt -f sarif -o results.sarif || true

      - name: Scan HF models
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
        run: python scripts/scan_models.py

      - name: Upload SARIF report
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: results.sarif

The scripts/scan_models.py script can iterate over a manifest file that lists every model your application loads:

import json
import sys
import requests

MODELS = [
    "meta-llama/Llama-3.1-8B",
    "sentence-transformers/all-MiniLM-L6-v2",
    "BAAI/bge-large-en-v1.5",
]

def main(token: str | None) -> int:
    failures = 0
    for repo_id in MODELS:
        headers = {"Authorization": f"Bearer {token}"} if token else {}
        url = f"https://huggingface.co/api/models/{repo_id}/scan"
        r = requests.get(url, headers=headers, timeout=30)
        if r.status_code != 200:
            print(f"[WARN] No scan available for {repo_id}")
            continue
        data = r.json()
        if data.get("status") == "infection":
            print(f"[FAIL] {repo_id} flagged as infected")
            failures += 1
        else:
            print(f"[OK] {repo_id} clean")
    return 1 if failures else 0

if __name__ == "__main__":
    import os
    sys.exit(main(os.environ.get("HF_TOKEN")))

Advanced Scanning Techniques

Detecting Typosquatting and Malicious Packages

LLM libraries are frequent targets for typosquatting attacks. Packages such as langchain-community have been impersonated by malicious lookalikes. Use pip-audit alongside tools like safety and socket to detect suspicious packages:

pip install safety
safety check --full-report --output json > safety-report.json

For npm projects, Socket.dev analyzes package behavior for network access, filesystem manipulation, and install-time scripts:

npx socket scan

Scanning LangChain and LlamaIndex Tool Dependencies

Orchestration frameworks dynamically load tools and integrations. Each integration pulls in its own dependencies. Audit them explicitly by generating a dependency graph:

pip install pipdeptree
pipdeptree --packages langchain,langchain-community,llama-index --graph svg > dep-graph.svg

Review the graph for unexpected packages, especially those with install-time hooks or network calls during import.

Runtime Monitoring With OWASP DepScan

Static scanning catches known CVEs but misses novel attacks. OWASP DepScan performs deep analysis including license risks and vulnerable call paths. Run it against your project:

pip install depscan
depscan --src . --reports-dir ./depscan-reports

Best Practices

Pin every dependency. Use lockfiles (requirements-lock.txt, poetry.lock, package-lock.json) and never resolve to floating ranges in production.
Prefer safetensors over pickle. When downloading models, reject any repository that only offers .bin or .pt files unless you fully trust the publisher.
Use private model registries. Mirror approved models into a private Hugging Face organization or an internal artifact store, and pull only from there in production.
Scan continuously, not once. New CVEs are published daily. Schedule nightly scans and subscribe to advisory feeds for your core libraries.
Enforce policy gates. Fail CI on critical and high severity findings. Treat medium severity as warnings that require acknowledgment before merge.
Monitor runtime behavior. Use egress filtering and sandboxing so that even a compromised dependency cannot exfiltrate data or reach arbitrary endpoints.
Maintain a Software Bill of Materials (SBOM). Generate SBOMs in CycloneDX or SPDX format for every release so you can respond quickly when a new vulnerability is disclosed.
Vet plugins and agents carefully. Any tool that executes code or calls external APIs should be reviewed, signed, and loaded from a trusted source.

To generate an SBOM for a Python LLM project, use cyclonedx-bom:

pip install cyclonedx-bom
cyclonedx-py requirements -r requirements-lock.txt -o sbom.json --schema-version 1.5

Conclusion

Securing LLM applications requires treating the model and its surrounding ecosystem as part of your software supply chain. By combining traditional SCA tools like pip-audit and safety with model-specific checks against Hugging Face's scanning API, runtime monitoring, and disciplined dependency pinning, you can dramatically reduce the risk of a compromised component taking down your application or leaking sensitive data. Build scanning into your CI/CD pipeline from day one, maintain an up-to-date SBOM, and remember that the model itself is just one dependency among many — every one of which deserves scrutiny.