Black October: Your SBOM Is Not Enough

October has been a miserable month to be responsible for keeping systems secure. During the week of October 19, teams were dealing with critical PHP flaws, a poisoned LiteLLM release, a critical GitLab patch, a Brotli vulnerability in Node.js, compromised certificate keys, and cloud platforms telling customers to rotate their secrets. Several disclosures described attacks that had started earlier. The remediation arrived together.

The industry has taken to calling it “Black October.” The “cybersecurity apocalypse” warnings predate this week, but the exhaustion is easy to understand. You could be halfway through rebuilding your application images when the next advisory told you to replace the credentials those applications use.

There is a data problem running through these incidents. Sensitive information sits in process memory, environment variables, agent configurations, CI systems, and backups. Some of that information is a credential granting access to another system. An attacker who obtains it can keep going after the original entry point has been patched.

We need software bills of materials. We also need to stop treating a software inventory as the end of the security review.

A week of different ways in

PHP2Shell is the most visible example. Wiz demonstrated unauthenticated code execution and, separately, the ability to read sensitive PHP process memory, including passwords, tokens, and API keys. The researchers say AI helped turn longstanding flaws into practical exploits across different deployments. The NSA called for immediate patching and advised considering taking affected systems offline when immediate updates were not feasible.

The same week, Node.js disclosed a Brotli decoder flaw in which malformed compressed input could corrupt memory, potentially causing a crash, memory disclosure, or remote code execution. The input your application decompresses is part of its attack surface. That includes data received while doing ordinary application work, not only requests to a login page.

GitLab's October 21 critical release fixed a maximum-severity flaw that could allow an unauthenticated attacker to execute code through a specially crafted regular expression. A vulnerability advisory establishes a dangerous capability, not proof that every exposed installation was breached. It still puts the system holding source code and release automation at the front of the patch queue.

Then there was eMudhra. Mozilla reported that private keys for eight intermediate certificates had been compromised. Someone able to intercept a connection could use fraudulent certificates to impersonate websites and potentially read or modify traffic. Mozilla blocked the affected certificates through OneCRL, then updated NSS to help protect applications using its trust store without OneCRL updates. Updating a browser and updating every application's certificate trust are different jobs.

The platform incidents made the credential problem explicit. Railway's preliminary report says attackers accessed encrypted environment variables and decrypted a subset using compromised credentials. Zeabur reported that attackers read customer environment variables from its platform database after entering with leaked internal credentials. Its report describes activity on October 14–15, before the PHP disclosure.

These were separate failures with different mechanisms. The reports do not establish one coordinated campaign or an AI cause for every incident. The workload for defenders was simultaneous anyway.

LiteLLM shows how a data leak becomes a software attack

The canonical LiteLLM investigation is worth reading all the way through. It corrects parts of the initial incident report, including the early theory that the first GitHub token came from logs.

In May, a configuration file containing an internal Agent Platform master key was committed to a public repository. The commit was overwritten. The key was identified as needing revocation. It was not revoked.

On October 10, the attacker used that key to query the Agent Platform. The response exposed a GitHub CLI OAuth token belonging to an organization owner. A system intended to let agents file pull requests had been given an owner's token without narrowing its scope.

The attacker used that access to extract more credentials through CI workflows. On October 21, stolen identities allowed the attacker to change the release workflow, overcome repository protections, and trigger and approve a malicious release. The pipeline signed and published the poisoned artifacts. Version 1.107.0 contained credential-stealing malware.

An SBOM helps downstream users identify that version. It does not revoke the master key exposed months earlier, restrict the owner's token, or prevent a compromised release system from signing malicious output.

The sequence crossed several systems: configuration data exposed a credential; that credential unlocked agent configuration containing another credential; repository access enabled further secret extraction; those identities controlled a release consumed by other organizations. The software supply chain and the movement of sensitive data were part of the same attack.

The backup copied the replacement key

Zeabur's report contains a detail that should change how teams think about incident response.

During containment, a new credential was put in place at 15:40. A daily cluster backup captured it at 16:00. The attacker used it at 20:30. Zeabur confirms the backup captured the new key and that the key was later used; downloading that backup is its assessed explanation, rather than a directly confirmed observation.

The team rotated a credential while another route to replacement credentials remained exposed. A backup operation meant to preserve recoverability also preserved sensitive access information.

The same report offers a bounded example of data minimization working. Projects that had moved their environment variables off the platform database after an earlier incident were outside the scope of that database read. Zeabur explicitly warns that this did not rule out exposure from a compromised host. Removing a copy reduced one exposure without making the entire system safe.

Railway's account reinforces the point about encryption. The variables were encrypted, but the attacker acquired credentials that enabled decryption of a subset. Encryption at rest matters. So does the separation between the encrypted data and the identities capable of retrieving its plaintext.

The inventory needs the actual data paths

CISA defines an SBOM as a nested inventory of software components. Keep it current and connect it to what is actually deployed. During a week like this, an accurate inventory saves responders from hunting through every team and every image to find an affected runtime.

Alongside it, we need evidence of what each workload can read, which identities it uses, where its output goes, and how long the resulting copies survive. A second spreadsheet is not sufficient either. Permissions and destination restrictions have to hold when tested.

For a data pipeline, that means following the raw input through transformations, logs, retry queues, caches, and backups. A job that needs one file should not inherit a credential that reads the entire archive. A text extraction worker should not carry release credentials. Sending a redacted result downstream is safer than sending the raw record and hoping every recipient applies the same policy.

Processing near the source can avoid unnecessary exports of raw data. It still requires constrained workloads and controlled outputs. A worker running beside a database with unrestricted access can do plenty of damage, and an aggregate can still disclose sensitive information. Location does not substitute for authorization.

AI applications add another reason to examine incoming data. Indirect prompt injection can turn a retrieved document into an attempt to redirect an assistant's behavior. This is an additional mechanism, not an explanation for the incidents above. A valid file format or known source does not make the contents safe to obey. The application must enforce access and action permissions outside the model.

A patch is one part of recovery

The surrounding incidents show why the work continues after an update. OpenCode reported a breach affecting a subset of Console users with external providers linked. ConnectWise's October 26 advisory described unauthorized access to Cloud Backup application credentials and continued to direct customers to investigate their Microsoft 365 environments. ConnectWise also said it had no indication that existing backup archives were compromised. Stolen application credentials and stolen backup contents are different claims, and responders need to know which one the evidence supports.

I build data infrastructure, so the operational details in these reports are where I keep stopping. A configuration API returns an owner's token. A backup copies a fresh secret. A compromised identity decrypts stored variables. These are specific data flows we can restrict, observe, and test.

Pick one production service at the next security review. Assume its process is compromised. Verify which records and credentials it can read, which records it can change, and where it can send them. Include backups and intermediate copies. Then test whether revocation holds after the next automated job runs.

Keep the SBOM open while you do it. The response is unfinished until you have checked what the vulnerable software could reach and closed the routes that remain useful after patching.

Want to learn how intelligent data pipelines can reduce your AI costs? Check out Expanso. Or don't. Who am I to tell you what to do.*

NOTE: I'm currently writing a book based on what I have seen about the real-world challenges of data preparation for machine learning, focusing on operational, compliance, and cost. I'd love to hear your thoughts!