In October, Railway experienced a significant security breach resulting in the estimated unauthorized usage of $740,000 due to compromised AI provider keys. The platform-as-a-service company announced its commitment to cover these costs if AI providers do not offer refunds.
Details of the Security Breach
Railway, which develops and manages applications for clients, reported that attackers decrypted environment variables in approximately 1.85% of workspaces, affecting around 100,000 instances. These variables included sensitive API keys targeted by the intruders.
In a statement on October 26, Railway informed customers that it could not refund charges incurred from the misuse of API keys. Customers were advised to contact the AI providers directly for potential reimbursements, with Railway offering to assist in escalating their requests.
Impact on Customer Data
The breach also exposed linked-account tokens for 1,852 users, including BYOK agent keys and GitHub OAuth tokens, all of which have been reset. Despite these vulnerabilities, there was no evidence of unauthorized logins to customer accounts using the stolen credentials.
The attackers also accessed stored histories of 353 builds, which contained various secrets and variables. However, Railway’s forensic analysis indicated no unauthorized access to customer databases or code repositories.
Response and Recommendations
Railway has taken steps to mitigate future risks, advising customers to revoke and replace sensitive variables stored on the platform before October 21. The company also recommended reviewing audit logs and access permissions on platforms such as GitHub and Docker Hub.
In response to the breach, Railway disabled the credentials exploited by the attackers, preventing further decryption of any unaccessed data. The company has also shared affected keys with providers to ensure comprehensive resets.
How the Breach Occurred
The attack was traced back to an exploit in Railway’s build system on October 20. An attacker leveraged a flaw to access files on a virtual machine, gaining a credential used for internal systems. This breach escalated, allowing the intruder to decrypt customer variables and identify AI provider keys.
Railway has since closed all vulnerabilities and reinforced its systems. They have filed a complaint with the FBI and are collaborating with a forensics firm to continue the investigation. The company aims to publicly share indicators of compromise upon law enforcement’s approval.
While the breach’s origin remains under investigation, it is suspected that stolen credentials were used by services reselling AI access. Railway’s incident is noted among several similar breaches in October, highlighting a trend of targeting stored environment variables for AI access.
