Strengthening Rust Infrastructure Security with Ubuntu Pro
The Rust Project depends on a broad set of infrastructure. Some of it follows modern practices: containers are periodically redeployed from images rebuilt with the latest updates, including the latest Ubuntu builds. Other parts carry some legacy weight; long-running Ubuntu servers that, in infrastructure terms, are still pets.
This is partly due to genuine complexity. For example, crater and docs.rs use rustwide to run untrusted code, which requires access to the Docker socket, meaning rustwide can’t run on AWS ECS or AWS Lambda. We have ideas for better isolating untrusted code, but none of them are low-hanging fruit.
Security work in open source projects has to be both effective and practical. We always want to move toward stronger isolation, more redundancy, and smoother maintenance workflows, but we also need improvements that can be deployed with the capacity we have today. That is why we adopted Ubuntu Pro across parts of the Rust infrastructure fleet.
What Ubuntu Pro adds
Ubuntu Pro extends the security maintenance available to Ubuntu systems. For the Rust infrastructure fleet, the most important benefits are Expanded Security Maintenance (ESM) and Livepatch.
As the name suggests, ESM expands security update coverage in two ways:
- It extends LTS release support from 5 years to 10 years (Ubuntu 18.04, for instance, is supported until 2028).
- It offers security patches for packages in Ubuntu’s Universe repository, whereas standard LTS maintenance only covers Main.
Livepatch automatically mitigates Linux kernel exploits while systems are running, without requiring a reboot.
How we added Ubuntu Pro
We integrated the Ubuntu Pro activation into our Ansible configuration, so that new EC2 activates Ubuntu Pro by default, and the Ubuntu Pro status is checked every time we run Ansible. Security updates are then applied periodically through unattended-upgrades, which helps keep machines patched without requiring every update to become a manual task for a team member.
Why automatic updates matter for Rust
Automatic security updates are valuable for any production system, but they are especially important for parts of Rust’s infrastructure.
Some of our systems intentionally run untrusted code from the wider Rust ecosystem. Docs.rs builds documentation for crates published by the community, and Crater runs experiments across the Rust ecosystem to help the Rust Project understand the impact of compiler changes. These services are designed with isolation in mind, but they still run in an environment where host security matters. Keeping the underlying operating system patched reduces the window of exposure when new vulnerabilities are disclosed.
Automatic updates also matter for the Rust dev desktops, high-end servers provided by the Rust Foundation’s cloud compute program. These servers are shared among maintainers, who connect via SSH. If one account is compromised, local privilege escalation vulnerabilities such as Copy Fail can turn a contained compromise into a much more serious incident.
Finally, some Rust services do not yet have the level of redundancy we would like. For services such as the Rust Playground and docs.rs, rebooting the EC2 instance means downtime for users. We want to improve this over time, but the Infrastructure Team has limited capacity, and most team members are volunteers. By applying kernel security fixes without requiring a reboot, Livepatch helps us keep our systems secure, reducing downtime. It does not replace future work to make services more highly available, but it improves our position today.
A pragmatic step forward
The Rust Project’s Infrastructure Team has more security improvements in mind, but in the meantime, we thank Rust Foundation Gold Member Canonical for donating 25 Ubuntu Pro subscriptions to the Rust Foundation.
This donation allows us to improve the security posture of Rust’s infrastructure with little operational effort, while also lowering the operational burden for the Infrastructure Team.
