This Week #9: Devbox 0.6, perf improvements, secrets and deploys

Lucille

Continuing the line of work for Jetpack Cloud Dashboard, I connected the Remix app to our Supabase DB and swapped out the mocked data with real ones. On a separate note, Stytch resolved the Node SDK issue I reported this week! It’s time to integrate it into our login flow. I also spent some effort patching our devbox installer GitHub action - props to Greg for catching the Nix store caching problem in our CICD!

Mohsen

This week was a short one for me. I was only online for half of the week. During that time, I worked on clarifying the architecture and endpoints for the deployments project. More specifically, the initiation process. On the side, I started working on adding HTTP endpoints to envsec service. These endpoints will allow envsec to integrate with other services we offer.

Greg

Last week, we shipped some improvements to how Devbox and Nixhub sort package versions. Sorting is a tricky problem due to the infinite number of versioning schemes in nixpkgs, but it’s important to get it right so that commands like devbox add @latest install the correct version.

I’m also working on a change to allow comments and trailing commas in devbox.json files. Comments have been a long-requested feature, so I hope we can get it into the next release.

Savil

This past week was quite satisfying as we made concrete progress on some important tasks.

The first is that the Devbox 0.6.0 release contains a faster way to install and update packages. This new approach should save users a lot of time: my estimate is over 30 seconds for each unique package (glossing over some details). The core insight came from Daniel and Greg, who realized that Nix contains an API called builtins.fetchClosure that lets us directly install the package's binaries. This allows us to bypass downloading the instance of each nixpkgs registry that a devbox package comes from, which was always a painfully slow step. We've worked on this off-and-on for most of the third quarter of this year, and took efforts from Greg, Rodrigo and myself to implement various sub-features to get it done. I was also pleased that this high-risk feature has yet to receive any user reports of failures so far. Yay!

The second task was finding a way to fix some glibc dynamic linking issues. This issue has affected a few users, and arises due to a mismatch between how various nix packages are built and the current glibc used in the environment. With Greg's wonderful collaboration, we were able to define a nix flake that can wrap a package to update it to use the very latest glibc. This relies on glibc having excellent backwards compatibility. That said, its not without its risks and I look forward to gaining some validation from real world use-cases that encounter this issue. You can find a writeup of the approach in the README to a proposed solution for a GitHub issue filed by a user.

John

Last week, we released Devbox 0.6, which improves our package installation performance by directly pulling packages from the Nix Binary Cache, without needing to evaluate the Nix repository first. Devbox 0.6 saves up to 30 seconds when installing a new package, and even more when updating projects with multiple packages. We plan further performance improvements for our next 0.7.0 release, which we hope to publish this week.

We also wrapped up planning for our next cycle, which includes a beta release of Devbox Secrets, powered by envsec. I'm looking forward to sharing this early work with beta testers soon!

Finally, I spoke with a few devbox users to learn more about how they use Devbox at their companies. We learned a lot about how they use Devbox locally and in the cloud, and developed a list of potential improvements for future releases.

Rodrigo

This past week, I continued work on the deployments tool. We’re ready to have it automatically deploy the service after merges to main. Now, we’re focusing on adding management capabilities (i.e., setting up a project for deployment, editing its configuration, etc.). We debated where this logic should be and tentatively decided to put it in the dashboard side of things (instead of in the go-service that houses the rest of the deployments infra). The next steps for me here are to figure out exactly how the GitHub integration would look, specifically for the new project flow, which is a bit less straightforward than the rest of the operations we need to support.

Mike

I worked on integrating runx into devbox! Runx is a new feature that allows installing any binary published as a release on a GitHub repo. We’re currently dogfooding this feature and hope to release it soon.

I also merged a few perf improvements to make bin wrappers and global shellenv faster. These improvements should be out in our next 0.7 release.