🪄 Install asdf: One Runtime Manager to Rule All Dev Environments#
Almost everyone I know who uses a computer, for fun or work, is overtaken by a slight nervousness when installing a new library, package, service or application on their operating system. The horror stories underpinning these reservations vary slightly between Linux, Unix, Windows and MacOS users, but everyone knows, regardless of preferred ecosystem, that installing software can do a number on their computer and the task should not be taken lightly. Especially for programmers, that struggle is no stranger. Therefore, any tool looking to simplify this task and eliminate potential points of failure is, in my book, a very welcome occurrence.
In case this sounds relatable and you can’t help but think about the time you’ve lost cosplaying as a development environment archeologist, I strongly suggest you give asdf.vm or a similar solution a try. This tutorial serves as an introduction to asdf, runtime management and, most importantly, should ellucidate the thought process behind my appreciation for such tools.
Discussion#
Note
Some folks kindly asked for a video version of this and similar tutorials on YouTube. I haven’t posted much to 🎞️ my YouTube channel, but that seems like a worthwhile endeavor!
I’ll try to find time to post it in the next couple of weeks. Feel free to subscribe and enable the notifications 👍
This tutorial was heavily discussed on HackerNews, Reddit, Tildes.net and Mastodon. Feel free to let me know in case I ought to include any other conversations relating to it.
Folks on Reddit brought mise-en-place to my attention, a runtime manager similar to asdf.vm which is written in Rust, doesn’t use shims and incorporates direnv as well as make. I was only aware of the existence of rtx, but apparently that is what mise used to be called! rtx was renamed in order to avoid confusion and make the project more recognizable[1].
Moreover, for those interested in alternatives to asdf.vm with Windows support, you are in luck! version-fox is another popular runtime manager which, unlike asdf.vm and mise-en-place, does support the Windows ecosystem.
I’ll keep an eye on both projects since they seem to be evolving quickly. Having said that, however, one of the reasons I like asdf.vm is how light-weight and minimalistic it is.
Introduction#
Throughout the years, my distaste for overly complex and non-reproducible development environments has grown considerably; without signs of ever slowling down. Needing to install npx globally in order to skip a global installation of ng, Angular’s CLI[2], is somewhat amusing [3] [4] [5]. On the other hand, needing to juggle two or more Python versions which are unable to co-exist and therefore break Ubuntu, or the operating system du Jour, is tragic [6] [7] [8].
Important
This is no way to spend our waking hours
Historical Detour: Using Sand Castles to Create “Dependable” Software#
As programmers we regularly acquire, install, test, use, and manage as many software components in order to prototype, program, test, and maintain a single application as a regular computer user likely has ever even considered installing. One might start working on a Python project X, using a certain framework F and library L. While working on version 0.1 of X and integrating L, the programmer realizes library L requires an operating system package P which is written in another programming language; let’s call that new hypothetical piece of the puzzle C for no apparent reason.
Thus, programmer S, for Sisyphus, ends up having to litter their carefully configured workstation with a Python development environment, with framework F and its dependencies D, library L but also operating system package P and the required tools to prepare said package; let’s call them build-essential for no reason at all!
Situations like the one described above are rarely something to write home about. That’s a daily occurrence in software development and anyone would be able to attest to that fact. Now, what does, indeed, grind my gears, is how often we, as an industry, write software, set infrastructure up and operate systems without regard for reproducibility and with little care for minimizing complexity.
If you are lucky, you might only have to discover the right version combination of nodejs and angular necessary in order to run the small service, mostly written in Python, a colleague of yours wrote some quarters ago. However, it may or may not use C Types to interact with a deprecated OS package, and no Dockerfile nor dependency pins were ever prepared. Yes, in case you asked, the Python version required also wasn’t listed in the README.md and the project uses grpc[9]. In this relatively simple scenario I can guarantee you, that hours if not days will be spent bringing the corresponding development environment to an acceptable state.
Linux users might worry about suddenly having multiple versions of the same library, or one version overwriting crucial symbolic links used by the previous one. Windows users know to respect the almighty registry and the ominous AppData user directory, both of which continue to grow and absorb data as the Gaia BH1 and Gaia BH2 black holes. MacOS users, bless us, will worry about homebrew being homebrew.
Before the advent of containerization and package management as inextricable elements of any respectable software development toolchain, the general advice was to spin disposable VMs and develop within them so, should things ever go south, they could be nuked from orbit without much hesitation. That approach, however, comes with a steep performance and convenience cost. One would create a development VM template, then configure it to one’s liking. Said template can then be cloned and used as general purpose development workstation or be customized on a per-project basis:
Docker, and the golden age of containerization it brought about, made working on and managing multiple projects less painful, but introduced new usability and maintainability issues to the Software Development Life Cycle(SDLC)[10]. For example, it took long until most prominent Integrated Development Environments(IDEs) could reliably attach themselves to a, let alone multiple, containers in order to develop from within them. Furthermore, continuosly tweaking, building, debugging and starting multiple contaniners during development is not a very efficient way to spend one’s time.
Although this historical diatribe might seem unnecessary, especially for such a brief tutorial, my intention with it is to emphasize the importance of 1️⃣ being able to effortlessly keep different development environments and switch between them as well as how 2️⃣ doing so directly from the programmer’s main operating system, without additional layers of abstraction, can be valuable and improve one’s overall workflow.
Enter Runtime Managers and asdf#
Runtime managers aren’t exactly a new concept. Ruby programmers were the first ones to try and tackle the problem of ergonomic runtime management by working on the Ruby Version Manager(RVM)[11] in 2007, but that idea quickly took over the world of interpreted languages; JavaScript, Python, and GoLang all followed with nvm[12], [13], pyenv[14] and goenv[15] respectively.
asdf would take four more years to come into play, with its first ever commit dating back to 2014-09-29T16:51:09Z[16]. Even then one can already tell what its creator @HashNuke was after:
The idea was simple but game-changing, to write a CLI wrapper which would allow the user to seamlessly install and use multiple versions of a given programming language runtime, and that’s precisely what asdf became:
Real Solutions for Real Problems#
Consider the following scenario: you work at a small start-up whose stack consists of multiple services written in Ruby and Python. For historical reasons, a considerable portion of the codebase used Python 2.7 until its final deprecation. However, that code, although useful, was never ported to newer Python versions and the company moved on. Nowadays, all services are written either for Ruby 3.2.2 or Python 3.10.0.
Here is where our hypothetical story gets interesting: you are tasked with rescuing one of the ancient Python 2.7 projects and test its interoperability with the modern parts of the codebase written in Ruby 3.2.2 and Python 3.10.0.
Your mind starts racing, considering how you could set up a development environment where you can easily execute, test, debug, package and ship the three projects without issues. You ponder whether virtual machines, docker containers or setting up the development environments natively on your local system might be the best approach. Since we are here to learn about asdf, you decide to explore the third option.
Right away, you get to work and clone the two projects to your local work directory ~/sisyphus/work/:
sisyphus
└── work
├── py310 # Project runnable under Python 3.10
│ ├── README.md
│ └── main.py
└── rb322 # Project runnable under Ruby 3.2.2
├── README.md
└── main.rb
Once opened with VS Code, or your editor of choice, a very rough approximation of how your system, the shell and your projects within VS code interact can be depicted as follows:
The direction of the arrows signify what component calls what component.
That seems perfectly fine, after all you installed Ruby 3.2.2 and Python 3.10.0 when you first joined the company and the projects run without issues.
Two Separate Ruby and Python Projects in VSCode#
Remember we still have one Python 2.7 project whose development environment must be set up. After placing that project under ~/sisyphus/work/, your directory structure looks as follows:
sisyphus
└── work
├── py27 # Project without a valid runtime environment
│ └── main.py
├── py310 # Project runnable under Python 3.10
│ ├── README.md
│ └── main.py
└── rb322 # Project runnable under Ruby 3.2.2
├── README.md
└── main.rb
Building upon the schematic introduced previously, we can imagine our current setup as shown below. Both the py310 as well as rb322 folders have been opened on VS Code and corresponding terminals were spawned. Project py27 is present within ~/sisyphus/work as well, but it has yet to be opened and configured in order to run properly. The available Python 3.10 as well as Ruby 3.2.2 installations are also depicted, since the terminals opened from within VS Code interact with them through the system’s terminal.
Runtimes with asdf#
We finally reach a point where we must ensure a Python 2.7 installation is available in order for project py27 to be opened and run from within VS Code; it’s time for asdf to save the day.
After we have installed[17] it, we verify it’s been properly added to our path:
$ asdf --version
v0.13.1
We proceed to add both the Python as well as Ruby asdf plugins and our desired runtime versions.
$ asdf plugin add python
Plugin named python already added
Usually, it’s a good practice to review the available versions and variants of the desired runtime. Note the presence of miniconda and miniforge; if this was Java we were talking about we’d also see different Java versions, JREs and JDKs.
$ asdf list-all python | grep 3.10.0
3.10.0
mambaforge-23.10.0-0
miniconda3-3.8-23.10.0-1
miniconda3-3.9-23.10.0-1
miniconda3-3.10-23.10.0-1
miniconda3-3.11-23.10.0-1
miniforge3-23.10.0-0
Having found the Python version of interest we want to install, we continue doing so:
$ asdf install python 3.10.0
python-build 3.10.0 /Users/sisyphus/.asdf/installs/python/3.10.0
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.0.tar.xz...
-> https://www.python.org/ftp/python/3.10.0/Python-3.10.0.tar.xz
...
python-build: use readline from homebrew
python-build: use ncurses from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.0 to /Users/sisyphus/.asdf/installs/python/3.10.0
The process for Ruby is very much the same, although in this case I had already installed it in advance:
$ asdf plugin add ruby
Plugin named ruby already added
$ asdf list-all ruby | grep 3.2.2
3.2.2
$ asdf install ruby 3.2.2
ruby 3.2.2 is already installed
Finally, it’s time to start using our Python and Ruby runtimes managed by asdf. Note both the python and ruby commands are aliased to so-called asdf shims, small shell scripts that will determine which Python and Ruby version to run depending on the versions installed and configured within asdf.
$ type python
python is /Users/sisyphus/.asdf/shims/python
$ type ruby
ruby is /Users/sisyphus/.asdf/shims/ruby
However, upon trying to use the Python and Ruby runtimes we just installed, asdf tells us it cannot decide which version to use since we haven’t specified one yet on the local nor the global level:
$ python --version
No version is set for command python
Consider adding one of the following versions in your config file at /Users/sisyphus/.tool-versions
python 3.10.0
$ ruby --version
No version is set for command ruby
Consider adding one of the following versions in your config file at /Users/sisyphus/.tool-versions
ruby 3.2.2
.tool-versions#
asdf runtime versions are selected through so-called .tool-versions files, these can be present in the local directory ./.tool-versions or in the user’s home directory ~/.tool-versions.
In the second case, these are called global versions and can be set using asdf global <language> <version> and removed by editing the ~/.tool-versions file.
$ cat ~/.tool-versions
$ asdf global python 3.10.0
$ asdf global ruby 3.2.2
$ cat ~/.tool-versions
python 3.10.0
ruby 3.2.2
After selecting 3.10 and 3.2.2 as global asdf versions for Python and Ruby respectively, we can now transparently use them:
$ python --version
Python 3.10.0
$ ruby --version
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin23]
That means asdf will from now on be in charge of managing our Python and Ruby versions, as well as any runtime for which a plugin is available and which we add to it. Going back go our trusty schematic, it now looks as follows:
Multiple Versions of the same Runtime#
Recall our initial objective, we are looking to manage three different projects with their respective runtimes effortlessly and we are still missing the Python 2.7 runtime. In order to add it to asdf’s repertoire we proceed as explained in the previous section:
$ asdf list-all python | grep 2.7
2.7.0
2.7-dev
2.7.1
...
2.7.18
Let’s go ahead with Python 2.7.18, the last 2.7 minor version before the corresponding major version was deprecated forever[19]:
$ asdf install python 2.7.18
python 2.7.18 is already installed
Overriding the Global ~/.tool-versions#
You might be asking yourself how to override the asdf global python 3.10.0 configuration we set earlier. That is trivially achieved by placing a .tool-versions file in the directory of the project where you wish asdf to use a different version to the one globally selected:
$ pwd
/Users/sisyphus/work/py27
$ ls
. .. .tool-versions README.md main.py
$ cat .tool-versions
python 2.7.18
$ type python
python is /Users/sisyphus/.asdf/shims/python
$ python --version
Python 2.7.18
With that, we bid farewell once and for all to potential Python, or Ruby, version conflicts and are free to install, develop in, and manage as many runtime environments as we desire without affecting our main operating system or atleast with the possiblity to resolve problems much easier. The final state of our hypothetical development environment looks as follows:
Two cloned Python 3.10.0 and Ruby 3.2.2 projects opened with VS Code
Interactions between VS Code and available runtimes without asdf
Interactions between VS Code, asdf and its managed runtimes
Final addition of Python 2.7.18
Consider the simplicity with which Python 2.7.18 has been added and how powerful this workflow is, since it can be extrapoladed to any other language supported by asdf. For example, adding a NodeJS project and its runtime to our increasingly complex but easy to manage ~/sisyphus/work/ directory is as simple as:
$ asdf plugin add nodejs
...
$ asdf install nodejs latest
...
$ asdf global nodejs latest
...
$ node --version
v22.0.0
That last example is the reason I will remain, for the foreseeable future at least, a passionate and enthusiastic user of asdf.
Acknowledgements#
@crabmusket on Reddit pointed out that the JavaScript community with nvm weren’t the ones to set the trend in motion, but that it was the Ruby community with rvm in 2007. Although I remembered vaguely that there was for sure something before rbenv, I couldn’t recall or find precisely what it was!
Final Words#
For the time being, that is all from me regarding asdf. I hope to have presented a compelling case as to why you should give multi-lingual runtime environment managers a try and get a bit farther from the “it works on my machine days” which hunt many of us in our dreams.
If you are interested in receiving updates whenever I write something new, subscribe by entering your email in the registration form on the sidebar!
Thank you for reading and remember that feedback is always welcome and appreciated; you may contact me via email or social media. Let me know if there’s anything else you’d like to know, there’s something you’d like to have corrected, translated, added or clarified further.