English
The ultimate showdown of the ultimate destiny!
All the good things are trilogies. Star Wars original trilogy, Nolan’s Batman trilogy, and The Hobbit trilogy. And now, Yocto & WSL trilogy. As per usual with the trilogies, the last part may be a bit of a letdown for the hardcore fans. This time we’re not doing really anything technical or exciting. Instead, we’re comparing some numbers while trying to decide if this whole exercise was worth it or not. If you’ve not read the previous parts, you can find the part number one here and the part numero dos here.
To judge the usefulness of WSL in the Yocto build context, we’re making a comparison between a WSL machine and a ”regular” virtual machine to see which one is actually better for building Yocto (if you’re forced to use Windows). Better means faster in this case. For this purpose, I created a VMWare virtual machine that matched my WSL machine’s specs: 4 cores, 8GB, and enough disk space to accommodate the build area. For the operating system, I chose Ubuntu Server 20.04.4, and as a reference build target, I used the simplest core-image-minimal recipe with both machines.
But wait, there’s more!
As a bit of an extra, I’ll be adding a comparison to a cloud build server as well for the same price! For this purpose, I prepared a similar 4-core 8GB build server on Hetzner. This is a virtual machine as well, but it’s interesting to have more things to compare to. Especially because the cloud instances provide an option to scale up in case they aren’t powerful enough. Also, like WSL and VMWare, they can be used as an alternative for a Linux build machine. Without further ado, let’s get into crunching numbers.
First, to actually make a worthwhile comparison between different build times, I’m going to separate the process of downloading the sources from the actual build itself. I did three runs of downloads because ”the third time’s the charm”. VMWare was using a NAT network connection here. The separation was done because the time it takes to download the sources depends on the load on the source code servers, how much my neighbour happens to be using the internet at the given moment, and the alignment of the stars. Or what do you think of these results:
| Mean | Median | Run 1 | Run 2 | Run 3 | |
| WSL | 44m 4s | 52m 1s | 52m 1s | 59m 29s | 20m 42s |
| VMWare | 46m 20s | 45m 5s | 37m 28s | 56m 29s | 45m 5s |
| Cloud build | 35m 18s | 33m 40s | 33m 40s | 41m 49s | 30m 26s |
Two packages became bottlenecks in these tests: cross-localedef-native and linux-yocto. Especially the former. Other packages were downloaded in a few minutes but cross-localedef-native took almost always at least half an hour to download. In the WSL run #3, cross-localedef-native took only about 15 minutes to download, and instantly the whole download process was a lot quicker. On average, I would still boldly claim that the Hetzner build is potentially the fastest. Or at least it would make sense, as the Internet pipes in their data centres are most likely wider than mine at my home.
Then, the actual build times. For these, I actually did four runs, because ”third time’s the charm and one more for a good measure” (this blog text isn’t quite as scientific as I make it out to be). As mentioned earlier, I built the core-image-minimal for all these runs. These builds were done without downloading the sources in an attempt to get the build times to be a bit more stable. Here’s the table:
| Mean | Median | Run 1 | Run 2 | Run 3 | Run 4 | |
| WSL | 109m 15s | 106m 48s | 106m 11s | 121m 26s | 107m 26s | 103m 37s |
| VMWare | 125m 51s | 125m 51s | 126m 33s | 126m 32s | 125m 10s | 125m 9s |
| Cloud build | 106m 30s | 105m 35s | 111m 10s | 105m 29s | 105m 3s | 105m 40s |
As perhaps expected, the virtual machine was constantly the slowest option. What, however, is slightly surprising is that the WSL was almost as fast as the cloud build server with the same resources. Or actually, with better resources, as the build server had an SSD disk while WSL was running on an old HDD I bought from a friend of mine for 30 euros some years ago.
But yeah, in the light of this evidence, I’d say that if you’ve got some dollars to spare, getting a cloud build server is the best option for building Yocto (if you can’t get a native Linux build machine, that is). It’s not only the fastest option, but it’s also scalable, so when you’re building something else than the ”trivial” core-image-minimal you can easily increase your build performance. I’ll admit that the time difference isn’t big in this comparison, but once you build more complex images, the differences in the build times will become more apparent.
The cloud server used in this comparison costs about 15€ a month. However, if you don’t have enough dollars for about two avocado toasts or three lattes every month, WSL is a better alternative than VMWare for building Yocto. Virtualbox seemed to be even worse than VMWare, as it didn’t even want to build anything for me. It spat out constant disk I/O errors that I couldn’t fix. Different caching options, physical disk drives, or even guest operating systems didn’t fix that.
But this is not all! You thought that you didn’t have to read any more of these tables, didn’t you? I did some generic performance tests to get a bit better understanding of the different build machines, so I’ll add their results here as a bit of a bonus. The first one is the network speed test. I used the Ookla’s Speedtest command line tool for these. The values are averages of ”about a dozen” runs.
| Latency | Download | Upload | |
| WSL | 3.76ms | 104.22 Mb/s | 59.85 Mb/s |
| VMWare | 3.94ms | 104.13 Mb/s | 59.88 Mb/s |
| Cloud build | 0ms | 9266 Mb/s | 9220 Mb/s |
There’s a bit of a difference in the latency between WSL and virtual machine, but the differences in the download and upload speeds are marginal. Cloud build is something entirely different from the other two, as expected.
Next is the hard drive speed, both reading and writing. For this, I used dd command-line tool. For the write test I used this command:
dd if=/dev/zero of=test bs=1G count=4 oflag=dsync
For the read test I used this command:
dd if=./test of=/dev/null
For the un-cached timings, I cleared the caches with this command between the runs:
echo 3 | sudo tee /proc/sys/vm/drop_caches
For the cached timings I didn’t. Here are the results:
| Write | Write (cached) | Read | Read (cached) | |
| WSL | 91.6 MB/s | 93.7 MB/s | 91.9 MB/s | 628 MB/s |
| VMWare | 50.6 MB/s | 107.5 MB/s | 338 MB/s | 559 MB/s |
| Cloud build | 1.2 GB/s | 1.3 GB/s | 603 MB/s | 1.2 GB/s |
This is where the cloud machine’s SSD seems to shine, even if it didn’t do so well when doing the actual build work. The differences between WSL and VMWare, on the other hand, are a bit varied. What these results seem to suggest is that the uncached write performance is quite important for Yocto build performance. However, results also seem to be showing that the mass storage performance isn’t quite as important for the Yocto builds.
Finally, the CPU speed test. This was done with sysbench, using the command sysbench cpu --threads=4 run. Here I compared the CPU events per second:
| CPU events per second | |
| WSL | 5359.31 |
| VMWare | 5284.58 |
| Cloud build | 14582.81 |
All things considered, it’s slightly surprising how small the cloud build’s margin of victory when comparing the build times actually was. Especially considering how much more performant the CPU is according to the raw numbers. The most important thing actually seemed to be the number of cores because a three-core cloud machine would already be slower for Yocto builds than a four-core VMWare virtual machine. So please remember this when you’re setting up your Hetzner cloud instance.
That’s all for now. This is also most likely the last text about Yocto builds with WSL for the time being. Next time it’s either something different that’s not related to Yocto at all or something similar that is related to Yocto but not WSL.
tl;dr: WSL is faster than a virtual machine, but I recommend using cloud servers if possible
The writer of this blog post is wondering how he ever managed to finish his thesis.
Check out Esa’s whole blog here: https://ejaaskel.dev/
