9

Short version

I would like to know the technical reasons why do Docker images need to be created for multiple architectures. Also, it is not clear whether the point here is creating an image for each CPU architecture or for an OS. Shouldn't the OS abstract the architecture?

Long version

I can understand why the Docker Engine must be ported to multiple architectures. It is a piece of software that will interact with the OS, make system calls, and ultimately it is just code that is represented as a sequence of instructions within a particular instruction set, for a particular architecture. So the Docker Engine must be ported to multiple OS/architectures much like, let's say, Microsoft Word would have to be ported.

The same thing would occur to - let's say - the JVM, or to VirtualBox.

But, different than with Docker, software written for the JVM on Windows would run on Linux. The JVM would abstract the differences of the underlying OS/architectures, and run the same code on both platforms.

Why isn't that the case with Docker images? Why can't the Docker Engine just abstract the differences, and provide a common interface, so the image itself wouldn't need to be compatible with a specific OS/architecture?

Is this a decision (like "let's make different images per architecture because it is better for reason X"), or a consequence of how Docker works (like "we need to do it this way because Docker requires Y")?

Note

  • I'm not crying "omg, why??". This is not a rant or criticism, I'm just looking for a technical explanation for the need of different images for different architectures.
  • I'm not asking how to create a multi-architecture image.
  • I'm not looking for an answer like "multi-architecture images are needed so you can run your images on various platforms", which answers "what for?", but not "why is that needed?" (which is my question).

Besides that, when you see an image, it usually has an os/arch in the digest, like this:

docker image digest

What exactly the image is targeting? The OS, the architecture, or both? Shouldn't the OS abstract the underlying architecture?


edit: I'm starting to assume that the need for different images per architecture is on the lines of: the image will contain applications inside it. Let's say, it will contain the Go compiler. The Go compiler itself is a binary that must have been complied to different architectures. The image for x86-64 will contain the Go compiler compiled to x86-64, and so on. Is this correct? If this is correct, is this the only reason?

6
  • there can be a big difference using the same base image for different architectures in the name of packages you install. example centos 7: the is a package called llvm-toolset-7 on intel and it is called llvm-toolset-7.0 on ppc64le. and this happens often. some repos do not even have packages for a specific architecture. it's an impossible task to abstract that.
    – clemens
    Jun 10, 2020 at 7:23
  • Thanks, but I still don't get the actual difference. You provided an answer talking about the names of packages for different architectures, but I don't know how does this relates to the operating system (does the operating system downloads different packages depending on the underlying architecture?), and what are actually the difference between the packages for different architectures (besides the name) Jun 11, 2020 at 15:59
  • Docker is not a VM, it just configures the OS so that a program is: a) constrained (cgroups); b) isolated (namespaces + chroot + layered fs). A container run inside the host OS directly. Different hosts have different CPUs. Docker doesn't sit between the CPU (or any other hardware) and the container. Jun 11, 2020 at 17:28
  • Thanks @MargaretBloom, that starts to answer my question. "Docker is not a VM" -> that is Docker 101, I think the level of my question should imply that I already know that. "Docker doesn't sit between the CPU" -> that is better information. Still though: what there is in the image itself that it needs to be ported to different OS/architecture? Are we talking about the compiled programs that the image might contain, or something else? And which one is the compatibility target here: the OS or the CPU? Shouldn't the OS abstract the CPU (honest question, don't know how it should be)? Jun 11, 2020 at 19:50
  • @RafaelEyng The compiled programs, of course. You compile a program for a specific CPU. That's why Java and CIL (.NET) bytecode is... bytecode and not machine code. Docker only supports Linux (it runs a Linux VM on Win and Mac) and while some features are provided by the docker images, the kernel seen by a container is the host kernel. So that's another source of possible incompatibility. The OS doesn't abstract the CPU as it is what run the os in the first place. Not even VMs totally abstract the CPU, emulators do. Jun 12, 2020 at 7:40

2 Answers 2

1

Why can't the Docker Engine just abstract the differences, and provide a common interface

Performance would be a major factor. Consider how slow Cygwin is for some things when providing a POSIX API on top of Windows by emulating some POSIX things that don't map directly to the Windows API. (e.g. fork() / exec separately, instead of CreateProcess).

And that's just source compatibility; the resulting binaries are specific to Cygwin on Windows. It's even worse if you want to do that at runtime (binary compat instead of source compat).


There's also the amount of complexity Docker would need to provide an efficient portable JIT-compiling VM on top of various OSes, especially across various CPU ISAs like x86-64 vs. AArch64 that don't even share common machine code.

If Docker had gone this route, it would really just be re-inventing a JVM or .NET CLR bytecode-based VM.

Or more likely, instead of reinventing that wheel, it would just use an existing VM and add image management on top of that. But then it couldn't work with native programs written in C, unless it transpiled them to Java or CLR bytecode.

12
  • After reading your answer, I still don't get it. "Consider how slow Cygwin" -> never used that in my life, and exactly a good detailing of the reason for that being slow is the kind of answer I need to understand this. "It's even worse if you want to do that at runtime" why? Is the reason behind it that I need to understand. I've never programmed OSes, so there is a ton going on that I actually don't understand. Just saying that something doesn't work or is "even worse" won't help me understand that. What I mean is: you are assuming too much previous knowledge in your answer. Jun 11, 2020 at 16:03
  • Also, your answer does not touch Docker images. I don't exactly understand how a Docker image is implemented, and why does it have to contain to be compatible with a specific architecture. Jun 11, 2020 at 16:05
  • @RafaelEyng: That's fair, my answer does require significant background knowledge to understand. I can't think of a simple way to explain it, though. Emulating a portable API on top of whatever each OS provides would add overhead for any case where there's a mismatch, and/or stop an application from taking advantage of the power of the native API in any case where the hypothetical portable API is more limited / simpler. If you don't want to take my word for that, you're going to need to understand how things like JVMs work on an asm level, and that's a much longer answer than I want to write Jun 11, 2020 at 16:07
  • 1
    @RafaelEyng: I was just talking about the software package inside the image. The parts of the docker image other than the programs (and their data) is just AFAIK metadata to be read by the docker engine, which already has to be ported to each platform. Jun 11, 2020 at 17:04
  • 1
    @RafaelEyng: BTW, I know this answer isn't complete or details, but I hoped it would be better than nothing, e.g. perhaps as pointers for further avenues of research. I was going to just comment but it was almost an answer so I posted it as such. If you're lucky, someone else who has the time and interest will go into more depth. Jun 11, 2020 at 17:07
1

All tough the promise of Docker is the elimination of differences when moving software between machines, you'll still face the problem that Docker runs with the host machine's CPU architecture, which can't be crossed in Docker.

Neither Docker, nor a virtual machine, abstract a CPU to enable full cross compatibility.

Emulators do. If both Docker and VM's would run on Emulators, they would be less performant as they are today.

The docker buildx command and --build-arg ARCH flag leverages the advantage of the qemu emulator, emulating the full system with an architecture during a build. The downside of emulation is that it runs much slower than normal builds.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.