Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane.
I'm curious what made think of it in the first place? What was your thought process that made you think that these Linux primitives give the opportunity to make it a platform for distributing and deploying packaged applications and eliminating a lot of the issue around "it works my machine" argument?
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
I'm a big fan of docker and mainly just want to say thanks for bringing it about.
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
To give a sense of time: Docker launched in 2013, but it was one of many iterations of the same project, which I had been working on continuously since 2008. Over those 5 years, virtually nobody cared or understood what the hell I was working on. I lived in France and had no connection to the tech industry.
So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)
I remember in 2013 the first time someone showed me Docker [im in Mexico]. My first thought is that we already had Vagrant for setting up and running virtual machines through console.
In those years in the systems group I was working in at a fortune 500, we were very interested in this type of technology. Mostly using Solaris Zones and cgroups and trying to do this ourselves with limited results.
When Docker came out it solved so many issues we were having. Thank you for scratching that itch.
I believe Docker was always called Docker, but it grew directly out of the previous generation of the technology called dotCloud (the company Solomon founded), which was using LXC with AUFS.
I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.
Do you think e.g. students learning docker syntax at a uni is beneficial in 2026 or you think LLM-assisted programming means that Docker will become an abstraction under an abstraction and its setup will be mostly automated by LLMs? I find myself very rarely debugging or handwriting dockerfiles nowadays!
I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.
If you were implementing an operating system (free from compatibility concerns), what would you do differently to better support lightweight containers?
Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
Oh wow, hi! I think my biggest “gripe” with docker is that everyone grabs it as a hammer and views many problems as a nail. Am I way off base with that? What was your original vision for how docker might be used in the wild?
You’re an interesting character Shykes. In that you’re the only intelligent person I know who is not a politician or grifter yet seems so committed to an ideology that if described under another name wouldn’t be called anything but racist, abhorrent, and downright disgusting.
I prefer Podman most of the time, but Docker is/was amazing and a great DX put on top of the so called "5 different linux kernel features in trenchcoat". That's the selling point, it's super lightweight and takes full advantage of the OS features that were already there (or being developed) but were awkward to use and required tons of specialized knowledge.
Always nice to see someone demystifying a complex thing by learning the building blocks it's made from.
Knowing this, you might find rootless Podman's fork-exec method of starting containers to be more sensible. I certainly do. No root, no daemon, no socket.
If you want to learn more about the Linux Kernel namespaces which work together to make a container, this LWN series is really good: https://lwn.net/Articles/531114/
And not a single mention of the actual weird decision, a client / server model which complicates permission management by at least two orders of magnitude.
A lot of people start using Docker as a black box and have the mental model of a VM, and when they start digging into the internals all the "weirdness" starts to exhibit itself.
That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking..
One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.
>>Images are just folders that Docker puts somewhere and then calls chroot (or pivot_root) to make that folder your container process’s new root directory.
And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?
I'm happy to answer any questions you may have.
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)
I didnt get it at that time haha.
When Docker came out it solved so many issues we were having. Thank you for scratching that itch.
I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.
https://jpetazzo.github.io/2017/02/24/from-dotcloud-to-docke...
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
https://docs.docker.com/reference/dockerfile/
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
Knowing this, you might find rootless Podman's fork-exec method of starting containers to be more sensible. I certainly do. No root, no daemon, no socket.
If you want to learn more about the Linux Kernel namespaces which work together to make a container, this LWN series is really good: https://lwn.net/Articles/531114/
Everything in that list feel natural when you know how things evolves over many years. Maybe it is just me getting older and less curious.
[1] "Docker implemented in around 100 lines of bash" https://github.com/p8952/bocker
One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.
Containers From Scratch • Liz Rice • GOTO 2018
https://www.youtube.com/watch?v=8fi7uSYlOdc
And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?