In my world of bootstrapped, smaller apps looking for market traction, even if things go well, a few Linodes should be enough to handle most of the traffic I'll ever need to deal with, so this kind of thing is kind of foreign to me. I'm curious how people utilize it in practice.
I (or rather, jenkins) builds all my software in it, I don't actually use it for containerizing final applications.
In a nutshell, for me the value is in trivial repeatability. I can reproduce the entire build toolchain, test environment and produce artifacts all from a few KiB git repo which centres around the Dockerfile and submodules to dependencies.
Some of my ARM stuff takes hours to cross-compile and involves enormous amounts of fiddly babysitting normally. Dockerfiles have RUN statements (think lines a shell script) which are cached. My adjustments toward the end of a Dockerfile take only seconds to test and produces the exact same result as if it had really run each statement from the start, which doesn't sound like much but turns out (for me) to be pretty liberating compared to constantly trying to fight other automation where you have to dance around short-circuiting stuff to re-use bits of a past build to save time and get only a handful of "pristine" iterations in a day (that might differ to the iterations you rolled by hand).
They've got a lot of idiosyncrasies at the moment, some of it working around the fact ADD some/directory/ could never be cached (so my build scripts maintain some.directory.tar.gz and those are ADDed instead), but I really should. I guess I'd put it under my github profile (I'm also csirac2 there).
There's two types of ARM builds, ones which can cross-compile and those that can't. The ones which can't cross-compile are done with qemu-binfmts and we chroot into an ARM filesystem and run the build there.
Perhaps the only useful contribution would be the fact that I persist the ccache up to the docker host with a shared ccache volume, and that helps enormously especially for the qemu-binfmts builds which can be quite slow.
(em)debian provides nearly enough of what I need most of the time. The trigger for going to a ARM chroot is when I can't get build dependencies installed properly on an amd64 host. Either that or the thing I'm compiling just isn't developed to be cross-compiler friendly and it's too much work to hotwire it to be so.
For example, say I need libfoo, I have an amd64 host (being the docker container). Sometimes I just can't get the libfoo:armhf or libfoo-dev:armhf package installed because it would break/conflict with the amd64 host's version of it in some way. xapt often helps but then sometimes screws up by re-packaging something that has an "all" arch (non-arch-specific) to something armhf specific (eg. foo-data). This ultimately either conflicts with the host or fails to be named properly in such a way that it meets the build-deps of the project.
Sometimes I know it would be easier in some cases to avoid the debian packaging ecosystem, but for my workflow and distribution requirements it brings a lot of benefits.
Self-hosted PaaS platforms like Dokku, Flynn, Deis, etc give you Heroku-like deployment & management without the cost of something like Heroku. All three of these are built on Docker.
I've been running Dokku on DigitalOcean lately and it has been great (though I'm planning on eventually moving to Deis, now that DigitalOcean supports CoreOS).
Right now I use it for demos... We use a stack that is little complex for someone wanting a quick taste. So provided they have a modern Linux with with docker installed they can quickly do a self demo.
And if they don't, well having a fresh image that is easily rebuildable makes it easier for me to give demos than to spin fresh VMs. Arguably I could have done the same with Vagrant, but really didn't need the overhead of the VM.
This is really cool to hear. If you're interested in having us know more about your project, I'd be more than happy to take the time. Information is in my profile.
Truth be told, to me those are not very meaningful, as those companies operate on a scale, and have requirements that are on another planet from mine. Is Docker useful for people/organizations that are not so large? The other responses seem to think so!
Bringing up a properly configured VM that stays up to date is actually a lot of work, and the more software you pile directly on top of that VM, the harder it is.
For me, my VMs are actually dead simple and relatively homogeneous. Fresh install + lock down SSH + lock down iptables + install Git + install Docker.
Then I build from the correct Dockerfile and open whichever port(s) on the VM. Bang, instant node that does [whatever]. Time to upgrade the DB/mail server/app? Build from the new Dockerfile, stop the existing container, bring up the new one. No worrying about installing/uninstalling the correct dependencies on the VM and getting into an inconsistent state.
That's also what any sane configuration management system does for you.
Don't get me wrong, I think docker is great, but this whole "I get simple idempotent machines" is a solved problem. Docker excels at bundling applications with all their dependencies as an modern analogue to statically linking, or a Linux equivalent to OSX's .app
Most configuration management systems actually make ongoing configuration management pretty hard. You end up with your supposedly declarative configuration files littered with upgrade scripts and "please uninstall that package", unless you simply wipe your systems and install them from scratch with each change.
I think the point of Docker is that it decouples that "destroy a system and start a new one" from the VPS/cloud/server provider you're using; you can talk about "logical" systems as separate from whatever they're actually running on.
Personally, my use of NixOS means that I don't have the problem, so I don't need Docker. My systems get generated from scratch (logically speaking) whenever I upgrade without having to tear down the VM, and then some simple logic figures out which services to restart.
Solved by what, Puppet and the like? It's simply too easy to get your machine into an inconsistent state as a result of someone fiddling around, and God forbid you switch from Ubuntu to CentOS or something like that.
I find it's easier to make the OS part as simple as possible and use Docker for everything else since you can strictly enforce that your Dockerfile will always build the same way regardless of which OS you build it on.
This is my experience with Chef/Vagrant and one of the principal reasons we're moving our deployment over to Docker with CoreOS. Using etcd/skydns for service discovery and fleet to deploy containers. So far it's going very well, much easier than Chef ever was.
One big difference is that if you don't declare a resource it doesn't exist anymore. With chef, puppet and friends there's a big gray area of things that are one the machine but not managed by the tool. In practice there are always things lingering on old machines that are not described by the SCM.
It changes your deployment "atom" so instead of deploying compressed artefacts, you deploy containers. This has some advantages in itself (eg. it makes it easier to do gradual upgrades of your full stack, and you can run containers side by side on the same host), but is especially nice in combination with Mesos and Marathon which enables you to scale out horizontally across your cluster.
Out of curiosity, how do you manage your servers? Chef, Puppet, Salt Stack? Simple shell scripts? Why not using something like Heroku if you want the flexibility of scaling only when needed?
In my world of bootstrapped, smaller apps looking for market traction, even if things go well, a few Linodes should be enough to handle most of the traffic I'll ever need to deal with, so this kind of thing is kind of foreign to me. I'm curious how people utilize it in practice.