Deployment research
In the future, deployment will happen at the touch of a button. If you want a new server set up, you will just touch this button, several magical woodland fairies will be sacrificed to the elder gods, and a server will just appear set up for you, completely automagical. When something goes wrong, you just destroy the server, and maybe send apologetic cards to the families of the etherial woodland creatures. This will greatly improve our scalability.
However we have no idea how this will happen yet. Watch here for details as this is researched.
Contents
Goals
In short:
- Failover / load balancing.
- If a server goes down, another should pick it up.
- If things are slow, another server should be able to pick things up.
- Boot and shoot approach
- Quick and mindless spin-up of a new server
- Ability to just take down a server / node if it's not working well
- Still have the ability to step in and debug things as much as we like
- An approach for spinning up live & devel servers using mostly the same setup
The cc.engine stack has some advantages. For the most part, there simply is no "changing database"... everything is stored in RDF files that are in a git repository / in python packages.
We also need to support CC Wordpress, but this can also be checked out of an svn repository on the fly. Assuming we do edits somewhere else and just push to the server, no need to do backups of these "node" servers, even!
(However maybe we will eventually want to use this setup with things that *do* have a database that matters, like cc network?)
Cleanness is also a goal. It should be clear enough what's happening in the deployment setup and how to adjust / reproduce things. From the talk Continuous Deployment, by Laurens Van Houtven, he talks about a setup wherein anything goes, and perl scripts wrap scheme scripts alongside erlang alongside java and PHP...:
"And developers' sense of decency and morals just completely falter, and they start implementing just completely anything, and then you have some giant shambling lovecraftian horror of a deployment system..."
That's the deployment setup we don't want :)
Deployment
Deployment covers:
- Server creation (spinning up a new server with our setup)
- Server management (which stuff is on which machine?)
- Server updating (update software / data on server)
The following tools are being considered:
Fabric
plain ol' ssh and bash
Old stuff
(Old stuff from Jurisdiction Management)
On live Perhaps in a future time, the follow will replace the above steps:
$ fab staging deploy
On staging
Perhaps in a future time, the following can replace the above steps:
$ fab production deploy $ fab idot imgsplat '''XXX It'd be nice to have this maintained in a fab file so we don't need extended instructions; maybe we need a repo of "deployment tools" that contain these scripts? Feels a bit like a inappropriate mixing of concerns'''
Silver Lining
I really like the approach that Silver Lining takes in many ways. The idea of being able to just take a virtualenv and push it to a new server is quite appealing. It's also fairly declarative and uses a config file format that's like paste deploy.
However, we need some very custom stuff. Our apache config files are large and bloated. Maybe they could be a little bit less bloated, but there is a ton of stuff in there like all the rewriting and static file serving we're doing that Silver Lining doesn't support and doesn't want to support. So for cc.engine at least, silver lining is right out.
Puppet
Puppet seems similar to Silver Lining in the sense that it has an abstracted config file setup that "describes" the server and you use that to push to a bunch of slave nodes. Unlike Silver Lining though it appears to be a lot less "simple" and allows things like having your own apache configs.
As I understand it, Puppet kind of has its own config "language" to describe the server setup. It also has a full node management system, which would be useful.
There are some advantages of this in that, as I understand it, you can have some existing nodes running, and if you change the description of the server, it can reconfigure the server for you to match the new "described" setup.
Or so I think.
Anyway that all seems pretty cool but also pretty complicated and abstract, maybe a lot more than we need initially.
Chef
I don't know much about Chef except that it's apparently a lot closer to Puppet. But insted of its abstracted config system and etc you write a lot of ruby.
For now, that makes me pretty disinterested in Chef. I don't understand yet how it could be better than Fabric on that end.
bcfg2
Where to deploy to
This section is about what we might end up deploying to in the end.
Ideally our deployment system will ge generic enough to deploy to multiple types of servers, but...
generic metal servers
Linode
openstack
Rackspace Cloud, for example, runs this.
