Server Configuration Management With Chef Solo

Having set up one Linux server from scratch too many, I decided to investigate configuration management solutions and try to automatise my setups.

I ended up choosing Chef Solo for the task, due to being a standalone solution not requiring a client/server setup. The following blog posts served as inspiration and reference for my implementation:

• Chef Solo tutorial: Managing a single server with Chef
• Simple Chef Solo Tutorial

Following up my previous post where I set up a base minimum Ubuntu installation with a SSH daemon running on the server (a requirement for my solution), I will now describe my implementation.

Revision control is a must for any project, so my first step was to create a git repository on github to maintain the Chef cookbooks and recipes as well as any additional files required by the solution. This also has the added advantage that cloning the git repository is an excellent way of deploying these files on the server.

The first component of the solution is a bash bootstrap script which runs on the local workstation, taking as its argument the remote user (which will need to have sudo privileges) and server address in the user@remote-ip-or-fqdn format, and through a SSH session takes care of automatising the following steps:

• Add the local user's public SSH key as a remote user's authorised key so the remote user's password doesn't have to be entered again (this requires typing the remote user's password).
• Update the remote system's package index and upgrade the installed packages to the latest available version.
• Install the git package and its dependencies.
• Clone the git repository and its sub-modules (not wanting to re-invent the wheel I make use of some Opscode Public Cookbooks which are referenced in the git repository as sub-modules).
• Hand over control to a second bash installation script that is available in the cloned repository.

Running on the remote server, this installation script automates, in turn, the following steps:

• Install the debconf-utils and python-software-properties system packages which simplify the tasks of scripting the installation of packages with interactive prompts and of adding additional package repositories from the command line (or in this case from a script).
• Add the OpsCode APT repository to the system's package repositories, from which we'll be able to install the Chef packages.
• Install the chef package and all its dependencies (the chef package includes Chef Solo), and create a link from its configuration file in our repository to its default location at /etc/chef.

Finally, with Chef Solo installed and configured, and with many thanks to our Bash scripts which took care of bootstrapping our Chef installation, we can now start using Chef recipes for all the remaining steps! The last line of the installation script simply invokes chef-solo with our default Chef JSON configuration file.

The Chef recipes perform (as they should) the bulk of the server installation and configuration (the previous steps were just the minimum required to install and configure Chef). The following are the actions performed by the Chef recipes:

• Disable running the chef client service on start-up. Since we will be running Chef Solo and aren't really connected to a Chef server and so having the Chef client running would be redundant and wasteful.
• Install the etckeeper package: keeping on with the revision control everywhere mantra, having the server's configuration files under /etc revision controlled seems like a very good idea and etckeeper does this automatically for you! The recipe also tweaks the etckeeper configuration to use git (which we already have installed) instead of Bazaar which is the default for the default DRCS for Ubuntu (although the author prefers git in the source distribution and recommends its use).
• Install the ufw (Uncomplicated Firewall) package. The default recipe blocks all incoming ports with the exception of the SSH port (22).
• Reconfigure the SSH daemon to disallow root logins and password authentication (it will only allow connections authenticated with a public key for non-root users). This and the previous steps are obvious security best practices.
• Install the acpid daemon package to allow host initiated graceful shutdowns on KVM virtual machine guests (which is the case of the this system).
• Install the byobu package to provide an enhanced terminal multiplexer experience when interactive sessions with the server are required.
• Install vim. Although I belong to the Church of Emacs, I am not a fundamentalist, and, because of its ubiquity, I tend to use an editor of the vi persuasion when wearing my systems administrator hat.
• Install htop for process and system resources monitoring.
• Install multitail (tail on steroids!) for an improved experience when following log files.
• Finally install the Apache2 web server (using the default recipe from the Opscode public cookbook) and reconfigure the firewall to allow HTTP (port 80) traffic in.

And that sums it up! A functional Ubuntu 12.04 LTS system with an Apache2 HTTP server setup, fully automated using Chef Solo and a couple of bootstrap shell scripts.

Minimal Ubuntu Precise

The purpose of the minimal Ubuntu installation is to server as a base that can be customised and configured automatically and reproducibly using a configuration management tool (in my case Chef in its Solo incarnation). For virtual machines, these manual steps can be easily automated with tools such as Vagrant or Ubuntu's JeOS VM Builder, but since my virtual server plan only supports booting the VM from a CD as the deployment method, I chose the minimal Ubuntu installation as the simplest alternative for the base installation.

These instructions are based on the installation of the Ubuntu 12.04 LTS (Precise Pangolin) Minimal CD and assume that the installed Ubuntu system will be the only one on the machine. After booting from the installation medium with the minimal CD image, the following steps were executed:

1. On the Installer boot menu choose: Install
2. On the Select a language screen choose your desired language.
3. On the Select your location screen choose your country, territory or area.
4. On the Configure the keyboard screen use one of the available methods to choose your keyboard layout.
5. On the Configure the network screen specify your hostname.
6. On the Choose a mirror of the Ubuntu archive series of screens choose the desired Ubuntu mirror to download the packages required for the installation. The best mirror is usually the one geographically closest to you. If you need to specify an http proxy this should also be done in the respective screen (leave the prompt blank for no proxy).
7. On the Set up users and passwords series of screens specify the account details for systems default non-root user: full name, username, password and whether the user's home directory should be encrypted or not. This user will be able to use the sudo command to execute commands as root.
8. On the Configure the clock confirm if the suggested time zone is correct or choose the desired one.
9. On the Partition disks series of screens choose the desired partitioning method, and the disk you use to partition for installation, doing if necessary any manual adjustments. In the end confirm your choices and write the new partitions to the disk.
10. On the Configuring discovery screen when asked how you want to manage upgrades on the system choose the option to install security updates automatically.
11. On the Software selection screen, choose only the OpenSSH server as the software to install.
12. On the Install the GRUB boot loader on a hard disk screen, when asked if you want to install the GRUB boot loader to the master boot record, answer Yes.
13. On the Finish the installation screen, when asked if the system clock is set to UTC, answer Yes.
14. The installation will now be finished. Follow the instruction on the screen to reboot the machine into your new installation.

Hello World!

Welcome to my new blog where I plan to share interesting tidbits, gleaned from my personal pet projects, on subjects potentially useful for others (and as a future reference for me).

Python and Linux will be recurring subjects of this blog and provide a fair amount of the content since they are two of my main interests.

The occasional random thought or opinion piece may also grace these pages.

That's all for now. Watch this space!