Easy deployment of custom debian packages with Puppet

The context

In modern computer science, applications often need to withstand large amount of traffic or process a lot of data. The way to achieve required performance is distributed computing, which employs many machines to carry out tasks in parallel. Large number of nodes means that there could be problems with keeping the production environment consistent on all of them.

The problem

We would like to be able to deploy a custom created debian package across multiple machines. This could be achieved by using scp to transfer a .deb file to every node and then running dpkg to install the package. Such manual installation could work well for a few machines, but as the number of nodes grows, an automatic solution is a must.

The solution

Deployment of custom packages can be achieved by creating a debian repository. Client would connect to this repository and install the latest version of given package:

$ sudo apt-get update
$ sudo apt-get install custom-package

Having in mind, that our package is going to be deployed to multiple machines, we would like to automate that process. Our solution will employ:

Puppet - installing components and managing configuration
Aptly - publishing and updating debian repository
Apache HTTPD - exposing Aptly repository to clients

Server side

The apt_repository::server class provides a Puppet recipy to host a debian repository on a given machine. This has been achieved in a few steps:

Installing Aptly

Atply can be easily installed using module from puppetforge:

class { 'aptly':
  config => {
    rootDir => $aptly_rootdir,
  }
}

This step is fairly straightforward, apart from one gotcha. Aptly is not available in default debian repositories of the operating system, so we need to add Aptly repository to /etc/apt/sources.list. Luckily, the Aptly module does this job, but does not trigger apt-get update. We ensure such update by adding a Puppet command:

Class['apt::update'] -> Package<| |>

Publishing repository

To publish a repository, we need to call:

vagrant@server:~$ sudo /usr/bin/aptly -architectures=amd64 -skip-signing=true publish repo pracuj-debs

Sadly, there is a problem. Even though the first command call works flawlessly:

...
Local repo pracuj-debs has been successfully published.
...

The following call of the same command fails:

...
ERROR: prefix/distribution already used by another published repo: ./precise [amd64] publishes {main: [pracuj-debs]}

To prevent such error, a simple shell script, publish_repo.sh has been provided in the module. It checks if given repository exists, and executes publish command if it does not. Execution of publish_repo.sh is idempotent, just like any command executed by Puppet should be.

The server module also provides a command to update Aptly repository with packages included in directory specified by debian_package_dir parameter:

vagrant@server:~$ sudo /usr/bin/update-apt-repository

Serving repository

Another puppetforge module, apache, is used to create a webservice to serve the contents of the aplty repository:

class { 'apache': }

apache::vhost { 'bigdata.aptrepo':
  port    => $server_port,
  docroot => "${aptly_rootdir}/public",
  require => Package['aptly'],
}

Client side

Thanks to the apt module, the apt_repository::client class is quite simple. All that needs to be done is adding the server host to /etc/hosts file and list the Aptly server as debian source:

host { 'apt-repository-server':
  ip => $server_address,
}

apt::source { 'atplyrepo':
  location       => "http://apt-repository-server:${server_port}/",
  architecture   => $repo_architecture,
  release        => $repo_release,
  repos          => 'main',
  allow_unsigned => true,
  require        => Host['apt-repository-server'],
}

The signing of the repositories was skipped when Aptly repository was created (using the -skip-signing=true flag), so for the client the allow_unsigned => true option has been set. This is fine as long as the Aptly repository will not be made public.

Configuration

Whole apt_repository module is configured with Hiera. Single set of parameters configures both client and server, which guarantees that they will work well together. Parameters are placed in .yaml files, that assign configuration to given nodes in hierarchical fashion.

Demo Time!

To follow the example presented below, Vagrant 1.7.4+ and SSH client are needed. Source code can be found on Github

Let’s check if the apt_repository module works as expected! Two simple debian packages have been created. pracuj-example_1.0-1_all.deb is placed in files/debian_package_dir (and hiera parameter debian_package_dir has been set to that directory). Other file, pracuj-example_1.2-1_all.deb has been placed outside of the package directory. Both those packages install simple bash command that prints version of package on console.

Vagrant needs to know, how to configure each node. For server we would like to install apt-repository::server and trigger update-apt-repository afterwards:

node 'server' {
  ...
  class { 'apt_repository::server': }
  
  exec { '/usr/bin/update-apt-repository':
    require => Class['apt_repository::server'],
  }
}

So, let’s run the server:

> vagrant up server

After a while, the server should be up and running. Client configuration is also simple.

node 'client' {
  ...
  class { 'apt_repository::client': }

  package {
    'pracuj-example':
      ensure  => latest,
      require => Class['apt_repository::client'],
  }
}

After running vagrant up client the latest version of the pracuj-example package should be installed:

> vagrant ssh client

vagrant@client:~$ pracuj-example
This is version 1.0-1
vagrant@client:~$ logout

Now, let’s add a newer version of the pracuj-example package to the Aptly repository:

> vagrant ssh server

vagrant@server:~$ cd /vagrant/files/
vagrant@server:/vagrant/files$ mv pracuj-example_1.2-1_all.deb debian_package_dir/
vagrant@server:/vagrant/files$ sudo update-apt-repository
...
Publish for local repo ./precise [amd64] publishes {main: [pracuj-debs]} has been successfully updated.
vagrant@server:/vagrant/files$ logout

Let’s simulate refresh of Puppet configuration on client. Such task can be achieved by running vagrant provision:

> vagrant provision client
> vagrant ssh client

vagrant@client:~$ pracuj-example
This is version 1.2-1
vagrant@client:~$ logout

Hey! It works :)

Now only thing left is to cleanup:

> vagrant destroy -f

Wrap-up

We have demonstrated how to deploy a custom package in small, virtual two machine environment. Thanks to Puppet, deploying such package on hundreds of nodes is as easy. Presented solution could be used to propagate debian packages generated using continuous integration tools e.g. Jenkins and provide a consistent production environment across many nodes.