x

Menu

Using Test Kitchen With Puppet

A short tutorial on how to use Test Kitchen With Puppet

published in: Agile Testing Methodologies

From the kitchen.ci homepage:

Your infrastructure deserves tests too.

So lets write some integration test for our puppet code. We will use a super simple puppet manifest just deploying ntp as a hello world example focusing on test-kitchen instead.

Prerequisites

You need git (you should use it ;-) , ruby (use > 2.0) and bundler installed

Cover Your Bases

So let us start by creating a directory and a github repo for our project:

mkdir puppet-kitchen-example
cd puppet-kitchen-example
git init
hub create # this is optional

Next up we want to leverage bundler so lets create a Gemfile

bundle init
Writing new Gemfile to puppet-kitchen-example/Gemfile

We want to use test-kitchen and the kitchen-puppet gem. Until the patch gets pulled in we use a fork here:

echo 'gem "test-kitchen"' >> Gemfile
echo 'gem "kitchen-puppet"' >> Gemfile

bundle install
Fetching gem metadata from https://rubygems.org/..........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using kitchen-puppet (0.0.7 804c905)
Using mixlib-shellout (1.4.0)
Using net-ssh (2.9.0)
Using net-scp (1.2.1)
Using safe_yaml (1.0.3)
Using thor (0.19.1)
Using test-kitchen (1.2.1)
Using bundler (1.5.3)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

No we are able to initialize test-kitchen

kitchen init
      create  .kitchen.yml
      create  test/integration/default
      create  .gitignore
      append  .gitignore
      append  .gitignore
      append  Gemfile
You must run `bundle install' to fetch any new gems.

So lets do as advised and install the new dependencies

bundle install

Resolving dependencies...
Using kitchen-puppet (0.0.8)
Using mixlib-shellout (1.4.0)
Using net-ssh (2.9.0)
Using net-scp (1.2.1)
Using safe_yaml (1.0.3)
Using thor (0.19.1)
Using test-kitchen (1.2.1)
Using kitchen-vagrant (0.15.0)
Using bundler (1.5.3)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Now we have to update the kitchen.yml file with puppet specific options from the kitchen-puppet gem See kitchen-puppet and provisioner options for details

cat > .kitchen.yml <<YAML
---
driver:
  name: vagrant

provisioner:
  name: puppet_apply
  manifests_path: manifests
  modules_path: modules
  hiera_data_path: hieradata

platforms:
  - name: nocm_ubuntu-12.04
    driver_plugin: vagrant
    driver_config:
      box: nocm_ubuntu-12.04
      box_url: http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-12042-x64-vbox4210-nocm.box
  - name: centos-6.5
    driver_plugin: vagrant
    driver_config:
      box: nocm_centos-6.5
      box_url: http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box

suites:
  - name: default
    manifest: site.pp
YAML

With this configuration in place we should be able to create the instances:

± kitchen create
-----> Starting Kitchen (v1.2.1)
-----> Creating <default-nocm-ubuntu-1204>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'nocm_ubuntu-12.04' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Adding box 'nocm_ubuntu-12.04' (v0) for provider: virtualbox
           default: Downloading: http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-12042-x64-vbox4210-nocm.box
       ==> default: Successfully added box 'nocm_ubuntu-12.04' (v0) for 'virtualbox'!
       ==> default: Importing base box 'nocm_ubuntu-12.04'...
       ==> default: Matching MAC address for NAT networking...
       ==> default: Setting the name of the VM: default-nocm-ubuntu-1204_default_1399308985043_82278
       ==> default: Clearing any previously set forwarded ports...
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2222 (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2222
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Connection timeout. Retrying...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
           default: The guest additions on this VM do not match the installed version of
           default: VirtualBox! In most cases this is fine, but in rare cases it can
           default: prevent things such as shared folders from working properly. If you see
           default: shared folder errors, please make sure the guest additions within the
           default: virtual machine match the version of VirtualBox you have installed on
           default: your host and reload your VM.
           default:
           default: Guest Additions Version: 4.2.10
           default: VirtualBox Version: 4.3
       ==> default: Setting hostname...
       Vagrant instance <default-nocm-ubuntu-1204> created.
       Finished creating <default-nocm-ubuntu-1204> (2m15.59s).
-----> Creating <default-centos-65>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'nocm_centos-6.5' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Adding box 'nocm_centos-6.5' (v0) for provider: virtualbox
           default: Downloading: http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box
       ==> default: Successfully added box 'nocm_centos-6.5' (v0) for 'virtualbox'!
       ==> default: Importing base box 'nocm_centos-6.5'...
       ==> default: Matching MAC address for NAT networking...
       ==> default: Setting the name of the VM: default-centos-65_default_1399309228645_16849
       ==> default: Fixed port collision for 22 => 2222. Now on port 2200.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2200 (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2200
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Connection timeout. Retrying...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       Vagrant instance <default-centos-65> created.
       Finished creating <default-centos-65> (4m15.89s).
-----> Kitchen is finished. (6m31.73s)

This looks very promising. Lets verify the current state kitchen list

± kitchen list
Instance                  Driver   Provisioner  Last Action
default-nocm-ubuntu-1204  Vagrant  PuppetApply  Created
default-centos-65         Vagrant  PuppetApply  Created

Add Puppet and librarian-puppet To The Mix

Since we are trying to test a puppet-apply run we have to add puppet to the mix. Unfortunately the librarian-puppet integration with kitchen-puppet got me some errors. Let us use it anyways to fetch the puppet modules and move it out of the way afterwards.

echo 'gem "puppet"' >> Gemfile
echo 'gem "librarian-puppet"' >> Gemfile
bundle install
Resolving dependencies...
Using CFPropertyList (2.2.7)
Using facter (2.0.1)
Using json_pure (1.8.1)
Using hiera (1.3.2)
Using highline (1.6.21)
Using json (1.8.1)
Using kitchen-puppet (0.0.8)
Using mixlib-shellout (1.4.0)
Using net-ssh (2.9.0)
Using net-scp (1.2.1)
Using safe_yaml (1.0.3)
Using thor (0.19.1)
Using test-kitchen (1.2.1)
Using kitchen-vagrant (0.15.0)
Using librarian (0.1.2)
Using librarian-puppet (1.0.1)
Using rgen (0.6.6)
Using puppet (3.5.1)
Using bundler (1.5.3)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Now we can use librarian-puppet to create our Puppetfile

± librarian-puppet init
      create  Puppetfile

We want to use the Puppetlabs ntp module. lets change the Puppetfile to fetch it.

# OSX/BSD
sed -i '' 's#modulefile#mod "puppetlabs/ntp"#' Puppetfile

# Linux
sed -i 's#modulefile#mod "puppetlabs/ntp"#' Puppetfile

cat Puppetfile
#!/usr/bin/env ruby
#^syntax detection

forge "http://forge.puppetlabs.com"

# use dependencies defined in Modulefile
mod "puppetlabs/ntp"

# mod 'puppetlabs/stdlib'

# mod 'ntp',
#   :git => 'git://github.com/puppetlabs/puppetlabs-ntp.git'

# mod 'apt',
#   :git => 'https://github.com/puppetlabs/puppetlabs-apt.git',
#   :ref => 'feature/master/dans_refactor'

run

librarian-puppet install

and it will fetch the needed modules

ls modules
ntp    stdlib

to actually install ntp we have to create a manifest. Our test suite will run site.pp so let us create a simple one.

mkdir manifests
cat > manifests/site.pp << SITE

class { '::ntp':
  servers => [ '0.pool.ntp.org', '1.pool.ntp.org' ],
}

SITE

First Step To Success: Convergence

The infrastructure code is in place. The infrastructure itself is ready (read created), so we can do our operations work and install ntp on the nodes.

I have truncated the output a little bit.

kitchen converge
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-nocm-ubuntu-1204>...
       Preparing files for transfer
       Preparing modules
       Preparing manifests
       Preparing hiera data
       Finished Preparing files for transfer
       Installing puppet, will try to determine platform os
--2014-05-08 16:12:02--  http://apt.puppetlabs.com/puppetlabs-release-precise.deb
Resolving apt.puppetlabs.com (apt.puppetlabs.com)... 198.58.114.168, 2600:3c00::f03c:91ff:fe69:6bf0
Connecting to apt.puppetlabs.com (apt.puppetlabs.com)|198.58.114.168|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3430 (3.3K) [application/x-debian-package]
Saving to: `puppetlabs-release-precise.deb'

100%[======================================>] 3,430       --.-K/s   in 0s

2014-05-08 16:12:03 (318 MB/s) - `puppetlabs-release-precise.deb' saved [3430/3430]

Selecting previously unselected package puppetlabs-release.

[ ... snipped content ...]

Fetched 5,163 kB in 12s (409 kB/s)
        package lists... Done
        package lists... Done
       g dependency tree
        state information... Done
The following extra packages will be installed:
  augeas-lenses debconf-utils facter hiera libaugeas-ruby libaugeas-ruby1.8
  libaugeas0 libjson-ruby libruby libruby1.8 libshadow-ruby1.8 puppet-common
  ruby ruby-json ruby-rgen ruby1.8 virt-what
Suggested packages:
  augeas-doc augeas-tools puppet-el vim-puppet ruby-selinux libselinux-ruby1.8
  librrd-ruby1.9.1 librrd-ruby1.8 ri ruby-dev ruby1.8-examples ri1.8
Recommended packages:
  rdoc
The following NEW packages will be installed:
  augeas-lenses debconf-utils facter hiera libaugeas-ruby libaugeas-ruby1.8
  libaugeas0 libjson-ruby libruby libruby1.8 libshadow-ruby1.8 puppet
  puppet-common ruby ruby-json ruby-rgen ruby1.8 virt-what
0 upgraded, 18 newly installed, 0 to remove and 123 not upgraded.
Need to get 3,771 kB of archives.
After this operation, 14.6 MB of additional disk space will be used.

[ ... snipped content ...]

Fetched 3,771 kB in 5s (629 kB/s)
Selecting previously unselected package augeas-lenses.
(Reading database ... 54000 files and directories currently installed.)
Unpacking augeas-lenses (from .../augeas-lenses_0.10.0-0ubuntu4_all.deb) ...
Selecting previously unselected package debconf-utils.
Unpacking debconf-utils (from .../debconf-utils_1.5.42ubuntu1_all.deb) ...
Selecting previously unselected package libruby1.8.
Unpacking libruby1.8 (from .../libruby1.8_1.8.7.352-2ubuntu1.4_amd64.deb) ...
Selecting previously unselected package ruby1.8.
Unpacking ruby1.8 (from .../ruby1.8_1.8.7.352-2ubuntu1.4_amd64.deb) ...
Selecting previously unselected package ruby.
Unpacking ruby (from .../apt/archives/ruby_4.8_all.deb) ...
Selecting previously unselected package virt-what.
Unpacking virt-what (from .../virt-what_1.11-1_amd64.deb) ...
Selecting previously unselected package facter.
Unpacking facter (from .../facter_2.0.1-1puppetlabs1_amd64.deb) ...
Selecting previously unselected package libaugeas0.
Unpacking libaugeas0 (from .../libaugeas0_0.10.0-0ubuntu4_amd64.deb) ...
Selecting previously unselected package libaugeas-ruby1.8.
Unpacking libaugeas-ruby1.8 (from .../libaugeas-ruby1.8_0.3.0-1.1ubuntu4_amd64.deb) ...
Selecting previously unselected package libaugeas-ruby.
Unpacking libaugeas-ruby (from .../libaugeas-ruby_0.3.0-1.1ubuntu4_all.deb) ...
Selecting previously unselected package ruby-json.
Unpacking ruby-json (from .../ruby-json_1.6.3-1_amd64.deb) ...
Selecting previously unselected package libjson-ruby.
Unpacking libjson-ruby (from .../libjson-ruby_1.6.3-1_all.deb) ...
Selecting previously unselected package libruby.
Unpacking libruby (from .../archives/libruby_4.8_all.deb) ...
Selecting previously unselected package libshadow-ruby1.8.
Unpacking libshadow-ruby1.8 (from .../libshadow-ruby1.8_1.4.1-8build1_amd64.deb) ...
Selecting previously unselected package hiera.
Unpacking hiera (from .../hiera_1.3.2-1puppetlabs1_all.deb) ...
Selecting previously unselected package ruby-rgen.
Unpacking ruby-rgen (from .../ruby-rgen_0.6.5-1puppetlabs1_all.deb) ...
Selecting previously unselected package puppet-common.
Unpacking puppet-common (from .../puppet-common_3.5.1-1puppetlabs1_all.deb) ...
Selecting previously unselected package puppet.
Unpacking puppet (from .../puppet_3.5.1-1puppetlabs1_all.deb) ...
Processing triggers for man-db ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Setting up augeas-lenses (0.10.0-0ubuntu4) ...
Setting up debconf-utils (1.5.42ubuntu1) ...
Setting up libruby1.8 (1.8.7.352-2ubuntu1.4) ...
Setting up ruby1.8 (1.8.7.352-2ubuntu1.4) ...
update-alternatives: using /usr/bin/ruby1.8 to provide /usr/bin/ruby (ruby) in auto mode.
Setting up ruby (4.8) ...
Setting up virt-what (1.11-1) ...
Setting up facter (2.0.1-1puppetlabs1) ...
Setting up libaugeas0 (0.10.0-0ubuntu4) ...
Setting up libaugeas-ruby1.8 (0.3.0-1.1ubuntu4) ...
Setting up libaugeas-ruby (0.3.0-1.1ubuntu4) ...
Setting up ruby-json (1.6.3-1) ...
Setting up libjson-ruby (1.6.3-1) ...
Setting up libruby (4.8) ...
Setting up libshadow-ruby1.8 (1.4.1-8build1) ...
Setting up hiera (1.3.2-1puppetlabs1) ...
Setting up ruby-rgen (0.6.5-1puppetlabs1) ...
Setting up puppet-common (3.5.1-1puppetlabs1) ...
Setting up puppet (3.5.1-1puppetlabs1) ...
 * Starting puppet agent
puppet not configured to start, please edit /etc/default/puppet to enable
                                                                         [ OK ]
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
-----> Installing Chef Omnibus to install busser to run tests
  % Total    % Received %        Xferd  Average Speed   Time    Time     Time  Current

         Dload  Upload   Total   Spent    Left  Speed

100 15934  100 15934           0     0  15250      0  0:00:01  0:00:01 --:--:-- 26035
Downloading Chef  for ubuntu...
downloading https://www.getchef.com/chef/metadata?v=&prerelease=false&nightlies=false&p=ubuntu&pv=12.04&m=x86_64
  to file /tmp/install.sh.2139/metadata.txt
trying wget...
url https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.12.4-1_amd64.deb
md5 c45e1d4f7842af1048f788c4452d6cc0
sha256  595cd1e884efd21f8f5e34bdbe878421a9d5c1c24abd3c669a84e8ed261317a3
downloaded metadata file looks valid...
downloading https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.12.4-1_amd64.deb
  to file /tmp/install.sh.2139/chef_11.12.4-1_amd64.deb
trying wget...
Comparing checksum with sha256sum...
Installing Chef
installing with dpkg...
Selecting previously unselected package chef.
(Reading database ... 56644 files and directories currently installed.)
Unpacking chef (from .../chef_11.12.4-1_amd64.deb) ...
Setting up chef (11.12.4-1) ...
Thank you for installing Chef!
       Transfering files to <default-nocm-ubuntu-1204>
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
Notice: Compiled catalog for default-nocm-ubuntu-1204.vagrantup.com in environment production in 0.32 seconds
Notice: /Stage[main]/Ntp::Config/File[/etc/ntp.conf]/content: content changed '{md5}32280703a4ba7aa1148c48895097ed07' to '{md5}4af276f148adc7960235ad4ba09cad48'
       Notice: /Stage[main]/Ntp::Service/Service[ntp]: Triggered 'refresh' from 1 events
Notice: Finished catalog run in 2.21 seconds

This is the first part. Converge on Ubuntu. An interesting point: the testing code we are about to add needs a ruby on the system. test-kitchen started as a community project around chef. So test-kitchen leverages the ruby which comes with chef. That’s why chef gets installed alongside puppet.

And the same command shines a light on test-kitchens beauty. We run it on as many OS as we put in the yml. How cool is that :-)

-----> Converging <default-centos-65>...
       Preparing files for transfer
       Preparing modules
       Preparing manifests
       Preparing hiera data
       Finished Preparing files for transfer
       Installing puppet, will try to determine platform os
       which: no puppet in (/usr/local/bin:/bin:/usr/bin)
       Retrieving https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
       warning: /var/tmp/rpm-tmp.TtvWua: Header V4 RSA/SHA1 Signature, key ID 4bd6ec30: NOKEY
Preparing...                #####  ########################################### [100%]
   1:puppetlabs-release            ########################################### [100%]
       Loaded plugins: fastestmirror, security
       Determining fastest mirrors
        * base: ftp.fau.de
        * extras: ftp.fau.de
        * updates: mirror.de.leaseweb.net
base                                                     | 3.7 kB     00:00
base/group_gz                                            | 220 kB     00:00
base/filelists_db                                        | 5.9 MB     00:01
base/primary_db                                          | 4.4 MB     00:00
base/other_db                                            | 2.8 MB     00:01
extras                                                   | 3.4 kB     00:00
extras/filelists_db                                      |  11 kB     00:00
extras/prestodelta                                       |  907 B     00:00
extras/primary_db                                        |  19 kB     00:00
extras/other_db                                          | 5.8 kB     00:00
puppetlabs-deps                                          | 2.5 kB     00:00
puppetlabs-deps/filelists_db                             | 182 kB     00:00
puppetlabs-deps/primary_db                               |  23 kB     00:00
puppetlabs-deps/other_db                                 |  16 kB     00:00
puppetlabs-products                                      | 2.5 kB     00:00
puppetlabs-products/filelists_db                         | 921 kB     00:00
puppetlabs-products/primary_db                           | 107 kB     00:00
puppetlabs-products/other_db                             | 145 kB     00:00
updates                                                  | 3.4 kB     00:00
updates/filelists_db                                     | 1.7 MB     00:00
updates/prestodelta                                      | 262 kB     00:00
updates/primary_db                                       | 2.6 MB     00:00
updates/other_db                                         |  21 MB     00:04
       Metadata Cache Created
       Loaded plugins: fastestmirror, security
       Loading mirror speeds from cached hostfile
        * base: ftp.fau.de
        * extras: ftp.fau.de
        * updates: mirror.de.leaseweb.net
       Setting up Install Process
       Resolving Dependencies
       --> Running transaction check
       ---> Package puppet.noarch 0:3.5.1-1.el6 will be installed
       --> Processing Dependency: facter >= 1:1.7.0 for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby >= 1.8.7 for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: hiera >= 1.0.0 for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby >= 1.8 for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby-rgen >= 0.6.5 for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby(selinux) for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: /usr/bin/ruby for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: rubygem-json for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby-shadow for package: puppet-3.5.1-1.el6.noarch
       --> Processing Dependency: ruby-augeas for package: puppet-3.5.1-1.el6.noarch
       --> Running transaction check
       ---> Package facter.x86_64 1:2.0.1-1.el6 will be installed
       ---> Package hiera.noarch 0:1.3.2-1.el6 will be installed
       ---> Package libselinux-ruby.x86_64 0:2.0.94-5.3.el6_4.1 will be installed
       ---> Package ruby.x86_64 0:1.8.7.352-13.el6 will be installed
       --> Processing Dependency: ruby-libs = 1.8.7.352-13.el6 for package: ruby-1.8.7.352-13.el6.x86_64
       --> Processing Dependency: libruby.so.1.8()(64bit) for package: ruby-1.8.7.352-13.el6.x86_64
       ---> Package ruby-augeas.x86_64 0:0.4.1-3.el6 will be installed
       --> Processing Dependency: augeas-libs >= 0.8.0 for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0(AUGEAS_0.12.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0(AUGEAS_0.8.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0(AUGEAS_0.11.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0(AUGEAS_0.1.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0(AUGEAS_0.10.0)(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       --> Processing Dependency: libaugeas.so.0()(64bit) for package: ruby-augeas-0.4.1-3.el6.x86_64
       ---> Package ruby-rgen.noarch 0:0.6.5-2.el6 will be installed
       ---> Package ruby-shadow.x86_64 1:2.2.0-2.el6 will be installed
       ---> Package rubygem-json.x86_64 0:1.5.5-1.el6 will be installed
       --> Processing Dependency: rubygems for package: rubygem-json-1.5.5-1.el6.x86_64
       --> Running transaction check
       ---> Package augeas-libs.x86_64 0:1.0.0-5.el6_5.1 will be installed
       ---> Package ruby-libs.x86_64 0:1.8.7.352-13.el6 will be installed
       --> Processing Dependency: libreadline.so.5()(64bit) for package: ruby-libs-1.8.7.352-13.el6.x86_64
       ---> Package rubygems.noarch 0:1.3.7-5.el6 will be installed
       --> Processing Dependency: ruby-rdoc for package: rubygems-1.3.7-5.el6.noarch
       --> Running transaction check
       ---> Package compat-readline5.x86_64 0:5.2-17.1.el6 will be installed
       ---> Package ruby-rdoc.x86_64 0:1.8.7.352-13.el6 will be installed
       --> Processing Dependency: ruby-irb = 1.8.7.352-13.el6 for package: ruby-rdoc-1.8.7.352-13.el6.x86_64
       --> Running transaction check
       ---> Package ruby-irb.x86_64 0:1.8.7.352-13.el6 will be installed
       --> Finished Dependency Resolution

       Dependencies Resolved

       ================================================================================
        Package            Arch     Version                Repository             Size
       ================================================================================
       Installing:
        puppet             noarch   3.5.1-1.el6            puppetlabs-products   1.2 M
       Installing for dependencies:
        augeas-libs        x86_64   1.0.0-5.el6_5.1        updates               309 k
        compat-readline5   x86_64   5.2-17.1.el6           base                  130 k
        facter             x86_64   1:2.0.1-1.el6          puppetlabs-products    84 k
        hiera              noarch   1.3.2-1.el6            puppetlabs-products    23 k
        libselinux-ruby    x86_64   2.0.94-5.3.el6_4.1     base                   99 k
        ruby               x86_64   1.8.7.352-13.el6       updates               534 k
        ruby-augeas        x86_64   0.4.1-3.el6            puppetlabs-deps        21 k
        ruby-irb           x86_64   1.8.7.352-13.el6       updates               314 k
        ruby-libs          x86_64   1.8.7.352-13.el6       updates               1.6 M
        ruby-rdoc          x86_64   1.8.7.352-13.el6       updates               377 k
        ruby-rgen          noarch   0.6.5-2.el6            puppetlabs-deps       237 k
        ruby-shadow        x86_64   1:2.2.0-2.el6          puppetlabs-deps        13 k
        rubygem-json       x86_64   1.5.5-1.el6            puppetlabs-deps       763 k
        rubygems           noarch   1.3.7-5.el6            base                  207 k

       Transaction Summary
       ================================================================================
       Install      15 Package(s)

       Total download size: 5.9 M
       Installed size: 14 M
       Downloading Packages:
(1/15): augeas-libs-1.0.0-5.el6_5.1.x86_64.rpm           | 309 kB     00:00
(2/15): compat-readline5-5.2-17.1.el6.x86_64.rpm         | 130 kB     00:00
(3/15): facter-2.0.1-1.el6.x86_64.rpm                    |  84 kB     00:00
(4/15): hiera-1.3.2-1.el6.noarch.rpm                     |  23 kB     00:00
(5/15): libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64.rpm    |  99 kB     00:00
(6/15): puppet-3.5.1-1.el6.noarch.rpm                    | 1.2 MB     00:01
(7/15): ruby-1.8.7.352-13.el6.x86_64.rpm                 | 534 kB     00:00
(8/15): ruby-augeas-0.4.1-3.el6.x86_64.rpm               |  21 kB     00:00
(9/15): ruby-irb-1.8.7.352-13.el6.x86_64.rpm             | 314 kB     00:00
(10/15): ruby-libs-1.8.7.352-13.el6.x86_64.rpm           | 1.6 MB     00:00
(11/15): ruby-rdoc-1.8.7.352-13.el6.x86_64.rpm           | 377 kB     00:00
(12/15): ruby-rgen-0.6.5-2.el6.noarch.rpm                | 237 kB     00:00
(13/15): ruby-shadow-2.2.0-2.el6.x86_64.rpm              |  13 kB     00:00
(14/15): rubygem-json-1.5.5-1.el6.x86_64.rpm             | 763 kB     00:00
(15/15): rubygems-1.3.7-5.el6.noarch.rpm                 | 207 kB     00:00
       --------------------------------------------------------------------------------
       Total                                           253 kB/s | 5.9 MB     00:23
       warning: rpmts_HdrFromFdno: Header V4 RSA/SHA512 Signature, key ID 4bd6ec30: NOKEY
       Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
       Importing GPG key 0x4BD6EC30:
        Userid : Puppet Labs Release Key (Puppet Labs Release Key) <info@puppetlabs.com>
        Package: puppetlabs-release-6-10.noarch (installed)
        From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
       Running rpm_check_debug
       Running Transaction Test
       Transaction Test Succeeded
       Running Transaction
       Warning: RPMDB altered outside of yum.
  Installing : augeas-libs-1.0.0-5.el6_5.1.x86_64                          1/15
  Installing : libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64                   2/15
  Installing : compat-readline5-5.2-17.1.el6.x86_64                        3/15
  Installing : ruby-libs-1.8.7.352-13.el6.x86_64                           4/15
  Installing : ruby-1.8.7.352-13.el6.x86_64                                5/15
  Installing : ruby-rgen-0.6.5-2.el6.noarch                                6/15
  Installing : 1:facter-2.0.1-1.el6.x86_64                                 7/15
  Installing : ruby-irb-1.8.7.352-13.el6.x86_64                            8/15
  Installing : ruby-rdoc-1.8.7.352-13.el6.x86_64                           9/15
  Installing : rubygems-1.3.7-5.el6.noarch                                10/15
  Installing : rubygem-json-1.5.5-1.el6.x86_64                            11/15
  Installing : hiera-1.3.2-1.el6.noarch                                   12/15
  Installing : 1:ruby-shadow-2.2.0-2.el6.x86_64                           13/15
  Installing : ruby-augeas-0.4.1-3.el6.x86_64                             14/15
  Installing : puppet-3.5.1-1.el6.noarch                                  15/15
  Verifying  : ruby-rgen-0.6.5-2.el6.noarch                                1/15
  Verifying  : puppet-3.5.1-1.el6.noarch                                   2/15
  Verifying  : ruby-1.8.7.352-13.el6.x86_64                                3/15
  Verifying  : 1:facter-2.0.1-1.el6.x86_64                                 4/15
  Verifying  : compat-readline5-5.2-17.1.el6.x86_64                        5/15
  Verifying  : ruby-rdoc-1.8.7.352-13.el6.x86_64                           6/15
  Verifying  : ruby-irb-1.8.7.352-13.el6.x86_64                            7/15
  Verifying  : libselinux-ruby-2.0.94-5.3.el6_4.1.x86_64                   8/15
  Verifying  : 1:ruby-shadow-2.2.0-2.el6.x86_64                            9/15
  Verifying  : rubygems-1.3.7-5.el6.noarch                                10/15
  Verifying  : ruby-libs-1.8.7.352-13.el6.x86_64                          11/15
  Verifying  : rubygem-json-1.5.5-1.el6.x86_64                            12/15
  Verifying  : hiera-1.3.2-1.el6.noarch                                   13/15
  Verifying  : ruby-augeas-0.4.1-3.el6.x86_64                             14/15
  Verifying  : augeas-libs-1.0.0-5.el6_5.1.x86_64                         15/15

       Installed:
         puppet.noarch 0:3.5.1-1.el6

       Dependency Installed:
         augeas-libs.x86_64 0:1.0.0-5.el6_5.1
         compat-readline5.x86_64 0:5.2-17.1.el6
         facter.x86_64 1:2.0.1-1.el6
         hiera.noarch 0:1.3.2-1.el6
         libselinux-ruby.x86_64 0:2.0.94-5.3.el6_4.1
         ruby.x86_64 0:1.8.7.352-13.el6
         ruby-augeas.x86_64 0:0.4.1-3.el6
         ruby-irb.x86_64 0:1.8.7.352-13.el6
         ruby-libs.x86_64 0:1.8.7.352-13.el6
         ruby-rdoc.x86_64 0:1.8.7.352-13.el6
         ruby-rgen.noarch 0:0.6.5-2.el6
         ruby-shadow.x86_64 1:2.2.0-2.el6
         rubygem-json.x86_64 0:1.5.5-1.el6
         rubygems.noarch 0:1.3.7-5.el6

       Complete!
-----> Installing Chef Omnibus to install busser to run tests
         % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
100 15934  100 15934    0     0   2756      0  0:00:05  0:00:05 --:--:-- 57111
       Downloading Chef  for el...
       downloading https://www.getchef.com/chef/metadata?v=&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
         to file /tmp/install.sh.2242/metadata.txt
       trying wget...
       url  https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.12.4-1.el6.x86_64.rpm
       md5  959acd5df77c25f4f69d1f786f3c7360
       sha256 d4eacc6b16c448a628367e7201922a4c58997f68808c5f698676e8a5eaf169b5
       downloaded metadata file looks valid...
       downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.12.4-1.el6.x86_64.rpm
         to file /tmp/install.sh.2242/chef-11.12.4-1.el6.x86_64.rpm
       trying wget...
       Comparing checksum with sha256sum...
       Installing Chef
       installing with rpm...
       warning: /tmp/install.sh.2242/chef-11.12.4-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing...                #####  ########################################### [100%]
   1:chef                          ########################################### [100%]
       Thank you for installing Chef!
       Transfering files to <default-centos-65>
       Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
       Notice: Compiled catalog for default-centos-65.vagrantup.com in environment production in 0.68 seconds
       Notice: /Stage[main]/Ntp::Config/File[/etc/ntp.conf]/content: content changed '{md5}7fda24f62b1c7ae951db0f746dc6e0cc' to '{md5}4af276f148adc7960235ad4ba09cad48'
       Notice: /Stage[main]/Ntp::Service/Service[ntp]/ensure: ensure changed 'stopped' to 'running'
       Notice: Finished catalog run in 0.36 seconds
       Finished converging <default-centos-65> (2m25.37s).
-----> Kitchen is finished. (3m44.54s)

Very cool. And is ntp running? Sure it is:

± kitchen login default-nocm-ubuntu-1204
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64)

 * Documentation:  https://help.ubuntu.com/
Last login: Thu May  8 16:13:18 2014 from 10.0.2.2
vagrant@default-nocm-ubuntu-1204:~$ ps -ef |grep ntp
ntp       2391     1  0 16:13 ?        00:00:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 103:108
vagrant   2514  2415  0 16:20 pts/0    00:00:00 grep --color=auto ntp

± kitchen login default-centos-65
Last login: Thu May  8 09:15:43 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos-65 ~]$ ps -ef|grep ntp
ntp       2485     1  0 09:15 ?        00:00:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
vagrant   2513  2495  0 09:21 pts/0    00:00:00 grep ntp

But this feels manual. It actually is. So lets automate the tests!

Create Some Integration Tests: Here Comes Bats

First we need a place for the testing code. Test kitchen creates a default location, so lets use that.

ls test/integration/default

This is empty and has to be filled with our testing code. Lets start simple and add our manual test. A very interesting testing framework for this kind of tests is Bats: Bash Automated Testing System

mkdir test/integration/default/bats

cat > test/integration/default/bats/verify_installed.bats <<TEST

@test 'ntp is up and running' {
  pgrep ntp
}

@test 'ntp.conf contains correct servers' {
  grep 0.pool.ntp.org /etc/ntp.conf
}

TEST

Verify What We Have Got

Test kitchen uses busser to automate the heavy lifting. It will sync the local test folder to the nodes, install the test dependencies, and invoke the test on the node.

kitchen verify
-----> Starting Kitchen (v1.2.1)
-----> Setting up <default-nocm-ubuntu-1204>...
Fetching: thor-0.19.0.gem (100%)
Fetching: busser-0.6.2.gem (100%)
Successfully installed thor-0.19.0
Successfully installed busser-0.6.2
2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin bats installed (version 0.2.0)
-----> Running postinstall for bats plugin
Installed Bats to /tmp/busser/vendor/bats/bin/bats
       Finished setting up <default-nocm-ubuntu-1204> (0m54.06s).
-----> Verifying <default-nocm-ubuntu-1204>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
Uploading /tmp/busser/suites/bats/verify_installed.bats (mode=0644)
-----> Running bats test suite
 ✓ ntp is up and running
 ✓ ntp.conf contains correct servers

2 tests, 0 failures
       Finished verifying <default-nocm-ubuntu-1204> (0m0.89s).
-----> Setting up <default-centos-65>...
Fetching: thor-0.19.0.gem (100%)
Fetching: busser-0.6.2.gem (100%)
       Successfully installed thor-0.19.0
       Successfully installed busser-0.6.2
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin bats installed (version 0.2.0)
-----> Running postinstall for bats plugin
       Installed Bats to /tmp/busser/vendor/bats/bin/bats
       Finished setting up <default-centos-65> (0m57.40s).
-----> Verifying <default-centos-65>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
       Uploading /tmp/busser/suites/bats/verify_installed.bats (mode=0644)
-----> Running bats test suite
 ✓ ntp is up and running
   ntp.conf contains correct servers                                        2/2
 ✓ ntp.conf contains correct servers

       2 tests, 0 failures
       Finished verifying <default-centos-65> (0m0.87s).
-----> Kitchen is finished. (1m53.64s)

Very cool. But we knew that already. And the coolest part of it: It will run on all the OS flavors specified. Hook this up to a CI/CB system and have some extra productive time back :-)

Add Some More Tests. Serverspec To The Rescue

mkdir test/integration/default/serverspec

cat > test/integration/default/serverspec/ntp_spec.rb <<TEST
require 'serverspec'

include Serverspec::Helper::Exec
include Serverspec::Helper::DetectOS

RSpec.configure do |c|
  c.before :all do
    c.path = '/sbin:/usr/sbin'
  end
end

describe package('ntp') do
  it { should be_installed }
end

describe service('ntp') do
  it { should be_running }
end
TEST

Running kitchen verify now will not do the right thing. It will copy the serverspec but not run it. Run a kitchen converge first and kitchen will pick up the serverspec tests.

kitchen verify
-----> Starting Kitchen (v1.2.1)
-----> Verifying <default-nocm-ubuntu-1204>...
       Removing /tmp/busser/suites/bats
       Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/bats/verify_installed.bats (mode=0644)
Uploading /tmp/busser/suites/serverspec/ntp_spec.rb (mode=0644)
-----> Running bats test suite
 ✓ ntp is up and running
 ✓ ntp.conf contains correct servers

2 tests, 0 failures
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/ntp_spec.rb --color --format documentation

Package "ntp"
  should be installed

Service "ntp"
  should be running

Finished in 0.06612 seconds
2 examples, 0 failures
       Finished verifying <default-nocm-ubuntu-1204> (0m1.95s).
-----> Verifying <default-centos-65>...
       Removing /tmp/busser/suites/bats
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/bats/verify_installed.bats (mode=0644)
       Uploading /tmp/busser/suites/serverspec/ntp_spec.rb (mode=0644)
-----> Running bats test suite
 ✓ ntp is up and running
   ntp.conf contains correct servers                                        2/2
 ✓ ntp.conf contains correct servers

       2 tests, 0 failures
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/ntp_spec.rb --color --format documentation

       Package "ntp"
         should be installed

       Service "ntp"
         should be running

       Finished in 0.0413 seconds
       2 examples, 0 failures
       Finished verifying <default-centos-65> (0m1.62s).
-----> Kitchen is finished. (0m3.96s)

Now we can be sure that we have a cross plattform puppet module :-)

Final Notes

Have a look at

Old Errors

bundle exec kitchen converge default-nocm-ubuntu-1204
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-nocm-ubuntu-1204>...
       Preparing files for transfer
       Preparing modules
       Resolving module dependencies with Librarian-Puppet 1.0.1...
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #converge action: [undefined method `puppet_gem_version' for Librarian::Puppet:Module]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Ouch. Nasty error. This is the librarian puppet integration not working for me on osx. So let us move it out of the way for now as we already have installed the modules.

mv Puppetfile Puppetfile.