Blog Move
I have moved my blog.
Topics include Opscode Chef, configuration management, Ruby, Linux/Unix administration. Opinions expressed here are my own and do not reflect those of my employer (Opscode, Inc.).
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
It has been a few months and several thousand page views since my original post on how I manage my workstations with Chef. I have made some changes to my repository, and have a few additional notes for working with Mac OS X Lion. I also added a new system in my network, an iMac.
First, I would like to point out the cookbook I created, mac_os_x. In the original post, I discussed using Pivotal Lab's workstation recipes, which use the Mac OS X user defaults system to update system preferences from the command-line (with an execute resource). One of the first things I did in the mac_os_x cookbook is create a lightweight resource and provider for managing user defaults. It can be used, for example, like this:
mac_os_x_userdefaults "dont show hard drives on the desktop" do domain "com.apple.finder" key "ShowHardDrivesOnDesktop" value "false" type "bool" end
While this does involve more typing than just entering the command, it has a couple advantages. First, it is easier to read for people not totally familiar with the defaults command. Second, behind the scenes it will check if the setting is already set and not update the resource if it isn't. This isn't a huge deal in terms of system resource usage, but depending on your Chef setup might end up with extra reporting on things that didn't need to change if you're using a report handler, which I do use.
This cookbook also includes a lightweight resource/provider for managing plist preferences files for ~/Library/Preferences. All plists in OS X can be manipulated through the defaults system, but it can be cumbersome for highly customized applications such as your Adium or 1Password configuration. Usage is very simple:
mac_os_x_plist_file "ws.agile.1Password.plist"
This is in my local 1password cookbook, which has the file in files/default/ws.agile.1Password.plist. If I make modifications to the preferences, I do have to copy it over to the cookbook and re-upload to the Chef Server, but since I now have 4 computers to change preferences on, this is much easier than remembering everywhere I clicked. I use this for managing my preferences for ghmac (GitHub for Mac), iterm2 and alfredapp.
Lion brought a lot of changes. In particular along with Xcode 4, Apple changed the gcc compiler to llvm. I'm not a C programmer, and don't really understand the differences yet, I just know that a number of things in Homebrew fail to build. The main thing I installed with Homebrew that wasn't working with Lion is Emacs. I actually only have Lion on my iMac, so I haven't updated any recipes for installing it via Chef. In fact, I actually removed it from my "workstation" recipe and *gasp* installed it manually. The story there is enough for another post though, so stay tuned and I'll write up my experience.
Relevant to how I manage workstations with Chef, I had to make sure that on Lion, I take care of some additional new preferences using my handy-dandy "mac_os_x_userdefaults" LWRP. These are in the mac_os_x cookbook recipe, "lion_tweaks." In my workstation recipe, I include this one only on Lion.
If you haven't read it yet, go back and read my post on how I manage my workstations with Chef. I hope if you're using Chef to manage a Mac that the mac_os_x cookbook is useful to you. Also, stay tuned for an update later this week about my experiences installing Emacs on Lion.
I wanted smartmontools installed to monitor the disk health of my LAN server at home. This is not an uncommon thing to want to do, so I thought I'd write and share a Chef cookbook for it. I also took this opportunity to write up the experience so I can illustrate how easy it is to write a cookbook for Chef.
The first thing to do when writing a cookbook is to create the cookbook directory structure with knife cookbook create. This command will create a README.rdoc by default, and I prefer Markdown, so I specify the -r md option.
knife cookbook create smartmontools -r md
By default, metadata and the default recipe are created with boilerplate content for author and copyright. I have configured the values in my knife.rb:
cookbook_copyright "Joshua Timberman" cookbook_license "apachev2" cookbook_email "cookbooks@housepub.org"
The resulting directory structure will be created:
% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
├── definitions
├── files
│ └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│ └── default.rb
├── resources
└── templates
└── default
10 directories, 3 filesI'm a big fan of Tom Preston-Werner's blog post on README driven development. I don't write the complete README before I start writing code for a new cookbook. I do write it as I go.
In order to write a proper README for a cookbook, and to write the cookbook itself, we'll need to know a bit more about the software we're installing. The best way to do that depends on the software, but often it is as simple as merely installing the package on a test system such as a virtual machine and explore its contents.
apt-get install smartmontools dpkg -L smartmontools
Of note for smartmontools, documentation is in /usr/share/doc/smartmontools and configuration is in /etc. In particular, /etc/smartd.conf and /etc/smartmontools.
For now assume that the cookbook README.md is being written along the way.
One of the things I do when I am writing a new cookbook and exploring the contents of a package is to be mindful of Chef Resources I want to manage in the recipe(s). In the case of smartmontools, at this point I have determined I need a few specific resources.
First, as I've installed the package, I clearly need a package. I'm pretty confident that this particular package will not break backwards compatibility, and can be safely upgraded to the latest version if necessary.
The next resources in the recipe are the configuration files. I want to dynamically configure these, so I am going to use templates.
I'm not going to write these from scratch. Instead, I will copy the source files from the installed package on my test system. These will go into templates/default in the cookbook.
% tree templates
templates
└── default
├── smartd.conf.erb
└── smartmontools.default.erbTemplates are dynamically generated using ERB, and they can use Node attributes. I can use the automatically detected attributes, or I can set new attributes for the node in the cookbook.
The attributes go in the attributes/default.rb file in the cookbook. The ones I use are:
Attributes are definitely something to document in the README.
In templates/default/smartd.conf.erb, I check if there's a list of devices to monitor, and if so iterate over the list passing in the default options (device_opts). The cookbook doesn't at this time support per-device options - the same ones are applied to all devices. If devices is empty, then the configuration will use DEVICESCAN.
<% if node['smartmontools']['devices'].length > 0 -%>
<% node['smartmontools']['devices'].each do |device| -%>
<%= "/dev/#{device} #{node['smartmontools']['device_opts']}" %>
<% end -%>
<% else -%>
DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
<% end -%>In templates/default/smartmontools.default.erb, the smartmontools daemon will be enabled based on the start_smartd attribute. Additional options will be passed per smartd_opts:
start_smartd=<%= node['smartmontools']['start_smartd'] %> <% if node['smartmontools']['smartd_opts'].length > 0 -%> smartd_opts="<%= node['smartmontools']['smartd_opts'] %>" <% else -%> #smartd_opts="--interval=1800" <% end -%>
Generally speaking when creating attributes and using them in a template, I use the default values that are found in the configuration file dropped off by the package. I also try to use the default settings over all.
However, a cookbook is a place to express opinions. I made some with this cookbook, such as enabling DEVICESCAN if there are no devices, despite the configuration file's comments indicating that the option shouldn't be used. The best thing about attributes is they allow other people using the cookbook to change the behavior based on their preferences much easier than manually modifying things dropped off by a package.
I strongly recommend documenting where a cookbook has behavior that is not default for the installed package or upstream documentation. I did this in the README for this cookbook discussing the DEVICESCAN option and otherwise where appropriate.
This cookbook has only one static file which will be deployed to /etc/smartmontools/run.d/10mail. The smartmontools package allows creating a number of scripts that go in the directory, and I have created an attribute for the list of scripts. These generally don't cookbook_file resources. Since the list is an attribute, I iterate over that with Ruby's Array#each loop.
The attribute:
The loop of resources:
Each filename in the array (currently only '10mail') needs to have a corresponding file in the files/default directory of the cookbook.
% tree files
files
└── default
└── 10mailThis file's contents are not particularly exciting, I used the same one that came out of the package.
Why would I want to manage a static file that came from the package? Perhaps I want to modify the script in some way that doesn't make sense in an attribute. In this case I don't, however creating the attribute and iterating over it makes it easy to extend this functonality in the cookbook.
Smartmontools comes with a service. That is, there's an init script that can be enabled and started to monitor disk devices. This is actually the whole point of the cookbook, so I'll make sure there's a resource to manage the service.
In this resource, I used the meta parameter supports for the service. I found out what the init script can do by simply running it with no options on my test system.
% /etc/init.d/smartmontools
Usage: /etc/init.d/smartmontools {start|stop|restart|reload|force-reload|status}The various options passed to the init script manage it in the familiar way. Telling Chef about it has a specific effect on the way the service provider functions when Chef manages it.
/etc/init.d/SERVICE_NAME status to determine if the service is running. If the return code is 0, its running. Otherwise, Chef checks the process table for a process running with the name of the service. reload action for a service if it actually supports reload. restart action is sent to a service, if it supports restart then Chef will use /etc/init.d/SERVICE_NAME restart. Otherwise, Chef will use stop and start. Earlier when we wrote the template resources, we notified the service to reload. Since the resource supports reload, we can do this. Also note that the resource has an array of actions. Each of these actions will be taken if necessary when Chef manages it. In this case, it will be enabled at boot time, and then started if it is not already running.
Now that we've written a nice recipe and understand what it's about to do, let's actually use it on a node we'd like to have smartmontools. Before uploading, I remove the boilerplate directories that were created by knife. The actual cookbook contents for upload are:
% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
│ └── default.rb
├── files
│ └── default
│ └── 10mail
├── metadata.rb
├── recipes
│ └── default.rb
└── templates
└── default
├── smartd.conf.erb
└── smartmontools.default.erb
6 directories, 7 filesI didn't mention the metadata yet as I haven't modified it yet. The knife command will create a default metadata.rb file as mentioned before. It will also populate it with some boilerplate content. The main thing I'm going to modify is the version and the platforms supported.
Now it is time to actually upload the cookbook and apply it to a node.
% knife cookbook upload smartmontools
Uploading smartmontools [0.5.0]
upload complete
% knife node run list add virt1test 'recipe[smartmontools]'
run_list:
role[ubuntu]
recipe[smartmontools]I'll use knife ssh to run chef-client on the nodes that have the smartmontools recipe applied.
% knife ssh 'recipes:smartmontools' 'sudo chef-client' [Sat, 27 Aug 2011 13:33:31 -0600] INFO: *** Chef 0.10.4 *** ... [Sat, 27 Aug 2011 13:33:35 -0600] INFO: Starting Chef Run for virt1test ... [Sat, 27 Aug 2011 13:34:25 -0600] INFO: Processing package[smartmontools] action upgrade (smartmontools::default line 20) [Sat, 27 Aug 2011 13:34:31 -0600] INFO: package[smartmontools] upgraded from uninstalled to 5.39.1+svn3124-2 [Sat, 27 Aug 2011 13:34:31 -0600] INFO: Processing template[/etc/default/smartmontools] action create (smartmontools::default line 24) [Sat, 27 Aug 2011 13:34:31 -0600] INFO: template[/etc/default/smartmontools] backed up to /var/chef/backup/etc/default/smartmontools.chef-20110827133431 [Sat, 27 Aug 2011 13:34:31 -0600] INFO: template[/etc/default/smartmontools] mode changed to 644 [Sat, 27 Aug 2011 13:34:31 -0600] INFO: template[/etc/default/smartmontools] updated content [Sat, 27 Aug 2011 13:34:31 -0600] INFO: Processing template[/etc/smartd.conf] action create (smartmontools::default line 32) [Sat, 27 Aug 2011 13:34:32 -0600] INFO: template[/etc/smartd.conf] backed up to /var/chef/backup/etc/smartd.conf.chef-20110827133432 [Sat, 27 Aug 2011 13:34:32 -0600] INFO: template[/etc/smartd.conf] mode changed to 644 [Sat, 27 Aug 2011 13:34:32 -0600] INFO: template[/etc/smartd.conf] updated content [Sat, 27 Aug 2011 13:34:32 -0600] INFO: template[/etc/smartd.conf] not queuing delayed action reload on service[smartmontools] (delayed), as it's already been queued [Sat, 27 Aug 2011 13:34:32 -0600] INFO: Processing cookbook_file[/etc/smartmontools/run.d/10mail] action create (smartmontools::default line 42) [Sat, 27 Aug 2011 13:34:32 -0600] INFO: cookbook_file[/etc/smartmontools/run.d/10mail] backed up to /var/chef/backup/etc/smartmontools/run.d/10mail.chef-20110827133432 [Sat, 27 Aug 2011 13:34:32 -0600] INFO: cookbook_file[/etc/smartmontools/run.d/10mail] mode changed to 755 [Sat, 27 Aug 2011 13:34:32 -0600] INFO: cookbook_file[/etc/smartmontools/run.d/10mail] created file /etc/smartmontools/run.d/10mail [Sat, 27 Aug 2011 13:34:32 -0600] INFO: Processing service[smartmontools] action enable (smartmontools::default line 51) [Sat, 27 Aug 2011 13:34:32 -0600] INFO: Processing service[smartmontools] action start (smartmontools::default line 51) [Sat, 27 Aug 2011 13:34:34 -0600] INFO: service[smartmontools] started [Sat, 27 Aug 2011 13:34:36 -0600] INFO: template[/etc/default/smartmontools] sending reload action to service[smartmontools] (delayed) [Sat, 27 Aug 2011 13:34:36 -0600] INFO: Processing service[smartmontools] action reload (smartmontools::default line 51) [Sat, 27 Aug 2011 13:34:36 -0600] INFO: service[smartmontools] reloaded [Sat, 27 Aug 2011 13:34:36 -0600] INFO: Chef Run complete in 61.179418 seconds [Sat, 27 Aug 2011 13:34:36 -0600] INFO: Running report handlers [Sat, 27 Aug 2011 13:34:36 -0600] INFO: Resources updated this run: [Sat, 27 Aug 2011 13:34:36 -0600] INFO: execute[apt-get update] [Sat, 27 Aug 2011 13:34:36 -0600] INFO: package[smartmontools] [Sat, 27 Aug 2011 13:34:36 -0600] INFO: template[/etc/default/smartmontools] [Sat, 27 Aug 2011 13:34:36 -0600] INFO: template[/etc/smartd.conf] [Sat, 27 Aug 2011 13:34:36 -0600] INFO: cookbook_file[/etc/smartmontools/run.d/10mail] [Sat, 27 Aug 2011 13:34:36 -0600] INFO: service[smartmontools]
We can verify the results of running the recipe by examining the resources on the target system(s). Chef's contract with you is that it will configure the resources in the manner specified in the recipe. You can be confident that it will completely configure every resource if it exits cleanly (which it did from the output above).
% cat /etc/default/smartmontools ... start_smartd=yes ... % cat /etc/smartd.conf ... DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner ... % sudo service smartmontools status * smartd is running
Now we can take this same recipe and apply it to other systems. I tested smartmontools on an Ubuntu system. Notice earlier that I had an ubuntu role on my node. I actually modified that role to include the smartmontools recipe, and then all my Ubuntu nodes were configured for smartmontools when they ran Chef again.
I hope this guide was helpful. I shared the cookbook on the Chef Community web site, and the source code is available on Chef Community web site. Note that while I said this would be easy, it certainly isn't trivial. There are a lot of steps involved in making cookbooks that are dynamic, easily customized and shareable with others. It takes practice, but after a few you get the hang of it. End to end, this cookbook took me about 2 hours to write, test, tweak (fix bugs) and document.
I recently had a chance to sit down and implemented an encrypted data bag in my personal environment. This should translate nicely to anyone that wants to use encrypted data bags in their environment.
I send mail out through an SASL authenticated SMTP server. My local network has a postfix SMTP relay that connects to the SASL auth relay. I'm using the Opscode postfix cookbook with the sasl_auth recipe, since I wrote it originally for this use case.
The postfix::sasl_auth recipe is applied in an "operations master" role. The attributes for configuring the user and password for SASL are attributes. Relevant lines from the role:
name "ops_master"
run_list("recipe[postfix::sasl_auth]")
override_attributes(
"postfix" => {
"relayhost" => "[smtp.example.com]:587",
"smtp_sasl_auth_enable" => "yes",
"smtp_sasl_passwd" => "AWESOME!!",
"smtp_sasl_user_name" => "MYUSER"
}
)Chef's data bags are a great way to store infrastructure wide, but not role or node specific information. Encrypted data bags are a great way to store sensitive information, like passwords. Here are the steps I followed to get the encrypted data bag set up.
First, I created the secret key file that is used to encrypt the contents of the data bag item. This file will not be stored in source control, as it is highly sensitive, and only gets copied to the systems that need it.
openssl rand -base64 512 > ~/.chef/encrypted_data_bag_secret
Next, I created the actual data bag.
knife data bag create secrets
Next I created the data bag item using the secret key. This is created directly on the Chef Server, rather than a plain text file.
knife data bag create secrets postfix --secret-file ~/.chef/encrypted_data_bag_secret
I'm not saving the plaintext data bag item, but will store the encrypted item, so I'll retrieve it from the Chef Server and redirect the output to a JSON file.
mkdir data_bags/secrets
knife data bag show secrets postfix -Fj > data_bags/secrets/postfix.json
cat data_bags/secrets/postfix.json
{
"id": "postfix",
"user": "encrypted string here",
"passwd": "encrypted string here"
}The current recipe doesn't support using an encrypted data bag item, so I had to modify it. First, load the encrypted data bag item.
postfix_creds = Chef::EncryptedDataBagItem.load("secrets","postfix",
Chef::EncryptedDataBagItem::DEFAULT_SECRET_FILE)This access the "secrets" data bag for the "postfix" item, and uses the default value for the secret key file (which is "/etc/chef/encrypted_data_bag_secret".
Next, I update the sasl_passwd template to pass in the user and password from the data bag item.
template "/etc/postfix/sasl_passwd" do
#...
variables(:smtp_sasl_passwd => smtp_sasl['passwd'],
:smtp_sasl_user_name => smtp_sasl['user'])
#...
endFinally, the template is updated to use the new values.
<%= node[:postfix][:relayhost] %> <%= @smtp_sasl_user_name %>:<%= @smtp_sasl_passwd %>
After uploading the cookbook and the role (I removed the username and password), I copied the secret key file over to the node I needed to run Chef on, then ran Chef Client.
sudo cat /etc/postfix/sasl_passwd [smtp.example.com]:587 MYUSER:AWESOME!!
Yay!
The full recipe is:
