Chef Fundamentals

[email protected] Copyright (C) 2013 Opscode, Inc.

v1.1.3

Monday, 4 November 13

Introductions

v1.1.3

Monday, 4 November 13

Instructor Introduction

Monday, 4 November 13

Introduce Yourselves
Name Current job role Previous job roles/background Experience with Chef and/or config management Favorite Text Editor

Monday, 4 November 13

Course Objectives and Style

v1.1.3

Monday, 4 November 13

Course Objectives
Upon completion of this course you will be able to Automate common infrastructure tasks with Chef Describe Chefs architecture Describe Chefs various tools Apply Chefs primitives to solve your problems

Monday, 4 November 13

How to learn Chef

You bring the domain expertise about your business and problems Chef provides a framework for solving those problems Our job is to work together to teach you how to express solutions to your problems with Chef

Monday, 4 November 13

Chef is a Language
Learning Chef is like learning the basics of a language 80% fluency will be reached very quickly The remaining 20% just takes practice The best way to learn Chef is to use Chef

Monday, 4 November 13

Training is really a discussion

We will be doing things the hard way Were going to do a lot of typing You cant be: Absent Late Left Behind We will troubleshoot and fix bugs on the spot The result is you reaching fluency fast
Monday, 4 November 13

Training is really a discussion

Ill post objectives at the beginning of a section Ask questions when they come to you Ask for help when you need it Youll get the slides after class

Monday, 4 November 13

Agenda

v1.1.3

Monday, 4 November 13

Topics
Overview of Chef Workstation Setup Test Node Setup Dissecting your first Chef run Introducing the Node object Writing your first cookbook

Monday, 4 November 13

Topics
Setting attributes, cookbook metadata, templates Idempotency, notifications, template variables Data bags, search Roles Environments Using community cookbooks

Monday, 4 November 13

Breaks!
Well take a break between each section, or every hour, whichever comes first Well obviously break for lunch :)

Monday, 4 November 13

Overview of Chef

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Describe how Chef thinks about Infrastructure Automation Define the following terms: Node Resource Recipe Cookbook Run List Roles Search
Monday, 4 November 13

Complexity

http://www.ickr.com/photos/michaelheiss/3090102907/
Monday, 4 November 13

Items of Manipulation (Resources)

Networking Files Directories Symlinks Mounts Registry Key Powershell Script Users Groups Packages Services Filesystems

Monday, 4 November 13

A tale of growth...
Application

Monday, 4 November 13

Add a database
Application

Application Database

Monday, 4 November 13

Make database redundant

Application

App Databases

Monday, 4 November 13

Application server redundancy

App Servers

App Databases

Monday, 4 November 13

Add a load balancer

App LB

App Servers

App Databases

Monday, 4 November 13

Webscale!
App LBs

App Servers

App Databases

Monday, 4 November 13

Now we need a caching layer

App LBs App Servers App DB Cache App DBs

Monday, 4 November 13

Infrastructure has a Topology

App LBs App Servers App DB Cache App DBs

Monday, 4 November 13

Your Infrastructure is a Snowflake

Round Robin DNS App Servers App DB Cache Floating IP? App DBs

Monday, 4 November 13

Complexity Increases Quickly

App LBs Cache App Servers < Shiny! DB Cache DB slaves DBs

e w re

o t i n mo

? ? g rin

Monday, 4 November 13

...and change happens!

App LBs Cache App Servers DB Cache DB slaves DBs

Monday, 4 November 13

...and change happens!

C a dd a r t en g o lL t s o H
App LBs Cache App Servers DB Cache DB slaves
Central Log Host

DBs

Monday, 4 November 13

...and change happens!

g o L l a r t n e n C o a f n d o d c . A g o l s y s s e e t d a o d N p U all t s o H
App LBs Cache App Servers DB Cache DB slaves
Central Log Host

DBs

Monday, 4 November 13

Chef Solves This Problem

But you already guessed that, didnt you?

Monday, 4 November 13

Managing Complexity
Organizations Environments Roles Nodes Recipes Cookbooks Search

Monday, 4 November 13

Organizations
My Infrastructure
Your Infrastructure

Their Infrastructure

Monday, 4 November 13

Organizations
Completely independent tenants of Enterprise Chef Share nothing with other organizations May represent different Companies Business Units Departments

Monday, 4 November 13

Environments
Development Staging Production

Monday, 4 November 13

Environments
Model the life-stages of your applications Every Organization starts with a single environment Environments to reflect your patterns and workflow Development Test Staging Production etc.
Monday, 4 November 13

Environments Define Policy

Environments may include data attributes necessary for configuring your infrastructure The URL of your payment services API The location of your package repository The version of the Chef configuration files that should be used

Monday, 4 November 13

Roles
Load Balancers Application Servers DB Cache Database

Monday, 4 November 13

Roles
Roles represent the types of servers in your infrastructure Load Balancer Application Server Database Cache Database Monitoring

Monday, 4 November 13

Roles Define Policy

Roles may include a list of Chef configuration files that should be applied. We call this list a Run List Roles may include data attributes necessary for configuring your infrastructure The port that the application server listens on A list of applications that should be deployed

Monday, 4 November 13

Nodes

Monday, 4 November 13

Nodes
Nodes represent the servers in your infrastructure Nodes may represent physical servers or virtual servers Nodes may represent hardware that you own or may represent compute instances in a public or private cloud

Monday, 4 November 13

Node
Each Node will belong to one Organization belong to one Environment have zero or more Roles

Monday, 4 November 13

Nodes Adhere to Policy

An application, the chef-client, runs on each node chef-client will gather current system configuration download the desired system configuration from the Chef server configure the node such that it adheres to the policy

Monday, 4 November 13

Chef is Infrastructure as Code

Programmatically provision and configure components Treat like any other code base Reconstruct business from code repository, data backup, and bare metal resources.

http://www.ickr.com/photos/louisb/4555295187/

Monday, 4 November 13

Configuration Code
Chef ensures each Node complies with the policy Policy is determined by the configurations included in each Nodes run list Reduce management complexity through abstraction Store the configuration of your infrastructure in version control

Monday, 4 November 13

Declarative Interface to Resources

You define the policy in your Chef configuration Your policy states what state each resource should be in, but not how to get there Chef-client will pull the policy from the Chef Server and enforce the policy on the Node

Monday, 4 November 13

Resources
A Resource represents a piece of the system and its desired state A package that should be installed A service that should be running A file that should be generated A cron job that should be configured A user that should be managed and more
Monday, 4 November 13

Resources in Recipes
Resources are the fundamental building blocks of Chef configuration Resources are gathered into Recipes Recipes ensure the system is in the desired state

Monday, 4 November 13

Recipes
Configuration files that describe resources and their desired state Recipes can: Install and configure software components Manage files Deploy applications Execute other recipes and more
Monday, 4 November 13

Example Recipe
package "apache2"" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end

Monday, 4 November 13

Cookbooks
Recipes are stored in Cookbooks Cookbooks contain recipes, templates, files, custom resources, etc Code re-use and modularity

http://www.flickr.com/photos/shutterhacks/4474421855/

Monday, 4 November 13

Run List
Enterprise Chef

What policy sho uld

I follow?
chef-client

Node

Monday, 4 November 13

Run List
Enterprise Chef

What policy sho uld

I follow?
chef-client

Node

"recipe[ntp::client]" "recipe[users]" "role[webserver]"

Monday, 4 November 13

Run List
Enterprise Chef

What policy sho uld

I follow?
chef-client

"recipe[ntp::client]" "recipe[users]" "role[webserver]"

Monday, 4 November 13

Run Lists Specifies Policy

The Run List is a collection of policies that the Node should follow. Chef-client obtains the Run List from the Chef Server Chef-client ensures the Node complies with the policy in the Run List

Monday, 4 November 13

Search
Search for nodes with Roles Find Topology Data IP addresses Hostnames FQDNs

http://www.ickr.com/photos/kathycsus/2686772625
Monday, 4 November 13

Search for Nodes

pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end
Monday, 4 November 13

Search for Nodes

pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end
Monday, 4 November 13

Pass results into Templates

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%>

Monday, 4 November 13

Pass results into Templates

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%>

Monday, 4 November 13

Pass results into Templates

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%>

Monday, 4 November 13

So when this...
Graphite Nagios

Jboss App Memcache Postgres Slaves Postgres Master

Monday, 4 November 13

...becomes this
Graphite Nagios

Jboss App Memcache Postgres Slaves Postgres Master

Monday, 4 November 13

...this can happen automatically

Graphite Nagios

Jboss App Memcache Postgres Slaves Postgres Master

Monday, 4 November 13

Count the Resources

Graphite Nagios Jboss App Memcache Postgres Slaves

12+ resource changes for 1 node addition

Monday, 4 November 13

Load balancer config Nagios host ping Nagios host ssh Nagios host HTTP Nagios host app health Graphite CPU Graphite Memory Graphite Disk Graphite SNMP Memcache firewall Postgres firewall Postgres authZ config

Manage Complexity
Determine the desired state of your infrastructure Identify the Resources required to meet that state Gather the Resources into Recipes Compose a Run List from Recipes and Roles Apply a Run List to each Node in your Environment Your infrastructure adheres to the policy modeled in Chef

Monday, 4 November 13

Configuration Drift
Configuration Drift happens when: Your infrastructure requirements change The configuration of a server falls out of policy Chef makes it easy to manage Model the new requirements in your Chef configuration files Run the chef-client to enforce your policies
Monday, 4 November 13

Review Questions
What is a Node? What is a Resource? What is a Recipe? How is it different from a Cookbook? What is a Run List? What is a Role?

Monday, 4 November 13

Workstation Setup
Getting started

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Login to Enterprise Chef View your Organization in Enterprise Chef Describe Knife, the Chef command line utility Use Knife on your Workstation

Monday, 4 November 13

Legend

v1.1.3

Monday, 4 November 13

Legend: Do I run that command on my workstation?

This is an example of a command to run on your workstation
$ whoami i-am-a-workstation

This is an example of a command to run on your target node via SSH.

user@hostname:~$ whoami i-am-a-chef-node

Monday, 4 November 13

Legend: Example Terminal Command and Output

$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 ! options=3<RXCSUM,TXCSUM> ! inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 ! inet 127.0.0.1 netmask 0xff000000 ! inet6 ::1 prefixlen 128 gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ! ether 28:cf:e9:1f:79:a3 ! inet6 fe80::2acf:e9ff:fe1f:79a3%en0 prefixlen 64 scopeid 0x4 ! inet 10.100.0.84 netmask 0xffffff00 broadcast 10.100.0.255 ! media: autoselect ! status: active p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304 ! ether 0a:cf:e9:1f:79:a3 ! media: autoselect ! status: inactive

Monday, 4 November 13

Legend: Example of editing a file on your workstation

OPEN IN EDITOR: ~/hello_world
Hi! I am a friendly file.

SAVE FILE!

Monday, 4 November 13

Landscape of Chef-managed Infrastructure

NODES

Monday, 4 November 13

Landscape of Chef-managed Infrastructure

NODES

Install Chef

Monday, 4 November 13

Install Chef
Install Chef (if not already installed) http://www.opscode.com/chef/install

Monday, 4 November 13

Install Chef

Monday, 4 November 13

Install on Mac OSX

Monday, 4 November 13

Install on Enterprise Linux

Monday, 4 November 13

Workstation Setup - Mac OS X / Linux

$ curl -L http://www.opscode.com/chef/install.sh | sudo bash

% Total

% Received % Xferd

100 6515 100 6515 0 0 Downloading Chef for ubuntu... Installing Chef Selecting previously unselected package chef. (Reading database ... 47446 files and directories currently installed.) Unpacking chef (from .../tmp.MqRJP6lz/chef__amd64.deb) ... Setting up chef (11.4.4-2.ubuntu.11.04) ... Thank you for installing Chef! Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-3.2.0-48-virtual

Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 20600 0 --:--:-- --:--:-- --:--:-- 31172

Monday, 4 November 13

Workstation Setup - Windows

Windows 2008 (Windows 7) or 2012 (Windows 8) i686 (32-bit) or x86_64 (64-bit) 11.6.2
Monday, 4 November 13

Install on Windows

Monday, 4 November 13

What just happened?

Chef and all of its dependencies installed via an operating system-specific package ("omnibus installer") Installation includes The Ruby language - used by Chef knife - Command line tool for administrators chef-client - Client application ohai - System profiler ...and more
Monday, 4 November 13

Workstation or Node?
NODES

chef-client ohai

knife

Monday, 4 November 13

Landscape of Chef-managed Infrastructure

NODES

1.Install Chef 2.Create User 3.Create Organization

Chef Installed

Monday, 4 November 13

Your Chef Server for this class...

Hosted Enterprise Chef http://opscode.com

Monday, 4 November 13

Create new account

Sign up for a new account Chef Organization provides multi-tenancy name must be globally unique

Monday, 4 November 13

Landscape of Chef-managed Infrastructure

NODES

Chef Server Ready

Chef Installed Congure Knife

Monday, 4 November 13

Download "Starter Kit"

You get a .zip file from clicking this Unzip the zipfile - youll get a "chefrepo" Put the "chef-repo" somewhere, e.g.: C:\Users\you\chef-repo (Win) /Users/you/chef-repo (Mac) /home/you/chef-repo (Linux)
Monday, 4 November 13

Knife is the command-line tool for Chef

Knife provides an interface between a local Chef repository and the Chef Server Knife lets you manage: Nodes Cookbooks and recipes Roles Stores of JSON data (data bags), including encrypted data Environments Cloud resources, including provisioning The installation of Chef on management workstations Searching of indexed data on the Chef Server
Monday, 4 November 13

A quick tour of the chef-repo

Every infrastructure managed with Chef has a Chef Repository ("chef-repo") Type all commands in this class from the chef-repo directory Lets see whats inside the chef-repo...

Monday, 4 November 13

Verify that knife is working

$ cd chef-repo

[~/chef-repo]$

Monday, 4 November 13

A quick tour of the chef-repo

$ ls -al
total 40 drwxr-xr-x@ 11 opscode drwxr-xr-x+ 92 opscode drwxr-xr-x@ 3 opscode drwxr-xr-x@ 5 opscode -rw-r--r--@ 1 opscode -rw-r--r--@ 1 opscode -rw-r--r--@ 1 opscode -rw-r--r--@ 1 opscode -rw-r--r--@ 1 opscode drwxr-xr-x@ 3 opscode drwxr-xr-x@ 3 opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode 374 3128 102 170 495 1433 2416 3567 588 102 102 Aug Aug Aug Aug Aug Aug Aug Aug Aug Aug Aug 15 09:42 . 15 09:43 .. 15 2013 .berkshelf 15 2013 .chef 15 2013 .gitignore 15 2013 Berksfile 15 2013 README.md 15 2013 Vagrantfile 15 2013 chefignore 15 2013 cookbooks 15 2013 roles

Monday, 4 November 13

Whats inside the .chef directory?

$ ls .chef

ORGNAME-validator.pem USERNAME.pem knife.rb

Monday, 4 November 13

Whats inside the .chef directory?

knife.rb is the configuration file for Knife. The other two files are certificates for authentication with the Chef Server Well talk more about that later.

Monday, 4 November 13

knife.rb
Default location ~/.chef/knife.rb %HOMEDRIVE%:%HOMEPATH%\.chef (Windows) Use a project specific configuration .chef/knife.rb of the current directory chef-repo/.chef/knife.rb http://docs.opscode.com/config_rb_knife.html
Monday, 4 November 13

knife.rb
OPEN IN EDITOR: chef-repo/.chef/knife.rb
current_dir = File.dirname(__FILE__) log_level :info log_location STDOUT node_name "USERNAME" client_key "#{current_dir}/USERNAME.pem" validation_client_name "ORGNAME-validator" validation_key "#{current_dir}/ORGNAME-validator.pem" chef_server_url "https://api.opscode.com/organizations/ORGNAME" cache_type 'BasicFile' cache_options( :path => "#{ENV['HOME']}/.chef/checksums" ) cookbook_path ["#{current_dir}/../cookbooks"]

Monday, 4 November 13

Verify Knife
$ knife --version Chef: 11.6.2
Your version may be different, thats ok!
NODES

$ knife client list ORGNAME-validator

Chef Server Ready

Knife Congured
Monday, 4 November 13

knife client list

NODE

Read the chef_server_url from knife.rb HTTP GET to #{chef_server_url}/clients Display the result

Monday, 4 November 13

Exercise: Run knife help list

$ knife help list
Available help topics are: bootstrap chef-shell client configure cookbook cookbook-site data-bag environment exec index knife node role search shef

Monday, 4 November 13

A few knife tips

Commands are always structured as follows knife NOUN (client) VERB (list) You can get more help with knife NOUN help knife --help just shows options
Monday, 4 November 13

Checkpoint
NODES

Monday, 4 November 13

Whats Next?
NODES

Monday, 4 November 13

Source Code Repository

NODES

Monday, 4 November 13

Checkpoint
NODES

Monday, 4 November 13

Review Questions
What is the chef-repo? What is knife? What is name of the knife configuration file? Where is the knife configuration file located? What is your favorite text editor? :)

Monday, 4 November 13

Organization Setup
Setup an Organization

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Explain the purpose of Organizations Manage your Chef Organization

Monday, 4 November 13

Checkpoint
NODES

Monday, 4 November 13

Organizations
My Infrastructure
Your Infrastructure

Their Infrastructure

Monday, 4 November 13

Organizations
Provide multi-tenancy in Enterprise Chef Nothing is shared between Organizations - they're completely independent May represent different Companies Business Units Departments

Monday, 4 November 13

Manage Organizations
Login to your Hosted Enterprise Chef

Monday, 4 November 13

Organizations

Monday, 4 November 13

Manage Organization
Reset Validation Key Generate Knife Config Leave Organization Starter Kit

Monday, 4 November 13

Review Questions
What is an Organization? How do you regenerate the Starter Kit for your Organization?

Monday, 4 November 13

Node Setup
Setup a Node to manage

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Install Chef nodes using "knife bootstrap" Explain how knife bootstrap configures a node to use the Organization created in the previous section Explain the basic configuration needed to run chefclient

Monday, 4 November 13

Nodes
NODES

Monday, 4 November 13

Nodes
Nodes represent the servers in your infrastructure these may be Physical or virtual servers Hardware that you own Compute instances in a public or private cloud

Monday, 4 November 13

We Have No Nodes Yet

Monday, 4 November 13

Training Node
The labs require a node to be managed We allow for two different options Bring your own Node Launch an instance of a public AMI on EC2

Monday, 4 November 13

Bring Your Own Node

Use your own Virtual Machine (VM) or Server Required for the labs: Ubuntu 10.04+ 512 MB RAM 15 GB Disk sudo or root level permissions

Monday, 4 November 13

EC2 Public AMI

Opscode publishes a public AMI on EC2 that may be used Search for oc-training-public m1.small should be sufficient Open ports 22, 80-90 in security group SSH Credentials Login: opscode Password: opscode
Monday, 4 November 13

EC2 Public AMI

Opscode publishes a public AMI on EC2 that may be used Ne ve Search for oc-training-public an r u y s t th hi e t m1.small should be sufficient an ng hi s t o Open ports 22, 80-90 in security group hi f t s c he or r l a SSH Credentials ss! Login: opscode Password: opscode
Monday, 4 November 13

Checkpoint
At this point you should have One virtual machine (VM) or server that youll use for the lab exercises The IP address or public hostname An application for establishing an ssh connection sudo or root permissions on the VM

Monday, 4 November 13

Checkpoint
NODES

Bootstrap a Node

Monday, 4 November 13

"Bootstrap" the Target Instance

$ knife bootstrap IPADDRESS --sudo -x opscode -P opscode -N "node1"

Bootstrapping Chef on ec2-54-211-119-145.compute-1.amazonaws.com ec2-54-211-119-145.compute-1.amazonaws.com knife sudo password: Enter your password: ... ... ec2-54-211-119-145.compute-1.amazonaws.com Converging 0 resources ec2-54-211-119-145.compute-1.amazonaws.com ec2-54-211-119-145.compute-1.amazonaws.com Chef Client finished, 0 resources updated ec2-54-211-119-145.compute-1.amazonaws.com
Monday, 4 November 13

$ knife bootstrap IPADDRESS --sudo -x USERNAME -P PASSWORD -N node1

SSH! local workstation managed node (VM)

chef_server_url validation_client_name Opscode Hosted Chef validation_key

chef-client bash -c ' install chef congure client run chef'

Monday, 4 November 13

What just happened?

Chef and all of its dependencies installed via an operating system-specific package ("omnibus installer") Installation includes The Ruby language - used by Chef knife - Command line tool for administrators chef-client - Client application ohai - System profiler ...and more
Monday, 4 November 13

Workstation or Node?
NODES

chef-client ohai

knife

Monday, 4 November 13

Verify Your Target Instances Chef-Client is Configured Properly

$ ssh opscode@IPADDRESS opscode@node1:~$ ls /etc/chef client.pem client.rb first-boot.json validation.pem opscode@node1:~$ which chef-client /usr/bin/chef-client

Monday, 4 November 13

Examine /etc/chef/client.rb
opscode@node1:~$ cat /etc/chef/client.rb

log_level :auto log_location STDOUT chef_server_url "https://api.opscode.com/organizations/ORGNAME" validation_client_name "ORGNAME-validator" node_name "node1"

Monday, 4 November 13

Change the log level on your test node

opscode@node1:~$ sudo vi /etc/chef/client.rb log_level :info log_location STDOUT chef_server_url "https://api.opscode.com/organizations/ORGNAME" validation_client_name "ORGNAME-validator" node_name "node1"

Set the default log level for chef-client to :info More configuration options can be found on the docs site: http://docs.opscode.com/config_rb_client.html
Monday, 4 November 13

View Node on Chef Server

Login to your Hosted Enterprise Chef

Monday, 4 November 13

View Node on Chef Server

Monday, 4 November 13

View Node on Chef Server

Monday, 4 November 13

Node
The node is registered with Chef Server The Chef Server displays information about the node This information comes from Ohai

Monday, 4 November 13

Ohai
"languages": { "ruby": { }, "perl": { "version": "5.14.2", "archname": "x86_64linux-gnu-thread-multi" }, "python": { "version": "2.7.3", "builddate": "Aug 1 2012, 05:14:39" }, "php": { "version": "5.3.10-1ubuntu3.6", "builddate": "(cli) (built: Mar" } },
Monday, 4 November 13

"kernel": { "name": "Linux", "release": "3.2.0-32-virtual", "version": "#51-Ubuntu SMP Wed Sep 26 21:53:42 UTC 2012", "machine": "x86_64", "modules": { "isofs": { "size": "40257", "refcount": "0" }, "acpiphp": { "size": "24231", "refcount": "0" } }, "os": "GNU/Linux" }, "os": "linux", "os_version": "3.2.0-32-virtual", "ohai_time": 1369328621.3456137,

"network": { "interfaces": {
"lo": { "mtu": "16436", "flags": [ "LOOPBACK", "UP","LOWER_UP" ], "encapsulation": "Loopback", "addresses": { "127.0.0.1": { "family": "inet", "netmask": "255.0.0.0", "scope": "Node" }, "::1": { "family": "inet6", "scope": "Node" } }, }, "eth0": { "type": "eth", "number": "0",

Checkpoint

Monday, 4 November 13

Review Questions
Where is the chef-client configuration file? What is the command to run chef? What does a knife bootstrap do?

Monday, 4 November 13

Chef 101 Terminology

So were on the same page...

v1.1.3

Monday, 4 November 13

Configured, or managed systems are called Nodes

Monday, 4 November 13

chef-client runs on your nodes

Monday, 4 November 13

chef-client talks to a Chef Server

Were using Hosted Chef today; you can buy Private Chef for your own data center

Monday, 4 November 13

API Clients authenticate with RSA keys

chef-client and knife are both API clients The server has the public key
Monday, 4 November 13

Knife is the command-line tool for Chef.

Monday, 4 November 13

Dissecting your first chef-client run

The Anatomy of a Chef run

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to List the steps taken by a chef-client during a run Explain the basic security model of Chef

Monday, 4 November 13

build node

authenticate Ohai! node_name platform platform_version

converge

load cookbooks expanded run list chef-client (recipes)

sync cookbooks

Yes success? No
Monday, 4 November 13

node.save notication handlers exception

Private Keys
Chef Server requires keys to authenticate. client.pem - private key for API client validation.pem - private key for ORGNAMEvalidator Next, lets see how those are used...

Monday, 4 November 13

No

No

/etc/chef/ client.pem?

/etc/chef/ validation.pem?

401!

Yes Yes

Request API Client

Sign Requests

client.pem

Monday, 4 November 13

Review Questions
What are the steps in a Chef Client run? How does a new machine get a private key with which to authenticate requests? If you have the right credentials in place, why else might you not be able to authenticate?

Monday, 4 November 13

Introducing the Node object

Attributes & Search

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Explain what the Node object represents in Chef List the Nodes in an organization Show details about a Node Describe what Node Attributes are Retrieve a node attribute directly, and via search

Monday, 4 November 13

What is the Node object

A node is any physical, virtual, or cloud machines that is configured to be maintained by a Chef When you are writing Recipes, the Node object is always available to you.

Monday, 4 November 13

Exercise: List nodes

$ knife node list

node1

Monday, 4 November 13

Exercise: List clients

$ knife client list

ORGNAME-validator node1

Monday, 4 November 13

Each node must have a unique name

Every node must have a unique name within an organization Chef defaults to the Fully Qualified Domain Name of the server, i.e. in the format server.domain.com We overrode it to "node1" to make typing easier

Monday, 4 November 13

Exercise: Show node details

$ knife node show node1
Node Name: Environment: FQDN: IP: Run List: Roles: Recipes: Platform: Tags:
Monday, 4 November 13

node1 _default ip-10-154-155-107.ec2.internal 54.242.35.165

ubuntu 12.04

What is the Node object

Nodes are made up of Attributes Many are discovered automatically (platform, ip address, number of CPUs) Many other objects in Chef can also add Node attributes (Cookbooks, Roles and Environments, Recipes, Attribute Files) Nodes are stored and indexed on the Chef Server
Monday, 4 November 13

Exercise: Run Ohai on node

opscode@node1:~$ sudo ohai | less
{ "languages": { "ruby": { }, "python": { "version": "2.7.3", "builddate": "Apr 10 2013, 06:20:15" }, "perl": { "version": "5.14.2", "archname": "x86_64-linux-gnu-thread-multi" } }, "kernel": {

Monday, 4 November 13

Exercise: Show all the node attributes

$ knife node show node1 -l
Node Name: Environment: FQDN: IP: Run List: Roles: Recipes: Platform: Tags: Attributes: tags: node1 _default ip-10-154-155-107.ec2.internal 54.242.35.165

ubuntu 12.04

Default Attributes: Override Attributes: Automatic Attributes (Ohai Data): block_device: loop0: removable: 0 size: 0

Monday, 4 November 13

Exercise: Show the raw node object

$ knife node show node1 -Fj
{ "name": "node1", "chef_environment": "_default", "run_list": [], "normal": {"tags":[]} }
Monday, 4 November 13

Exercise: Show only the fqdn attribute

$ knife node show node1 -a fqdn

node1: fqdn: ip-10-154-155-107.ec2.internal

Monday, 4 November 13

Exercise: Use search to find the same data

$ knife search node "*:*" -a fqdn

1 items found node1: fqdn: ip-10-154-155-107.ec2.internal

Monday, 4 November 13

Review Questions
What is the Node object? What is a Node Attribute? How do you display all the attributes of a Node? Can you search for the cpu attribute of your node?

Monday, 4 November 13

Writing an Apache cookbook

Packages, Cookbook Files, and Services

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Describe what a cookbook is Create a new cookbook Explain what a recipe is Describe how to use the package, service, and cookbook_file resources Upload a cookbook to the Chef Server Explain what a run list is, and how to set it for a node via knife Read the output of a chef-client run
Monday, 4 November 13

What is a cookbook?
A cookbook is like a "package" for Chef recipes. It contains all the recipes, files, templates, libraries, etc. required to configure a portion of your infrastructure Typically they map 1:1 to a piece of software or functionality.

Monday, 4 November 13

The Problem and the Success Criteria

The Problem: We need a web server configured to serve up our home page. Success Criteria: We can see the homepage in a web browser.

Monday, 4 November 13

Required steps
Install Apache Start the service, and make sure it will start when the machine boots Write out the home page

Monday, 4 November 13

Exercise: Create a new Cookbook

$ knife cookbook create apache

** ** ** **

Creating Creating Creating Creating

cookbook apache README for cookbook: apache CHANGELOG for cookbook: apache metadata for cookbook: apache

Monday, 4 November 13

Exercise: Explore the cookbook

$ ls -la cookbooks/apache
total 24 drwxr-xr-x drwxr-xr-x -rw-r--r--rw-r--r-drwxr-xr-x drwxr-xr-x drwxr-xr-x drwxr-xr-x -rw-r--r-drwxr-xr-x drwxr-xr-x drwxr-xr-x drwxr-xr-x 13 5 1 1 2 2 3 2 1 2 3 2 3 opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode opscode 442 170 412 1447 68 68 102 68 276 68 102 68 102 Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar Mar 24 24 24 24 24 24 24 24 24 24 24 24 24 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 21:25 . .. CHANGELOG.md README.md attributes definitions files libraries metadata.rb providers recipes resources templates

Monday, 4 November 13

Exercise: Open the default recipe in your editor

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb

# # # # # # # #

Cookbook Name:: apache Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

Monday, 4 November 13

Recipe Naming
The "default.rb" recipe for a given cookbook is referred to by the name of the cookbook (apache) If we added another recipe to this cookbook named "mod_ssl.rb", we would refer to it as apache::mod_ssl

Monday, 4 November 13

Exercise: Add package resource to install Apache

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
# # # # # # # # Cookbook Name:: apache Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

package "apache2" do action :install end

SAVE FILE!
Monday, 4 November 13

Chef Resources
package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Monday, 4 November 13

Chef Resources
Have a type
package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Monday, 4 November 13

Chef Resources
Have a type Have a name
package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Monday, 4 November 13

Chef Resources
Have a type Have a name Have parameters
package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Monday, 4 November 13

Chef Resources
Have a type Have a name Have parameters Take action to put the resource into the desired state
package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

Monday, 4 November 13

Chef Resources
Have a type Have a name Have parameters Take action to put the resource into the desired state Can send notifications to other resources
Monday, 4 November 13

package "haproxy" do action :install end template "/etc/haproxy/haproxy.cfg" do source "haproxy.cfg.erb" owner "root" group "root" mode "0644" notifies :restart, "service[haproxy]" end service "haproxy" do supports :restart => :true action [:enable, :start] end

So the resource we just wrote...

package "apache2" do action :install end

Monday, 4 November 13

So the resource we just wrote...

Is a package resource

package "apache2" do action :install end

Monday, 4 November 13

So the resource we just wrote...

Is a package resource Whose name is apache2

package "apache2" do action :install end

Monday, 4 November 13

So the resource we just wrote...

Is a package resource Whose name is apache2 With an install action

package "apache2" do action :install end

Monday, 4 November 13

Notice we didnt say how to install the package

Resources are declarative - that means we say what we want to have happen, rather than how Chef uses the platform the node is running to determine the correct provider for a resource

Monday, 4 November 13

Exercise: Add a service resource to ensure service is started & enabled at boot
OPEN IN EDITOR: cookbooks/apache/recipes/default.rb

... # All rights reserved - Do Not Redistribute # package "apache2" do action :install end service "apache2" do action [:enable, :start] end
SAVE FILE!
Monday, 4 November 13

So the resource we just wrote...

service "apache2" do action [:enable, :start] end

Monday, 4 November 13

So the resource we just wrote...

Is a service resource
service "apache2" do action [:enable, :start] end

Monday, 4 November 13

So the resource we just wrote...

Is a service resource Whose name is apache2
service "apache2" do action [:enable, :start] end

Monday, 4 November 13

So the resource we just wrote...

Is a service resource Whose name is apache2 With two actions: start and enable
service "apache2" do action [:enable, :start] end

Monday, 4 November 13

Order Matters
Resources are executed in order
1st

2nd

Body Level One Body Level Two template "/etc/haproxy/haproxy.cfg" do Body Level Three source "haproxy.cfg.erb" owner "root" Body Level Four group "root" mode "0644" Body Level Five notifies :restart, "service[haproxy]"
package "haproxy" do action :install end end service "haproxy" do supports :restart => :true action [:enable, :start] end

3rd

Monday, 4 November 13

Exercise: Add a cookbook_file resource to copy the homepage in place

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb

... service "apache2" do action [:enable, :start] end

cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

SAVE FILE!
Monday, 4 November 13

So the resource we just wrote...

cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

Monday, 4 November 13

So the resource we just wrote...

Is a cookbook_file resource
cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

Monday, 4 November 13

So the resource we just wrote...

Is a cookbook_file resource Whose name is /var/www/index.html
cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

Monday, 4 November 13

So the resource we just wrote...

Is a cookbook_file cookbook_file "/var/www/index.html" resource source "index.html" mode "0644" Whose name is end /var/www/index.html With two parameters: source of index.html mode of "0644"
do

Monday, 4 November 13

Full contents of the apache recipe

# # # # # # # # Cookbook Name:: apache Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

package "apache2" do action :install end service "apache2" do action [:enable, :start] end cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

Monday, 4 November 13

Exercise: Add index.html to cookbooks files/default directory

OPEN IN EDITOR: cookbooks/apache/files/default/index.html

<html> <body> <h1>Hello, world!</h1> </body> </html>

SAVE FILE!
Monday, 4 November 13

Whats with the default subdirectory?

Chef allows you to select the most appropriate file (or template) within a cookbook according to the platform of the node it is being executed on host node name (e.g. foo.bar.com) platform-version (e.g. redhat-6.2.1) platform-version_components (e.g. redhat-6.2, redhat-6) platform (e.g. redhat) default 99% of the time, you will just use default
Monday, 4 November 13

Exercise: Upload the cookbook

$ knife cookbook upload apache

Uploading apache Uploaded 1 cookbook.

[0.1.0]

Monday, 4 November 13

Exercise: Add apache recipe to test nodes run list

$ knife node run_list add node1 "recipe[apache]"

node1: run_list: recipe[apache]

Monday, 4 November 13

The Run List

The Run List is the ordered set of recipes and roles that the Chef Client will execute on a node Recipes are specified by "recipe[name]" Roles are specified by "role[name]"

Monday, 4 November 13

Exercise: Run chef-client on your test node

opscode@node1:~$ sudo chef-client
Starting Chef Client, version 11.4.4 [2013-06-25T04:20:22+00:00] INFO: *** Chef 11.4.4 *** [2013-06-25T04:20:23+00:00] INFO: [inet6] no default interface, picking the first ipaddress [2013-06-25T04:20:23+00:00] INFO: Run List is [recipe[apache]] [2013-06-25T04:20:23+00:00] INFO: Run List expands to [apache] [2013-06-25T04:20:23+00:00] INFO: Starting Chef Run for node1 [2013-06-25T04:20:23+00:00] INFO: Running start handlers [2013-06-25T04:20:23+00:00] INFO: Start handlers complete. resolving cookbooks for run list: ["apache"] [2013-06-25T04:20:24+00:00] INFO: Loading cookbooks [apache] Synchronizing Cookbooks: [2013-06-25T04:20:24+00:00] INFO: Storing updated cookbooks/apache/recipes/default.rb in the cache. [2013-06-25T04:20:24+00:00] INFO: Storing updated cookbooks/apache/CHANGELOG.md in the cache. [2013-06-25T04:20:25+00:00] INFO: Storing updated cookbooks/apache/metadata.rb in the cache. [2013-06-25T04:20:25+00:00] INFO: Storing updated cookbooks/apache/README.md in the cache. - apache Compiling Cookbooks... Converging 3 resources Recipe: apache::default * package[apache2] action install[2013-06-25T04:20:25+00:00] INFO: Processing package[apache2] action install (apache::default line 9) - install version 2.2.22-1ubuntu1 of package apache2

Monday, 4 November 13

Exercise: Verify homepage works

Open a web browser Type in the the URL for your test node

Monday, 4 November 13

Congratulate yourself!
You have just written your first Chef cookbook! (clap!)

Monday, 4 November 13

Reading the output of a chef-client run

Starting Chef Client, version 11.6.0 [2013-06-25T04:20:22+00:00] INFO: *** Chef 11.6.0 *** [2013-06-25T04:20:23+00:00] INFO: [inet6] no default interface, picking the first ipaddress [2013-06-25T04:20:23+00:00] INFO: Run List is [recipe[apache]] [2013-06-25T04:20:23+00:00] INFO: Run List expands to [apache] [2013-06-25T04:20:23+00:00] INFO: Starting Chef Run for node1 [2013-06-25T04:20:23+00:00] INFO: Running start handlers [2013-06-25T04:20:23+00:00] INFO: Start handlers complete.

The run list is shown The expanded Run List is the complete list, after nested roles are expanded
Monday, 4 November 13

Reading the output of a chef-client run

resolving cookbooks for run [2013-06-25T04:20:24+00:00] Synchronizing Cookbooks: [2013-06-25T04:20:24+00:00] cache. [2013-06-25T04:20:24+00:00] cache. [2013-06-25T04:20:25+00:00] [2013-06-25T04:20:25+00:00] - apache Compiling Cookbooks... list: ["apache"] INFO: Loading cookbooks [apache] INFO: Storing updated cookbooks/apache/recipes/default.rb in the INFO: Storing updated cookbooks/apache/CHANGELOG.md in the INFO: Storing updated cookbooks/apache/metadata.rb in the cache. INFO: Storing updated cookbooks/apache/README.md in the cache.

Load the cookbooks in the order specified by the run list Download any files that are missing from the server
Monday, 4 November 13

Reading the output of a chef-client run

Converging 3 resources Recipe: apache::default * package[apache2] action install[2013-06-25T04:20:25+00:00] INFO: Processing package[apache2] action install (apache::default line 9) - install version 2.2.22-1ubuntu1 of package apache2

Check to see if the package apache2 is installed It was not, so version 2.2.22-1ubuntu1 of the package was installed (yours may be different)

Monday, 4 November 13

Reading the output of a chef-client run

* service[apache2] action enable[2013-06-25T04:20:40+00:00] INFO: Processing service[apache2] action enable (apache::default line 13) (up to date) * service[apache2] action start[2013-06-25T04:20:40+00:00] INFO: Processing service[apache2] action start (apache::default line 13) (up to date)

Check to see if apache2 is already enabled to run at boot - it is, take no further action Check to see if apache2 is already started - it is, take no further action

Monday, 4 November 13

Idempotence
Actions on resources in Chef are designed to be idempotent This means they can be applied multiple times but the end result is still the same - like multiplying by 1 in mathematics! Chef is a "desired state configuration" system - if a resource is already configured, no action is taken. This is called convergence
Monday, 4 November 13

Reading the output of a chef-client run

* cookbook_file[/var/www/index.html] action create[2013-06-25T04:20:40+00:00] INFO: Processing cookbook_file[/var/www/index.html] action create (apache::default line 17) [2013-06-25T04:20:41+00:00] INFO: cookbook_file[/var/www/index.html] backed up to /var/chef/backup/var/ www/index.html.chef-20130625042041 [2013-06-25T04:20:41+00:00] INFO: cookbook_file[/var/www/index.html] mode changed to 644 [2013-06-25T04:20:41+00:00] INFO: cookbook_file[/var/www/index.html] created file /var/www/index.html - create a new cookbook_file /var/www/index.html --- /var/www/index.html 2013-06-25 04:20:37.036043860 +0000 +++ /var/chef/cache/cookbooks/apache/files/default/index.html 2013-06-25 04:20:40.904043861 +0000 @@ -1,4 +1,5 @@ -<html><body><h1>It works!</h1> -<p>This is the default web page for this server.</p> -<p>The web server software is running but no content has been added, yet.</p> -</body></html> ...

Check for an index.html file There is already one in place, backup the file Set permissions on the file A diff of the written file is shown with the modified lines called out

Monday, 4 November 13

Reading the output of a chef-client run

[2013-06-25T04:20:41+00:00] INFO: [2013-06-25T04:20:41+00:00] INFO: [2013-06-25T04:20:41+00:00] INFO: Chef Client finished, 2 resources Chef Run complete in 17.432572377 seconds Running report handlers Report handlers complete updated

Time to complete the Chef run is displayed Report and exception handlers are now run

Monday, 4 November 13

Exercise: Re-run Chef Client

opscode@node1:~$ sudo chef-client
Converging 3 resources Recipe: apache::default * package[apache2] action install[2013-06-25T04:49:38+00:00] INFO: Processing package[apache2] action install (apache::default line 9) (up to date) * service[apache2] action enable[2013-06-25T04:49:39+00:00] INFO: Processing service[apache2] action enable (apache::default line 13) (up to date) * service[apache2] action start[2013-06-25T04:49:39+00:00] INFO: Processing service[apache2] action start (apache::default line 13) (up to date) * cookbook_file[/var/www/index.html] action create[2013-06-25T04:49:39+00:00] INFO: Processing cookbook_file[/var/www/ index.html] action create (apache::default line 17) (up to date) [2013-06-25T04:49:39+00:00] INFO: Chef Run complete in 0.670331523 seconds [2013-06-25T04:49:39+00:00] INFO: Removing cookbooks/apache/files/default/index.html from the cache; it is no longer needed by chef-client. [2013-06-25T04:49:39+00:00] INFO: Running report handlers [2013-06-25T04:49:39+00:00] INFO: Report handlers complete Chef Client finished, 0 resources updated

Monday, 4 November 13

Review Questions
What is a cookbook? How do you create a new cookbook? What is a recipe? What is a resource? How do you upload a cookbook to the Chef Server? What is a run list? What do the "Processing" lines in the chef-client output mean?
Monday, 4 November 13

Writing an MOTD Cookbook

Cookbook Attributes, Cookbook Dependencies, Attribute Precedence, and ERB Templates

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Describe Cookbook Attribute files Use ERB Templates in Chef Explain Attribute Precedence Describe Cookbook Metadata Specify cookbook dependencies Perform the cookbook creation, upload, and test loop
Monday, 4 November 13

The Problem and the Success Criteria

The Problem: We need to add a message that appears at login that states: "This server is property of COMPANY" "This server is in-scope for PCI compliance" if the server is, in fact, in scope. Success Criteria: We see the message when we log in to the test node

Monday, 4 November 13

We have a small problem...

We dont have an attribute for Company, nor one that reflects whether it is in or out of scope for PCI Compliance

Monday, 4 November 13

Possible Solutions
Well factored cookbooks only contain the information relevant to their domain We know we will likely have other things related to PCI (security settings, for example) So the best thing to do is create a PCI cookbook, and add our attribute there

Monday, 4 November 13

Exercise: Create a cookbook named motd

$ knife cookbook create motd
** ** ** ** Creating Creating Creating Creating cookbook motd README for cookbook: motd CHANGELOG for cookbook: motd metadata for cookbook: motd

Monday, 4 November 13

Exercise: Create a default.rb attribute file

OPEN IN EDITOR: cookbooks/motd/attributes/default.rb

default['company'] = "Opscode"
SAVE FILE!

Creates a new Node attribute: node[company] Sets the values to the string "Opscode"

Monday, 4 November 13

Exercise: Open the default recipe in your editor

OPEN IN EDITOR: cookbooks/motd/recipes/default.rb

# # # # # # # #

Cookbook Name:: motd Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

Monday, 4 November 13

What resource should we use?

We could try and use a cookbook_file here, and rely on the file copy rules. Create a file per server, basically. Obviously, thats dramatically inefficient. Instead, we will render a template - a file that is a mixture of the contents we want, and embedded Ruby code

Monday, 4 November 13

Exercise: Add template resource for /etc/motd.tail

Use a template resource The name is "/etc/motd.tail" The resource has two parameters source is "motd.tail.erb" mode is "0644"

Monday, 4 November 13

The template[/etc/motd.tail] resource

OPEN IN EDITOR: cookbooks/motd/recipes/default.rb
# # # # # # # # Cookbook Name:: motd Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

template "/etc/motd.tail" do source "motd.tail.erb" mode "0644" end

SAVE FILE!
Monday, 4 November 13

Exercise: Open motd.tail.erb in your Editor

OPEN IN EDITOR: cookbooks/motd/templates/default/motd.tail.erb

This server is property of <%= node['company'] %> <% if node['pci']['in_scope'] -%> This server is in-scope for PCI compliance <% end -%>
SAVE FILE!

"erb" stands for "Embedded Ruby"

Monday, 4 November 13

Exercise: Open motd.tail.erb in your Editor

OPEN IN EDITOR: cookbooks/motd/templates/default/motd.tail.erb

This server is property of <%= node['company'] %> <% if node['pci']['in_scope'] -%> This server is in-scope for PCI compliance <% end -%>

To embed a value within an ERB template: Start with <%= Write your Ruby expression - most commonly a node attribute End with %>
Monday, 4 November 13

Exercise: Open motd.tail.erb in your Editor

OPEN IN EDITOR: cookbooks/motd/templates/default/motd.tail.erb

This server is property of <%= node['company'] %> <% if node['pci']['in_scope'] -%> This server is in-scope for PCI compliance <% end -%>

You can use any Ruby construct in a template Starting with <% will evaluate the expression, but not insert
the result

Ending with -%> will not insert a line in the resulting file
Monday, 4 November 13

Templates Are Used For Almost All Configuration Files

Templates are very flexible ways to create your configuration files Coupled with Chefs attribute precedence rules, you can create very effective, data-driven cookbooks

Monday, 4 November 13

Best Practice: Recipes contain the pattern, attributes supply the details

Recipes contain the pattern for how to do something. ("How we deploy tomcat") Attributes contain the details. ("What port do we run tomcat on?")

Monday, 4 November 13

Exercise: Upload the motd cookbook

$ knife cookbook upload motd Uploading motd Uploaded 1 cookbook. [0.1.0]

Monday, 4 November 13

Exercise: Create a cookbook named pci

$ knife cookbook create pci
** ** ** ** Creating Creating Creating Creating cookbook pci README for cookbook: pci CHANGELOG for cookbook: pci metadata for cookbook: pci

Monday, 4 November 13

Exercise: Create a default.rb attribute file

OPEN IN EDITOR: cookbooks/pci/attributes/default.rb

default['pci']['in_scope'] = false
SAVE FILE!

Creates a new Node attribute: node[pci][in_scope] Sets the value to the Ruby false literal

Monday, 4 November 13

Node Attributes have four levels of precedence

Automatic attributes are those discovered by Ohai Override attributes are the strongest way to set an attribute - use sparingly Normal attributes are those set directly on a Node object Default attributes are typically set in Cookbooks, Roles and Environments
Monday, 4 November 13

"rsyslog": { "log_dir": "/srv/rsyslog", "server": false, "rsyslog": { "protocol": "tcp", "log_dir": "/srv/rsyslog", "port": 1514, "server": false, "rsyslog": { "conf": {} "protocol": "tcp", "log_dir": "/srv/rsyslog", } "port": 1514, Automatic "server": false, "rsyslog": { "conf": {} "protocol": "tcp", "log_dir": "/srv/rsyslog", } "port": 1514, Override "server": false, "conf": {} "protocol": "tcp", } "port": 1514, Normal "conf": {} }

Default

Best Practice: Always use default attributes in your cookbooks

When setting an attribute in a cookbook, it should (almost) always be a default attribute There are exceptions, but they are rare

Monday, 4 November 13

Best Practice: Make sure cookbooks have default values

If a cookbook needs an attribute to exist, it should 1. Define a default value for it in an attribute file, or 2. Depend on another cookbook that does Never rely on an attribute being created manually

Monday, 4 November 13

Exercise: Upload the PCI cookbook

$ knife cookbook upload pci Uploading pci Uploaded 1 cookbook. [0.1.0]

Monday, 4 November 13

Exercise: Add the motd recipe to your test nodes run list
$ knife node run_list add node1 "recipe[motd]"

node1: run_list: recipe[apache] recipe[motd]

Monday, 4 November 13

Exercise: Add the motd recipe to your test nodes run list

$ knife node show node1

Node Name: Environment: FQDN: IP: Run List: Roles: Recipes: Platform: Tags: node1 _default ip-10-154-155-107.ec2.internal 54.242.35.165 recipe[apache], recipe[motd]

ubuntu 10.04

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1:~$ sudo chef-client
Starting Chef Client, version 11.4.4 INFO: *** Chef 11.4.4 *** INFO: [inet6] no default interface, picking the first ipaddress INFO: Run List is [recipe[apache], recipe[motd]] INFO: Run List expands to [apache, motd] INFO: Starting Chef Run for node1.local INFO: Running start handlers INFO: Start handlers complete. resolving cookbooks for run list: ["apache", "motd"] INFO: Loading cookbooks [apache, motd] Synchronizing Cookbooks: INFO: Storing updated cookbooks/motd/recipes/default.rb in the cache. INFO: Storing updated cookbooks/motd/CHANGELOG.md in the cache. INFO: Storing updated cookbooks/motd/metadata.rb in the cache. INFO: Storing updated cookbooks/motd/README.md in the cache. - motd - apache Compiling Cookbooks... Converging 4 resources ... Recipe: motd::default * template[/etc/motd.tail] action create INFO: Processing template[/etc/motd.tail] action create (motd::default line 10)

FAIL!

================================================================================ Error executing action `create` on resource 'template[/etc/motd.tail]'

Monday, 4 November 13

You probably see this at the bottom of your screen...

Resource Declaration: --------------------# In /var/chef/cache/cookbooks/motd/recipes/default.rb 10: template "/etc/motd.tail" do 11: source "motd.tail.erb" 12: mode "0644" 13: end Compiled Resource: -----------------# Declared in /var/chef/cache/cookbooks/motd/recipes/default.rb:10:in `from_file' template("/etc/motd.tail") do provider Chef::Provider::Template action "create" retries 0 retry_delay 2 path "/etc/motd.tail" backup 5 source "motd.tail.erb" cookbook_name "motd" recipe_name "default" mode "0644" end ERROR: Running exception handlers

Monday, 4 November 13

Stack Traces
A stack trace tells you where in a program an error occurred They can (obviously) be very detailed They can also be intensely useful, as they supply the data you need to find a problem

Monday, 4 November 13

Scroll up
In this case, Chef actually knows exactly what went wrong. Scroll up to find out.
================================================================================ Error executing action `create` on resource 'template[/etc/motd.tail]' ================================================================================ Chef::Mixin::Template::TemplateError -----------------------------------undefined method `[]' for nil:NilClass Resource Declaration: --------------------# In /var/chef/cache/cookbooks/motd/recipes/default.rb 10: template "/etc/motd.tail" do 11: source "motd.tail.erb" 12: mode "0644" 13: end

Monday, 4 November 13

We do not have the attribute we are using in the conditional

INFO: Run List is [recipe[apache], recipe[motd]] INFO: Run List expands to [apache, motd] INFO: Starting Chef Run for node1.local INFO: Running start handlers INFO: Start handlers complete. resolving cookbooks for run list: ["apache", "motd"] INFO: Loading cookbooks [apache, motd]

Can anyone guess why? We did not load the PCI cookbook!
Monday, 4 November 13

Exercise: Add a dependency on the PCI cookbook to the MOTD cookbook

OPEN IN EDITOR: cookbooks/motd/metadata.rb
maintainer maintainer_email license description long_description version "YOUR_COMPANY_NAME" "YOUR_EMAIL" "All rights reserved" "Installs/Configures motd" IO.read(File.join(File.dirname(__FILE__), README.md)) "0.1.0"

Monday, 4 November 13

Exercise: Add a dependency on the PCI cookbook to the MOTD cookbook

OPEN IN EDITOR: cookbooks/motd/metadata.rb
maintainer maintainer_email license description long_description version depends "pci" "YOUR_COMPANY_NAME" "YOUR_EMAIL" "All rights reserved" "Installs/Configures motd" IO.read(File.join(File.dirname(__FILE__), README.md)) "0.1.0"

Monday, 4 November 13

Cookbook Metadata
OPEN IN EDITOR: cookbooks/motd/metadata.rb
maintainer maintainer_email license description long_description version depends "pci" "YOUR_COMPANY_NAME" "YOUR_EMAIL" "All rights reserved" "Installs/Configures motd" IO.read(File.join(File.dirname(__FILE__), README.md)) "0.1.0"

Monday, 4 November 13

Cookbook Metadata
OPEN IN EDITOR: cookbooks/motd/metadata.rb
maintainer maintainer_email license description long_description version depends "pci" "YOUR_COMPANY_NAME" "YOUR_EMAIL" "All rights reserved" "Installs/Configures motd" IO.read(File.join(File.dirname(__FILE__), README.md)) "0.1.0"

SAVE FILE!

Cookbooks that depend on other cookbooks will cause the dependent cookbook to be downloaded to the client, and evaluated
Monday, 4 November 13

Cookbook Attributes are applied for all downloaded cookbooks!

Cookbooks downloaded as dependencies will have their attribute files evaluated Even if there is no recipe from the cookbook in the run-list

Monday, 4 November 13

Exercise: Upload the motd cookbook

$ knife cookbook upload motd Uploading motd Uploaded 1 cookbook. [0.1.0]

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1:~$ sudo chef-client
Starting Chef Client, version 11.4.4 INFO: *** Chef 11.4.4 *** INFO: [inet6] no default interface, picking the first ipaddress INFO: Run List is [recipe[apache], recipe[motd]] INFO: Run List expands to [apache, motd] INFO: Starting Chef Run for node1.local INFO: Running start handlers INFO: Start handlers complete. resolving cookbooks for run list: ["apache", "motd"] INFO: Loading cookbooks [apache, motd, pci] Synchronizing Cookbooks: INFO: Storing updated cookbooks/pci/recipes/default.rb in the cache. INFO: Storing updated cookbooks/pci/attributes/default.rb in the cache. INFO: Storing updated cookbooks/pci/CHANGELOG.md in the cache. INFO: Storing updated cookbooks/pci/metadata.rb in the cache. INFO: Storing updated cookbooks/pci/README.md in the cache. - pci INFO: Storing updated cookbooks/motd/metadata.rb in the cache. - motd - apache Compiling Cookbooks... Converging 4 resources ... Recipe: motd::default * template[/etc/motd.tail] action create INFO: Processing template[/etc/motd.tail] action create (motd::default line 10) INFO: template[/etc/motd.tail] updated content INFO: template[/etc/motd.tail] mode changed to 644 - create template[/etc/motd.tail]

WIN!

Monday, 4 November 13

Exercise: Check your work

opscode@node1:~$ cat /etc/motd.tail

This server is property of opscode

Monday, 4 November 13

Exercise: Show your test nodes pci attribute

$ knife search node "pci:*" -a pci

1 items found id: node1 pci: in_scope:

Monday, 4 November 13

false

Exercise: Set attribute to true

OPEN IN EDITOR: cookbooks/pci/attributes/default.rb

default['pci']['in_scope'] = true
SAVE FILE!

Set the attribute to true

Monday, 4 November 13

Exercise: Upload the PCI cookbook

$ knife cookbook upload pci Uploading pci Uploaded 1 cookbook. [0.1.0]

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1:~$ sudo chef-client
Starting Chef Client, version 11.4.4 INFO: *** Chef 11.4.4 *** INFO: [inet6] no default interface, picking the first ipaddress INFO: Run List is [recipe[apache], recipe[motd]] INFO: Run List expands to [apache, motd] INFO: Starting Chef Run for node1.local INFO: Running start handlers INFO: Start handlers complete. resolving cookbooks for run list: ["apache", "motd"] INFO: Loading cookbooks [apache, motd, pci] Synchronizing Cookbooks: - pci - motd - apache Compiling Cookbooks... Converging 4 resources ... Recipe: motd::default * template[/etc/motd.tail] action create INFO: Processing template[/etc/motd.tail] action create (motd::default line 10) INFO: template[/etc/motd.tail] backed up to /var/chef/backup/etc/motd.tail.chef-20130624003110 INFO: template[/etc/motd.tail] updated content INFO: template[/etc/motd.tail] mode changed to 644 - update template[/etc/motd.tail] from 46ca00 to d6fdcd --- /etc/motd.tail 2013-06-24 00:26:27.261234214 +0000

Monday, 4 November 13

Exercise: Check your work

opscode@node1:~$ cat /etc/motd.tail

This server is property of opscode This server is in-scope for PCI compliance

Monday, 4 November 13

Exercise: Show your test nodes pci attribute

$ knife node show node1 -a pci 1 items found id: node1 pci: in_scope:
Monday, 4 November 13

true

Congratulations!
You now know the 3 most important resources in the history of configuration management Package Template Service

Monday, 4 November 13

Review Questions
What goes in a cookbooks attribute files? What are the 4 different levels of precedence? When do you need to specify a cookbook dependency? What does <%= mean, and where will you encounter it? What are the 3 most important resources in configuration management?

Monday, 4 November 13

Refactoring the Apache Cookbook

Execute, Not If/Only If, Directories, Notifications, Template Variables, and the Chef Docs Site

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to
Use the execute resource Control idempotence manually with not_if and only_if Navigate the Resources page on docs.opscode.com Describe the Directory resource Use resource notifications Explain what Template Variables are, and how to use them Use Ruby variables, loops, and string expansion

Monday, 4 November 13

The Problem and the Success Criteria

The Problem: We need to deploy multiple custom home pages running on different ports Success Criteria: Be able to view our custom home page

Monday, 4 November 13

Exercise: Change the cookbooks version number in the metadata

OPEN IN EDITOR: cookbooks/apache/metadata.rb
maintainer maintainer_email license description long_description version "YOUR_COMPANY_NAME" "YOUR_EMAIL" "All rights reserved" "Installs/Configures apache" IO.read(File.join(File.dirname(__FILE__), README.md)) "0.2.0"

SAVE FILE!

Major, Minor, Patch Semantic Versioning Policy: http://semver.org/

Monday, 4 November 13

Exercise: Create a default.rb attribute file

OPEN IN EDITOR: cookbooks/apache/attributes/default.rb

default['apache']['sites']['clowns'] = { "port" => 80 } default['apache']['sites']['bears'] = { "port" => 81 }

SAVE FILE!

We add information about the sites we need to deploy One about Clowns, running on port 80 One about Bears, running on port 81
Monday, 4 November 13

Exercise: Open the default apache recipe in your editor

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
# # Cookbook Name:: apache # Recipe:: default # # Copyright 2013, YOUR_COMPANY_NAME # # All rights reserved - Do Not Redistribute # package "apache2" do action :install end service "apache2" do action [:enable, :start] end cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

SAVE FILE!
Monday, 4 November 13

Exercise: Use execute resource to disable the default Apache virtual host
OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
service "apache2" do action [:enable, :start] end execute "a2dissite default" do only_if do File.symlink?("/etc/apache2/sites-enabled/000-default") end notifies :restart, "service[apache2]" end cookbook_file "/var/www/index.html" do

SAVE FILE!

Runs the command "a2dissite default", but only if the symlink exists If the action succeeds, restart Apache
Monday, 4 November 13

Execute resources are generally not idempotent

Chef will stop your run if a resource fails Most command line utilities are not idempotent - they assume a human being is interacting with, and understands, the state of the system The result is - its up to you to make execute resources idempotent

Monday, 4 November 13

Enter the not_if and only_if metaparameters

only_if do File.symlink?("/etc/apache2/sites-enabled/000-default") end

The only_if parameter causes the resources actions to be taken only if its argument returns true The not_if parameter is the opposite of only_if the actions are taken only if its argument returns false
Monday, 4 November 13

Best Practice: The Chef Docs Site

The Chef Docs Site is the home for all of the documentation about Chef. It is very comprehensive It has a page on every topic http://docs.opscode.com Lets use the docs to learn more about not_if and only_if

Monday, 4 November 13

Exercise: Search for more information about Resources

Monday, 4 November 13

Search for "Resources"

Exercise: Search for more information about Resources

Find "Resources and Providers Reference"

Monday, 4 November 13

The Resources Page

Monday, 4 November 13

Notifications
notifies :restart, "service[apache2]"
Resource Notifications in Chef are used to trigger an action on a resource when the current resources actions are successful. "If we delete the site, restart apache" The first argument is an action, and the second argument is the string representation of a given resource Like not_if and only_if, notifies is a resource metaparameter - any resource can notify any other
Monday, 4 November 13

Exercise: Iterate over each apache site

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
execute "a2dissite default" do only_if do File.symlink?("/etc/apache2/sites-enabled/000-default") end notifies :restart, "service[apache2]" end cookbook_file "/var/www/index.html" do source "index.html" mode "0644" end

Delete the cookbook_file resource

Monday, 4 November 13

Exercise: Iterate over each apache site

OPEN IN EDITOR: cookbooks/apache/recipes/default.rb
execute "a2dissite default" do only_if do File.symlink?("/etc/apache2/sites-enabled/000-default") end notifies :restart, "service[apache2]" end node['apache']['sites'].each do |site_name, site_data| document_root = "/srv/apache/#{site_name}"

SAVE FILE!

Delete the cookbook_file resource node['apache']['sites'] is a ruby hash, with keys and values
Monday, 4 November 13

Exercise: Iterate over each apache site

node['apache']['sites'].each do |site_name, site_data| document_root = "/srv/apache/#{site_name}"

Calling .each loops over each site default['apache']['sites']['clowns'] = { "port" => 80 } default['apache']['sites']['bears'] = { "port" => 81 } First pass
= clowns = { "port" =>

site_name site_data
80 }
Monday, 4 November 13

Second pass site_name site_data

81 }

= bears = { "port" =>

Exercise: Iterate over each apache site

node['apache']['sites'].each do |site_name, site_data| document_root = "/srv/apache/#{site_name}"

Create a variable called document_root #{site_name} means "insert the value of site_name here" First pass The value is the string "/srv/apache/clowns" Second pass The value is the string "/srv/apache/bears"
Monday, 4 November 13

Exercise: Add a template for Apache virtual host configuration

node['apache']['sites'].each do |site_name, site_data| document_root = "/srv/apache/#{site_name}" template "/etc/apache2/sites-available/#{site_name}" do source "custom.erb" mode "0644" variables( :document_root => document_root, :port => site_data['port'] ) notifies :restart, "service[apache2]" end

Monday, 4 November 13

Template Variables
Not all data you might need in a template is necessarily node attributes The variables parameter lets you pass in custom data for use in a template

Monday, 4 November 13

Exercise: Add an execute resource to enable new virtual host

template "/etc/apache2/sites-available/#{site_name}" do source "custom.erb" mode "0644" variables( :document_root => document_root, :port => site_data['port'] ) notifies :restart, "service[apache2]" end execute "a2ensite #{site_name}" do not_if do File.symlink?("/etc/apache2/sites-enabled/#{site_name}") end notifies :restart, "service[apache2]" end
Monday, 4 November 13

Exercise: Add a directory resource to create the document_root

Use a directory resource The name is document_root The resource has two parameters mode is "0755" recursive is true Use the Resources page on the Docs Site to read more about what recursive does.

Monday, 4 November 13

The directory resource

execute "a2ensite #{site_name}" do not_if do File.symlink?("/etc/apache2/sites-enabled/#{site_name}") end notifies :restart, "service[apache2]" end directory document_root do mode "0755" recursive true end

Monday, 4 November 13

Exercise: Add a template resource for the virtual hosts index.html

directory document_root do mode "0755" recursive true end template "#{document_root}/index.html" do source "index.html.erb" mode "0644" variables( :site_name => site_name, :port => site_data['port'] ) end end
Monday, 4 November 13

Dont forget the last "end"

template "#{document_root}/index.html" do source "index.html.erb" mode "0644" variables( :site_name => site_name, :port => site_data['port'] ) end end
See the correct, whole file at x
Monday, 4 November 13

Exercise: Add custom.erb to your templates directory

OPEN IN EDITOR: cookbooks/apache/templates/default/custom.erb

Note the two template variables are prefixed with an @ symbol Our first conditional if! If you are feeling hardcore, type it. https://gist.github.com/2866454

<% if @port != 80 -%> Listen <%= @port %> <% end -%> <VirtualHost *:<%= @port %>> ServerAdmin webmaster@localhost DocumentRoot <%= @document_root %> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory <%= @document_root %>> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> </VirtualHost>

SAVE FILE!
Monday, 4 November 13

Exercise: Add index.html.erb to your templates directory

OPEN IN EDITOR: cookbooks/apache/templates/default/index.html.erb <html> <body> <h1>Welcome to <%= node['company'] %></h1> <h2>We love <%= @site_name %></h2> <%= node['ipaddress'] %>:<%= @port %> </body> </html> SAVE FILE!

Note the two template variables are prefixed with an @ symbol https://gist.github.com/2866421
Monday, 4 November 13

Exercise: Upload the Apache cookbook

$ knife cookbook upload apache

Uploading apache Uploaded 1 cookbook.

[0.2.0]

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1:~$ sudo chef-client
Compiling Cookbooks... Converging 12 resources Recipe: apache::default * package[apache2] action install INFO: Processing package[apache2] action install (apache::default line 10) (up to date) * service[apache2] action start INFO: Processing service[apache2] action start (apache::default line 14) (up to date) * service[apache2] action enable INFO: Processing service[apache2] action enable (apache::default line 14) (up to date) * execute[a2dissite default] action run INFO: Processing execute[a2dissite default] action run (apache::default line 18) Site default disabled. To activate the new configuration, you need to run: service apache2 reload INFO: execute[a2dissite default] ran successfully - execute a2dissite default
Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1:~$ sudo chef-client
INFO: execute[a2dissite default] sending restart action to service[apache2] (delayed) Recipe: apache::default * service[apache2] action restart INFO: Processing service[apache2] action restart (apache::default line 14) INFO: service[apache2] restarted - restart service service[apache2] INFO: Chef Run complete in 5.559923708 seconds INFO: Running report handlers INFO: Report handlers complete Chef Client finished, 10 resources updated
Monday, 4 November 13

Exercise: Verify the two sites are working!

Monday, 4 November 13

Best Practice: Recipes contain the pattern, attributes supply the details

Recipes contain the pattern for how to do something. ("How we deploy apache virtual hosts") Attributes contain the details. ("What virtual hosts should we deploy?")

Monday, 4 November 13

Review Questions
How do you control the idempotence of an Execute resource? Where can you learn the details about all the core resources in Chef? What is a notification? What is a template variable? What does #{foo} do in a Ruby string?

Monday, 4 November 13

Writing a Users cookbook

Users, Groups, Data Bags, Recipe Inclusion and Search

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Explain what Data Bags are, and how they are used Use the User and Group resources Use include_recipe Describe the role Search plays in recipes

Monday, 4 November 13

The Problem and the Success Criteria

The Problem: Employees should have local user accounts created on servers, along with custom groups Success Criteria: We can add new employees and groups to servers dynamically

Monday, 4 November 13

Where should we store the user data?

As weve seen, we could start by storing information about users as Node Attributes This is sort of a bummer, because we would be duplicating a lot of information - every user in the company would be stored in every Node object! Additionally, it would be very hard to integrate such a solution with another source of truth about users

Monday, 4 November 13

Introducing Data Bags

A data bag is a container for items that represent information about your infrastructure that is not tied to a single node Examples Users Groups Application Release Information

Monday, 4 November 13

Make a data_bags directory

$ mkdir data_bags

(No output)

Monday, 4 November 13

Exercise: Create a data bag named users

$ mkdir data_bags/users $ knife data_bag create users

Created data_bag[users]

Monday, 4 November 13

Exercise: Create a user item in the users data bag

OPEN IN EDITOR: data_bags/users/bobo.json

{ "id": "bobo", "comment": "Bobo T. Clown", "uid": 2000, "gid": 0, "home": "/home/bobo", "shell": "/bin/bash" }

SAVE FILE!
Monday, 4 November 13

Exercise: Create the data bag item

$ knife data_bag from file users bobo.json

Updated data_bag_item[users::bobo]

Monday, 4 November 13

Exercise: Create another user in the users data bag

OPEN IN EDITOR: data_bags/users/frank.json

{ "id": "frank", "comment": "Frank Belson", "uid": 2001, "gid": 0, "home": "/home/frank", "shell": "/bin/bash" }

SAVE FILE!
Monday, 4 November 13

Exercise: Create the data bag item

$ knife data_bag from file users frank.json

Updated data_bag_item[users::frank]

Monday, 4 November 13

Exercise: Show all the items in users data bag

$ knife search users "*:*"
2 items found chef_type: comment: data_bag: gid: home: id: shell: uid: chef_type: comment: data_bag: gid: home: id: shell: uid: data_bag_item Frank Belson users 0 /home/frank frank /bin/bash 2001 data_bag_item Bobo T. Clown users 0 /home/bobo bobo /bin/bash 2000

Monday, 4 November 13

Exercise: Find Bobos shell in Chef

$ knife search users "comment:\"Bobo T. Clown\"" -a shell

1 items found data_bag_item_users_bobo: shell: /bin/sh

Monday, 4 November 13

Exercise: Create a data bag named groups

$ mkdir data_bags/groups $ knife data_bag create groups

Created data_bag[groups]

Monday, 4 November 13

Exercise: Create a group item in the group data bag

OPEN IN EDITOR: data_bags/groups/clowns.json

{ ! "id": "clowns", ! "gid": 3000, ! "members": ["bobo", "frank"] }

SAVE FILE!
Monday, 4 November 13

Exercise: Create the data bag item

$ knife data_bag from file groups clowns.json

Updated data_bag_item[groups::clowns]

Monday, 4 November 13

Exercise: Show all the groups in Chef

$ knife search groups "*:*"
1 items found chef_type: data_bag: gid: id: members: bobo frank
Monday, 4 November 13

data_bag_item groups 3000 clowns

Exercise: Create a cookbook named users

$ knife cookbook create users

** ** ** **

Creating Creating Creating Creating

cookbook users README for cookbook: users CHANGELOG for cookbook: users metadata for cookbook: users

Monday, 4 November 13

Exercise: Open the default recipe in your editor

OPEN IN EDITOR: cookbooks/users/recipes/default.rb

# # # # # # # #

Cookbook Name:: users Recipe:: default Copyright 2013, YOUR_COMPANY_NAME All rights reserved - Do Not Redistribute

SAVE FILE!
Monday, 4 November 13

Exercise: Open the default recipe in your editor

search(:users, "*:*").each do |user_data| user user_data['id'] do comment user_data['comment'] uid user_data['uid'] gid user_data['gid'] home user_data['home'] shell user_data['shell'] end end include_recipe "users::groups"

We use the same search we just tried with Knife in the recipe Each item is bound to user_data Use the Chef Docs for information about the user resource

Monday, 4 November 13

Exercise: Open the default recipe in your editor

search(:users, "*:*").each do |user_data| user user_data['id'] do comment user_data['comment'] uid user_data['uid'] include_recipe ensures another recipes resources gid user_data['gid'] are complete before we home user_data['home'] continue shell user_data['shell'] Chef will only include each end recipe once end

include_recipe "users::groups"

include_attribute does the same, but for attribute files

Monday, 4 November 13

Best Practice: Use include_recipe and include_attribute liberally

If there is a pre-requisite for your recipe that resides in another recipe (the JVM existing for your Java application, for example) Always use include_recipe to include it specifically, even if you put it in a run list The same goes for include_attribute

Monday, 4 November 13

Exercise: Open the users::group recipe in your editor

OPEN IN EDITOR: cookbooks/users/recipes/groups.rb

search(:groups, "*:*").each do |group_data| group group_data['id'] do gid group_data['gid'] members group_data['members'] end end
SAVE FILE!

This file follows the same pattern as the default users recipe Use the Chef Docs for information about the group resource
Monday, 4 November 13

Exercise: Upload the users cookbook

$ knife cookbook upload users

Uploading users Uploaded 1 cookbook.

[0.1.0]

Monday, 4 November 13

Exercise: Add the users recipe to your test nodes run list
$ knife node run_list add 'recipe[users]'

node1: run_list: recipe[apache] recipe[motd] recipe[users]

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client
* template[/etc/apache2/sites-available/bears] action create (up to date) * execute[a2ensite bears] action run (skipped due to not_if) * directory[/srv/apache/bears] action create (up to date) * template[/srv/apache/bears/index.html] action create (up to date) Recipe: motd::default * template[/etc/motd.tail] action create (up to date) Recipe: users::default * user[frank] action create - create user user[frank] * user[bobo] action create - create user user[bobo]

Monday, 4 November 13

Exercise: Verify the users and groups exist

opscode@node1:~$ cat /etc/passwd

frank:x:2001:0:Frank Belson:/home/frank:/bin/bash bobo:x:2000:0:Bobo T. Clown:/home/bobo:/bin/bash

opscode@node1:~$ cat /etc/group

clowns:x:3000:bobo,frank

Monday, 4 November 13

Lets review real quick..

We just created a centralized user and group repository, from scratch (Thats kind of like what LDAP and Active Directory do, only they are, well, fancier.) Between Data Bags and Node Attribute precedence, Chef provides a plethora of ways to inform the patterns you use to configure your infrastructure

Monday, 4 November 13

Review Questions
What are Data Bags? How are they used? What does the User resource do? What is include_recipe, and why is it useful? How does search work inside a recipe? What other applications do you see for search? How could we have used Data Bags in the refactored Apache recipe? Where would you go to find out more?
Monday, 4 November 13

Roles
Role-based Attributes and Merge Order Precedence

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Explain what Roles are, and how they are used to provide clarity Discuss the Role Ruby DSL Show a Role with Knife Merge order affects the precedence hierarchy Describe nested Roles

Monday, 4 November 13

What is a Role?
So far, weve been just adding recipes directly to a single node But thats not how your infrastructure works - think about how you refer to servers "Its a web server" "Its a database server" "Its a monitoring server"

Monday, 4 November 13

What is a Role?
Roles allow you to conveniently encapsulate the run lists and attributes required for a server to "be" what you already think it is In practice, Roles make it easy to configure many nodes identically without repeating yourself each time

Monday, 4 November 13

Best Practice: Roles live in your chef-repo

Like Data Bags, you have options with how to create a Role The best practice is that all of your Roles live in the roles directory of your chef-repo They can be created via the API and Knife, but its nice to be able to see them evolve in your source control history

Monday, 4 November 13

Exercise: Create the webserver role

OPEN IN EDITOR: roles/webserver.rb

A Role has a: name description run_list

name "webserver" description "Web Server" run_list "recipe[apache]" default_attributes({ "apache" => { "sites" => { "admin" => { "port" => 82 } } } })

SAVE FILE!
Monday, 4 November 13

Exercise: Create the webserver role

OPEN IN EDITOR: roles/webserver.rb

You can set default node attributes within a role.

name "webserver" description "Web Server" run_list "recipe[apache]" default_attributes({ "apache" => { "sites" => { "admin" => { "port" => 82 } } } })

SAVE FILE!
Monday, 4 November 13

Exercise: Create the role

$ knife role from file webserver.rb

Updated Role webserver!

Monday, 4 November 13

Exercise: Show the role with knife

$ knife role show webserver
chef_type: default_attributes: apache: sites: admin: port: 82 description: env_run_lists: json_class: name: override_attributes: run_list: recipe[apache] role

Web Server Chef::Role webserver

Monday, 4 November 13

Exercise: Search for roles with recipe[apache] in their run list

$ knife search role "run_list:recipe\[apache\]"
1 items found chef_type: default_attributes: apache: sites: admin: port: 82 description: env_run_lists: json_class: name: override_attributes: run_list: recipe[apache] role

Web Server Chef::Role webserver

Monday, 4 November 13

Exercise: Replace recipe[apache] with role[webserver] in run list

Click the Nodes tab then select node node1 Click Edit Run List from left navigation bar Drag Apache over from Current Run List to Available Recipes Drag webserver over from Available Roles to the top of Current Run List Click Save Run List

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client

INFO: *** Chef 11.4.4 *** INFO: Run List is [role[webserver], recipe[motd], recipe[users]] INFO: Run List expands to [apache, motd, users]

Monday, 4 November 13

Exercise: Re-run the Chef Client

INFO: Processing template[/etc/apache2/sites-available/clowns] action create (apache::default line 28) INFO: Processing execute[a2ensite clowns] action run (apache::default line 38) INFO: Processing directory[/srv/apache/clowns] action create (apache::default line 45) INFO: Processing template[/srv/apache/clowns/index.html] action create (apache::default line 50) INFO: Processing template[/etc/apache2/sites-available/bears] action create (apache::default line 28) INFO: Processing execute[a2ensite bears] action run (apache::default line 38) INFO: Processing directory[/srv/apache/bears] action create (apache::default line 45) INFO: Processing template[/srv/apache/bears/index.html] action create (apache::default line 50) INFO: Processing template[/etc/apache2/sites-available/admin] action create (apache::default line 28) INFO: template[/etc/apache2/sites-available/admin] updated content INFO: template[/etc/apache2/sites-available/admin] mode changed to 644 INFO: Processing execute[a2ensite admin] action run (apache::default line 38) Enabling site admin. To activate the new configuration, you need to run: service apache2 reload INFO: execute[a2ensite admin] ran successfully INFO: execute[a2ensite admin] not queuing delayed action restart on service[apache2] (delayed), as it's already been queued INFO: Processing directory[/srv/apache/admin] action create (apache::default line 45) INFO: directory[/srv/apache/admin] created directory /srv/apache/admin INFO: directory[/srv/apache/admin] mode changed to 755 INFO: Processing template[/srv/apache/admin/index.html] action create (apache::default line 50) INFO: template[/srv/apache/admin/index.html] updated content INFO: template[/srv/apache/admin/index.html] mode changed to 644

Monday, 4 November 13

Node Attributes that are hashes are merged

The apache cookbooks attribute file contains:
default['apache']['sites']['clowns'] = { "port" => 80 } default['apache']['sites']['bears'] = { "port" => 81 }
default_attributes({ "apache" => { "sites" => { "admin" => { "port" => 82 } } } })

While our role has...

Monday, 4 November 13

Exercise: Display the apache.sites attribute on all nodes with webserver role
$ knife search node "role:webserver" -a apache.sites

1 items found node1: apache.sites: admin: port: 82 bears: port: 81 clowns: port: 80
Monday, 4 November 13

Exercise: Edit the webserver role

OPEN IN EDITOR: roles/webserver.rb

Do not forget the comma after the admin site Change the value of the bears site to be 8081

default_attributes({ "apache" => { "sites" => { "admin" => { "port" => 82 }, "bears" => { "port" => 8081 } } } })

SAVE FILE!
Monday, 4 November 13

Exercise: Create the role

$ knife role from file webserver.rb

Updated Role webserver!

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client
[2012-10-23T03:28:34+00:00] INFO: Processing template[/etc/apache2/sites-available/bears] action create (apache::default line 28) [2012-10-23T03:28:34+00:00] INFO: template[/etc/apache2/sites-available/bears] backed up to / var/chef/backup/etc/apache2/sites-available/bears.chef-20121023032834 [2012-10-23T03:28:34+00:00] INFO: template[/etc/apache2/sites-available/bears] updated content [2012-10-23T03:28:34+00:00] INFO: template[/etc/apache2/sites-available/bears] owner changed to 0 [2012-10-23T03:28:34+00:00] INFO: template[/etc/apache2/sites-available/bears] group changed to 0 [2012-10-23T03:28:34+00:00] INFO: template[/etc/apache2/sites-available/bears] mode changed to 644 [2012-10-23T03:28:34+00:00] INFO: Processing execute[a2ensite bears] action run (apache::default line 38) [2012-10-23T03:28:34+00:00] INFO: Processing directory[/srv/apache/bears] action create (apache::default line 45)

Monday, 4 November 13

Exercise: Display the apache sites attribute on all nodes with the webserver role
$ knife search node 'role:webserver' -a apache.sites

1 items found node1: apache.sites: admin: port: 82 bears: port: 8081 clowns: port: 80
Monday, 4 November 13

When you combine merge order and precedence rules, you get this:
Monday, 4 November 13

Merge Order and Precedence

Monday, 4 November 13

Best Practice: Roles get default attributes

While it is awesome that you can use overrides, in practice there is little need If you always set default node attributes in your cookbook attribute files You can almost always set default node attributes in your role, and let merge order do the rest

Monday, 4 November 13

Best Practice: Have "base" roles

In addition to obvious roles, such as "webserver", it is a common practice to group any functionality that "goes together" in a role The most common example here is a base role, where you include all the recipes that should be run on every node

Monday, 4 November 13

Exercise: Create the base role

OPEN IN EDITOR: roles/base.rb

name "base" description "Base Server Role" run_list "recipe[motd]", "recipe[users]"

SAVE FILE!

Monday, 4 November 13

Exercise: Create the role

$ knife role from file base.rb

Updated Role base!

Monday, 4 November 13

Exercise: Add the base role to the webserver roles run list
OPEN IN EDITOR: roles/webserver.rb

name "webserver" description "Web Server" run_list "role[base]", "recipe[apache]" default_attributes({ "apache" => {

Put role[base] at the front of the run_list

SAVE FILE!
Monday, 4 November 13

Exercise: Update the role

$ knife role from file webserver.rb

Updated Role webserver!

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client

INFO: *** Chef 11.4.4 *** INFO: Run List is [role[webserver], recipe[motd], recipe[users]] INFO: Run List expands to [motd, users, apache]
Monday, 4 November 13

Best Practice: Be explicit about what you need or expect

Chef will only execute a recipe the first time it appears in the run list So be explicit about your needs and expectations either by nesting roles or using include_recipe

Monday, 4 November 13

Exercise: Set the run list to just role[webserver]

Remove all the entries in the run list other than role[webserver]

Monday, 4 November 13

Review Questions
What is a Role? What makes for a "good" role? How do you search for roles with a given recipe in their run list? How many times will Chef execute a recipe in the same run?

Monday, 4 November 13

Environments
Cookbook Version Constraints and Override Attributes

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Describe what an Environment is, and how it is different from an Organization Set cookbook version constraints Explain when to set attributes in an environment

Monday, 4 November 13

Environments
Development Staging Production

Organization
Monday, 4 November 13

Environments
Every Organization starts with a single environment Environments reflect your patterns and workflow Development Test Staging Production etc.

Monday, 4 November 13

Environments Define Policy

Each environment may include attributes necessary for configuring the infrastructure in that environment Production needs certain Yum repos QA needs different Yum repos The version of the Chef cookbooks to be used

Monday, 4 November 13

Environment Best Practice

Best Practice: If you need to share cookbooks or roles, you likely want an Environment rather than an organization Environments allow for isolating resources within a single organization

Monday, 4 November 13

Exercise: Use knife to show the available cookbook versions

$ knife cookbook show apache

apache

0.2.0

0.1.0

Monday, 4 November 13

Exercise: List current environments

$ knife environment list

_default

The _default environment is read-only, and sets no policy at all

Monday, 4 November 13

Make an environments directory

$ mkdir environments

(No output)

Monday, 4 November 13

Exercise: Create a dev environment

OPEN IN EDITOR: environments/dev.rb

name "dev" description "For developers!" cookbook "apache", "= 0.2.0"

SAVE FILE!

Environments have names Environments have a description Environments can have one or more cookbook constraints
Monday, 4 November 13

Cookbook Version Constraints

= Equal to There are other options but equality is the recommended practice. Learn more at http://docs.opscode.com/chef/ essentials_cookbook_versions.html

Monday, 4 November 13

Exercise: Create the dev environment

$ knife environment from file dev.rb

Updated Environment dev

Monday, 4 November 13

Exercise: Show your dev environment

$ knife environment show dev
chef_type: cookbook_versions: apache: = 0.2.0 default_attributes: description: json_class: name: override_attributes:
Monday, 4 November 13

environment

{} For developers! Chef::Environment dev {}

Exercise: Change your nodes environment to "dev"

Click the Nodes tab then select node node1 Select dev from the Environments drop-down list Click Save

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client

INFO: Chef Run complete in 1.587776095 seconds INFO: Running report handlers INFO: Report handlers complete

Monday, 4 November 13

Exercise: Create a production environment

OPEN IN EDITOR: environments/production.rb

Make sure the apache cookbook is set to version 0.1.0 Set an override attribute for being in scope - no matter what, you are in scope

name "production" description "For Prods!" cookbook "apache", "= 0.1.0" override_attributes({ "pci" => { "in_scope" => true } })

SAVE FILE!
Monday, 4 November 13

Exercise: Create the production environment

$ knife environment from file production.rb

Updated Environment production

Monday, 4 November 13

Exercise: Change your nodes environment to "production"

Click the Nodes tab then select node node1 Select production from the Environments drop-down list Click Save

Monday, 4 November 13

Exercise: Re-run the Chef Client

opscode@node1$ sudo chef-client
INFO: Loading cookbooks [apache, motd, pci, users] Synchronizing Cookbooks: ... Recipe: motd::default * template[/etc/motd.tail] action create INFO: Processing template[/etc/motd.tail] action create (motd::default line 10) ... * cookbook_file[/var/www/index.html] action create INFO: Processing cookbook_file[/var/www/index.html] action create (apache::default line 17) ... INFO: Chef Run complete in 1.903297305 seconds INFO: Removing cookbooks/apache/templates/default/custom.erb from the cache; it is no longer needed by chefclient. INFO: Removing cookbooks/apache/templates/default/index.html.erb from the cache; it is no longer needed by chef-client. INFO: Removing cookbooks/apache/attributes/default.rb from the cache; it is no longer needed by chef-client. INFO: Running report handlers INFO: Report handlers complete Chef Client finished, 2 resources updated

Monday, 4 November 13

Rollbacks and Desired State Best Practice

Chef is not magic - it manages state for declared resources We just rolled back to an earlier version of the apache cookbook While the recipe applied fine, investigating the system will reveal Apache is still configured as it was in the 0.2.0 cookbook A better way to ensure a smooth rollback: write contraresources to clean up, and have a new version of the cookbook.
Monday, 4 November 13

Review Questions
What is an Environment? How is it different from an Organization? What kind of node attributes do you typically set from an Environment?

Monday, 4 November 13

Using Community Cookbooks

Open Source: Saving you time!

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to Use the Opscode Chef Community site to find, preview and download cookbooks Use knife to work with the Community Site API Download, extract, examine and implement cookbooks from the Community site

Monday, 4 November 13

The easy way...

We've been writing some cookbooks so far... Hundreds already exist for a large number of use cases and purposes. Many (but only a fraction) are maintained by Opscode. Think of it like RubyGems.org, CPAN.org, or other focused plugin-style distribution sites.

Monday, 4 November 13

Exercise: Find and preview cookbooks on the community site

Cookbooks!

Monday, 4 November 13

Exercise: Search for a chef-client cookbook

Search for: chef-client

Monday, 4 November 13

Search Results...

Were probably looking for this one

Monday, 4 November 13

Viewing a cookbook

Browse Source Code

README displayed on the page (if it has one)

Monday, 4 November 13

You can download cookbooks directly from the site...

You can download cookbooks directly from the community site, but: It doesn't put them in your Chef Repository It isn't fast if you know what you're looking for (click, click...) It isn't necessarily fast if you don't know what you're looking for. You're already using knife for managing cookbooks and other things in your Chef Repository.
Monday, 4 November 13

Introducing Knife Cookbook Site plugin

Knife includes a "cookbook site" plugin with some sub-commands: search show download ... and more!

Monday, 4 November 13

Download and use chef-client cookbook

v1.1.3

Monday, 4 November 13

Exercise: Use knife to search the community site

$ knife cookbook site search chef-client
chef: cookbook: cookbook_description: cookbook_maintainer: cookbook_name: chef-client: cookbook: cookbook_description: cookbook_maintainer: cookbook_name: chef-client-cron: cookbook: cookbook_description: cookbook_maintainer: cookbook_name: http://cookbooks.opscode.com/api/v1/cookbooks/chef Installs and configures Chef for chef-client and chef-server opscode chef http://cookbooks.opscode.com/api/v1/cookbooks/chef-client Manages client.rb configuration and chef-client service opscode chef-client http://cookbooks.opscode.com/api/v1/cookbooks/chef-client-cron Manages aspects of only chef-client bryanwb chef-client-cron

Monday, 4 November 13

Exercise: Use knife to show the chef-client cookbook on the community site

$ knife cookbook site show chef-client

average_rating: 4.85714 category: Utilities created_at: 2010-12-16T23:00:45Z description: Manages client.rb configuration and chef-client service external_url: github.com/opscode-cookbooks/chef-client latest_version: http://cookbooks.opscode.com/api/v1/cookbooks/chef-client/versions/ 3_1_0 maintainer: opscode name: chef-client updated_at: 2013-10-01T03:14:41Z versions: http://cookbooks.opscode.com/api/v1/cookbooks/chef-client/versions/3_1_0 http://cookbooks.opscode.com/api/v1/cookbooks/chef-client/versions/3_0_6 http://cookbooks.opscode.com/api/v1/cookbooks/chef-client/versions/3_0_4

Monday, 4 November 13

Exercise: Download the chef-client cookbook

$ knife cookbook site download chef-client
Downloading chef-client from the cookbooks site at version 3.1.0 to /Users/YOU/chefrepo/chef-client-3.1.0.tar.gz Cookbook saved: /Users/YOU/chef-repo/chefclient-3.1.0.tar.gz
Monday, 4 November 13

Exercise: Extract chef-client cookbook tarball

$ tar -zxvf chef-client*.tar.gz -C cookbooks/
x x x x x x x x x x x x x x x x x x x x x x chef-client/ chef-client/attributes/ chef-client/CHANGELOG.md chef-client/CONTRIBUTING chef-client/LICENSE chef-client/metadata.json chef-client/metadata.rb chef-client/README.md chef-client/recipes/ chef-client/templates/ chef-client/templates/arch/ chef-client/templates/default/ chef-client/templates/windows/ chef-client/templates/default/debian/ chef-client/templates/default/redhat/ chef-client/templates/default/solaris/ chef-client/templates/arch/conf.d/ chef-client/templates/arch/rc.d/ chef-client/recipes/config.rb chef-client/recipes/cron.rb chef-client/recipes/default.rb chef-client/recipes/delete_validation.rb

Monday, 4 November 13

What we just did...

Cookbooks are distributed as a versioned .tar.gz archive. The latest version is downloaded by default (you can specify the version). Extract the cookbook into the "cookbooks" directory with tar. Next, let's examine the contents.

Monday, 4 November 13

Best practice: well written cookbooks have a README!

Documentation for cookbooks doesn't need to be extensive, but a README should describe some important aspects of a cookbook: Expectations (cookbooks, platform, data) Recipes and their purpose LWRPs, Libraries, etc. Usage notes Read the README first!
Monday, 4 November 13

Best Practice: This runs as root!

So, you just downloaded source code from the internet. As root. To load in the magic machine that: Makes your computers run code Read the entire cookbook first!

Monday, 4 November 13

Examining the chef-client cookbook

We're going to use two recipes on the node from the chef-client cookbook. delete_validation service (via default)

Monday, 4 November 13

Exercise: View the chef-client::delete_validation recipe

OPEN IN EDITOR: cookbooks/chef-client/recipes/delete_validation.rb
unless chef_server? file Chef::Config[:validation_key] do action :delete backup false only_if { ::File.exists?(Chef::Config[:client_key]) } end end

SAVE FILE!
Monday, 4 November 13

Exercise: Add chef-client::delete_validation to your base role

OPEN IN EDITOR: roles/base.rb
name "base" description "Base Server Role" run_list "recipe[chef-client::delete_validation]", "recipe[motd]", "recipe[users]"

SAVE FILE!

Add the delete_validation recipe

Monday, 4 November 13

Best Practice: Delete the validation certificate when it isn't required

Once Chef enters the actual run, synchronizing cookbooks, it has register its own API client with the validation certificate. That certificate is no longer required. We do this first because in case the run fails for another reason, we know at least the validation certificate is gone.

Monday, 4 November 13

Exercise: View the chef-client::default recipe

OPEN IN EDITOR: cookbooks/chef-client/recipes/default.rb
# # # # # # # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

include_recipe "chef-client::service"

SAVE FILE!
Monday, 4 November 13

Best Practice: Sane defaults do "pretty much" what you expect

The main point of the "chef-client" cookbook is managing the "chef-client" program. It is designed that it can run as a daemonized service. The least surprising thing for most users is that the default recipe starts the service. You can manage the service in a number of ways, see the cookbook's README.md.

Monday, 4 November 13

Exercise: View the chef-client::service recipe

OPEN IN EDITOR: cookbooks/chef-client/recipes/service.rb

The recipe supports a number of service providers and styles. It works on a lot of platforms. Everything is controllable through attributes.
Monday, 4 November 13

supported_init_styles = [ 'arch', 'bluepill', 'bsd', 'daemontools', 'init', 'launchd', 'runit', 'smf', 'upstart', 'winsw' ] init_style = node["chef_client"]["init_style"] # Services moved to recipes if supported_init_styles.include? init_style include_recipe "chef-client::#{init_style}_service" else log "Could not determine service init style, manual intervention required to start up the chef-client service." end

Best Practice: Well-written cookbooks change behavior based on attributes

Ideally, you don't have to modify the contents of a cookbook to use it for your specific use case. Look at the attributes directory for things you can override through roles to affect behavior of the cookbook. Of course, well written cookbooks have sane defaults, and a README to describe all this.

Monday, 4 November 13

Exercise: Upload the chef-client cookbook

$ knife cookbook upload chef-client
Uploading chef-client [3.1.0] ERROR: Cookbook chef-client depends on cookbook 'cron' version '>= 1.2.0', ERROR: which is not currently being uploaded and cannot be found on the server.

Monday, 4 November 13

Exercise: Download the cron cookbook

$ knife cookbook site download cron

Downloading cron from the cookbooks site at version 1.2.8 to /Users/YOU/chef-repo/cron-1.2.8.tar.gz Cookbook saved: /Users/YOU/chef-repo/ cron-1.2.8.tar.gz

Monday, 4 November 13

Exercise: Extract cron cookbook tarball

$ tar -zxvf cron*.tar.gz -C cookbooks/
x x x x x x x x x x x x x x x cron/ cron/CHANGELOG.md cron/README.md cron/metadata.json cron/metadata.rb cron/providers cron/providers/d.rb cron/recipes cron/recipes/default.rb cron/recipes/test.rb cron/resources cron/resources/d.rb cron/templates cron/templates/default cron/templates/default/cron.d.erb

Monday, 4 November 13

Exercise: Upload the cron cookbook

$ knife cookbook upload cron

Uploading cron Uploaded 1 cookbook.

[1.2.8]

Monday, 4 November 13

Exercise: Upload the chef-client cookbook

$ knife cookbook upload chef-client
Uploading chef-client [3.1.0] ERROR: Cookbook chef-client depends on cookbook 'logrotate' version '>= 1.2.0', ERROR: which is not currently being uploaded and cannot be found on the server.
Monday, 4 November 13

Exercise: Download the logrotate cookbook

$ knife cookbook site download logrotate

Downloading logrotate from the cookbooks site at version 1.4.0 to /Users/johnfitzpatrick/ cheftraining/chef-repo/logrotate-1.4.0.tar.gz Cookbook saved: /Users/YOU/chef-repo/ logrotate-1.4.0.tar.gz

Monday, 4 November 13

Exercise: Extract logrotate cookbook tarball

$ tar -zxvf logrotate*.tar.gz -C cookbooks/
x x x x x x x x x x x x x x x x x x logrotate/ logrotate/CHANGELOG.md logrotate/README.md logrotate/attributes logrotate/attributes/default.rb logrotate/definitions logrotate/definitions/logrotate_app.rb logrotate/libraries logrotate/libraries/logrotate_config.rb logrotate/metadata.json logrotate/metadata.rb logrotate/recipes logrotate/recipes/default.rb logrotate/recipes/global.rb logrotate/templates logrotate/templates/default logrotate/templates/default/logrotate-global.erb logrotate/templates/default/logrotate.erb

Monday, 4 November 13

Exercise: Upload the logrotate cookbook

$ knife cookbook upload logrotate

Uploading logrotate Uploaded 1 cookbook.

[1.4.0]

Monday, 4 November 13

Exercise: Upload the chef-client cookbook

$ knife cookbook upload chef-client

Uploading chef-client Uploaded 1 cookbook.

[3.1.0]

Monday, 4 November 13

Exercise: Add chef-client recipe to base role

OPEN IN EDITOR: roles/base.rb
name "base" description "Base Server Role" run_list "recipe[chef-client::delete_validation]", "recipe[chef-client]", "recipe[motd]", "recipe[users]"

SAVE FILE!

Monday, 4 November 13

Exercise: Upload the base role

$ knife role from file base.rb

Updated Role base!

Monday, 4 November 13

Exercise: Re-run the Chef Client

... Recipe: chef-client::delete_validation * file[/etc/chef/validation.pem] action delete INFO: Processing file[/etc/chef/validation.pem] action delete (chef-client::delete_validation line 25) INFO: file[/etc/chef/validation.pem] deleted file at /etc/chef/validation.pem - delete file /etc/chef/validation.pem ... * service[chef-client] action enable INFO: Processing service[chef-client] action enable (chef-client::init_service line 31) INFO: service[chef-client] enabled - enable service service[chef-client] * service[chef-client] action start INFO: Processing service[chef-client] action start (chef-client::init_service line 31) INFO: service[chef-client] started - start service service[chef-client] ... INFO: template[/etc/init.d/chef-client] sending restart action to service[chef-client] (delayed) Recipe: chef-client::init_service * service[chef-client] action restart INFO: Processing service[chef-client] action restart (chef-client::init_service line 31) INFO: service[chef-client] restarted - restart service service[chef-client]

Monday, 4 November 13

Exercise: Verify chef-client is running

opscode@node1$ ps awux | grep chef-client

root 8933 0.3 2.2 130400 37816 ? Sl 03:19 0:01 /opt/chef/embedded/bin/ruby /usr/bin/chef-client -d -P / var/run/chef/client.pid -c /etc/chef/client.rb -i 1800 -s 300 -L /var/log/chef/client.log

Monday, 4 November 13

Convergent infrastructure
Our node is now running chef-client as a daemon, and it will converge itself over time on a (by default) 30 minute interval. The amount of resources converged may vary with longer intervals, depending on configuration drift on the system. Because Chef resources are idempotent, it will only configure what it needs to each run.
Monday, 4 November 13

Download and use ntp cookbook

v1.1.3

Monday, 4 November 13

NTP Cookbook
Network time protocol - keeps system clocks in sync Chef Server authentication is time sensitive!

Monday, 4 November 13

Exercise: Download the ntp cookbook

$ knife cookbook site download ntp Downloading ntp from the cookbooks site at version 1.5.0 to /Users/YOU/ chef-repo/ntp-1.5.0.tar.gz Cookbook saved: /Users/YOU/chef-repo/ ntp-1.5.0.tar.gz
Monday, 4 November 13

Exercise: Extract the ntp cookbook

$ tar -zxvf ntp*.tar.gz -C cookbooks/
x x x x x x x x x x x x x x x x x x x ntp/ ntp/CHANGELOG.md ntp/README.md ntp/attributes ntp/attributes/default.rb ntp/files ntp/files/default ntp/files/default/ntp.ini ntp/files/default/ntp.leapseconds ntp/files/default/tests ntp/files/default/tests/minitest ntp/files/default/tests/minitest/default_test.rb ntp/files/default/tests/minitest/support ntp/files/default/tests/minitest/support/helpers.rb ntp/files/default/tests/minitest/undo_test.rb ntp/metadata.json ntp/metadata.rb ntp/recipes ntp/recipes/default.rb

Monday, 4 November 13

Examining the ntp cookbook

The cookbook is quite flexible, but for this exercise we're just interested in the most basic use, an NTP client. default recipe

Monday, 4 November 13

Exercise: View the ntp::default recipe

node['ntp']['packages'].each do |ntppkg| package ntppkg end
service node['ntp']['service'] do supports :status => true, :restart => true action [:enable, :start] end

template node['ntp']['conffile'] do source 'ntp.conf.erb' owner node['ntp']['conf_owner'] group node['ntp']['conf_group'] mode '0644' notifies :restart, "service[#{node['ntp']['service']}]" end
Monday, 4 November 13

All packages are installed The service is enabled and started The template notifies the service

Exercise: Upload the ntp cookbook

$ knife cookbook upload ntp

Uploading ntp Uploaded 1 cookbook.

[1.5.0]

Monday, 4 November 13

Exercise: Add the ntp recipe to the base role

OPEN IN EDITOR: roles/base.rb
name "base" description "Base Server Role" run_list "recipe[chef-client::delete_validation]", "recipe[chef-client]", "recipe[ntp]", "recipe[motd]", "recipe[users]"

SAVE FILE!

Monday, 4 November 13

Exercise: Upload the base role

$ knife role from file base.rb

Updated Role base!

Monday, 4 November 13

Exercise: Re-run the Chef Client

Recipe: ntp::default * package[ntp] action install INFO: Processing package[ntp] action install (ntp::default line 21) - install version 1:4.2.6.p3+dfsg-1ubuntu3.1 of package ntp * package[ntpdate] action install INFO: Processing package[ntpdate] action install (ntp::default line 21) (up to date) ... * template[/etc/ntp.conf] action create INFO: Processing template[/etc/ntp.conf] action create (ntp::default line 44) INFO: template[/etc/ntp.conf] backed up to /var/chef/backup/etc/ntp.conf.chef-20130702034747 INFO: template[/etc/ntp.conf] updated content INFO: template[/etc/ntp.conf] owner changed to 0 INFO: template[/etc/ntp.conf] group changed to 0 INFO: template[/etc/ntp.conf] mode changed to 644 - update template[/etc/ntp.conf] from 4eb9a0 to 7805aa ... INFO: template[/etc/ntp.conf] sending restart action to service[ntp] (delayed) Recipe: ntp::default * service[ntp] action restart INFO: Processing service[ntp] action restart (ntp::default line 33) INFO: service[ntp] restarted - restart service service[ntp]

Monday, 4 November 13

Review Questions
What is the Chef Community site URL? What are two ways to download cookbooks from the community site? What is the first thing you should read when downloading a cookbook? Who vets the cookbooks on the community site? Who has two thumbs and reads the recipes they download from the community site?
Monday, 4 November 13

Further Resources

v1.1.3

Monday, 4 November 13

Further Resources: Cookbooks and Plugins

Useful cookbooks DNS: djbdns, pdns, dnsimple, dynect, route53 Monitoring: nagios, munin, zenoss, zabbix Package repos: yum, apt, freebsd Security: ossec, snort, cis_benchmark Logging: rsyslog, syslog-ng, logstash, logwatch

Application cookbooks: application, database python, java, php, ruby Plugins Cloud: knife-ec2, kniferackspace, knife-openstack, knife-hp Windows: knife-windows More listed on docs.opscode.com

Monday, 4 November 13

Further Resources
http://opscode.com/ http://community.opscode.com/ http://docs.opscode.com/ http://learnchef.com http://lists.opscode.com http://youtube.com/user/Opscode irc.freenode.net #chef, #chef-hacking, #learnchef Twitter #opschef
Monday, 4 November 13

Food Fight Show

http://foodfightshow.org The Podcast Where DevOps Chef Do Battle Regular updates about new Cookbooks, Knifeplugins, and more Best Practices for working with Chef

Monday, 4 November 13

Opscode Community Summit 2013

November 12-13, 2013 - Hyatt Olive 8, Seattle, WA Sorry its during AWS re:Invent :-( Community "un-conference" style Topic ideas are being proposed on the Opscode Community Wiki (wiki.opscode.com) Register: https://www.regonline.com/opscodesummit-2013

Monday, 4 November 13

Local Meetup Groups

Fill in local meetup groups here

Monday, 4 November 13

Appendix: Just Enough Ruby for Chef

A quick & dirty crash course

v1.1.3

Monday, 4 November 13

Lesson Objectives
After completing the lesson, you will be able to
Explain what irb is, and how to use it. Describe the function of: Variable Assignment Regular Expressions Basic Arithmetic Conditionals Strings Method Declaration Truthiness Classes Operators Objects Hashes
Monday, 4 November 13

irb - the interactive ruby shell

irb is the interactive ruby shell It is a REPL for Ruby Read-Eval-Print-Loop LISP, Python, Erlang, Clojure, etc. An interactive programming environment Super-handy for trying things out

Monday, 4 November 13

Exercise: Start irb on your target node

opscode@node1$ /opt/chef/embedded/bin/irb

irb(main):001:0>

Monday, 4 November 13

Variable assignment
opscode@node1$ /opt/chef/embedded/bin/irb

irb(main):001:0> x = "hello" => "hello" irb(main):002:0> puts x hello => nil

Monday, 4 November 13

Arithmetic

irb(main):003:0> 1 + 2 => 3

Monday, 4 November 13

Arithmetic

irb(main):004:0> 18 - 5 => 13

Monday, 4 November 13

Arithmetic

irb(main):005:0> 2 * 7 => 14

Monday, 4 November 13

Arithmetic

irb(main):006:0> 5 / 2 => 2

Monday, 4 November 13

Arithmetic

irb(main):007:0> 5 / 2.0 => 2.5

Monday, 4 November 13

Arithmetic
irb(main):008:0> 5.class => Fixnum irb(main):009:0> 5.0.class => Float

Monday, 4 November 13

Arithmetic

irb(main):010:0> 1 + (2 * 3) => 7

Monday, 4 November 13

Strings

irb(main):011:0> 'jungle' => "jungle"

Monday, 4 November 13

Strings

irb(main):012:0> 'it\'s alive' => "it's alive"

Monday, 4 November 13

Strings

irb(main):013:0> "animal" => "animal"

Monday, 4 November 13

Strings
irb(main):014:0> "\"so easy\"" => "\"so easy\"" irb(main):015:0> puts "\"so easy\"" "so easy" => nil

Monday, 4 November 13

Strings
irb(main):016:0> x = "pretty" => "pretty" irb(main):017:0> "#{x} nice" => "pretty nice" irb(main):018:0> '#{x} nice' => "\#{x} nice"
Monday, 4 November 13

Truthiness
irb(main):019:0> true => true irb(main):020:0> false => false irb(main):021:0> nil => nil irb(main):022:0> !!nil => false irb(main):023:0> !!0 => true irb(main):024:0> !!x => true

Monday, 4 November 13

Operators
irb(main):025:0> 1 == 1 => true irb(main):026:0> 1 == true => false irb(main):027:0> 1 != true => true irb(main):028:0> !!1 == true => true
Monday, 4 November 13

Operators
irb(main):029:0> 2 < 1 => false irb(main):030:0> 2 > 1 => true irb(main):031:0> 4 >= 3 => true irb(main):032:0> 4 >= 4 => true irb(main):033:0> 4 <= 5 => true irb(main):034:0> 4 <= 3 => false

Monday, 4 November 13

Operators
irb(main):035:0> 5 <=> 5 => 0 irb(main):036:0> 5 <=> 6 => -1 irb(main):037:0> 5 <=> 4 => 1
Monday, 4 November 13

Arrays
irb(main):038:0> x = ["a", "b", "c"] => ["a", "b", "c"] irb(main):039:0> x[0] => "a" irb(main):040:0> x.first => "a" irb(main):041:0> x.last => "c"
Monday, 4 November 13

Arrays
irb(main):042:0> x + ["d"] => ["a", "b", "c", "d"] irb(main):043:0> x => ["a", "b", "c"] irb(main):044:0> x = x + ["d"] => ["a", "b", "c", "d"] irb(main):045:0> x => ["a", "b", "c", "d"]
Monday, 4 November 13

Arrays
irb(main):046:0> x << "e" => ["a", "b", "c", "d", "e"] irb(main):047:0> x => ["a", "b", "c", "d", "e"]

Monday, 4 November 13

Arrays
irb(main):048:0> x.map { |i| "the letter #{i}" } => ["the letter a", "the letter b", "the letter c", "the letter d", "the letter e"] irb(main):049:0> x => ["a", "b", "c", "d", "e"]

Monday, 4 November 13

Arrays
irb(main):050:0> x.map! { |i| "the letter #{i}" } => ["the letter a", "the letter b", "the letter c", "the letter d", "the letter e"] irb(main):051:0> x => ["the letter a", "the letter b", "the letter c", "the letter d", "the letter e"]

Monday, 4 November 13

Hashes
irb(main):052:0> h = { irb(main):053:1* "first_name" => "Gary", irb(main):054:1* "last_name" => "Gygax" irb(main):055:1> } => {"first_name"=>"Gary", "last_name"=>"Gygax"}

Monday, 4 November 13

Hashes
irb(main):056:0> h.keys => ["first_name", "last_name"] irb(main):057:0> h["first_name"] => "Gary" irb(main):058:0> h["age"] = 33 => 33 irb(main):059:0> h.keys => ["first_name", "last_name", "age"]
Monday, 4 November 13

Hashes

irb(main):060:0> h.values => ["Gary", "Gygax", 33]

Monday, 4 November 13

Hashes
irb(main):061:0> h.each { |k, v| puts "#{k}: #{v}" } first_name: Gary last_name: Gygax age: 33 => {"first_name"=>"Gary", "last_name"=>"Gygax", "age"=>33}

Monday, 4 November 13

Regular Expressions
irb(main):062:0> x = "I want to believe" => "I want to believe" irb(main):063:0> x =~ /I/ => 0 irb(main):064:0> x =~ /lie/ => 12 irb(main):065:0> x =~ /smile/ => nil irb(main):066:0> x !~ /smile/ => true

Monday, 4 November 13

Regular Expressions
irb(main):067:0> x.sub(/t/, "T") => "I wanT to believe" irb(main):068:0> puts x I want to believe => nil irb(main):069:0> x.gsub!(/t/, "T") => "I wanT To believe" irb(main):070:0> puts x I wanT To believe => nil

Monday, 4 November 13

Conditionals
irb(main):071:0> x = "happy" => "happy" irb(main):072:0> irb(main):073:1> irb(main):074:1> irb(main):075:1> irb(main):076:1> irb(main):077:1* irb(main):078:1> Sure am! => nil if x == "happy" puts "Sure am!" elsif x == "sad" puts "Boo!" else puts "Therapy?" end

Monday, 4 November 13

Conditionals
irb(main):079:0> irb(main):080:1> irb(main):081:1> irb(main):082:1> irb(main):083:1> irb(main):085:1> irb(main):086:1> irb(main):087:1> irb(main):088:1> irb(main):089:1> irb(main):090:1> Sure Am! => 1 case x when "happy" puts "Sure Am!" 1 when "sad" puts "Boo!" 2 else puts "Therapy?" 3 end

Monday, 4 November 13

Method Definition
irb(main):091:0> def metal(str) irb(main):092:1> puts "!!#{str} is metal!!" irb(main):093:1> end => nil irb(main):094:0> metal("ozzy") !!ozzy is metal!! => nil

Monday, 4 November 13

Classes
irb(main):095:0> class Person irb(main):096:1> attr_accessor :name, :is_metal irb(main):097:1> irb(main):098:1> def metal irb(main):099:2> if @is_metal irb(main):100:3> puts "!!#{@name} is metal!!" irb(main):101:3> end irb(main):102:2> end irb(main):103:1> end => nil

Monday, 4 November 13

Classes
irb(main):104:0> p = Person.new => #<Person:0x891ab4c> irb(main):105:0> p.name = "Adam Jacob" => "Adam Jacob" irb(main):106:0> p.is_metal = true => true irb(main):107:0> p.metal !!Adam Jacob is metal!! => nil irb(main):108:0> p.is_metal = false => false irb(main):109:0> p.metal => nil

Monday, 4 November 13

Review Questions
What is irb? What is true in ruby? What is false? How do you press for the truth? What does >= do? How do you define a method? What is a class? What is an object? The book you want: "Programming Ruby 1.9" http:// pragprog.com/book/ruby3/programming-ruby-1-9

Monday, 4 November 13

v1.1.3

Monday, 4 November 13