Chef and Solaris
Stephen Nelson-Smith
[email protected]
The Message
The Messanger
Solaris System Administrator since 1996
Ruby Programmer
Managed Solaris-based internet infrastructures
for some of the largest UK and US companies
(Diageo, British Gas, Motorola, Novartis)
Founded Atalanta Systems 6 years ago, still
Principal Consultant and non-executive
director
Stephen Nelson-Smith
@LordCope
[email protected]
[email protected]
Joined Opscode March 2012 to enhance Solaris
support
Overview of Chef
What is this thing again?
Applications
http://www.flickr.com/photos/steffenz/337700069/
http://www.flickr.com/photos/kky/704056791/
Infrastructure
http://www.flickr.com/photos/sbh/462754460/
Nodes
Networking
Files
Directories
Symlinks
Mounts
Collection of Resources
Routes
Users
Groups
Tasks
Packages
Software
Services
Configurations
Stuff
http://www.flickr.com/photos/stevekeys/3123167585/
Acting in concert
http://www.flickr.com/photos/glowjangles/4081048126/
To provide a Service
http://www.flickr.com/photos/28309157@N08/3743455858/
And it evolves
http://www.flickr.com/photos/16339684@N00/2681435235/
See Node
Application
See Nodes
Application
Application Database
See Nodes Grow
Application
App Databases
See Nodes Grow
App Servers
App Databases
See Nodes Grow
App LB
App Servers
App Databases
See Nodes Grow
App LBs
App Servers
App Databases
See Nodes Grow
App LBs
App Servers
App DB Cache
App DBs
Stitched together with configs
App LBs
App Servers
App DB Cache
App DBs
Your Infrastructure is a snow flake
App LB
App Servers
App DB Cache
Floating IP?
App DBs
Complexity increases quickly
App LBs
Cache
App Servers
DB Cache
NoSQL
DB slaves
DBs
Complexity increases very quickly
DC2
DC1
DC3
Configuration Management
http://www.flickr.com/photos/philliecasablanca/3354734116/
Golden Images are not the answer
Gold is heavy
Hard to transport
Hard to mold
Easy to lose
configuration detail
http://www.flickr.com/photos/garysoup/2977173063/
Typical Boring Infrastructure
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master
New Compliance Mandate
Graphite
Nagios
Jboss App
Move SSH off port 22
Lets put it on 2022
Memcache
Postgres Slaves
Postgres Master
6 Golden Image Updates
Graphite
Nagios
Jboss App
4
Memcache
Postgres Slaves
edit /etc/ssh/sshd_config
Postgres Master
12 Instance Replacements
Graphite
3
Delete, launch
Repeat
Typically manually
Nagios
Memcache
10
11
Postgres Slaves
12
Postgres Master
Jboss App
Done in Maintenance Windows
Graphite
3
Dont break anything!
Bob just got fired =(
Nagios
Memcache
5
10
11
Postgres Slaves
12
Postgres Master
Jboss App
Different IP Addresses?
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Invalid configs!
Postgres Master
Configuration Desperation
http://www.flickr.com/photos/francoforeshock/5716969942/
Chef Solves This Problem
But you already
guessed that, didnt
you?
Programs!
Generate
configurations directly
on nodes
Reduce management
complexity
Version control the
programs
http://www.flickr.com/photos/ssoosay/5126146763/
Declarative Interface to Resources
Define policy
Say what, not how
Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/
Chef is Infrastructure as Code
Programmatically
provision and configure
Treat like any other code
base
Reconstruct business from
code repository, data
backup, and bare metal
resources.
http://www.flickr.com/photos/louisb/4555295187/
That looks like this
package "ntp" do
action :install
end
template "/etc/ntpd.conf" do
source "ntpd.conf.erb"
owner "root"
group "root"
mode 0644
action :create
variables(:time_server => time.example.com)
notifies :restart, service[ntpd]
end
service "ntpd" do
action [:enable,:start]
end
Or this
package "net-snmp" do
action :install
end
template "/etc/snmpd.conf" do
source "snmpd.conf.erb"
owner "root"
group "root"
mode 0644
action :create
variables(:community_string => not_public)
notifies :restart, service[snmpd]
end
service "snmpd" do
action [:enable,:start]
end
"hostname": "server-1",
"fqdn": "server-1.example.com",
"domain": "example.com",
"network": {
"interfaces": {
"eth0": {
"type": "eth",
"number": "0",
"encapsulation": "Ethernet",
"addresses": {
"00:0C:29:43:26:C5": {
"family": "lladdr"
},
"192.168.177.138": {
"family": "inet",
"broadcast": "192.168.177.255",
"netmask": "255.255.255.0"
},
"fe80::20c:29ff:fe43:26c5": {
"family": "inet6",
"prefixlen": "64",
"scope": "Link"
}
},
"memory": {
"swap": {
"cached": "0kB",
"total": "4128760kB",
"free": "4128760kB"
},
"total": "2055676kB",
"free": "1646524kB",
"buffers": "35032kB",
"cached": "210276kB",
"active": "125336kB",
"inactive": "142884kB",
"dirty": "8kB",
"writeback": "0kB",
"anon_pages": "22976kB",
"mapped": "8416kB",
"slab": "121512kB",
"slab_reclaimable": "41148kB",
"slab_unreclaim": "80364kB",
"page_tables": "1784kB",
"nfs_unstable": "0kB",
"bounce": "0kB",
"commit_limit": "5156596kB",
"committed_as": "74980kB",
"vmalloc_total": "34359738367kB",
"vmalloc_used": "274512kB",
"vmalloc_chunk": "34359449936kB"
},
Ohai!
"block_device": {
"ram0": {
"size": "32768",
"removable": "0"
},
"ram1": {
"size": "32768",
"removable": "0"
},
"ram2": {
"size": "32768",
"removable": "0"
},
Decide what to declare
execute "load sysctl" do
command "/sbin/sysctl -p"
action :nothing
end
bytes = node[memory][total].split("kB")[0].to_i * 1024 / 3,
pages = node[memory][total].split("kB")[0].to_i * 1024 / 3 / 2048
# adjust shared memory and semaphores
template "/etc/sysctl.conf" do
source "sysctl.conf.erb"
variables(
:shmmax_in_bytes => bytes,
:shmall_in_pages => pages
)
notifies :run, "execute[load sysctl]", :immediately
end
Multiphase Execution
size = ((2 * 3) * 4) / 2
99.downto(1) do |i|
beer_bottle "bottle-#{i}" do
oz size
action [ :take_down, :pass_around ]
end
end
Recipes and Cookbooks
Recipes are collections of
Resources
Cookbooks contain
recipes, templates, files,
custom resources, etc
Code re-use and
modularity
http://www.flickr.com/photos/shutterhacks/4474421855/
Run Lists
Server
Server
chef-server
Server
Server
chef-client
API
ntp
client.rb
node
recipe[ntp::client]
Run Lists
Server
Server
chef-server
Server
Server
chef-client
API
ntp
openssh
node
client.rb
server.rb
ntp::client,
openssh::server
Run Lists
Server
Server
chef-server
Server
Server
chef-client
API
ntp
openssh
client.rb
node
apache
server.rb
php
default.rb
default.rb
recipe[ntp::client],
recipe[openssh::server],
recipe[apache],
recipe[php]
Roles
name "webserver"
description "webserver server"
run_list [
"role[base]",
"recipe[nginx::server]"
]
name "base"
description "base"
run_list [
"recipe[selinux::disabled]",
"recipe[etchosts]",
"recipe[yum::epel]",
"recipe[debugtools]"
]
Roles
Server
Server
chef-server
Server
Server
Role
Recipe
Role
Recipe
Recipe
Recipe
API
Role
Role
Knife
Recipe
Recipe
Recipe
Run Lists
Server
Server
chef-server
Server
Server
chef-client
API
ntp
openssh
client.rb
node
apache
server.rb
php
default.rb
default.rb
recipe[ntp::client],
recipe[openssh::server],
recipe[apache],
recipe[php]
Roles
Server
Server
chef-server
Server
Server
chef-client
API
ntp
openssh
client.rb
node
apache
server.rb
php
default.rb
default.rb
role[base],
role[webserver]
Roles
Server
Server
chef-server
Server
Server
ntp
openssh
client.rb
API
chef-client
apache
server.rb
php
default.rb
default.rb
ntp
openssh
client.rb
mysql
role[webserver]
node
chef-client
server.rb
server.rb
node
role[database]
Search
Search for nodes
with Roles
Find configuration
data
IP addresses
Hostnames
FQDNs
http://www.flickr.com/photos/kathycsus/2686772625
Search for nodes
pool_members = search("node","role:webserver)
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end
Pass results into Templates
# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>
munin::server example
node.set[:munin][:server] = true
munin_clients = search(:node, "munin_client:true")
cookbook_file "/etc/cron.d/munin" do
source "munin-cron"
mode "0644"
owner "root"
group "root"
end
template "/etc/munin/munin.conf" do
source "munin.conf.erb"
mode 0644
variables(:munin_clients => munin_clients)
end
munin::client example
node.set[:munin][:client] = true
munin_servers = search(:node, "munin_server:true")
unless munin_servers.empty?
package "munin-node" do
action :install
end
template "/etc/munin/munin-node.conf" do
source "munin-node.conf.erb"
mode 0644
variables :munin_servers => munin_servers
notifies :restart, "service[munin-node]"
end
service "munin-node" do
supports :restart => true
action [ :enable, :start ]
end
end
So when this
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master
Becomes this
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master
This can happen automatically
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master
Count the resources
Graphite
Nagios
Jboss App
Memcache
Postgres Slaves
12+ resource changes for 1 node addition
Load balancer config
Nagios host ping
Nagios host ssh
Nagios host HTTP
Nagios host app health
Graphite CPU
Graphite Memory
Graphite Disk
Graphite SNMP
Memcache firewall
Postgres firewall
Postgres authZ config
Questions!
What is a Node?
What is a Run List?
What is a Resource?
What is a Recipe? How is it different from a
Cookbook?
What is a Role?
Whats this got to do with Solaris?
Opscode Solaris Engineering
Fully Supported
As of 0.10.8, Solaris 9, 10 and 11 are fully
supported
Also works well with SmartOS, OpenIndiana and
OmniOS
SVR4 packages available for SPARC and Intel
http://www.opscode.com/chef/install
Improving Support
Native Resources
RBAC
Zones / Containers
SMF
IPS
Lightweight Resources and Providers (LWRP)
ZFS
Crossbow
Cookbooks
What do you want to see?
Who is using Chef?
On Solaris:
Joyent
OmniTi (and clients)
Wells Fargo
Expeditors International
Atalanta Systems (and clients)
Others
Others
With Opscode
Chef Cookbooks and
recipes, we were able to
definitively accelerate
our time-to-value and
time-to-market, which
results in operational
efficiency and cost
savings.
-Leandro Reox, MercadoLibre
How do I start?
http://www.opscode.com/chef/install
http://wiki.opscode.com
IRC: freenode.irc.net - #chef #chef-hacking
Mailing Lists
Test Driven Infrastructure with Chef (2nd edition
out soon)
Questions?
