UNIVERSITY OF ROMA TRE

Department of Computer Science and Automation

Emulating Computer Networks with Netkit Massimo Rimondini

Computer Networks Research Group http://www.dia.uniroma3.it/~compunet

4th International Workshop on Internet Performance, Simulation, Monitoring, and Measurement

Copyright notice
All the slides in this presentation, including but not limited to, images, photos, animations, videos, sounds, music, and text (hereby referred to as material) are protected by copyright This material, with the exception of some multimedia elements licensed by other organizations, is property of the authors and organizations appearing in the first slide This material, or its parts, can be reproduced and used for didactical purposes within universities and schools, provided that this happens for non-profit purposes Information contained in this material cannot be used within network design projects or other products of any kind Any other use is prohibited, unless explicitly authorized by the authors on the basis of a written agreement Authors assume no responsibility for the contents of this material, which may be subject to changes This copyright notice must always be redistributed together with the material, or its portions
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Prerequisites
Very basic knowledge of the Linux OS Networking
iso-osi stack routing protocols (rip, bgp) basic network tools

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Whats this Natkeet thing?

10.1.0.17 10.0.0.101 10.0.0.1 10.1.0.1 10.1.0.18 10.1.0.13 10.2.0.1 10.2.0.101 10.1.0.2 10.0.0.102 10.1.0.5 10.1.0.9 10.1.0.6 10.1.0.14

10.1.0.10

This is not Netkit

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Whats this Natkeet thing?

This is Netkit! This is Netkit!

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Yet another boring emulator toy? From Wikipedia:

Bochs, portable open source x86 and AMD64 PCs emulator FAUmachine Microsoft Virtual PC Microsoft Virtual Server OPEN COLINUX - Linux inside Windows Parallels QEMU SVISTA TRANGO Real-time Embedded Hypervisor twoOStwo User-mode Linux Virtuozzo VM from IBM, apparently the first true virtual machine system and still in wide use today. VMWare Xen
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Yet another boring emulator toy?

Product Xen License Open Source Pros Performance Cons Requires replacing the host kernel Requires porting the guest OS kernel Complex configuration Performance Requires privileges to be installed Not lightweight Development stalled

VMWare Bochs QEMU

Commercial Open Source (almost) Open Source

Near native performance Good support for many guest OSes Emulates multiple architectures Comes with an accelerator module Lightweight

Plex86

Open Source

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Success stories involving Netkit Networking courses

routing protocols application level services (dns, web, email, etc.)

Emulation of the italian academic research network (GARR)

assignment of OSPF weights

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Outline
Understanding Netkit
Architecture overview

Setting up Netkit
Download and installation

Using Netkit
Getting acquainted with Netkit commands How to prepare Netkit labs

Sample scenarios
A virtual network running BGP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

UNIVERSITY OF ROMA TRE

Department of Computer Science and Automation

Understanding Netkit

A Netkit network
virtual host virtual host
vhost:~# vhost:~#

virtual host
vhost:~#

virtual host
vhost:~#

virtual host
vhost:~#

virtual host
vhost:~#

virtual host
vhost:~#

virtual host
vhost:~#

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A Netkit network
virtual host virtual host
running routing daemon...

virtual host virtual host

ethernet bridge configured

virtual host virtual host

vhost:~#

real host real host

running virtual hub...

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A closer look at virtual machines

real host real host

HOST OS
virtual host

GUEST OS

virtual hub
(collision domain)

host disk(s)

host ram
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

host NIC

Interconnecting virtual machines

real host real host

HOST OS
virtual host

virtual host

ROUTER

virtual host

PC3

PC1 virtual hub

virtual host

virtual hub
(collision domain)

PC2

(collision domain)

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A few technicalities
Virtual machines run a user-mode-linux kernel
Linux kernel compiled for running as a userspace process Thin emulation layer better performance http://user-mode-linux.sourceforge.net/

The guest OS is a Debian GNU/Linux unstable

Most popular networking tools are included Copy-On-Write
Damage-free filesystem

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Cables & Interfaces

Virtual machines can be equipped with an arbitrary number of network interfaces

Network Layer
Handled by the UML kernel IPv4/IPv6

Data-link Layer
Ethernet No collisions

Physical layer
Virtual hub daemon (uml_switch) Communication over unix sockets No delay, loss, reordering
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

UNIVERSITY OF ROMA TRE

Department of Computer Science and Automation

Setting up Netkit

System requirements
Host CPU RAM i386 Linux host >600 MHz
lower freqs impact performance

~10MB for each virtual machine

depends on the amount of emulated memory

Hard disk space Others

~650 MB + ~1-20 MB for each virtual machine

depends on vm usage

ext2/ext3 filesystems preferred

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A tiresome procedure
Just three steps 1. Download @ http://www.netkit.org
netkit-2.4.tar.bz2 netkit-filesystem-F2.2.tar.bz2 netkit-kernel-K2.2.tar.bz2
large file

2. cd to a directory of your choice and unpack the files

host machine host machine
foo@host:~$ cd netkit netkitfoo@host:~/netkit$ tar xjf netkit-2.4.tar.bz2 netkit-filesystemfoo@host:~/netkit$ tar xjf netkit-filesystem-F2.2.tar.bz2 large file netkit-kernelfoo@host:~/netkit$ tar xjf netkit-kernel-K2.2.tar.bz2 foo@host:~$
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A tiresome procedure
3. Set some environment variables
set NETKIT_HOME to the path where you installed Netkit set PATH to the string $PATH:$NETKIT_HOME/bin set MANPATH to the string :$NETKIT_HOME/man Example (using bash):
host machine host machine
foo@host:~/netkit$ foo@host:~/netkit$ foo@host:~/netkit$ foo@host:~/netkit$ export NETKIT_HOME=/home/foo/netkit/netkit2 export PATH=$PATH:$NETKIT_HOME/bin export MANPATH=:$NETKIT_HOME/man

Thats all folks!

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Test your installation

cd to the Netkit directory

Run check_configuration.sh
host machine host machine
foo@host:~/netkit$ cd netkit2 foo@host:~/netkit/netkit2$ ./check_configuration.sh ...... [ READY ] Congratulations! Your Netkit setup is now complete! Enjoy Netkit! foo@host:~/netkit/netkit2$

Test failures are accompanied by a short description of the fix

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

UNIVERSITY OF ROMA TRE

Department of Computer Science and Automation

Using Netkit
(at last)

A toolkit made up of tools

Virtual machines can be controlled by using two interfaces
ltools vtools UML kernel (virtual machine) uml_switch (virtual hub)

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Vtools
vstart vconfig vlist vhalt vcrash vclean Start a virtual machine with a given configuration Attach a network interface to a running virtual machine List running virtual machines Gracefully shut down a virtual machine Kill a virtual machine Panic button
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Booting a virtual machine

vstart [options] MACHINE_NAME

Most common options:

--ethN=collision_domain
Interfaces on the same collision domain can exchange traffic

-M memory_amount -p
Just show what would be done
host machine host machine
--eth0=CD_A --eth1=CD_B foo@host:~$ vstart --eth0=CD_A --eth1=CD_B M 256 pc1 ============= Starting virtual machine "pc1" ============= /home/max/netkit2/kernel/netkitKernel: /home/max/netkit2/kernel/netkit-kernel Modules: /home/max/netkit2/kernel/modules ......
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Look: its booting

The virtual machine terminal window automatically pops up
pc1 pc1
--- Starting Netkit phase 2 startup script Virtual host pc1 ready. --- Netkit phase 2 init script terminated pc1 login: root (automatic login) Linux pc1 2.6.11.7 #1 Tue Sep 13 18:38:01 CEST 2005 i686 GNU/Linux Welcome to Netkit pc1:~#
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Where VMs write things

Copy-On-Write: every change to the model filesystem is written to pc1.disk Sparse files: zeros do not consume disk space
host machine host machine
foo@host:~$ ls -l rw- --r -rw-r--r-- 1 foo rw- --r -rw-r--r-- 1 foo foo@host:~$ -h du 884K pc1.disk 4.0K pc1.log foo@host:~$ pc1.* 2006-02foo 630358016 2006-02-18 19:53 pc1.disk 2006-02foo 98 2006-02-18 19:57 pc1.log pc1.*

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Getting rid of a virtual machine

Gracefully shut down from the inside
pc1 pc1
pc1:~# halt Broadcast message from root (vc/0) (Sat Feb 18 19:46:13 2006): The system is going down for system halt NOW! INIT: Switching to runlevel: 0 INIT: Sending processes the TERM signal

or from the outside

host machine host machine
foo@host:~$ vhalt pc1 Halting virtual machine "pc1" (PID 8598) owned by foo [... foo@host:~$
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Getting rid of a virtual machine

or harshly unplug its power cord
host machine host machine
foo@host:~$ vcrash pc1 ========== Crashing virtual machine "pc1" (PID 9741) ========= Virtual machine owner: foo Virtual machine mconsole socket: /home/foo/.netkit/mconsole/pc1/mconsole Crashing... done. foo@host:~$

Quick Screws virtual machine filesystem Remember: Copy-On-Write

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A Netkit lab
is a set of pre-configured virtual machines that can be launched/stopped all together consists of a hierarchy of directories allows to set up complex network experiences can be controlled by using the ltools

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Ltools
lstart ltest lhalt lcrash linfo lclean Start a Netkit lab or just some of its machines Start a Netkit lab in test mode Gracefully shut down (some of) the virtual machines of a lab Kill (some of) the virtual machines of a lab Display info about a lab without starting it; sketch the network topology Remove temporary files (no panic!)
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Mastering your own Netkit Lab

How to prepare a lab: 1. define topology 2. assign addresses 3. configure network services 4.

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

My Own Lab, scene 1, take 1 Network topology Sketch your planned topology before implementing it
LEGEND
1 eth1

B
9 eth0

collision domain name network address

A
30.3.3.0/24

11.0.0.8/30 11.0.0.8/30

B
10 eth0

9 eth0

IP, last byte Interface

A
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

*AN

My Own Lab, scene 1, take 2 Network topology A lab consists of a hierarchy of directories Each (even empty) directory represents a virtual machine
host machine host machine
foo@host:~/lab$ as10r1 as10r2 foo@host:~/lab$ ls lab.conf

A lab consisting of two virtual machines (as10r1, as10r2) Check with linfo

Link-level connections are described inside the file lab.conf (in the lab root)
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

My Own Lab, scene 1, take 3 Network topology

lab.conf syntax
vm[if]=cd
vm: virtual machine name (e.g., as10r1) if: interface number (e.g., 0) cd: collision domain name (arbitrary string)

vm[opt]=val
opt: the name of a vstart option (e.g., mem) val: a value for that option

Other optional items

Informational: LAB_DESCRIPTION, LAB_VERSION, LAB_AUTHOR, LAB_EMAIL, LAB_WEB Explicit list of virtual machines (machines)

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

My Own Lab, scene 1, take 4 Network topology Sample lab.conf

1 eth1 host machine host machine
as10r1[0]=B as10r1[1]=A as10r2[0]=B lab.conf

A
30.3.3.0/24

9 eth0

B
10 eth0

11.0.0.8/30

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

My Own Lab, scene 2 Assigning addresses, etc.

On startup, virtual machine vm runs shared.startup vm.startup Interfaces may be configured inside vm.startup
host machine host machine
11.0.0.11 ifconfig eth0 11.0.0.9 netmask 255.255.255.252 broadcast 11.0.0.11 up /etc/init.d/zebra start as10r1.startup

Network services may be started up inside vm.startup too

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

My Own Lab, scene 3 Configuring network services

On startup, Netkit copies the directory hierarchy in vm/ (on the host) to / on virtual machine vm
host machine host machine
foo@host:~/lab$ find . . ./as10r1 ./as10r1/root ./as10r1/root/this_is_a_file ./as10r2 ./lab.conf foo@host:~/lab$

as10r1 as10r1
as10r1:~# pwd /root as10r1:~# ls this_is_a_file as10r1:~#

Useful to alter routing software configuration files

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Ignition!
To start up a lab:
host machine host machine
foo@host:~$ cd lab foo@host:~/lab$ lstart

or

host machine host machine

foo@host:~$ lstart d lab

To crash a lab (use lhalt to halt):

host machine host machine
foo@host:~$ cd lab foo@host:~/lab$ lcrash

host machine host machine

or

foo@host:~$ lcrash d lab

To get rid of temporary files

host machine host machine
foo@host:~$ cd lab foo@host:~/lab$ lclean

(.disk, .log):

host machine host machine

or

foo@host:~$ lclean d lab

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

UNIVERSITY OF ROMA TRE

Department of Computer Science and Automation

Playing Around with Netkit

A sample lab
A multihomed network
isp 1 isp 2

link 1

link 2

BGP

customer
(200.1.0.0/16)

RIP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A sample lab
A multihomed network, designed to:
Prohibit transit traffic
isp 1 isp 2

link 1

link 2

BGP

customer
(200.1.0.0/16)

RIP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A sample lab
A multihomed network, designed to:
isp 1 isp 2

Prohibit transit traffic Be fault tolerant BGP

link 1

link 2

customer
(200.1.0.0/16)

RIP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A sample lab
A multihomed network, designed to:
isp 1 isp 2

Prohibit transit traffic Be fault tolerant Perform loadsharing BGP

inbound: by announcing /17s

link 1

link 2

customer
(200.1.0.0/16)

RIP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

A sample lab
A multihomed network, designed to:
isp 1 isp 2

Prohibit transit traffic Be fault tolerant Perform loadsharing BGP

inbound: by announcing /17s outbound: by nearest exit

link 1

link 2

customer
(200.1.0.0/16)

RIP
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

Making routers route traffic

Routing is performed by the zebra software
bgpd
bgpd> bgpd> ripd> ripd>

ripd

ospfd

ospfd> ospfd>

zebra> zebra>

file access routing updates

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

10 eth0

14 eth0 11.0.0.8/30

P
11.0.0.12/30

40.4.4.0/24

N
30.3.3.0/24 9 eth0

O
13 eth0

AS300

customer
(200.1.0.0/16) 2 2 eth0 eth1

1 2 eth0 eth1

1 eth1

1 eth1

Y
200.1.0.0/18

X
200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

10 eth0

14 eth0 11.0.0.8/30

P
11.0.0.12/30

40.4.4.0/24

N
30.3.3.0/24 9 eth0

O
13 eth0

AS300

as300r1 as300r1

customer
(200.1.0.0/16)

1 2 eth0 eth1

1 eth1

ping 11.0.0.26 ping ping 11.0.0.29 ping 1 2 2 eth0 eth1 traceroute 11.0.0.29 traceroute eth1 Y observeX nearest exit
200.1.64.0/18

200.1.0.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

10 eth0

14 eth0 11.0.0.8/30

P
11.0.0.12/30

40.4.4.0/24

N
30.3.3.0/24 9 eth0

O
13 eth0

AS300

as300r1 as300r1

customer
(200.1.0.0/16)

1 2 eth0 eth1

telnet localhost bgpd (pw:zebra) telnet 1 2 2 show ip bgp 1 show eth1 eth1 eth0 observe local routes eth1 the default and Yenable X enable show running-config show running200.1.0.0/18 200.1.64.0/18 observe prefix-lists that prevent prefix-

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

transit traffic and let /16 and200.1.128.0/17 /17 M be announced inspect ripd configuration

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

R
as30r1 as30r1

10 eth0 11.0.0.8/30

14 eth0

P
11.0.0.12/30

40.4.4.0/24

traceroute 11.0.0.14N traceroute 30.3.3.0/24 traffic does not traverse the

O
13 eth0

customer

9 eth0

AS300

customer
(200.1.0.0/16) 2 2 eth0 eth1

1 2 eth0 eth1

1 eth1

1 eth1

Y
200.1.0.0/18

X
200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30

as1r1 as1r1
30 traceroute eth0 traceroute 200.1.0.2

isp 1

S
25 eth2

traceroute 200.1.128.1 traceroute 11.0.0.28/30 Q loadsharing at work

29 eth2 1 eth1

AS30
1 eth1

isp 2
40.4.4.0/24

AS40 P
11.0.0.12/30

10 eth0

14 eth0 11.0.0.8/30

N
30.3.3.0/24 9 eth0

O
13 eth0

AS300

customer
(200.1.0.0/16) 2 2 eth0 eth1

1 2 eth0 eth1

1 eth1

1 eth1

Y
200.1.0.0/18

X
200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30

as1r1 as1r1
30 telnet eth0 telnet localhost bgpd (pw:zebra)

isp 1

S
25 eth2

show ip bgp show 11.0.0.28/30 Q observe the two /17s and the
29 two eth2

alternatives for the /16

AS30
1 eth1

isp 2

AS40
1 eth1

10 eth0

14 eth0 11.0.0.8/30

P
11.0.0.12/30

40.4.4.0/24

N
30.3.3.0/24 9 eth0

O
13 eth0

AS300

customer
(200.1.0.0/16) 2 2 eth0 eth1

1 2 eth0 eth1

1 eth1

1 eth1

Y
200.1.0.0/18

X
200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

10 eth0

14 eth0 11.0.0.8/30

P
11.0.0.12/30

40.4.4.0/24

N
30.3.3.0/24 9 eth0

O
13

as300r1 as300r1
telnet localhost bgpd (pw:zebra) telnet eth0 AS300 enable enable configure terminal configure (200.1.0.0/16) router bgp 300 router 1 2 2 neighbor 11.0.0.10 shutdown neighbor eth1 eth0 eth1 simulating failure of link N Y shutting down the interface is X

customer

1 2 eth0 eth1

1 eth1

also 200.1.0.0/18

possible 200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

AS1
26 eth1 11.0.0.24/30 30 eth0

isp 1

S
25 eth2

Q
29 eth2

11.0.0.28/30

AS30
1 eth1

isp 2
AS40
1 eth1

R
as300r1 as300r1

10 eth0

14 eth0

P
11.0.0.12/30

40.4.4.0/24

11.0.0.8/30 after a while... N traceroute 11.0.0.25 traceroute 30.3.3.0/24 the backup route is being used 9 eth0

O
13 eth0

AS300

customer
(200.1.0.0/16) 2 2 eth0 eth1

1 2 eth0 eth1

1 eth1

1 eth1

Y
200.1.0.0/18

X
200.1.64.0/18

M
IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet

200.1.128.0/17

Wanna play?
Visit http://www.netkit.org
Other ready-to-use labs Lecture slides Netkit updates Other resources (NetML)

Forthcoming lab topology:

Tha

IPS MoMe 2006 - 2006 Massimo Rimondini & Compunet