MPLS OAM Tutorial
Sam K Aldrin
[email protected]
June 3-6, 2012
NANOG55
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
What is OAM
Means different things to different people and
organizations.
Worst, some times it means different things to
different people within the same organization
IETF standardized the meaning of OAM within the
IETF
June 2011, RFC 6291
June 3-6, 2012
NANOG55
IETF definition of OAM
Operations: Operational activities to keep
network up and running. E.g. Monitoring, finding
faults
Administration: Involves keeping track of
network resources. E.g. Bookkeeping, (available ports,
BW)
Maintenance: Involves repair and upgrades.
E.g. Software upgrades, configurations, corrective and
preventive measures.
June 3-6, 2012
NANOG55
Scope of the Tutorial
Todays presentation mainly focus on IETF
defined Operations aspects of MPLS OAM.
Various OAM operations and techniques are
presented for MPLS networks
June 3-6, 2012
NANOG55
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
June 3-6,
Tools
Galore
2012
NANOG55
Important Terminologies
Before we dive deeper, it is important to
understand some of the terminologies and their
meanings
What are they ?
Various organizations (IEEE, ITUT, IETF) all have their
version
We will discuss here selected set of definitions from
RFC 5860, RFC 6371 and draft-ietf-opsawg-oamoverview-05
Good understanding of these Terminologies will
help us to appreciate modern OAM protocols
better.
June 3-6, 2012
NANOG55
Important Terminologies
Maintenance Point (MP)
Is a functional entity that is defined within a node that either
initiate or react to a OAM message
Maintenance Entity (ME)
Point to Point relationship between two MP
In MPLS this is LSP, In BFD this is session
Maintenance Point can be either MEP or MIP
Maintenance End Point (MEP)
Can either initiate or react to OAM Messages
MEP are the two end points of the ME
Maintenance Intermediate Point (MIP)
Is an intermediate MP between two MEP
It can only respond to NANOG55
OAM messages
June 3-6, 2012
Relationship of MP
(ME)
(MEP)
(MIP)
(MIP)
(MEP)
B
traceroute to B
Request
Response
June 3-6, 2012
NANOG55
Important Terminologies (contd..)
Continuity Check
Ability of endpoint to monitor liveliness of a path (BFD )
Connectivity Verification
Ability of an endpoint to verify it is connected to a specific endpoint.
(BFD,Ping)
Route Tracing
This is also known as path tracing, allows to identify the path taken
from one MEP to another MEP (traceroute)
Fault Verification
Exercised on demand to validate the reported fault. (Ping)
Fault Isolation
Localizing and isolating the failure domain/point (traceroute)
Performance
Includes Packet Loss Measurements and Packet Delay
Measurements
E.g. IP Performance Metrics (IPPM) (RFC 2330)
June 3-6, 2012
NANOG55
10
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
11
Ping
Ping refers to tools that allows to detect liveliness of a
remote host
Most commonly known Ping is based on ICMP Echo
Request and Response
Security policies and firewalls sometimes prevent
forwarding of ICMP messages.
UDP/TCP version of the Ping has surfaced to circumvent
barriers introduced by security policies and Firewalls on
ICMP Echo Requests
RFC 4379 use UDP port 3503 for LSP Ping
Different implementations of Ping has different options
June 3-6, 2012
NANOG55
12
Ping traceroute simulation
Ping an IP address with increasing the TTL count at each step.
In the example below TTL increased by 1 at each iteration..
ping -c 1 -t 2 -n www.yahoo.com
PING any-fp3-real.wa1.b.yahoo.com (98.139.127.62) 56(84) bytes of data.
From 10.35.78.17 icmp_seq=0 Time to live exceeded
--- any-fp3-real.wa1.b.yahoo.com ping statistics --1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms,
pipe 2
ping -c 1 -t 3 -n www.yahoo.com
PING any-fp3-real.wa1.b.yahoo.com (98.139.127.62) 56(84) bytes of data.
From 10.34.159.13 icmp_seq=0 Time to live exceeded
--- any-fp3-real.wa1.b.yahoo.com ping statistics --1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms,
pipe
2
June
3-6, 2012
NANOG55
13
Traceroute
Design to trace the path taken from a node A to a
node B.
Probe packets are generated with monotonically
increasing TTL value
Forcing ICMP TTL expiry message from each
intermediate node.
In Linux Echo request packet is UDP (default
destination port is UDP:33434)
In some other platforms it can be ICMP Echo
request.
June 3-6, 2012
NANOG55
14
traceroute sample output linux
traceroute -n 10.35.78.17
traceroute to 10.35.78.17 (10.35.78.17), 30 hops max, 46 byte
packets
1 10.35.75.3 0.292 ms 0.366 ms 0.213 ms
TTL=1
2 10.35.78.17 0.642 ms 0.429 ms 0.369 ms
TTL=2
traceroute -n -I 10.35.78.17
traceroute to 10.35.78.17 (10.35.78.17), 30 hops max, 46 byte
packets
1 10.35.75.3 0.271 ms 0.219 ms 0.213 ms
TTL=1
2 10.35.78.17 0.442 ms 0.265 ms 0.351 ms
TTL=2
June 3-6, 2012
NANOG55
15
traceroute
2
1. ICMP_TME_EXCEED
1. Echo Req, TTL=1
2. Echo Req, TTL=2
2. ICMP_TME_EXCEED
3. Echo Req,TTL=3
4. ICMP_TME_EXCEED
3. ICMP_TME_EXCEED
4. Echo Req, TTL=4
June 3-6, 2012
NANOG55
16
Challenges
Over the years networking has evolved with that
comes OAM challenges
ECMP (Equal Cost Multi Path)
Multicast
Tunneling (MPLS, PW, VPN, TRILL)
Firewalls
ICMP and more traditional OAM are designed for
unicast traffic with single path to the destination.
June 3-6, 2012
NANOG55
17
Equal Cost Multipath
Equal Cost Multi Path (ECMP) allows
Protection against failures
Increased overall end-end BW
ECMP is becoming increasingly popular
Devices typically use fields in the MAC or IP header
to select the forwarding path among multiple equal
cost paths
Connectivity and Continuity verification messages
MUST follow the same path as user data.
How can we accomplish this ?
There is no standard way of doing this in IP world
MPLS RFC 4379 has payload discovery approach
June 3-6, 2012
NANOG55
18
ECMP
Ping From A to B
User Data A to B with
UDP Src/Dest Port X/Y
A
User Data A to B with
UDP Src/Dest Port A/B
June 3-6, 2012
NANOG55
19
ECMP Failure Example
Ping From A to B
User Data A to B with
UDP Src/Dest Port X/Y
A
User Data A to B with
UDP Src/Dest Port A/B
3
Can not utilize end-end connectivity
tools to quickly detect the failure
May need to wait until control protocol
time-out
If it is an oversubscribed link that causing
intermittent traffic drops, protocols would
not timeout
June 3-6, 2012
NANOG55
20
ECMP Monitoring Challenges
Ping From A to B
User Data A to B with
UDP Src/Dest Port X/Y
A
User Data A to B with
UDP Src/Dest Port A/B
Challenges:
Ingress Node (A) may not even know how many ECMP from intermediate node (1)
Monitoring probes SHOULD take the same path as the normal data
Different vendors utilize different hash algorithms in selection ECMP paths
June 3-6, 2012
NANOG55
21
ECMP challenges
Conclusion
No standard method to exercise end-end
continuity and connectivity verifications that
covers all of the ECMP in IP networks
June 3-6, 2012
NANOG55
22
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
23
What is MPLS
MPLS is acronym for Multi Protocol Label Switching
Forwards traffic using labels
Provides virtual connection (LSP) within the network
Labels are allocated based on FEC
Different types of label distribution
An LSP is usually unidirectional
Ingress, Transit and Egress router types
Traditional MPLS networks support PHP processing
Supports different traffic types like ATM, FR, IP etc
Private services like VPN for scalable service provider
requirements
June 3-6, 2012
NANOG55
24
MPLS LSP signaling protocols
Resourced Reservation Protocol (RSVP)
Extended to support Traffic Engineering
Labels are assigned for identified path
Explicit bandwidth reservation and paths
Label Distribution Protocol (LDP)
Labels are exchanged between neighbors
IGP identifies the shortest path
Constrained Routing LDP (CR-LDP)
Traffic Engineering support using LDP
June 3-6, 2012
NANOG55
25
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
June 3-6,
Tools
Galore
2012
NANOG55
26
What is MPLS-TP
MPLS
RFC 5654
MPLS TP
MPLS TP is a subset of MPLS
MPLS network enhanced to support Transport
requirements
Bidirectional LSPs with a highly reliable protection
schemes
Inter-op with existing MPLS Technologies
Transport agnostic protocol extensions
June 3-6, 2012
NANOG55
27
What is being solved by MPLS-TP?
Next Generation networks are moving
SONET/SDH to Packet Switching
Bandwidth hungry
Lower cost with network resource sharing
OPEX and CAPEX
Provisioning of paths
OAM capabilities
Fault detection and recovery mechanisms
Path computation
SLA requirements
June 3-6, 2012
NANOG55
28
MPLS TP New additions to MPLS
Operations
NMS Driven
provisioning
Static Network
setup
Associated and corouted
bidirectional paths
June 3-6, 2012
MPLS- TP
Reliability and
Resiliency
OAM Requirements
In band OAM
Loss and delay
measurements for
SLA
Fault notification
and Alarm
indication
NANOG55
Linear, Ring and
Mesh protection
schemes
Fast switchover to
standby paths
50msec switchover
support
29
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
30
Problems in MPLS Networks
Control Plane is working, Data Plane is broken
IGP working but MPLS control protocol is broken
Proactive monitoring of End-to-End MPLS LSPs
Identifying the End-to-End packet path
Unlabelled interface
MTU issues
Performance degradation and unable to provide QoS
Black holes
ECMP Verification
June 3-6, 2012
NANOG55
31
Primitive Debugging Methods
ICMP provides connectivity verification
VRF aware ping could test VPN path connectivity
UDP ping could test the UDP transport
Route table and Label table provides label entries
programmed
Interface status verification
MPLS control plane protocols provides control plane
information
June 3-6, 2012
NANOG55
32
ICMP ping
X
LSP
PE1
P1
P2
L0:10.10.10.10
P3
PE2
L0:20.20.20.20
ICMP ping emulates the data but can only verify IP layer
It cannot verify if MPLS path is broken but IP is working
It cannot verify ECMP
It cannot validate control plane to data plane
It cannot verify various MPLS control plane protocols
It cannot verify for unlabelled interface, black-holes, control
plane to data plane mismatch, etc.
June 3-6, 2012
NANOG55
33
VRF aware ping
->label 50
->label 60
->label 70
->Pop
LSP
PE1
L0:10.10.10.10
P1
P2
P3
X
PE2
L0:20.20.20.20
VRF aware could emulate VPN traffic
Could test VPN connectivity
Cannot detect LSP breakage
If IP connectivity is working and MPLS is broken, it
cannot detect
Can detect if there is no label path, but not in all cases
JuneCannot
detect ECMP failures,
CP to DP mismatch, etc.34
3-6, 2012
NANOG55
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
35
What is MPLS OAM
Operations, Administration and Maintenance of
MPLS Networks
Perform proactive and on-demand troubleshooting
of MPLS Networks and devices
Ability to measure MPLS network and aid user in
managing the network
Ability to diagnose defects which cannot be done at
other layers or using non-MPLS specific toolset
Provide carrier class tool set to manage MPLS
networks
June 3-6, 2012
NANOG55
36
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
37
LSP ping
Requirements
Detect LSP failures
Detect label mismatch
Detect CP to DP mismatch
Pin point the failure
Detect MTU failures
Applications
Verify all MPLS FEC types
Verify PE, P, MPLS TP devices
Ability to verify MPLS VPN,
TE, LDP, TP, P2MP, etc., LSPs.
June 3-6, 2012
Solution
LSP ping to detect
connectivity checks
LSP ping based traceroute for
path verification
LSP ping based topology tree
verification
Standards
RFC4379 and all other
extensions
NANOG55
38
LSP Ping What is it?
Function
Modeled like ICMP ping but based on UDP
Connectivity between two end points of an LSP
Format
Encapsulated like data frame for the FEC
The IP destination of the packet is local host address
Behavior
Cannot leak out onto non-MPLS interface
Response packet contains a code indicating the reason
Destination IP address used as entropy simulate ECMP
OAM packets are treated the same as data packets
TTL field is used to test intermediate hops
June 3-6, 2012
NANOG55
39
LSP Ping What can it verify?
Sub-Type
Length
Value field
LDP IPv4 Prefix
17
LDP IPv6 Prefix
20
RSVP IPV4 Prefix
56
RSVP IPv6 Prefix
Not Assigned
13
VPN IPv4 Prefix
25
VPN IPv6 Prefix
14
L2 VPN endpoint
10
FEC 128 PW (Deprecated)
10
14
FEC 128 PW
11
16+
FEC 129 PW
12
BGP Labeled IPv4 Prefix
13
17
BGP Labeled IPv6 Prefix
14
Generic IPv4 Prefix
15
Generic IPv6 Prefix
Nil FEC NANOG55
June 3-6,
162012
40
LSP Ping Constructs
LSP ping packet is encapsulated to simulate data
packet in order to test a LSP
Two types Echo Request and Echo Response
The FEC to be verified
The Label stack for the FEC/LSP
A UDP/IP packet with LSP ping payload to be send on
the LSP
The interface information on which the packet has to
be forwarded
Forwarding and interface information for the FEC for
verification purposes
June 3-6, 2012
NANOG55
41
LSP Ping Response Codes
Value
Meaning
-------
-----------
No return code
Malformed echo request received
One or more TLV's not understood
Replying router is egress for the FEC
No mapping for the FEC
DSMAP mismatch
Unknown upstream index
Reserved
Label switched at stack depth <RSC>
Label switched but no MPLS forwarding at stack depth <RSC>
10
Mapping for this FEC is not the given label at stack depth <RSC>
11
No label entry at stack depth <RSC>
12
Protocol not associated with interface at FEC stack depth <RSC>
13
Premature termination of ping due to label stack shrinking to a single label
June 3-6, 2012
NANOG55
42
LSP Ping Echo Request
Echo Request is sent by the router to test LSP of a given FEC
MPLS encapsulation
MPLS encapsulated IP/UDP packet
Label stack is same as data packet for the FEC.
Default TTL value for the label is 255
FEC TLV contains the details of the FEC to be verified
IP Encapsulation
IP/UDP Packet
Source address: Valid source address
Destination address: Local host address
Destination Port: 3503
RA option : Enable
TTL : 1
June 3-6, 2012
NANOG55
43
LSP Ping Echo Reply
Echo Reply is sent by the router to responding to the Echo Request
Reply Modes
IP reply
No Reply
IP reply with RA option
Control Channel
Packet Format
IP source address : Replying router IP address
Destination address : Received Source address
Source port : 3503/other chosen port
Destination Port : Port number in the echo request
TTL : 255
June 3-6, 2012
NANOG55
44
Downstream Mapping
15
23
31
Downstream Router ID
MTU
Addr Type
Downstream interface address is IP
address of outgoing interface for the LSP
Downstream label is the outgoing label
for the LSP
Protocol associated with the label
DDMAP is enhanced version of the
DSMAP TLV (Deprecated)
DS Index
Downstream Interface Address
Hash Key
Depth Limit
MultiPath Length
IP Address or Next Label
More IP addresses or next labels
Downstream Label
Protocol
.
Downstream Label
DSMAP TLV
Protocol
0
7
MTU
15
23
Addr Type
31
DS Flags
Downstream Address (4 or 16 octets)
Downstream Interface Address (4 or 16 octets)
R Code
Return SC
SubTLV Length
IP Address or Next Label
List of SubTLVs
1. Multipath
2. Label Stack
3. FEC Stack change
DDMAP TLV
June 3-6, 2012
NANOG55
45
Downstream Mapping TLV Example
2
E0/0:10.131.151.1
50
E0/0:10.131.161.1
60
E0/1:10.131.151.2
PE1
PE1 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.151.1
DS Label : 50
E0/0:10.131.171.1
3
E0/1:10.131.161.2
P1
P2
P1 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.161.1
DS Label : 60
E0/1:10.131.171.2
PE2
P2 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.171.1
DS Label : 3
Note: No DSMAP TLV is sent by Egress router
June 3-6, 2012
NANOG55
46
Theory of Operation
LSP
60
50
127/8
SA
SA
127/8
Echo Req
SA
127/8
Echo Req
Echo Req
60
50
P1
P2
PE1
PE2
SA
127/8
Echo Reply
Packet is encodes with the same label stack as data packet
The destination header of the packet is set as local host address
The packet is forwarded on Egress interface identified for the FEC
The packet get labeled switched on transit routers
No special treatment of OAM packets on transit routers
The Echo reply is sent as IP as default
June 3-6, 2012
NANOG55
47
LSP ping as diagnostic tool
LSP
SA
50
127/8
SA
127/8
Echo Req
Echo Req
60
50
P1
PE1
SA
127/8
P2
Echo Reply
PE2
LSP could be broken due to various reasons
No MPLS interface
No LDP adjacency
Label mismatch
Control Plane and Data Plane mismatch
LSP ping Echo Request cannot get label forwarded due to LSP
breakage
Echo request gets locally processed due to local address
Reply sent by the processing router with appropriate error code
June 3-6, 2012
NANOG55
48
LSP ping for Control Plane Data
Plane Mismatch
LSP
PE1
127/8
70
50
PE1
127/8
Echo Req
PE1 127/8 Echo Req
60
Echo Req
P2
3
50
PE
1
P1
70
P2
PE1
Echo Reply
LSP control plane and data plane mismatch
Control plane advertises label 60 to PE2 FEC
Data Plane takes different path with label 70
Though packets reach PE2, they traverse different path
LSP ping with DSMAP or Trace validation
When LSP ping with DSMAP is set hop by hop, it can
identify the fault
June 3-6, 2012
NANOG55
DSMAP mismatch error will
be return upon this error
PE2
49
Trace with LSP Ping
Label TTL : 2
Label TTL: 3
Label TTL: 1
E0/0:10.131.151.1
50
E0/0:10.131.161.1
E0/1:10.131.151.2
PE1
PE1 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.151.1
DS Label : 50
60
E0/0:10.131.171.1
E0/1:10.131.161.2
P1
P1 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.161.1
DS Label : 60
P2
E0/1:10.131.171.2
PE2
P2 Downstream for
PE2
MTU: MRU of E0/0
Address type: 1
DS Intf Addr:
10.131.171.1
DS Label : 3
LSP Ping with TTL is used to validate every hop of the LSP
Downstream TLV is used to validate and request downstream info
If the responding router is Egress of the FEC, a return code of 3 is
returned.
JuneNo
DSMAP TLV is sent in the NANOG55
response by Egress router for the FEC
3-6, 2012
50
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
51
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
PE1
1
TTL = 1
DA: 127.0.0.0
MapSize/hash:
32/8
Bitmap:0xFFFF
June 3-6, 2012
NANOG55
52
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
PE1
1
TTL = 1
DA: 127.0.0.0
MapSize/hash:
32/8
Bitmap:0xFFFF
June 3-6, 2012
E0/0
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
P1
2
MultiPath1
[E0/0]
Bitmap: 0x00FF
Multipath2[E2/0]
Bitmap: 0xFF00
NANOG55
53
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
E0/0
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
PE1
3
TTL = 2
DA: 127.0.0.24
Mapsize/Hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
NANOG55
54
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
PE1
3
TTL = 2
DA: 127.0.0.24
Mapsize/Hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
E0/0
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
P2
MultiPath1 [E0/0]
Bitmap: 0x00FF
4
NANOG55
55
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
PE1
5
TTL = 3
DA: 127.0.0.24
Mapsize/hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
NANOG55
56
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
P1
E0/0
E2/0
E2/0
P4
PE2
P5
P3
MultiPath1
[E0/0]
Bitmap: 0x00FF
PE1
5
TTL = 3
DA: 127.0.0.24
Mapsize/hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
E1/0
E1/0
NANOG55
57
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
P1
E1/0
E1/0
E0/0
E2/0
E2/0
P4
P5
PE2
PE1
7
TTL = 4
DA: 127.0.0.24
Mapsize/Hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
NANOG55
58
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
E0/0
E1/0
E2/0
E1/0
PE1
P1
E0/0
E2/0
E2/0
P4
PE1
7
TTL = 4
DA: 127.0.0.24
Mapsize/Hash:
32/8
Bitmap:0x00FF
June 3-6, 2012
E1/0
E1/0
P5
PE2
PE2
Egress of the FEC
8
NANOG55
59
LSP ping in ECMP topology
P2
P3
E0/0
E1/0
E0/0
E1/0
E2/0
E0/0
E0/0
PE1
1
TTL = 1
DA: 127.0.0.0
MapSize/hash: 32/8
Bitmap:0xFFFF
3
PE1
TTL = 2
DA: 127.0.0.24
Mapsize/Hash: 32/8
Bitmap:0x00FF
PE1
5
TTL = 3
DA: 127.0.0.24
Mapsize/hash: 32/8
Bitmap:0x00FF
7
PE1
TTL = 4
DA: 127.0.0.24
Mapsize/Hash: 32/8
Bitmap:0x00FF
June 3-6, 2012
E1/0
E1/0
E2/0
E1/0
PE1
E1/0
E0/0
9
PE1
TTL = 2
DA: 127.0.0.0
Mapsize/Hash: 32/8
Bitmap:0xFF00
11
PE1
TTL = 3
DA: 127.0.0.0
Mapsize/Hash: 32/8
Bitmap:0xF000
PE1
13
TTL = 4
DA: 127.0.0.0
Mapsize/Hash: 32/8
Bitmap:0xF000
E2/0
E2/0
P1
P1
MultiPath1 [E0/0]
Bitmap: 0x00FF
Multipath2[E2/0]
Bitmap: 0xFF00
E0/0
P4
2
P2
MultiPath1 [E0/0]
Bitmap: 0x00FF
P4
MultiPath1 [E0/0]
Bitmap: 0xF000
MultiPath2 [Eth2/0]
Bitmap: -0x0F00
15
PE1
TTL = 3
DA: 127.0.0.4
Mapsize/Hash: 32/8
Bitmap:0x0F00
PE2
P5
10
P3
MultiPath1 [E0/0]
Bitmap: 0x00FF
P3
MultiPath1 [E0/0]
Bitmap: 0xF000
P5
MultiPath1 [E0/0]
Bitmap: 0x0F00
PE2
Egress of the FEC
12
PE2
Egress of the FEC
16
PE2
Egress of the FEC
14
18
PE1
17
TTL = 4
DA: 127.0.0.4
Mapsize/Hash: 32/8
Bitmap:0x0F00
NANOG55
60
FEC types support
LSP ping supports various FEC types
FEC Type
LSP Ping
LSP Trace
ECMP Trace
LDP IPv4 and
IPv6
Yes
Yes
Yes
RSVP TE v4 and
v6
Yes
Yes
N/A
PW v4 and v6
Yes
MSPW(Yes)
Entropy Label
VPN v4 and v6
Yes
Yes
N/A
BGP v4 and v6
Yes
Yes
N/A
P2MP TE and
mLDP
Yes
Yes
N/A
MPLS-TP
Yes
Yes
N/A
June 3-6, 2012
NANOG55
61
LSP ping for Pseudowire FEC
Requirement
Solution
Applications
Solution
June 3-6, 2012
Provide end-to-end fault detection and diagnostic features for
emulated Pseudowire service
P2P PWE3
MS-PW end-to-end Ping and Trace
Static and Dynamic Pseudowires
VCCV provides control channel to allow control packets over
Pseudowires
VCCV capabilities are signalled using control protocols
Ability to support Control Word encapsulation
Router Alert labeled packets are to be punted
TTL exhaustion causes the packet to be processed
Layer 2 transport over MPLS
EoMPLS
FRoMPLS
ATMoMPLS
RFC5085
NANOG55
62
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
63
Bidirectional Forward Detection
(BFD)
Simple fixed-field, hello protocol
Packets are periodically transmitted over respective
directions of the path
If a node stops receiving BFD packets, some component
of the bidirectional path is assumed to have failed.
Several modes of operation
June 3-6, 2012
NANOG55
64
BFD protocol Overview
Typical hello protocol
Neighbors continuously negotiate transmit and receive
rates in micro seconds
Dynamic rate adaption
Neighbor is declared down when hello packets dont
show up
Uses UDP/IP or Non IP packets as BFD packets
Ability to support single-hop and multi-hop
June 3-6, 2012
NANOG55
65
BFD Timer negotiation
Neighbors continuously negotiate transmit and receive
rates
Designated UDP ports 3784 and 3785 are assigned to
BFD
Ability to support single-hop and multi-hop
Negotiated Rate
Desired Received rate = 50msec
Desired Transmit rate = 100msec
Negotiated Transmit rate = 100msec
June 3-6, 2012
Desired Received rate = 60msec
Desired Transmit rate = 40msec
Negotiated Transmit rate = 50msec
NANOG55
66
BFD for MPLS
Ability to verify LSP
BFD to verify TE tunnels, TP tunnels, PW LSPs etc
VCCV to be used to verify PW LSPs
BFD could be used to complement or replace use of
RSVP hellos for MPLS FRR Link/Node protection
BFD to carry AIS, RDI errors to end points of TP tunnels
BFD the primary mechanism to make fast switchover
and meet transport requirements
BFD to play complimentary role to provide OAM within
MPLS
June 3-6, 2012
NANOG55
67
LSP ping & BFD for MPLS-TP
LSP ping got enhanced to support TP LSPs
LSP ping plays crucial role in static TP LSPs.
Ability to perform MEP-MEP, MIP-MEP and MIP-MIP OAM
functions
BFD is used to fast detect failures
GAL label(13) to identify OAM and BFD packets
MIP
MEP
MIP
MEP
LSP Ping
BFD
LSP Ping
BFD
LSP Ping
BFD
June 3-6, 2012
LSP Ping & EndEnd BFD
NANOG55
68
Agenda
Introduction
Terms and Terminology
An Introduction to Tools
Introduction to MPLS
MPLS TP 101
Troubleshooting MPLS
MPLS OAM
LSP Ping
ECMP troubleshooting
BFD for MPLS
Tools Galore
June 3-6, 2012
NANOG55
69
Tools
CC and CV for MPLS networks using LSP Ping
Fault Isolation using traceroute with LSP Ping
Performance monitoring based on Y.1731 model
1:1, 1+1, 1:n and m:n protection supported using BFD
All FEC types supported using LSP ping
Provides support for IPv4 and IPv6
Automated tools built around LSP ping and other OAM
tools
No CCIE expertise required to use these tools
June 3-6, 2012
NANOG55
70
Summary of OAM tools
Continuity
Check
ICMP
BFD
Connectivity
Verification
Path
Discovery
Echo (Ping)
Traceroute
Defect
Performance
Indications Monitoring
BFD control BFD Echo
LSP Ping
Ping
Traceroute
IPPM
MPLS-TP
OAM
-Delay
- Packet loss
CC
CV
Traceroute -Alarm
Reporting
- Client
failure Ind
- Remote
Defect
-Delay
- Packet loss
Ref: draft-ietf-opsawg-oam-overview-05
June 3-6, 2012
NANOG55
71
Summary
June 3-6, 2012
NANOG55
72
Summary
MPLS OAM covers all types of MPLS networks
No CCIEs required to manage MPLS networks
Already built into major vendors MPLS devices
Deployed and being used in major carrier networks
Inter-op tests carried out at various labs prove the OAM
technologies WORK
MPLS-TP brought forth the usefulness of OAM in
transport networks
MPLS OAM a proven technology
June 3-6, 2012
NANOG55
73
Questions
June 3-6, 2012
NANOG55
74
