2019 International Workshop on Fiber Optics in Access Networks (FOAN)

Advancing IP/MPLS with Software Defined

Network in Wide Area Network
Irena Šeremet, Samir Čaušević
Communication Technology
Faculty for traffic and communication
Sarajevo, Bosnia and Herzegovina
[email protected] , [email protected]
Abstract— New technologies introduce more services configured for a particular VPN, each VPN has
which will call for significant advances and changes in separate tables, rules, and policies that control its
Multiprotocol Label Switching (MPLS) networks. operation [2]. MPLS as a technique is very flexible,
Networks will have to go in the direction of network adaptive, reliable and scales very quickly. With
programmability, virtualization and cloud-based IP/MPLS, the paths between end-points are dynamic
services. In this paper, we examined Software Defined and extremely resilient to failures; IP/MPLS will
Wide Area Network solution, its architecture and find a path as long as one exists, regardless of the
benefits of use. In order to analyze benefits of using number and locations of failures in the network [3].
SD-WAN two scenarios are compared: (i) Policy-based LSPs from source to destination are pre-determined
routing on WAN links in classical IP/MPLS network, so devices in LSP do not have to make decision on
and (ii) Using SDN controller to manage traffic on every hop. This allows faster data transfer and less
WAN links. Aim in both scenarios is to use the most load for routers. In spite of the many advantages of
optimal path for specific network.
MPLS, the development of new technologies results
Key words: IP, MPLS, SD-WAN
in different needs and requirements of end users.
When MPLS was created, applications were not in
I. INTRODUCTION the cloud and users were not accessing corporate
applications from mobile devices [4]. MPLS was an
Multiprotocol Label Switching is a protocol-
adequate technology for that-time demands. Today,
agnostic technique designed to direct data from
new technologies introduce more services which
source to destination based on labels rather than IP
will call for significant advances and changes in
addresses. All packet-forwarding decisions are
MPLS networks. Networks will have to go in the
made on the contents of assigned label, without the
direction of network programmability, virtualization
need to open and examine the IP packet. Routers in
and cloud-based services [5]. In second section of
an MPLS network exchange label information with
this paper, we examine programmable networks,
each other using Label Distribution Protocol (LDP)
especially Software Defined Networks - SDN. In the
protocol. When sending packets to specific network
third section, we describe using SDN in WAN and
over MPLS, devices check Label Forwarding
SD-WAN architecture. In the fourth section, we
Information Base (LFIB) tables and examine which
compare routing and traffic management in two
label to use for specific network. One of the key
scenarios. In first scenario policy-based routing is on
features that MPLS support are traffic engineering
WAN links in IP/MPLS network used, and in second
(TE), Virtual Routing and Forwarding (VRF) and
scenario control of routing and traffic management
L2/L3 Virtual Private Networks (VPNs). Traffic
over WAN links is delegated to a SDN controller.
engineering enables service providers to route
We conclude our paper focusing on benefits of using
network traffic offering the best service to their
Software Defined Wide Area Networks in section
users in terms of throughput and delay. MPLS traffic
five.
engineering automatically establishes and maintains
Label Switched Paths - LSPs across the network, II. PROGRAMMABLE NETWORKS
using Resource Reservation Protocol-RSVP.
Depending on customer requirements, MPLS VPNs With higher demands in network predictability,
can be (i) point-to-point (ii) Layer 2 (iii) Layer 3 [1]. reliability, and performance, better management in
Layer 2 and Layer 3 MPLS VPNs enable customers networks became crucial. Service providers started
to have point-to-multipoint VPN connections. A to implement more intelligent, flexible and
VRF consists of one or more routing tables, a programmability-enabled networks.
derived forwarding table, the interfaces that use the Programmability-enabled networks [6] are driven by
forwarding table, and the policies and routing intelligent software and use Application
protocols that determine what goes into the Programmable Interfaces (APIs) which serve as the
forwarding table. Because each instance is interface to the device or controller in order to gather

978-1-7281-1563-4/19/$31.00 ©2019 IEEE

56
 2019 International Workshop on Fiber Optics in Access Networks (FOAN)

data or intelligently build configurations. Software virtual overlay networks that may use a different
Defined Networking (SDN) is an architecture that underlay service/technology, e.g., Dedicated
decouples control plane and data plane achieving Internet Access, Broadband Internet (Cable, DSL or
flexible and intelligent networks. Control plane is PON), Internet over LTE, MPLS over T1s, or MPLS
responsible for building and maintaining routing over fiber. Because IP based SD-WANs are virtual
table while data plane is responsible for actual overlay networks, no modifications have to be made
forwarding packets. In traditional IP networks, to any of the underlay networks. SD-WANs also
every network device has its own control and data support any topology, e.g., full/partial mesh and hub
plane. In full SDN solution, network devices will
& spoke [8].
only have data plane and be responsible for
forwarding data. Control plane for whole network A. SD-WAN architecture
segment will be centralized and placed in one or
more SDN controllers. SDN controllers will make As shown in the Figure 1 and Figure 2, SD-WAN
routing decision and maintain routing table. architecture contains four main parts: (i) a
Through different protocols, controllers will instruct mechanism to orchestrate connectivity
network devices how to handle the packet. SDN (orchestrator), (ii) appliance to provide management
architecture contains three layers: (i) infrastructure capabilities for this environment (management), (iii)
layer, which represents physical routers and mechanism to enforce all different topologies and
switches; (ii) control layer, which is centralized policy enforcements (control), and (iv) a carrier for
controller responsible for managing devices in applications and data (data).
infrastructure layer and (iii) application layer with
applications interacting with lower layers. Orchestrator, management and control parts are
Applications communicate with controller through virtual entities and can reside on premise or within
northbound interfaces or APIs and controller the cloud. Each of these entities will be responsible
communicates with infrastructure devices through for providing specific functions and it will be
southbound interfaces such as OpenFlow, Border consumed by a customer as a service. In data part
Gateway Protocol-Link State (BGP-LS), Path are actual network elements such as actual physical
Computation Element Protocol (PCEP), Netflow, or virtual router. When network device is on, it is
Netconf, etc. The SDN provides programmability of already programmed according to the customer’s
a control plane and automation of configurations needs and basic customer’s configuration is
through a centralized controller and open APIs. installed. Then encrypted channel between network
Network operators can implement their own device and orchestrator is established. Through that
protocols, rules and policies with common channel, orchestrator checks authentication,
programming languages achieving flexible control authorization and certificates of the router. The main
over network services such as routing, traffic role of orchestrator is to validate identity of network
engineering, QOS and security [5]. devices in the data part. By now, router has
permission to access network, but it does not have
III. SOFTWARE DEFINED WIDE AREA any configuration. Management entity has the role
NETWORKS of configuring network devices.

With SD-WAN, the advantages of SDN are no

longer limited to the data center. SD-WAN is a Orchestrator
concept of implementing SDN to WAN connections entity
such as broadband internet, MPLS, 4 or 5G mobile
networks etc. SD-WAN is managed by a centralized
controller and uses SDN to automatically determine Management
the best route between two sites. Also it has the entity
ability to monitor links and if needed, dynamically
route traffic to links with enough bandwidth for each
application’s demand. Unlike other network
connectivity services, SD-WANs use application- Control
driven networking where application traffic is entity
forwarded over different WANs based on QoS,
Security and Business priority policies [7]. SD-
WANs use policies to make application routing
LTE

decisions for SD-WANs tunnels over each WAN

link [8]. Policy considers an application’s or
customer’s different requirements such as QoS
performance or security requirements. For example,
a QoS policy may be set so voice packets are
forwarded over any WAN as long as its QoS
performance requirements, e.g., packet latency and Figure 1 SD-WAN architecture
loss, are met. SD-WAN provide secure, IP-based Figure 1 SD-WAN architecture

57
 2019 International Workshop on Fiber Optics in Access Networks (FOAN)

As shown in the Figure 3, topology in both

Network scenarios containes six routers connected with eight
control entity
links. Due to insufficient number of physical test
Network Network devices, configuration in both scenarios was
orchestrator management implemented on virtual routers in Cisco’s Cloud
entity entity virtual environment [9]. Used routers are Cisco’s
ASR 9000 series with IOS XR. Each router is
configured with loopback interface as an ID, where
Router 1 is configured with 1.1.1.1/32, Router 2 with
2.2.2.2/32 and so on. On the routers IS-IS [10] is
configured as a routing protocol, and MPLS TE
process in enabled. In scenarios, router 6 is sending
two additional prefixes in the network: 7.7.7.7/32
and 8.8.8.8/32. Communication from R1 to prefix
7.7.7.7/32 is latency-sensitive communication and it
has higher priority than communication from R1 to
Network devices 8.8.8.8/32. Also, in both scenarios, links R2 - R4 and
providing data R4-R5 are high latency links and R1-R3, R3-R5 are
low latency links. Aim is to avoid high latency links
when sending traffic from R1 to prefix 7.7.7.7, and
Figure 2: SD-WAN components use any other paths when sending traffic from R1 to
Network device establishes encrypted channel prefix 8.8.8.8.
between device and management entity. Through
that channel, network device sends to management A. Scenario 1 – Classical IP/MPLS network
entity data about telemetry, availability, statistics In this scenario classical IP/MPLS network is
etc. At the same time, management entity sends configured using BGP routing policies. As shown in
configuration to network device such as vlans, the Figure 3, R6 is sending two prefixes via BGP:
interfaces, routing protocols etc. After network 7.7.7.7/32 and 8.8.8.8/32. For R1, prefix 7.7.7.7 has
device is configured, routing and policy updates higher priority. On R6 route policy
from the system are needed. Network device create LOW_LATENCY is configured, marked with
encrypted channel with control entity to get system 100:100 community and attached to prefix 7.7.7.7/32
updates. The control part of architecture is the most under BGP process. Under BGP process, also prefix
intelligent part, containing one or more controllers 8.8.8.8/32 is configured without any attached routing
that are in charge of managing routing processes, policies. On R1 two TE attributes are defined: (i)
creating tunnels, policy enforcement and so on. LOW_LATENCY; (ii) ANY_OTHER. On routers
Network device exchange different information with R3 and R5 LOW_LATENCY attribute is defined,
controller such as: connected circuits (Internet, and on R2 and R4 HIGH_LATENCY attribute is
MPLS, LTE), VPNs or VRFs and all learned defined. First attribute on R1 is for tunnels which
networks in specific VRF etc. have to avoid links with HIGH_LATENCY attribute
(defined on R2 and R4), and second attribute is for
After identifying, getting configurations and all other tunnels. When R1 receives prefixes from R6, it
needed updates from system, network device is maps those prefixes to one of two attributes based on
ready to communicate with other network devices in the 100:100 community presence. If community
the network. Controller has visibility about all 100:100 is received, R1 mapps prefix on
attributes and information in the network. This gives LOW_LATENCY attribute and send traffic to R3.
opportunity to a network provider to configure With these routing policies, traffic from R1 to prefix
several connections over different circuits defining 7.7.7.7/32 will avoid path with HIGH_LATENCY
different policies. attribute defined, which are R2 and R4, and use path
with LOW_LATENCY attribute defined, which are
IV. COMPARING CLASSICAL IP/MPLS R3 and R5.
AND SD-WAN SOLUTION
B. Scenario 2 – Using SDN controller
Policy-based routing is possible to configure in As mentioned before, SDN architecture
classical IP/MPLS networks using routing policies containes three layers and two interfaces connecting
under Interior Gateway Protocol (IGP) or BGP. But those three layers. Starting from the bottom of
thiese solutions have some limitations. In order to architecture, on infrastructure layer same 6 routers
achieve optimal usage of WAN links, some from Scenario 1 are used. As a southbound
requirements have to be satified: (i) Global view of protocols, BGP and PCEP [11], [12], [13] are used.
the network, (ii) Constant measurment of link states, On the controller layer, OpenDayLight (ODL) [14],
(iii) Dynamic rerouting traffic to links with better [15], [16] controller is used. It is configured on
parameters. In this section, defined routing policies Ubuntu virtual machine. On the application layer,
on WAN links are configured in two scenarios: (i) due to lack of any SDN application, only basic
In classical IP/MPLS network and (ii) Using SDN scripts are used with RESTCONF APIs of the ODL
controller. controller as a northbound interface.

58
 2019 International Workshop on Fiber Optics in Access Networks (FOAN)

Figure 3 Used topology in both scenarios

Figure 4 Configuration in the first scenario

All scripts are based on a Newman [17], which On the one hand, BGP-LS provides TED
is open source REST CLI client previously installed (Traffic Engineering Database) information on
on the ODL server. BGP Link-State (BGP-LS), also topology and link conditions (bandwidth, cost,
known as BGP for Traffic Engineering or BGP-TE, existing LSPs, TE metrics, etc.), while PCEP
is a new BGP address family allowed to carry link- communicates between PCE and PCC. After
state information. This link state information is installing PCEP plugin on ODL, session between
acquired from the interior gateway protocol – IGP, routers (PCCs) and ODL (PCE) has to be
which is IS-IS in this case. PCEP is protocol for established.
communication between two elements: (i) Path
Computation Element - PCE and (ii) Path
Computation Client - PCC. Path Computation
Element is a server from the perspective of PCEP.
PCE has the global view of network topology and
resources enabling centralized path computation and
applying TE policies. On the other hand, Path
Computation Clients are network devices that take
instructions from PCE and make those instructions
as local configurations. In this scenario, routers
(R1-R6) are configured as clients – PCCs, and ODL
controller is configured as PCE. Instructions from
PCE to PCC are transferred via PCEP protocol. In
this scenario, BGP-LS and PCEP work together as Figure 5 Topology in Scenario 2 using SDN controller
southbound protocols in this SD-WAN solution.

59
 2019 International Workshop on Fiber Optics in Access Networks (FOAN)

When PCEP session is established, using script to analyze benefits of using SDN in WAN. Both
with HTTP GET request and corresponding REST scenarios contain same number of routers and links,
API URL ODL gets whole topology of network. On and have the same requested routing policy:
routers, all control over routing and creating tunnels Communication from Router 1 to prefix 7.7.7.7/32
is delegated to PCE element. With previously (communication A) request low latency and has
written JSON scripts for managing routing or traffic higher priority than communication from Router 1
engineering process only command to run a specific to prefix 8.8.8.8/32 (communication B). In order to
script using ODL RESTCONF API is needed. satisfy low-latency requirement for Communication
Controller can be programmed to send IP SLA A, tasks in the network are related to route
probes in order to examine loss, latency and jitter Communication A over better links. In first scenario,
on the network. Also, different dynamical routing routing policy is configured using BGP SR-TE
policies can be programmed on the controller. These routing policies. Each communication is manually
policies can define different types of traffic and marked with specific BGP community and sent over
ways of treating different types of traffic. In our specific path. Measurments of the network are
example, traffic from R1 do 7.7.7.7/32 has higher previously done, and results showed that R1-R3, R3-
priority than R1-8.8.8.8/32. Further on, R5 are low latency links, so Communication A is sent
communication from Router 1 to prefix 7.7.7.7/32 is through that links using 100:100 BGP community. In
presented as Communication A, and communication the second scenario, SDN controller is programmed
from Router 1 to prefix 8.8.8.8/32 is presented as to constantly measure states on link and dynamically
Communication B. Network management entity route Communication over better link. If quality
policy route for Communication A is configured to level on link R1-R2-R4-R5 ever become better,
transfer packets over better link, and at the same prioritized traffic is rerouted automatically on that
time continuously examines congestions, loss, and link, which is impossible to achieve in the first
latency on links. If quality level on link R2-R4, R4- scenario. In the first scenario, this automatization is
5 ever become better, prioritized traffic is rerouted impossible because we manually had to configure
automatically on that link. Controller automatically routing policies over high-latency or low-latency
manages different parameters of network based on paths. Communication A will always use R1-R3-
configured policies. In order to test the second R5-R6, even if latency on that link increases.
scenario, we started ping from R1 to 7.7.7.7/32 and Comparing these two scenarios, the benefits of using
we analyzed MPLS forwarding table in two cases: SDN in WAN are obvious. SDN controller allows
(i) when R1-R3-R5-R6 is low-latency link, (ii) when much more possibilities when configuring routing
R1-R2-R4-R6 is low latency link. R1 is connected policies than BGP. In the first scenario, there is no
to R2 via Gi0/0/0/0, and with R3 via Gi0/0/0/1 port. network element that has global view of the
In order to easier analyse labels and outgoing network. Each router makes decisions
interfaces, we manually configured that label on R1 independently for itself without being aware of the
to R2 is 16002, R1 to R3 16003 and so on. Since other network routers’ requirements. SDN
prefix 7.7.7.7/32 is directly connected on R6, controller, on the other hand, has global view of
analyzed label is label number 16006. In the first whole network topology and all WAN links. Also,
case, when analyzing the output of show mpls in the first scenario, static route policy under BGP
forwarding command, label 16006 has two outgoing process is manually added.
interfaces (Gi0/0/0/0, which connects R1 and R2;
Gi0/0/0/1 which connects R1 and R3), but only
Gi0/0/0/1 is used. We can conclude that because we
can see that 100 bytes is switched through Gi0/0/0/1
and 0 bytes is switched through Gi0/0/0/0. That
means that communication is going through R1-R3-
R5-R6 link. Output of show mpls forwarding
command is presented in Figure 6. In the second
case, we increased latency on R1-R3 link by
manually decreasing the bandwidth. After
increasing the latency on link and clearing MPLS Figure 6 Output of show mpls forwarding command-Case I
forwarding counters, we started ping from R1 to
7.7.7.7/32 and analyzed the output of show mpls
forwarding command again. This time, 100 bytes
switched through Gi0/0/0/0 and 0 bytes through
Gi0/0/0/1, which means that R1-R2-R4-R6 link is
used. Output of show mpls forwarding command in
the second case is shown on the Figure 7.
V. CONCLUSION
In this paper, routing policy on WAN links in
classical IP/MPLS network is compared to scenario Figure 7 Output of show mpls forwarding command-Case II
where SDN controller is added to a network in order

60
 2019 International Workshop on Fiber Optics in Access Networks (FOAN)

If any quality changes on path R1-R3-R5 occurs,

communication A will not use better path until [7] S. W. Sanjay Uppal, S. Woo and D. Pitt, Software-
Defined WAN SD-WAN. ISBN: 978‐1‐119‐10148‐2
network engineer changes routing policy manually. [8] MEF, “Understanding SD-WAN Managed Services,”
On the other hand, PCE on ODL can be programmed no. July, p. 15, 2017., Available:
to monitor links and dynamically route different https://www.mef.net/resources/download?id=45&filei
types of traffic in accordance with requirements. d=file1
Advancing MPLS with SD-WAN has many [9] “Cisco dCloud Virtual Environment.” [Online].
Available: https://dcloud2-
benefits. First of all, since SD-WAN is agnostic to
lon.cisco.com/content/demo/246986.
the underlying physical transport, implementation is [10] “Implementing IS-IS on Cisco ASR 9000 Series
easy to deploy without replacing and changing the Routers,” Cisco. [Online]. Available:
existing MPLS network. Also, SD-WAN is less https://www.cisco.com/c/en/us/td/docs/routers/asr900
expensive solution, improves application 0/software/routing/configuration/guide/rcasr9kisis.htm
performance, simplifies the network by automating l.
[11] J. A. A. Farrel, J.-P. Vasseur, “A Path Computation
site deployments, configurations and operations. Element (PCE)-Based Architecture.” p. 40, 2006.
SD-WAN improves application performance [12] F. Paolucci, F. Cugini, A. Giorgetti, N. Sambo, and P.
through a combination of WAN optimization Castoldi, “A survey on the path computation element
techniques and its ability to dynamically shift traffic (pce) architecture,” IEEE Commun. Surv. Tutorials,
to links with bandwidth sufficient enough to vol. 15, no. 4, pp. 1819–1841, 2013.
[13] Cisco, “Dynamic Path Computation Client,” Cisco.
accommodate each application's requirements. [Online]. Available:
REFERENCES https://www.cisco.com/c/en/us/td/docs/routers/asr920/
configuration/guide/segment-routing/16-6-1/segment-
[1] “MPLS VPNs.” [Online]. Available: routing-book/seg-routing-dynamic-pcc.html.
https://en.wikipedia.org/wiki/MPLS_VPN. [14] J. Medved, R. Varga, A. Tkacik, and K. Gray,
[2] Juniper, “MPLS VPN Overview.” [Online]. Available: “OpenDaylight: Towards a model-driven SDN
https://www.juniper.net/documentation/en_US/junos/t controller architecture,” Proceeding IEEE Int. Symp. a
opics/concept/mpls-security-vpn-overview.html. World Wireless, Mob. Multimed. Networks 2014,
[3] Packet Design, “Understanding and Managing IP / WoWMoM 2014, 2014.
MPLS Mobile Backbone and Backhaul Networks [15] OpenDayLight, “Downloading and installing
White Paper.” 2015. Available: OpenDaylight,” OpenDaylight Project, 2016. [Online].
https://www.blueplanet.com/?src=packetdesign Available: https://test-odl-
[4] Expereo, “Reasons to reconsider mpls.” . Available: docs.readthedocs.io/en/latest/getting-started-
https://www.expereo.com/9-reasons-make-switch- guide/installing_opendaylight.html.
mpls/ [16] OpenDayLight, “OpenDaylight User Interface
[5] I. Šeremet and S. Čaušević, “Evolving IP / MPLS (DLUX),” OpenDaylight Project, 2016. [Online].
network in order to meet 5G requirements,” no. March, Available: https://docs.opendaylight.org/en/stable-
pp. 20–22, 2019., Available: nitrogen/getting-started-guide/common-
https://infoteh.etf.ues.rs.ba/zbornik/2019/radovi/KST- features/dlux.html.
1/KST-1-2.pdf [17] “Command line integration with Newman,” Postman
[6] T. Ryan, Programming and Automating Cisco learning center. [Online]. Available:
Networks. Cisco Press, 2017. https://learning.getpostman.com/docs/postman/collecti
on_runs/command_line_integration_with_newman/.

61