Cisco Support Community
Expert Series Webcast
Troubleshooting Dynamic Multipoint VPN
Hamzah Kardame, CCIE Security
Frank DeNofa
June 7, 2016
Upcoming Ask the Expert Events
IOS-XR Architecture and Troubleshooting
Details - https://supportforums.cisco.com/discussion/13025746/ask-expert-ios-xr-architecture-and-
troubleshooting
Raj Pathak Sudhir Kumar
Become an Event Top Contributor
Participate in Live
Interactive
Technical Events
and much more
http://bit.ly/1jlI93B
https://supportforums.cisco.com/expert-corner/top-contributors
Now your ratings on documents, videos,
Rate and blogs count give points to the
authors!!!
Content So, when you contribute and receive
ratings you now get the points in your
profile.
Help us to recognize the quality content
in the community and make your
searches easier. Rate content in the
Encourage and community.
acknowledge people https://supportforums.cisco.com/blog/154746
who generously share
their time and expertise
Cisco Support Community Expert Series Webcast
Hamzah Kardame
CCIE Security #35596
Frank DeNofa
Cisco TAC
Question Managers
Danny Lia
Cisco TAC
Ask the Expert Event following the Webcast
Now through June 17th
https://supportforums.cisco.com/event/13023496/ask-
expert-dynamic-multipoint-vpn-dmvpn-troubleshooting
Join the discussion for these Ask The Expert Events:
http://bit.ly/events-webinar
Thank You For Joining Us
Today!
If you would like a copy of the presentation slides, click the
PDF file link in the chat box on the right or go to:
Need the link here
Submit Your Questions Now!
Please take a moment
Use the Q & A panel to submit your to complete the survey
questions and the panel of experts at the end of the
webcast
will respond.
Cisco Support Community
Expert Series Webcast
Troubleshooting Dynamic Multipoint VPN
Hamzah Kardame, CCIE Security
Frank DeNofa
June 7, 2016
DMVPN Phase 1/2/3
Overview
Deep Dive on Phase 3
Layered Troubleshooting
Approach for DMVPN
Agenda Live Demonstration of Phase
3 Operations
Troubleshooting Common
Scenarios
Best Practices
QnA
Which DMVPN phase allows for direct
spoke to spoke communication?
Polling A. Only Phase 3
B. Phase 1 and Phase 2
Question 1 C. Phase 2 and Phase 3
D. Phase 1 and Phase 3
DMVPN Components
Next Hop Resolution Protocol (NHRP)
Creates a distributed (NHRP) mapping database of all the spokes tunnel to real (public interface)
addresses
Multipoint GRE Tunnel Interface (mGRE)
Single GRE interface to support multiple GRE/IPsec tunnels
Simplifies size and complexity of configuration
IPsec tunnel protection
Dynamically creates and applies encryption policies
Routing
Dynamic advertisement of branch networks; almost all routing protocols are supported
DMVPN Example
192.168.0.0/24
Static Spoke-to-hub tunnels
.1 LANs can have
private addressing
Dynamic Spoke-to-spoke tunnels
Physical: 172.17.0.1
Tunnel0: 10.0.0.1
Static known
IP address
Physical: dynamic
Tunnel0: 10.0.0.12
Dynamic
unknown
IP addresses Spoke B
.1
192.168.2.0/24
Physical: dynamic
Tunnel0: 10.0.0.11
Spoke A
.1 ...
192.168.1.0/24
Phase 1 Phase 2 Phase 3
Hub and spoke Spoke to spoke Spoke to spoke
functionality functionality functionality
p-pGRE interface on mGRE interface on Spokes dont need
spokes
spokes, mGRE on full routing table
hubs Direct spoke to spoke can summarize
data traffic reduces load
Spokes dont need on hubs Spoke-spoke tunnel
full routing table triggered by hubs
can summarize on Spoke must have full
routing table no NHRP routes/next-
hubs summarization hops in RIB
Spoke-spoke tunnel
triggered by spoke itself
DMVPN Phase 3 Deep Dive
1. Originating spoke
IP Data packet is forwarded out tunnel interface to destination via Hub (NHS)
2. Hub (NHS)
Receives and forwards data packet on tunnel interfaces with same NHRP Network-id.
Sends NHRP Redirect message to originating spoke.
3. Originating spoke
Receives NHRP redirect message
Sends NHRP Resolution Request for Data IP packet destination
4. Destination spoke
Receives NHRP Resolution Request
Builds spoke-spoke tunnel
Sends NHRP Resolution Reply over spoke-spoke tunnel
Phase 3 NHRP Redirects
Data packet 192.168.0.1/24 10.0.0.11 172.16.1.1
NHRP Redirect 10.0.0.12 172.16.2.1
NHRP Resolution
Physical: 172.17.0.1 192.168.0.0/24 Conn.
NHRP mapping Tunnel0: 10.0.0.1 192.168.1.0/24 10.0.0.11
192.168.2.0/24 10.0.0.12
CEF FIB Table
Physical: 172.16.2.1
(dynamic)
Physical: 172.16.1.1
(dynamic) Tunnel0: 10.0.0.12
Tunnel0: 10.0.0.11
Spoke A Spoke B 192.168.2.1/24
192.168.1.1/24
10.0.0.1 172.17.0.1
10.0.0.1 172.17.0.1
192.168.2.1 ???
192.168.2.0/24 Conn.
192.168.1.0/24 Conn. 192.168.0.0/16 10.0.0.1
192.168.0.0/16 10.0.0.1
Phase 3 NHRP Resolution Request
Data packet 192.168.0.1/24 10.0.0.11 172.16.1.1
NHRP Redirect 10.0.0.12 172.16.2.1
NHRP Resolution
Physical: 172.17.0.1 192.168.0.0/24 Conn.
NHRP mapping Tunnel0: 10.0.0.1 192.168.1.0/24 10.0.0.11
192.168.2.0/24 10.0.0.12
CEF FIB Table
Physical: 172.16.2.1
(dynamic)
Physical: 172.16.1.1
(dynamic) Tunnel0: 10.0.0.12
Tunnel0: 10.0.0.11
Spoke A Spoke B 192.168.2.1/24
192.168.1.1/24
10.0.0.1 172.17.0.1
10.0.0.1 172.17.0.1 10.0.0.11 172.16.1.1
192.168.2.1 ???
192.168.2.0/24 Conn.
192.168.1.0/24 Conn. 192.168.0.0/16 10.0.0.1
192.168.0.0/16 10.0.0.1
Phase 3 NHRP Resolution Reply
Data packet 192.168.0.1/24 10.0.0.11 172.16.1.1
NHRP Redirect 10.0.0.12 172.16.2.1
NHRP Resolution
Physical: 172.17.0.1 192.168.0.0/24 Conn.
NHRP mapping Tunnel0: 10.0.0.1 192.168.1.0/24 10.0.0.11
192.168.2.0/24 10.0.0.12
CEF FIB Table
Physical: 172.16.2.1
(dynamic)
Physical: 172.16.1.1
(dynamic) Tunnel0: 10.0.0.12
Tunnel0: 10.0.0.11
Spoke A Spoke B 192.168.2.1/24
192.168.1.1/24
10.0.0.1 172.17.0.1
10.0.0.1 172.17.0.1 10.0.0.11 172.16.1.1
10.0.0.12 172.16.2.1 192.168.1.0/24 172.16.1.1
192.168.2.1 ???
192.168.2.0/24 172.16.2.1
192.168.2.0/24 Conn.
192.168.1.0/24 Conn. 192.168.0.0/16 10.0.0.1
192.168.0.0/16 10.0.0.1
DMVPN Phase 3 Deep Dive
When the spoke-spoke DMVPN establishes successfully, NHRP
introduces the shortcuts into the routing table in two ways
H or NHRP routes
% or next-hop override routes
These shortcuts allow the router to forward traffic directly over the
spoke-spoke DMVPN tunnels rather than over the spoke-hub-spoke
path.
DMVPN Phase 3 Deep Dive
[ NHRP learned prefix-length ] [ NHRP learned prefix-length ] [ NHRP learned prefix-length ]
> = <
[ current prefix-length in RIB ] [ current prefix-length in RIB ] [ current prefix-length in RIB ]
[ NHRP learned prefix-length ] < [ NHRP learned prefix-length ]
[current prefix-length in RIB] = [current prefix-length in RIB]
NHRP overrides the next-
NHRP will adjust the prefix-
NHRP will insert a new, hop of the route present in
length it learns to be equal
more specific route marked the routing table. The %
to that present in the
with H and own it will appear for the route in
routing table
question.
NHRP then overrides the
NHRP will monitor the Route is not 'owned' by next-hop of that route. The
parent route NHRP % will appear for the route
in question
DMVPN Phase 3 Deep Dive
Spoke1#show ip route eigrp
21.0.0.0/24 is subnetted, 1 subnets
D % 21.21.21.0 [90/76800256] via 10.1.1.100, 00:12:12, Tunnel1
Spoke1#show ip route next-hop-override
21.0.0.0/24 is subnetted, 1 subnets
D % 21.21.21.0 [90/76800256] via 10.1.1.100, 00:11:16, Tunnel1
[NHO][90/255] via 10.1.1.2, 00:04:42, Tunnel1
DMVPN Phase 3 Deep Dive
Spoke1#show ip route eigrp
21.0.0.0/16 is subnetted, 1 subnets
D 21.21.0.0 [90/76800256] via 10.1.1.100, 00:00:24, Tunnel1
Spoke1#show ip route next-hop-override
21.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
D 21.21.0.0/16 [90/76800256] via 10.1.1.100, 00:01:43, Tunnel1
H 21.21.21.0/24 [250/255] via 10.1.1.2, 00:00:41, Tunnel1
Layered Troubleshooting Methodology
Physical and routing layer
IPsec encryption layerIPsec/ISAKMP
GRE encapsulation layerNHRP
VPN routing layerrouting and IP data
DMVPN Layers
Before You Begin
Sync up timestamps between the routers in question.
Enable msec level timestamping for debugs and syslogs
service timestamps debug datetime msec
service timestamps log datetime msec
Enable timestamping for show command outputs
terminal exec prompt timestamp
Review configuration on both endpoints to ensure that nothing has
been missed, altered or incorrectly configured.
Layered Troubleshooting Methodology
Is connectivity present between the two Ping | Traceroute
routers NBMA IP addresses over UDP 500,
UDP 4500, ESP/IP Proto 50 bi-directionally? Netflow/FNF | EPC | ACL hitcounts
show dmvpn | show crypto isakmp sa detail | show crypto ikev2 sa |
Is the VPN tunnel UP and stable? show crypto session detail
Do the encrypt and decrypt counters Do pings between the tunnel interface IP addresses work?
increment for the given IPsec SA, at both show crypto ipsec sa peer x.x.x.x
routers?
debug dmvpn all crypto*
Hub-Spoke : Does the NHRP entry at the hub for the
given spoke's tunnel IP, show as 'registered'? show ip nhrp detail | show ip nhrp traffic |
show ip nhrp nhs
Spoke-Spoke : Does each spoke have a 'dynamic'
NHRP entry present for the other, or, does it show as debug dmvpn all nhrp*
'incomplete?
show ip route [eigrp | ospf | rip]
Is the routing protocol neighborship UP and
stable? show ip [eigrp | ospf] neighbor
Are routes being learned at both routers, over the show ip nhrp multicast
Tunnel interface? ping [ 224.0.0.10 (eigrp) | 224.0.0.5
(ospf) | 224.0.0.9 (rip) ]
Layered Troubleshooting Methodology : IPsec
Encryption Layer
Is the DMVPN tunnel UP?
IKEv1
# show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
172.16.101.1 172.16.21.1 QM_IDLE 1010 ACTIVE
IKEv2
# show crypto ikev2 sa
IPv4 Crypto IKEv2 SA
Tunnel-id Local Remote fvrf/ivrf Status
1 172.16.1.1/500 172.16.1.2/500 none/none READY
Encr: AES-CBC, keysize: 256, Hash: SHA96, DH Grp:14, Auth sign: RSA,
Auth verify: RSA
Life/Active Time: 86400/53 sec
Layered Troubleshooting Methodology :
IPsec Encryption Layer
Check for IPsec SAs via show crypto ipsec or:
# show crypto session detail
Interface: Tunnel0
Uptime: 00:04:41
Session status: UP-ACTIVE
Peer: 172.16.101.1 port 500 fvrf: (none) ivrf: (none)
Phase1_id: 172.16.101.1
Desc: (none)
Session ID: 0
IKEv1 SA: local 172.16.21.1/500 remote 172.16.101.1/500 Active
Capabilities:(none) connid:1050 lifetime:23:55:18
IPSEC FLOW: permit 47 host 172.16.21.1 host 172.16.101.1
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 5 drop 0 life (KB/Sec) 4284407/3318
Outbound: #pkts enc'ed 5 drop 0 life (KB/Sec) 4284407/3318
Layered Troubleshooting Methodology : GRE
Encapsulation Layer
All traffic passing over the DMVPN tunnel will be GRE encapsulated and then
encrypted
NHRP will be the first component to utilize this layer
Check if spoke registered successfully with the hub
Spoke# show ip nhrp nhs
Legend: E=Expecting replies, R=Responding, W=Waiting
Tunnel1:
10.1.1.100 RE priority = 0 cluster = 0
10.1.1.123 E priority = 0 cluster = 0
Hub# show ip nhrp
10.1.1.1/32 via 10.1.1.1
Tunnel1 created 00:03:33, expire 01:56:26
Type: dynamic, Flags: unique registered used nhop
NBMA address: 172.16.11.100
(Claimed NBMA address: 10.10.10.1)
A DMVPN spoke fails to successfully
establish routing protocol neighborship
to the hub over the tunnel but pings from
the spokes tunnel IP to the hubs tunnel
IP work. Which layer(s) do we need to
Polling troubleshoot?
A. Only the Routing Layer
Question 2 B. Both Encryption and Routing Layers
C. Both GRE/NHRP and Routing
Layers
D. All the four layers
Troubleshooting
A TAC approach
Identify the exact issue
What traffic is affected? Hub to Spoke? Some Spoke to Spoke? All Spoke to Spoke? Host to
host? All Internet-destined traffic?
Is routing stable?
How long has the issue been occurring? Any notable changes?
Confirm a data plane or control plane issue
Does routing look correct?
Does NHRP/crypto trigger and complete?
Is our data plane path correct?
Could there be loss in the path?
Troubleshooting
General crypto failures
UDP500/4500/ESP traffic blocked in transit
While ICMP may pass, its very possible other protocols could be blocked
ISAKMP profile/IPsec profile misconfigurations
All ISAKMP/IPsec policy settings must match exactly
If using PKI for authentication, MM5/MM6 messages will be larger and are
subject to fragmentation
Troubleshooting
Progression of show crypto isakmp sa states during six packet
exchange
MM_NO_STATE
MM_SA_SETUP
MM_KEY_EXCH
QM_IDLE
Troubleshooting
MM1/MM2 failure
Typically indicates misconfiguration or UDP500 connectivity issue
Crypto_Initiator#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
172.16.101.1 172.16.11.100 MM_NO_STATE 0 ACTIVE
Crypto_Responder#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
172.16.101.1 172.16.11.100 MM_SA_SETUP 0 ACTIVE
Troubleshooting
MM5/MM6 failure
Typically indicates authentication or UDP4500 connectivity issue
Crypto_Initiator#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
172.16.101.1 10.10.10.1 MM_KEY_EXCH 1005 ACTIVE
Crypto_Responder#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
172.16.101.1 172.16.11.100 MM_KEY_EXCH 1007 ACTIVE
172.16.101.1 172.16.21.1 QM_IDLE 1010 ACTIVE ### QM_IDLE MAY BE MISLEADING
*Jun 1 14:10:42.703: %CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from 172.16.11.100 failed its sanity
check or is malformed
Troubleshooting
NAT issues
While Spoke to Spoke tunneling can work through NAT, there are some
caveats
If a Spoke is behind dynamic NAT, it must be the peer which initiates crypto in
order to create the necessary translations
Spoke to Spoke will not build with both Spokes behind dynamic NAT
Troubleshooting
NHRP issues
Barring any configuration issues, NHRP usually fails as a result of an issue
external to NHRP (GRE, IPsec, transit network)
debug nhrp packet can be helpful in reviewing packet flow
Troubleshooting Flexible NetFlow (FNF)
flow record test-record interface GigabitEthernet2
match ipv4 protocol ip vrf forwarding ISP_1
match ipv4 source address ip address 172.16.101.1 255.255.255.0
match ipv4 destination address ip flow monitor test-monitor input
match transport source-port ip flow monitor test-monitor output
match transport destination-port negotiation auto
collect flow direction end
collect counter packets
flow monitor test-monitor
record test-record
Troubleshooting Flexible NetFlow (FNF)
Hub1#show flow monitor test-monitor cache format table
Cache type: Normal (Platform cache)
Cache size: 200000
Current entries: 6
High Watermark: 6
Flows added: 6
Flows aged: 0
IPV4 SRC ADDR IPV4 DST ADDR TRNS SRC PORT TRNS DST PORT IP PROT flow dirn pkts
=============== =============== ============= ============= ======= ========= ==========
172.16.11.100 172.16.101.1 61211 52842 50 Input 48
172.16.101.1 172.16.11.100 55433 59192 50 Output 47
172.16.101.1 172.16.21.1 4500 4501 17 Output 47
172.16.21.1 172.16.101.1 4501 4500 17 Input 48
172.16.101.1 224.0.0.5 0 0 89 Output 23
172.16.101.254 224.0.0.5 0 0 89 Input 23
Troubleshooting Flexible NetFlow (FNF)
A nifty, TAC trick!
Hub1#show flow monitor test-monitor cache format table | include SRC|172.16.21.1
IPV4 SRC ADDR IPV4 DST ADDR TRNS SRC PORT TRNS DST PORT IP PROT flow dirn pkts
172.16.101.1 172.16.21.1 4500 4501 17 Output 47
172.16.21.1 172.16.101.1 4501 4500 17 Input 48
Troubleshooting - Embedded Packet Capture (EPC)
Spoke1(config)#ip access-list extended CAPACL
Spoke1(config-ext-nacl)#permit icmp any any
Spoke1(config-ext-nacl)#end
Spoke1#monitor capture PCAP access-list CAPACL interface GigabitEthernet3 both
Spoke1#monitor capture PCAP start
*Jun 1 02:31:26.091: %BUFCAP-6-ENABLE: Capture Point PCAP enabled.
Spoke1#monitor capture PCAP stop
*Jun 1 02:31:53.718: %BUFCAP-6-DISABLE: Capture Point PCAP disabled.
Spoke1#monitor capture PCAP limit packet-len ?
<64-9500> length(in bytes) : Min 64 : Max 9500
Troubleshooting - Embedded Packet Capture (EPC)
Spoke1#show monitor capture PCAP buffer
buffer size (KB) : 10240
buffer used (KB) : 128
packets in buf : 10
packets dropped : 0
packets per sec : 135
Spoke1#show monitor capture PCAP buffer brief
-------------------------------------------------------------
# size timestamp source destination protocol
-------------------------------------------------------------
0 114 0.000000 11.11.11.1 -> 10.1.1.100 ICMP
1 114 0.005004 10.1.1.100 -> 11.11.11.1 ICMP
2 114 0.005996 11.11.11.1 -> 10.1.1.100 ICMP
3 114 0.009002 10.1.1.100 -> 11.11.11.1 ICMP
4 114 0.010009 11.11.11.1 -> 10.1.1.100 ICMP
5 114 0.012999 10.1.1.100 -> 11.11.11.1 ICMP
Troubleshooting - Embedded Packet Capture (EPC)
Spoke1#monitor capture PCAP export ?
bootflash: Location of the file
flash: Location of the file
ftp: Location of the file
http: Location of the file
https: Location of the file
pram: Location of the file
rcp: Location of the file
scp: Location of the file
tftp: Location of the file
Spoke1#monitor capture PCAP export tftp://1.1.1.1/capture.pcap
Troubleshooting IOS-XE Packet Trace
Useful in troubleshooting data plane path of packets
Shows each feature which we hit in processing (fia-trace)
Similar to packet-tracer feature on ASA
Download slides for configuration guide and examples
Troubleshooting Example - Topology
Hub Information
Tunnel1: 10.1.1.100
NBMA: 172.16.101.1
Hub
Internet
Spoke1 Information Spoke2 Information
S1 NAT S2 NAT Tunnel2: 10.1.1.2
Tunnel1: 10.1.1.1
Real NBMA: 10.10.10.1 Real NBMA: 20.20.20.1
NATd NBMA: 172.16.11.100 NATd NBMA: 172.16.21.1
LAN: 11.11.11.0/24 LAN: 21.21.21.0/24
Spoke1 Spoke2
Troubleshooting Example Physical Configuration
All Routers DMVPN Spoke1
ip vrf ISP_1 interface GigabitEthernet2
ip vrf forwarding ISP_1
ip address 10.10.10.1 255.255.255.0
#### NATd to 172.16.11.x
negotiation auto
end
DMVPN Hub
DMVPN Spoke2
interface GigabitEthernet2
ip vrf forwarding ISP_1 interface GigabitEthernet2
ip address 172.16.101.1 255.255.255.0 ip vrf forwarding ISP_1
negotiation auto ip address 20.20.20.1 255.255.255.0
end #### NATd to 172.16.21.x
negotiation auto
end
Troubleshooting Example Crypto Configuration
crypto keyring ISP_1_KEYRING vrf ISP_1
pre-shared-key address 0.0.0.0 0.0.0.0 key ISP_1_KEY
crypto isakmp policy 1
encr aes 256
hash sha256
authentication pre-share
crypto isakmp keepalive 10 ##### Spokes Only
crypto ipsec transform-set DMVPN-TSET esp-aes 256 esp-sha256-hmac
mode transport
crypto ipsec profile DMVPN-IPSEC
set transform-set DMVPN-TSET
Troubleshooting Example Tunnel Configuration
DMVPN Hub DMVPN Spokes
interface Tunnel1 interface Tunnel1
bandwidth 50000 bandwidth 50000
ip address 10.1.1.100 255.255.255.0 ip address 10.1.1.1 255.255.255.0
no ip redirects no ip redirects
ip mtu 1400 ip mtu 1400
no ip split-horizon eigrp 1 ip nhrp authentication NHRPAUTH
ip nhrp authentication NHRPAUTH ip nhrp network-id 1
ip nhrp map multicast dynamic ip nhrp holdtime 300
ip nhrp network-id 1 ip nhrp nhs 10.1.1.100 nbma 172.16.101.1 multicast
ip nhrp holdtime 300 ip nhrp shortcut
ip nhrp redirect ip tcp adjust-mss 1360
ip summary-address eigrp 1 21.21.0.0 255.255.0.0 delay 100000
ip tcp adjust-mss 1360 tunnel source GigabitEthernet2
delay 100000 tunnel mode gre multipoint
tunnel source GigabitEthernet2 tunnel key 1
tunnel mode gre multipoint tunnel vrf ISP_1
tunnel key 1 tunnel protection ipsec profile DMVPN-IPSEC
tunnel vrf ISP_1
tunnel protection ipsec profile DMVPN-IPSEC
Troubleshooting Example in VIRL
Which port(s)/protocol(s) need to be
permitted/opened between two DMVPN
endpoints if there is a device present
along the network path between them,
performing NAT for all traffic?
Polling A. UDP 500 and UDP 4500 only
Question 3 B. UDP 500 and ESP/IP Protocol 50
only
C. UDP 500, 4500 and ESP/IP Protocol
50
D. Only UDP 4500
Best Practices - General
MTU=1400, adjust-MSS=1360
No GRE keepalives, unsupported with tunnel protection
GRE tunnel keys with mGRE (unique triplet)
Use VRFs to segregate traffic, when possible and convenient
Best Practices - ISAKMP
ISAKMP SA lifetime should be longer than that of IPsec SA lifetime
Dead Peer Detection (DPDs)
Security Settings
Use strong encryption and authentication, such as AES and SHA2
PKI/Certificate based authentication over PSK
Best Practices - IPsec
Transform set should be configured for mode transport rather than
default of mode tunnel
Use strongest settings supported by all devices
Do not combine ESP with AH, only ESP should be used
Best Practices - Routing
Unique routing instance for DMVPN cloud
Summarization on Hub
Submit Your Questions Now!
Use the Q & A panel to submit your questions and our expert will respond
Ask the Expert Event following the Webcast
Now through June 17th
https://supportforums.cisco.com/event/13023496/ask-
expert-dynamic-multipoint-vpn-dmvpn-troubleshooting
Join the discussion for these Ask The Expert Events:
http://bit.ly/events-webinar
Collaborate Facebook- http://bit.ly/csc-facebook
within our Twitter- http://bit.ly/csc-twitter
Social Media You Tube http://bit.ly/csc-youtube
Google+ http://bit.ly/csc-googleplus
LinkedIn http://bit.ly/csc-linked-in
Instgram http://bit.ly/csc-instagram
Learn About
Upcoming Events Newsletter Subscription
http://bit.ly/csc-newsletter
Cisco has
support
communities in
other languages! Spanish
https://supportforums.cisco.com/community/spanish
If you speak Spanish, Portuguese,
Japanese, Russian or Chinese we invite Portuguese
you to participate and collaborate in your https://supportforums.cisco.com/community/portuguese
language
Japanese
https://supportforums.cisco.com/community/csc-japan
Russian
https://supportforums.cisco.com/community/russian
Chinese
http://www.csc-china.com.cn
More IT Training Videos and Technical
Seminars on the Cisco Learning Network
View Upcoming Sessions Schedule
https://cisco.com/go/techseminars
Thank you for participating!
. Redeem your 35% discount offer by entering
code: CSC when checking out: Cisco Press
Visit Cisco Press at:
http://bit.ly/csc-ciscopress-2016
Please take a moment to complete
the survey
Thank you for Your Time!
