1 | © 2012 Oracle Corporation – Proprietary and Confidential

Safe Harbor Statement

The following is intended to outline our general product
direction. It is intended for information purposes only, and
may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing
decisions.
The development, release, and timing of any features or
functionality described for Oracle’s products remains at the
sole discretion of Oracle.

2 | © 2012 Oracle Corporation – Proprietary and Confidential

 Oracle Training Materials – Usage Agreement
Use of this Site (“Site”) or Materials constitutes agreement with the following terms and conditions:

1. Oracle Corporation (“Oracle”) is pleased to allow its business partner (“Partner”) to download and copy the information,
documents, and the online training courses (collectively, “Materials") found on this Site. The use of the Materials is restricted to
the non-commercial, internal training of the Partner’s employees only. The Materials may not be used for training, promotion, or
sales to customers or other partners or third parties.

2. All the Materials are trademarks of Oracle and are proprietary information of Oracle. Partner or other third party at no time has
any right to resell, redistribute or create derivative works from the Materials.

3. Oracle disclaims any warranties or representations as to the accuracy or completeness of any Materials. Materials are provided
"as is" without warranty of any kind, either express or implied, including without limitation warranties of merchantability, fitness
for a particular purpose, and non-infringement.

4. Under no circumstances shall Oracle or the Oracle Authorized Boot Camp Training Partner be liable for any loss, damage,
liability or expense incurred or suffered which is claimed to have resulted from use of this Site of Materials. As a condition of use
of the Materials, Partner agrees to indemnify Oracle from and against any and all actions, claims, losses, damages, liabilities
and expenses (including reasonable attorneys' fees) arising out of Partner’s use of the Materials.

5. Reference materials including but not limited to those identified in the Boot Camp manifest can not be redistributed in any format
without Oracle written consent.

3 | © 2012 Oracle Corporation – Proprietary and Confidential

 Oracle VM 3 for SPARC Network Concept
and Management
Presenter Name
4 | © 2012 Oracle Corporation – Proprietary and Confidential
 Module Objectives
• Introduction of Network Concept in OVM for SPARC
• Network Management in OVM for SPARC
– Planning physical network
• Network Roles
– vNet provisioning
• Virtual Network I/O Implementation
– Virtual Switch Implementation

• vNet MAC address uniqueness in OVM SPARC

5 | © 2012 Oracle Corporation – Proprietary and Confidential

Network Concept in Oracle
VM

6 | © 2012 Oracle Corporation – Proprietary and Confidential

 Oracle VM Network Concept
OVM Server OVM Server • Virtual Network
– Virtual NICs in Guest VMs
Guest VM Guest VM Guest VM Guest VM
• vNIC is physical network independent
– Ties to physical network on the fly
• Will be changed after migration

• Physical Network
– Physical Ethernet Ports on server
hardware
– Physically Wired
– Connects physical hardware
• Network Components
– Switches, Firewalls
Shared Storage • OVM Servers
• OVM Manager
Sun Storage Partner Storage • IP-based Storage Devices
– IP-SAN
– NAS

7 | © 2012 Oracle Corporation – Proprietary and Confidential

 Oracle VM Network Concept
Network Configuration
• Goals
– Define the behavior of physical network
• Topology, Cabling
• IP Setting, VLAN, Routing
• QoS, Firewall policies
– Mapping between physical and virtual network
• General Steps
– Physical networks should be planned and setup first
– Then provisioned to Virtual NICs

8 | © 2012 Oracle Corporation – Proprietary and Confidential

Network Management in
Oracle VM

9 | © 2012 Oracle Corporation – Proprietary and Confidential

 Oracle VM Network Management
Plan Physical Network on OVM SPARC Server
• Goals
– Plan on how to use physical network devices transferring all possible
network communications on the server
• Guidelines to plan physical network
– Define possible network communications
• Aka, network roles in OVM Manager Concept
– Map communications to physical network devices
• Topology, cabling and routing strategy can also be determined
– Choose Virtual NIC provision methods
• Either virtual I/O or physical I/O

10 | © 2012 Oracle Corporation – Proprietary and Confidential

 Network Roles in OVM for SPARC
Network Required or Max
IP Settings Description
Roles not Occurrence
Server Required Single Must provide IP setting Endpoint in Control Domain for receiving any management
Management commands

Live Migrate Required Single Endpoint in Control Domain for transferring guest VM
runtime state between OVM for SPARC

Storage Optional Multiple Network access to iSCSI, FCoE Storage and File Servers
• Not applicable if using only FC SAN-based Storage

Virtual Required Multiple Traffic endpoint is Guest VM, Access to/from Guest VMs
Machine do not need to assign IP

11 | © 2012 Oracle Corporation – Proprietary and Confidential

 Virtual NIC Provisioning Model
• Physical I/O
OVM Server
– Virtual NIC implementation
Guest VM Guest VM Guest VM • Control the whole physical network device (Direct I/O)
• Consume function of the physical network device (SR-IOV)
– Pros & Cons
• Better network performance
Proxy Proxy
Service Service • Will limit guest VM live migration & HA capabilities

• Virtual I/O (Proxy I/O)

– Proxy Service in OVM Server
• Provide simulated network device for guest VM
• Normally route network traffics through physical network device
– Pros & Cons
• Support guest VM live migration & HA
• Simulation will consume more resources and lower the network
performance

12 | © 2012 Oracle Corporation – Proprietary and Confidential

 Network Virtual I/O Architecture
• In Service Domain
Service Domain Guest Domain – The Virtual Switch (VSW) component is
created in Service Domain
• Will use a dedicated backend network
Aggr
device in service domain
vNet vNet vNet vNet
• A backend device can only be used for a
VSW VSW
specified VSW component

• In Guest Domain
– Virtual Network (vNet) device can be
created connecting to specified Virtual
Hypervisor Switch

SPARC Server • In Hypervisor

– A LDC channel will be allocated for each
vNet/VSW pair

13 | © 2012 Oracle Corporation – Proprietary and Confidential

 Backend device implementation
• Backend device in service domain will leverage Data Link
implementation in Solaris OS
– Supported data link types
• Single Port
• Link Aggregation
• Etherstub

• Can define capabilities for those backend devices

– Supported Capabilities
• MTU for the data link
– vNets over the data link will by default share same capabilities

14 | © 2012 Oracle Corporation – Proprietary and Confidential

 Link Aggregation
• Increase throughput with higher availability
– Two 1 Gb ports = 2 Gb bandwidth
– Two 10 Gb ports = 20 Gb bandwidth
• Use IEEE 802.3ad standard
– Must be configured on both OVM for SPARC and Network Switch
– Note:
• Before configuring dynamic link aggregation bonding on a server, make sure
that the network switch supports LACP protocol, that it is licensed, and the
switch has it enabled!
• Active-Active configurations are normally unsupported across different
switches.

15 | © 2012 Oracle Corporation – Proprietary and Confidential

 Virtual Switch
• Virtual Switch is a software-based component created in a
Service Domain
– Provision vNet devices for guest domains
• Setup and listen at the allocated LDC channel endpoint
• Generate MAC addresses for each vNet
• Awareness of vNet and can apply QoS to vNet
– L2 Switching
• Packet switching based on both MAC address and VLAN tags
• Enable communication between guest domains and external world

16 | © 2012 Oracle Corporation – Proprietary and Confidential

 Virtual Switch Implementation
Solaris 10
• Separate MAC and L2 switching
– MAC Layer
• Leverage OS MAC Layer directly
• Single MAC for each vNet device
– L2 Switching
• Implemented in Virtual Switch component
• Have to plumb the Virtual Switch Component manually in Service Domain to
enable communication between Service Domain and Guest Domain

17 | © 2012 Oracle Corporation – Proprietary and Confidential

 Virtual Switch Implementation
Solaris 11
• Use Crossbow Stack for both MAC and L2 switching
– MAC Layer
• Support vNIC on vNet
– Can create multiple MAC address for a single vNet device
– Require guest domain running Solaris 11 as well
– L2 Switching
• In fact, if adding multiple vNet devices will create a L2 switch in Crossbow
• Automatically recognized the VSW as a data link in Solaris 11

• The Solaris 11 OS is recommended for more features and better

performance
– Latest performance enhancement only implemented in Solaris 11 service domain

18 | © 2012 Oracle Corporation – Proprietary and Confidential

 Create vSwitch
• Creating Virtual Switch
– # ldm add-vsw net-dev=<backend data link> [option=value]* <vSwitch name> <service
domain name>
– Available Options
• MAC address
• MTU
• Inter-vNet Channel
• VLAN
• Physical link state update

19 | © 2012 Oracle Corporation – Proprietary and Confidential

 Create vNet
• Creating vNet
– # ldm add-vnet [option=value]* <vnet name> <vsw name> <guest domain>
– Available Options
• MAC Address
• MTU
• vNIC on vNet
• VLAN
• Physical link state update

20 | © 2012 Oracle Corporation – Proprietary and Confidential

 Inter-vNet Channel
• LDC Topology for vNets
– By default, two types of LDC channels are established for vNets connecting to the
same Virtual Switch
• vNet ↔ vSwitch
• vNet ↔ vNet
– vNet implementation in guest domain will use this channel as a shortcut to other vNet connecting to
same vSwitch

• LDC channels between vNets will improve performance if there are

lot of inter-communication among those vNets, however
– It will consume a lot of LDC resource
– You can turn off inter-vNet communication by
• # ldm set-vsw inter-vnet-link=off <vswitch>

21 | © 2012 Oracle Corporation – Proprietary and Confidential

 vNIC on vNet
• In Solaris 11 Service Domain, taking advantage of the Crossbow
implementation, we can create multiple MAC address for a single
vNet device
– You can use those MAC addresses inside your guest domain to provision MAC
addresses for different vNICs
• For example, vNICs inside a Solaris Zone
– Guest OS that can leverage multiple MAC addresses should be Solaris 11
• Create multiple MAC address for vNet device
– # ldm add-vnet <vnet name> alt-mac-addrs=“auto,atuo,auto,auto” <vsw name> <guest
domain>

22 | © 2012 Oracle Corporation – Proprietary and Confidential

 Physical Link State Update
• It is very possible that backend physical connection is down
– By default both VSW and vNet will use link state unaware driver
• It does not support monitoring of the physical connection state
• Link-based IPMP feature can not be supported in Guest Domain

• You can turn on link state update for both VSW and vNet
– For VSW
• # ldm add-vsw net-dev=<backend device> linkprop=phys-state <vsw name> <service domain>
• # ldm set-vsw linkprop=phys-state <vsw name>
– For vNet
• # ldm add-vnet <vnet name> linkprop=phys-state <vsw name> <guest domain>
• # ldm set-vnet linkprop=phys-state <vnet name>

23 | © 2012 Oracle Corporation – Proprietary and Confidential

 VLAN Concept
• VLAN is an Ethernet standard (802.1Q) to define separated virtual
LANs in a physical network fabric
– By tagging network traffics in the fabric, could be
• Tagged: Have a specific VLAN ID (2-4094) in packet
• Untagged: Does not have a VLAN ID in packet
– VLAN Trunking Protocol (VTP) enables transferring packets with different VLAN
IDs in same data link
• Will segment the data link as multiple data channels
– Network Switch and Server will help destine a packet to right peer
• Broadcast will be limited within same virtual LAN

• Use VLAN is you want to

– Isolate network traffics for security reason
– Create logical channels on limited physical network ports

24 | © 2012 Oracle Corporation – Proprietary and Confidential

 VLAN in OVM for SPARC
• Both VSW and vNet components are VLAN enabled
– Implemented in device driver in service and guest domain
– VLAN functions are controlled by following parameters
• pvid : Default VLAN ID for untagged traffic
– Filter outbound traffic to append default VLAN ID in packet
– Filter inbound traffic to remove VLAN ID from IP packet if VLAN ID in the packet matches default
VLAN ID
• vid : Supported VLAN IDs in tagged communication
– VLAN ID in an IP packet must be within the VLAN ID list otherwise will be dropped
• However VSW will not drop those packets not from data links over the VSW components

– Examples
• # ldm add-vsw net-dev=net0 pvid=20 vid=21,22,23,24 primary-vsw0 primary
• # ldm add-vnet pvid=21 vnet1 primary-vsw0 gvm2
• # ldm add-vnet pvid=20 vid=21,22 vnet1 primary-vsw0 gvm1

25 | © 2012 Oracle Corporation – Proprietary and Confidential

 MAC Address Uniqueness
• vNet in a guest domain will be seen a standard network card by
guest OS
– So need to guarantee MAC address uniqueness
• OVM for SPARC has several means to protect the uniqueness
– It is enforced by LDom Manager
• LDom Manager will only assign MAC address within a special OVM SPARC specific ranges
– 512K addresses : 00:14:4F:F8:00:00 - 00:14:4F:FF:FF:FF
– The lower part will be assinged automatically
– The higher part will be used by manual request
• When there may be more that one LDom Manager in the same data center
– LDom Manager will try detect duplicate MAC address by broadcasting message within specified TTL

26 | © 2012 Oracle Corporation – Proprietary and Confidential

27 | © 2012 Oracle Corporation – Proprietary and Confidential
28 | © 2012 Oracle Corporation – Proprietary and Confidential
29 | © 2012 Oracle Corporation – Proprietary and Confidential