vSphere Standard Switch Architecture

You can create abstracted network devices called vSphere standard switches. A standard switch can..

1. Route traffic internally between virtual machines and link to external networks
2. Combine the bandwidth of multiple network adaptors and balance communications traffic among them.
3. Handle physical NIC failover.
4. Have a default number of logical ports which for a standard switch is 120. You can
5. Connect one network adapter of a virtual machine to each port. Each uplink adapter associated with a standard switch uses one port.
6. Each logical port on the standard switch is a member of a single port group.
7. Have one or more port groups assigned to it.
8. When two or more virtual machines are connected to the same standard switch, network traffic between them is routed locally. If an uplink adapter is attached to the standard switch, each virtual machine can access the external network that the adapter is connected to.
9. vSphere standard switch settings control switch-wide defaults for ports, which can be overridden by port group settings for each standard switch. You can edit standard switch properties, such as the uplink configuration and the number of available ports.

Standard Switch

vSphere Distributed Switch Architecture

A vSphere distributed switch functions as a single switch across all associated hosts. This enables you to set network configurations that span across all member hosts, and allows virtual machines to maintain consistent network configuration as they migrate across multiple hosts

Like a vSphere standard switch, each vSphere distributed switch is a network hub that virtual machines can use.

• Enterprise Plus Licensed feature only
• VMware vCenter owns the configuration of the distributed switch
• Distributed switches can support up to 350 hosts
• You configure a Distributed switch on vCenter rather than individually on each host
• Provides support for Private VLANs
• Enable networking statistics and policies to migrate with VMs during vMotion
• A distributed switch can forward traffic internally between virtual machines or link to an external network by connecting to physical Ethernet adapters, also known as uplink adapters.
• Each distributed switch can also have one or more distributed port groups assigned to it.
• Distributed port groups group multiple ports under a common configuration and provide a stable anchor point for virtual machines connecting to labeled networks.
• Each distributed port group is identified by a network label, which is unique to the current datacenter. A VLAN ID, which restricts port group traffic to a logical Ethernet segment within the physical network, is optional.
• Network resource pools allow you to manage network traffic by type of network traffic.
• In addition to vSphere distributed switches, vSphere 5 also provides support for third-party virtual switches.

TCP/IP Stack at the VMkernel Level

The VMware VMkernel TCP/IP networking stack provides networking support in multiple ways for each of the services it handles.

The VMkernel TCP/IP stack handles iSCSI, NFS, and vMotion in the following ways for both Standard and Distributed Virtual Switches

• iSCSI as a virtual machine datastore
• iSCSI for the direct mounting of .ISO files, which are presented as CD-ROMs to virtual machines.
• NFS as a virtual machine datastore.
• NFS for the direct mounting of .ISO files, which are presented as CD-ROMs to virtual machines.
• Migration with vMotion.
• Fault Tolerance logging.
• Port-binding for vMotion interfaces.
• Provides networking information to dependent hardware iSCSI adapters.
• If you have two or more physical NICs for iSCSI, you can create multiple paths for the software iSCSI by configuring iSCSI Multipathing.

Networking Policies

Policies set at the standard switch or distributed port group level apply to all of the port groups on the standard switch or to ports in the distributed port group. The exceptions are the configuration options that are overridden at the standard port group or distributed port level.

• Load Balancing and Failover Policy
• VLAN Policy
• Security Policy
• Traffic Shaping Policy
• Resource Allocation Policy
• Monitoring Policy
• Port Blocking Policies
• Manage Policies for Multiple Port Groups on a vSphere Distributed Switch

Networking Best Practices

• Separate network services from one another to achieve greater security and better performance. Put a set of virtual machines on a separate physical NIC. This separation allows for a portion of the total networking workload to be shared evenly across multiple CPUs. The isolated virtual machines can then better serve traffic from a Web client, for example
• Keep the vMotion connection on a separate network devoted to vMotion. When migration with vMotion occurs, the contents of the guest operating system’s memory is transmitted over the network. You can do this either by using VLANs to segment a single physical network or by using separate physical networks (the latter is preferable).
• When using passthrough devices with a Linux kernel version 2.6.20 or earlier, avoid MSI and MSI-X modes because these modes have significant performance impact.
• To physically separate network services and to dedicate a particular set of NICs to a specific network service, create a vSphere standard switch or vSphere distributed switch for each service. If this is not possible, separate network services on a single switch by attaching them to port groups with different VLAN IDs. In either case, confirm with your network administrator that the networks or VLANs you choose are isolated in the rest of your environment and that no routers connect them.
• You can add and remove network adapters from a standard or distributed switch without affecting the virtual machines or the network service that is running behind that switch. If you remove all the running hardware, the virtual machines can still communicate among themselves. If you leave one network adapter intact, all the virtual machines can still connect with the physical network.
• To protect your most sensitive virtual machines, deploy firewalls in virtual machines that route between virtual networks with uplinks to physical networks and pure virtual networks with no uplinks.
• For best performance, use vmxnet3 virtual NICs.
• Every physical network adapter connected to the same vSphere standard switch or vSphere distributed switch should also be connected to the same physical network.
• Configure all VMkernel network adapters to the same MTU. When several VMkernel network adapters are connected to vSphere distributed switches but have different MTUs configured, you might experience network connectivity problems.

How Many NIC Ports should I use?

Whether you are purchasing new servers or trying to reuse existing servers you need to determine how many NIC ports you want/need and what speed NIC’S; 10GB, 1GB, fibre, etc. I would try and install as many NICs as possible and combine NIC ports across switches

• Redundancy

You want to be able to remove all single points of failure in your network.  You can team NIC’S together to achieve redundancy and use Link Aggregration or Etherchannel to compliment this on your physical switches

• Throughput

The speed of your NICs is extremely important depending on the amount of network traffic you anticipate creating on your networks. NFS is a consideration along with backup and replication traffic, let alone normal network traffic.

• Flexibility

You can provision more NIC’s as demand for certain services increase or decrease.

NIC Considerations

• Jumbo Frames
• TOE (TCP Offload Engine)
• Boot from SAN
• iSCSI or Fibre
• 1GB, 10GB ethernet or Fibre

Data Plane and Control Planes

vSphere network switches can be broken into two logical sections. These are the data plane and the management plane.

• The data plane implements the actual packet switching, filtering, tagging, etc.
• The management plane is the control structure used to allow the operator to configure the data plane functionality.
• With the vSphere Standard Switch (VSS), the data plane and management plane are each present on each standard switch. In this design, the administrator configures and maintains each VSS on an individual basis.

Virtual Standard Switch Control and Data Plane

With the release of vSphere 4.0, VMware introduced the vSphere Distributed Switch. VDS eases the management burden of per host virtual switch configuration by treating the network as an aggregated resource. Individual host-level virtual switches are abstracted into a single large VDS that spans multiple hosts at the Datacenter level. In this design, the data plane remains local to each VDS, but the management plane is centralized with vCenter Server acting as the control point for all configured VDS instances.

Virtual Distributed Switch Control and Data Plane

Limits