Building a flat L2 network between VMs and hardware in different locations

With the help of virtual machines, you can solve a huge number of problems. In addition, they are quite cheap to use, quickly deployed, easily scaled, and can be easily transferred between different locations.

For some tasks, dedicated servers may be required. They are productive, well suited for heavy applications, and the cost of expanding physical storage for them is lower.

In some cases, you will have to combine these approaches altogether. This task is easy to solve if there is enough L3 connectivity between virtual and physical servers. For example, between a scalable, fault-tolerant cluster of web applications on virtual machines (VMs) and a cluster of databases on physical servers.

But sometimes you need to combine services into a L2 network. For example, you have DCImanager installed on your VM — our platform for managing IT infrastructure. DCImanager should manage physical servers. If all your hardware is physically located in one place and you have access to the network equipment, all that remains is to configure VLAN on switch ports and in the hypervisor (for example, in the VMmanager virtualization management system).

Learn more about our infrastructure management software

In this article we talk about building a flat network in an infrastructure based on ISPsystem products:

• DCImanager — a physical infrastructure management platform that helps you take control of all your racks, servers, network equipment, PDUs and UPSs, physical and virtual networks.

• VMmanager — a virtualization management platform that allows you to build a fault-tolerant virtual infrastructure.

All our products can be tested absolutely free of charge: a 30-day trial and other ways to get acquainted with the solutions are available to everyone.

What to do if your equipment is located in different locations and there is no access to the switches and router of the data center? In this case, you need to set up L2VPN between your sites. Below I will share one of the ways to solve this problem.

How it works

To connect networks we will use VxLAN tunnels and the BGP EVPN dynamic routing protocol.

If you already have VMmanager, enable it in your cluster “Virtual Networks” technology — VMmanager will configure the BGP FRR daemon on the cluster nodes and create VxLAN tunnels.

Let's look at an example.

Let's assume we have a physical switch Juniper QFX5100 with VxLAN and BGP EVPN support. Let's connect a physical server to it, for example, to port xe-0/0/1.

The diagram shows a VMmanager cluster with two nodes, Node1 and Node2, connected to a switch. Node1 contains a virtual machine connected to a virtual network. On another site, there is a physical server connected to a Juniper QFX switch on port xe-0/0/1. A routed network is configured between the QFX and Node1 and Node2.

Description of settings

Setting up the switch

1. Connect QFX to a routed network so that the address on the lo0.0 interface is accessible from the VMmanager cluster nodes (in my example, this is 10.3.0.47).

2. Select the AS number that we will use. For example, I took 62200:

routing-options {
  router-id 10.3.0.47;
  autonomous-system 62200;
}

3. Configure VTEP:

switch-options {
  vtep-source-interface lo0.0;
  route-distinguisher 10.3.0.47:1;
  vrf-target target:62200:1;
}

4. Configure BGP – as neighbor specify all nodes of your VMmanager cluster):

protocols {
    bgp {
        group VMmanager {
            type internal;
            local-address 10.3.0.47;
            family evpn {
                signaling;
            }
            peer-as 62200;
            neighbor 172.31.33.2 {
                cluster 10.3.0.47;
            }
            neighbor 172.31.33.3 {
                cluster 10.3.0.47;
            }
        }
}

Setting up VMmanager

1. Enable “Virtual Networks” in the cluster settings in Route Reflector mode.

2. Set the local autonomous system to 62200.

3. Add our QFX 10.3.0.47 with autonomous system 62200 to the neighbors.

I have a regular VMmanager user — ilya@ispsystem.net. Under the administrator:

1. Create a virtual network for it 192.168.10.0/24.

2. Create a virtual machine for it in this virtual network.

3. After creating the VM, go to the “Network settings” section, copy the VxLAN number. In my example, the interface is connected to vxbr1194142then our VNI will be 1194142.

Back to setting up QFX

1. Add EVPN protocol settings:

protocols {
    evpn {
        vni-options {
            vni 1194142 {
                vrf-target target:62200:1194142;
            }
        }
        encapsulation vxlan;
        multicast-mode ingress-replication;
        extended-vni-list all;
    }
}

Please note: it is important to specify here vrf-target = target:ASN:VNIotherwise the routes will not be established and there will be no connectivity.

2. Create VLAN:

vlans {
    test {
        vlan-id 100;
        vxlan {
            vni 1194142;
            ingress-node-replication;
        }
    }
}

3. Connect the port to this VLAN:

interfaces {
    xe-0/0/1 {
        ether-options {
            auto-negotiation;
        }
        unit 0 {
            family ethernet-switching {
                interface-mode access;
                vlan {
                    members test;
                }
            }
        }
    }
}

Done. Now the created VM and the server are in the same flat network.

Examination

1. Let's check the connection on Juniper:

root> show evpn database
Instance: default-switch
VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
     1194142    00:1e:67:ca:6d:a9  xe-0/0/1.0                     Mar 12 15:44:40  192.168.13.3
     1194142    52:54:00:6b:33:a6  172.31.33.2                    Mar 12 14:12:26

2. Let's check on the VMmanager node, run the vtysh command and inside it:

show evpn mac vni 1194142
Number of MACs (local and remote) known for this VNI: 2
Flags: N=sync-neighs, I=local-inactive, P=peer-active, X=peer-proxy
MAC               Type   Flags Intf/Remote ES/VTEP            VLAN  Seq #'s
00:1e:67:ca:6d:a9 remote       10.3.0.47                            0/0
52:54:00:6b:33:a6 local        vm13706_vxlan0                       0/0

We can add other servers to this network by simply assigning a VLAN to the desired port. We can also create several more virtual machines for the user in this VxLAN – all of them will be inside the virtual network.

What's the bottom line?

We have combined virtual machines and physical servers located on different sites into a L2 network – flat and non-routable. Now we can configure physical server management via DCImanager or deploy a gray network between virtual and physical servers to solve any other problem.

The main feature of this approach is the absence of routing. When using VxLAN, traffic is encapsulated and looks like UDP traffic between two points to external routers. I hope this will work for your needs.