Hi, Habr. I am finishing a series of articles, launching the course “Network engineer” from OTUS, using VxLAN EVPN technology for routing inside the fabric and using a Firewall to restrict access between internal services
The previous parts of the cycle can be found at the links:
- Part 1 of the cycle – L2 connectivity between servers
- 2 part of the cycle – Routing between VNIs
- 2.5 part of the cycle – Theoretical digression
Today we will continue to study the routing logic inside the VxLAN fabric. In the previous part, we looked at routing within a factory within a single VRF. However, the network can have a huge number of client services and all must be distributed to different VRFs to differentiate access between them. In addition to network demarcation, a business may need to connect a Firewall to restrict access between these services. Yes, this cannot be called the best solution, but modern realities require “modern solutions”.
Consider two options for routing between VRFs:
- Routing without leaving the VxLAN fabric;
- Routing on external equipment.
Let’s start with the inter-VRF routing logic. There are a certain number of VRFs. To route between VRFs, you need to allocate a device in the network that will know about all VRFs (or the parts between which you need routing). Such a device can be, for example, one of the Leaf switches (or all at once). This topology will look like this:
What are the disadvantages of such a topology?
That’s right, every Leaf needs to know about all the VRFs (and all the information in them) on the network, which leads to memory loss and increased network load. Indeed, quite often each Leaf switch does not need to know about everything that is on the network.
However, we will consider this method in more detail, since this option is quite suitable for small networks (if there are no specific business requirements)
At this point, you may have a question about how to transfer information from VRF to VRF, because the meaning of this technology is precisely that the dissemination of information should be limited.
And the answer lies in functions such as export and import of routing information (setting up this technology was considered in the second part of the cycle). I will briefly repeat:
When setting VRF in AF, you must specify
route-target for import and export routing information. It can be specified in automatic mode. Then ASN BGP and L3 VNI bound to VRF will be included in the value. This is convenient when you have only one ASN used in your factory:
vrf context PROD20 address-family ipv4 unicast route-target export auto ! В автоматическом режиме экспортируется RT-65001:99000 route-target import auto
However, if you have more than one ASN and you need to transfer routes between them, then manual configuration would be a more convenient and scalable option.
route-target… The recommendation in manual setting is the first number, use a convenient one, for example,
The second should be made equal to the VNI for this VRF.
Configure as follows:
vrf context PROD10 address-family ipv4 unicast route-target export 9999:99000 route-target import 9999:99000 route-target import 9999:77000 ! Пример 1 import из другого VRF route-target import 9999:77000 ! Пример 2 import из другого VRF
What it looks like in the routing table:
Leaf11# sh ip route vrf prod <.....> 192.168.20.0/24, ubest/mbest: 1/0 *via 10.255.1.20%default, [200/0], 00:24:45, bgp-65001, internal, tag 65001 (evpn) segid: 99000 tunnelid: 0xaff0114 encap: VXLAN ! префикс доступен через L3VNI 99000
Consider the second option for routing between VRFs – through external equipment, for example, a Firewall.
We can assume several options for working through an external device:
- The device knows what VxLAN is and we can add it to part of the factory;
- The device knows nothing about VxLAN.
We will not dwell on the first option, since the logic will be almost the same as shown above – we bring all VRFs to the Firewall and configure routing between VRFs on it.
Consider the second option, when our Firewall does not know anything about VxLAN (now, of course, equipment with VxLAN support appears. For example, Checkpoint announced its support in version R81. You can read about it here, but this is all at the testing stage and there is no confidence in stability work).
When connecting an external device, we get the following scheme:
As you can see from the diagram, a bottleneck appears at the junction with the Firewall. This should be taken into account in the future when planning the network and optimizing network traffic.
However, back to the original inter-VRF routing problem. As a result of adding Firewall, we come to the fact that Firewall must know about all VRFs. To do this, all VRFs must also be configured on the border Leafs, and we connect the Firewall to each VRF with a separate link.
As a result, the scheme with Firewall:
That is, on the Firewall it is necessary to configure the interface to each VRF located in the network. In general, the logic does not look complicated and the only thing that you may not like here is the huge number of interfaces on the Firewall, but now it’s time to think about automation.
Good. We connected Firewall, added it to all VRFs. But now how to get traffic from each Leaf to go through this Firewall?
On a Leaf connected to the Firewall, there will be no problems, since all routes are local:
0.0.0.0/0, ubest/mbest: 1/0 *via 10.254.13.55, [1/0], 6w5d, static ! маршрут по-умолчанию через Firewall
However, what about the deleted Leaf? How do I pass them the default external route?
That’s right, via EVPN route-type 5, like any other prefix on the VxLAN fabric. However, this is not so simple (if we are talking about cisco, as I have not tested with other vendors)
The default route must be announced from the Leaf to which the Firewall is connected. However, in order to transmit the route, Leaf must know it itself. And here a certain problem arises (perhaps only for me), the route must be registered statically in the VRF where you want to announce such a route:
vrf context PROD10 ip route 0.0.0.0/0 10.254.13.55
Further in the BGP configuration, set this route in AF IPv4:
router bgp 65001 vrf prod address-family ipv4 unicast network 0.0.0.0/0
However, this is not all. Thus, the default route will not fall into the family.
l2vpn evpn… In addition to this, you need to configure redistribution:
router bgp 65001 vrf prod address-family ipv4 unicast network 0.0.0.0/0 redistribute static route-map COMMON_OUT
We indicate which prefixes will get into BGP through redistribution
route-map COMMON_OUT permit 10 match ip address prefix-list COMMON_OUT ip prefix-list COMMON_OUT_GATE seq 10 permit 0.0.0.0/0
Now the prefix
0.0.0.0/0 gets into EVPN route-type 5 and is passed to the rest of the Leaf:
0.0.0.0/0, ubest/mbest: 1/0 *via 10.255.1.5%default, [200/0], 5w6d, bgp-65001, internal, tag 65001, segid: 99000 tunnelid: 0xaff0105 encap: VXLAN ! 10.255.1.5 - Виртуальный адрес Leaf(так как Leaf выступают в качестве VPС пары), к которому подключен Firewall
In the BGP table, we can also observe the received route-type 5 with the default route through 10.255.1.5:
* i::::[0.0.0.0]/224 10.255.1.5 100 0 i *>i 10.255.1.5 100 0 i
This concludes the series of articles dedicated to EVPN. In the future I will try to consider the work of VxLAN in conjunction with Multicast, since this method is considered more scalable (at the moment, a controversial statement)
If you still have questions / suggestions on the topic, consider any EVPN functionality – write, we will consider additionally.