A little more about using VPN in Beeline
Hey! In this article, I will talk about two more VPN options that we use to enable clients using Multisim Reservations, these are L2TP and L2-VPN. The first is used to enable Internet services, the second is used to enable L2 MPLS channels.
The very first is L2TP
Historically, this option was the very first implemented option for providing the Internet service to clients, it was on it that we ran all the business processes of inclusions, filled the first bumps with the complexity of setting up CPE, the processes of activating Sim cards and registering them in dedicated APNs, choosing the most optimal settings and configurations of CPE and PE equipment, in total for the first six months we prepared eight different versions of equipment configurations. This iterative approach allowed us to find a balance between the needs of our clients and our capabilities, we looked at what and how to set up, how to build processes, and what is better not to tackle at all.
The architectural diagram looks like this:
Description:
Two Sim cards are installed in the router, APNs are registered and the radio interface is configured, plus the WAN interface for the incoming wired channel, if any, is configured. Sim cards are registered in typical APNs, all APNs of different telecom operators are combined into one VRF and receive standardized private IP addresses from their APNs.
On our RADIUS server, when connecting, a session account is created for the client, information about this account will be used by BRAS when building a session. The router begins to build an L2TP session either through a wired WAN interface (Internet channel), or through a radio interface, depending on the type of work (Fix + LTE / 3G or “pure” LTE / 3G), the session is terminated on BRAS with standard settings prescribed on the RADIUS server.
It is also possible to configure Framed Route with a public Internet subnet on the RADIUS server, which is configured on the LAN interface of the router and broadcast to the client. Thus, each client has one static IP address on the tunnel with private addressing and NAT on three LAN interfaces, and an additionally registered public subnet (usually / 31 or / 30 subnet) on one LAN interface.
Since the tunnel can be built through a cable connection (Internet channel) or a Sim card, and the subnet is routed behind the tunnel, the client receives static settings regardless of the transport network used.
If the cable channel is lost, the router transfers traffic to the radio interface and the first Sim-card in 10-20 seconds, and in case of problems with it, it changes the working Sim-card to a backup one, usually this change takes 60-90 seconds.
The built-in NQA (Network Quality Analysis) tool and the preference route values are responsible for choosing the traffic route between the cable and the radio network:
#
nqa test-instance admin inet
test-type icmp
destination-address ipv4 4.2.2.2
frequency 20
source-interface GigabitEthernet0/0/4
start now
#
#
ip route-static 0.0.0.0 0.0.0.0 Virtual-Template1 preference 40
ip route-static 4.2.2.2 255.255.255.255 GigabitEthernet0/0/4 dhcp
ip route-static 85.21.4.234 255.255.255.255 GigabitEthernet0/0/4 dhcp track nqa admin inet
ip route-static 85.21.4.235 255.255.255.255 GigabitEthernet0/0/4 dhcp track nqa admin inet
ip route-static 100.64.0.0 255.255.254.0 Cellular0/0/0 preference 70
ip route-static 100.64.0.0 255.255.254.0 NULL0 track nqa admin inet
#And the auto-switching script written in Python is responsible for the selection of active Sim-cards.
The scheme also has its limitations: the access speed is limited to 30-50 Mbit / s, and MTU = 1450.
A further direction of development is the implementation of the same scheme with redundancy of wired channels, but using BGP, for the announcement of a LAN network either through a cable or through a tunnel, this will allow using the router as a reserve on broadband channels of 100-200 Mbit / s and separation of traffic routes over the cable and through the tunnel. Naturally, this will only work on wired channels from Beeline; to work through other providers, we send all traffic through the L2TP tunnel.
The most difficult is L2 MPLS
This option turned out to be the latest in our solutions to date, and was the most difficult from the point of view of implementation. The protocol used for this type of connection on Huawei AR129 is SVPN L2TPv2, on the counter part there is Cisco ASR1001-X, which is the counter part of the tunnel and outputs traffic to the MEN network.
Architecturally, the network looks like this:
Setting up this mode on a Huawei AR129 router looks like this:
#
interface Tunnel0/0/0
tunnel-protocol svpn
encapsulation l2tpv3
l2tpv3 local session-id 3
l2tpv3 remote session-id 4
tunnel-source 100.64.16.63
tunnel-destination 100.64.0.43
#The most difficult for our implementation due to the peculiarities of the protocol operation, a session for L2 MPLS is initiated from the IP address of the SIM card, which is active at a particular moment for this tunnel, and the values on both the CPE and the response host PE are bound to the IP address, and in In case of problems with a working LTE / 3G network, the IP on the SIM card is changed.
We had the following options for getting out of this situation:
Use a GRE tunnel, and launch a tunnel with L2 MPLS through this tunnel.
Assign the same IP addresses to Sim cards during setup.
Make the configuration of the tunnels change in the router depending on the SIM card.
Although the first two options looked optimal from the point of view of the network architecture (a separate tunnel is a separate pair of ports on the MEN network), from the point of view of process synchronization and technical support, this is a much bigger problem.
Accordingly, we had to implement the reconfiguration of the router “on the fly.” Considering that the Python script we are using to automatically switch Sim cards already performed most of the logic, we just need to add a couple of functions to it:
When changing Sim cards, it was necessary to call the process of changing in the “remote session-id” and “tunnel-source” tunnel settings, and when changing Sim cards back, return these values:
def l2_sim2(o, cli):
log_syslog(o, ops.CRITICAL, "New parameters applied: " + l2_lses2 + " " + l2_rses2 + " " + l2_vcid2 + " " + l2_src2 + " " + l2_dst2)
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Vlanif1")
output, n1, n2 = o.cli.execute(cli, "undo link-bridge Tunnel0/0/0")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
result = o.timedelay(seconds=60, milliseconds=0)
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Tunnel0/0/0")
output, n1, n2 = o.cli.execute(cli, "tunnel-protocol svpn")
output, n1, n2 = o.cli.execute(cli, "encapsulation l2tpv3")
output, n1, n2 = o.cli.execute(cli, "interface Tunnel0/0/0")
output, n1, n2 = o.cli.execute(cli, "l2tpv3 local session-id " + str(l2_lses2))
output, n1, n2 = o.cli.execute(cli, "l2tpv3 remote session-id " + str(l2_rses2))
output, n1, n2 = o.cli.execute(cli, "tunnel-source " + str(l2_src2))
output, n1, n2 = o.cli.execute(cli, "tunnel-destination " + str(l2_dst2))
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Vlanif1")
output, n1, n2 = o.cli.execute(cli, "link-bridge Tunnel0/0/0 vc-id " + str(l2_vcid2) + " tagged")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "save", {"continue": "Y"})
def l2_sim1(o, cli):
log_syslog(o, ops.CRITICAL, "New parameters applied: " + l2_lses1 + " " + l2_rses1 + " " + l2_vcid1 + " " + l2_src1 + " " + l2_dst1)
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Vlanif1")
output, n1, n2 = o.cli.execute(cli, "undo link-bridge Tunnel0/0/0")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
result = o.timedelay(seconds=60, milliseconds=0)
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Tunnel0/0/0")
output, n1, n2 = o.cli.execute(cli, "tunnel-protocol svpn")
output, n1, n2 = o.cli.execute(cli, "encapsulation l2tpv3")
output, n1, n2 = o.cli.execute(cli, "l2tpv3 local session-id " + str(l2_lses1))
output, n1, n2 = o.cli.execute(cli, "l2tpv3 remote session-id " + str(l2_rses1))
output, n1, n2 = o.cli.execute(cli, "tunnel-source " + str(l2_src1))
output, n1, n2 = o.cli.execute(cli, "tunnel-destination " + str(l2_dst1))
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "system-view")
output, n1, n2 = o.cli.execute(cli, "interface Vlanif1")
output, n1, n2 = o.cli.execute(cli, "link-bridge Tunnel0/0/0 vc-id " + str(l2_vcid1) + " tagged")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "quit")
output, n1, n2 = o.cli.execute(cli, "save", {"continue": "Y"})What are the advantages of this scheme of working in L2 MPLS mode:
The router works only with transparent Ethernet up to our PE, the scheme can also be used for both the Internet and IP VPN-inclusions.
MTU is now larger than the IPSec variant.
It is possible to carry out an easy seamless replacement of cable channels in case of accidents.
Taking into account all the options for providing services from the previous articles, we were able to build a universal, reliable and scalable access network infrastructure using the advantages of a VPN, which we are going to develop in the future.
In the following articles, we will talk about USB Deployment – a tool for simple configuration of routers, a Python script that we use to switch Sim cards, and the integration of our Multisim solution with a Cloud PBX (the router has a built-in SIP client and two FXS ports, we just couldn’t help but try to use them).
In one of the comments they asked about the speed of the router via IPSec…
According to the results of our tests, the speed is more than 50 Mbps (IPSec in IKEv2 mode, iperf3 – server), the same for the L2TP tunnel mode. The speeds were raised even higher, but we did not plan to offer our customers more than 50 Mbps according to our decision, because firstly, we run into the “hygienic” limit for LTE, and secondly, the routers were planned to be used together with the additional setting. services such as SIP PBX (we will tell in the next article), and thirdly, Huawei AR129 is being replaced by Huawei AR617, which is more productive, which we have yet to test.
