Cloud storage of components: a new stage in the development of the ICS Internet gateway

Start

The popularity of modern UTM solutions is primarily associated with their multifunctionality. The ability to quickly solve all IT tasks of a company, and even from one web interface, attracts many system administrators.

Our solution, Internet Control Server, also belongs to the class of UTM products, combining functions of network protection, Internet connection management, content filtering and many others.

Despite the obvious advantages, the use of such solutions has a downside – they are often monolithic, which does not allow responding to market changes in a more flexible way.

From this point of view, a more interesting product is the UTM gateway, which can be built by the user himself, based on the real tasks of the company.

In this case, the sysadmin builds his gateway from the “building blocks” of the components, installing only the modules he needs. All tools are in the developer’s cloud, and the user downloads and installs the required packages. This principle has become fundamental for our future developments.

The historical FreeBSD ecosystem was fine and well understood. We have chosen the traditional FreeBSD package installer, pkg, as the most appropriate method for distributing packages. A repository is collected in our cloud, and access to it is registered on the client in the traditional pkg config.

What did we plan to do

The system should be a constructor in the form of a core on which the components are strung. The user receives a distribution kit that includes only the minimum required plugins (router, fierwall, etc.). The rest is installed as needed.

For the system to work, we needed to implement the following components:

• system – modified FreeBSD kernel
• core – a special package with product settings, which is a parent dependency for all plugins
• plugins – each of them is a compiled package, which is installed in a regular way by the pkg utility

Plugin structure

As stated, a plugin is a regular package. It can be installed with the pkg utility.
In order to maintain backward compatibility, it was decided that the new major version of the plugin would be a separate package, not related to the previous major version. for instance plugin-v1 and plugin-v2… The reason is that they may have completely different dependencies, perhaps even contradicting each other.

The minor version simply changes the version number of the package, everything is as usual here.

Working with plugins

In order for the system to work as we need, an add-on utility was created on top of pkg.

The main tool for work will be teams pkg query and pkg rquery…

The first collects information on plugins installed in the system, the second accesses the repository.

In order for the utility to control the correct execution of commands in the system, the execution of each command is checked for a return code. If a return code of 0 is received, then the command was executed, if something else, then an error occurred. Thus, you can track, for example, network connection problems.

An interesting nuance arose here. If, for example, we search for all packages by pattern:

then in case no installed package matches the template condition, error 69 is returned without an error message. The developers of the utility considered that if the search did not return anything, then this is abnormal behavior. Well, OK. I had to handle such a case in a special way.

Update issues

Then problems with versioning begin when updating plugins.

First, the pkg utility when executing the command pkg upgrade also updates ALL direct package dependencies, this behavior is built by the developers. But in our case, this includes the update and core, if a minor version has been released for it, which is undesirable, since core changes system parameters, and also requires a system reboot after the update.

That is, if we have 2 packages installed on our system – pkg-1 and pkg-2, and in dependencies pkg-1 indicated pkg-2, then if we execute the command pkg upgrade pkg-1, then it will also update pkg-2…

Let’s go the other way.

Let’s build the package dependency tree up and down.
We find the names of all packages that our package depends on:

pkg rquery% rn

Now all packages that depend on our package:

pkg rquery% dn

Let’s remove all package-dependent entities. Next, we will begin to delete the entities on which our package depends up the tree until we find ourselves in core (we will not delete it, of course). You can now restore the tree by installing the latest minor versions of all packages in it.

For example, in the picture above, we see that the package ics-plugin-a-v1 depends on the plugin ics-plugin-b-v1… If we need to update the package to version ics-plugin-v2 then this will entail an update of the package ics-plugin-b-v1for which there are 2 major versions – ics-plugin-b-v2 and ics-plugin-b-v3… That being said, none of them support the plugin. ics-plugin-c-v1… That is, the update will first install the package ics-plugin-b-v2 or -v3then ics-plugin-a-v1, and ics-plugin-c will be uninstalled and will not be installed.

In addition, core may have a major version, in which case the entire set of plugins must be updated to match.

To install a package ics-plugin-awhich depends on ics-core-v2 you need to update ics-core, after which only major packages that depend on ics-core-v2.

Database backup

When working with the repository, a situation is possible when during the update the connection will be lost (system error code 70 or 3 occurs). In addition, in order to correctly uninstall the plugin, the system needs a valid pkg base on the system. When executing the command pkg updateif there is no connection, the base reports an error and even locally does not perform its functions until the update is executed correctly.

To avoid such situations, we will use the utility pkg backup… Before any operation, during which there is a possibility of not getting the desired result, save the base:

pkg backup -d

If the operation was completed incorrectly, we return the base to its place:

pkg backup -r

Core

So far, everything is quite simple. If there is an update for the kernel, then:

1. download the new kernel image in the archive
2. create a new dataset in zfs
3. mount the dataset into the system
4. unpack the image
5. install a special package with the necessary kernel options for normal operation
6. register the new image as bootable
7. ???

Profit?

Not yet.

We need to restore the system plugins installed earlier by the user (if they are supported by the new kernel, of course). Accordingly, you need to create a separate plugin repository for each kernel version.

But (as always) there is a nuance.
We have not yet installed the kernel of the future version, and we cannot install the plugin for another version. If we boot the system from a new image, then we will not be able to access the repository (for historical and technical reasons, routing is also a plugin).

What to do?

We raise the server with api, which will serve up the archive with the plugin repository for the specified kernel version.

Then our utility will generate a repository config file with the path to the folder with the unpacked archive.

For FreeBSD this will be the file .conf in folder / usr / local / etc / repos or / etc / repos… It is worth noting here that the path is written as follows: url: “file: /// path_to_repo” (3 slashes!)

We save the data about the installed plugins and check their compatibility with the future version of the kernel (if there are incompatible ones, we will inform the user about it).

Now you can reboot.

Last thing

Pkg requires initialization (bootstrap) after system upgrade. Therefore, if you execute any command, you will be prompted for it. In our case, the binding utility will not understand that they want it and will consider that the operation was performed incorrectly.

To do this, we made an initialization error handler, fortunately, it has a separate system operation code 1. Therefore, if the utility encounters such a code, it simply executes

pkg -y

Pkg makes bootstrap, and then you can work normally.

Total

This is how we built our repository. While this is a prototype, and in the future it will most likely change and become more complex, but the design base has been laid and will remain unchanged.

The described technology will be applied in new developments of Internet Control Server, making it even more convenient for use in corporate networks of companies. You can download and test the most current version of ICS by link…

Trial period, free version for 9 users, online demo and responsive technical support.

Follow the news and stay with us!