Trimming Lines: Migrating from Puppet Enterprise to Ansible Tower. Part 1

The National Environmental Satellite Information Service (NESDIS) has reduced its Red Hat Enterprise Linux (RHEL) configuration management costs by 35% by moving from Puppet Enterprise to Ansible Tower. In this video of the “how we did it” category, system engineer Michael Rau substantiates the implementation of this migration, shares useful tips and experience gained as a result of the transition from one SCM to another.

From this video you will learn:

  • How to justify management the feasibility of moving from Puppet Enterprise to Ansible Tower;
  • what strategies to use for the smoothest transition;
  • tips for transcoding PE manifests in Ansible Playbook;
  • Best practices for installing Ansible Tower.

Greetings to all, my name is Michael Rau, I am a senior systems engineer at ActioNet, which works for the National Oceanic and Atmospheric Administration (NOAA) of NESDIS. Today we’ll talk about line trimming — my own experience migrating from Puppet Enterprise to Ansible Tower. The theme of this presentation is to “look at my scars” left over after I made this transition at the beginning of the year. I want to tell you what I learned during this process. So when you do this, using my experience, you can make the transition without too much difficulty.

You see slides similar to this at the beginning of every presentation at Ansible Fest. This slide shows the history of my company’s automation. I am not new to this business because I have been using Puppet / Puppet Enterprise since 2007. I started working with Ansible in 2016, and I, like many other users of this product, was attracted by the possibility of “tricks” using the command line and simple scripts (playbooks). At the end of 2017, I turned to my leadership about the good reasons for moving to Ansible Tower. In a minute I will talk about the reasons that prompted me to take this step. After obtaining the consent of the management, it took several more months to fulfill the plan, and I made the transition in January-February of this year. So, we completely abandoned Puppet in favor of Ansible, and this is a great thing.

What attracts me most about Ansible is the ability to write and use roles and scripts (playbooks). Roles are great for creating different, but interrelated tasks (tasks) and placing all the data related to these tasks in one place. Playbook is YAML syntax, a script file that describes actions for one or more hosts. I talk about these features to users, primarily software developers. Ansible Tower makes it possible to say: “No, you do not have access to shell access, but I give you the opportunity to start all Tower processes and restart the service when you need it.” I will tell you about the working environment and the equipment we use.

This is a federal LAN, 7 physical sites connected via cloud-based MPLS, 140 RHEL servers, 99% of which are virtual (vSphere), SuperMicro hardware, NexentaStore NAS, a set of Cisco, Arista and Cumulus switches and unified Fortinet UTM threat management tools on each site .

The federal network means that I must use all means of information protection provided for by legislative acts. You should bear in mind that Puppet Enterprise does not support most of the equipment we use. We are forced to use budget hardware because government agencies are having problems financing this item of expenditure. Therefore, we buy SuperMicro class hardware and assemble our equipment from individual parts, the maintenance of which is guaranteed by government contracts. We use Linux, and this is one of the important reasons for switching to Ansible.

Our history with Puppet is this.

In 2007, we had a small network of 20-25 nodes, in which we deployed Puppet. Basically, these nodes were just “boxes” of RedHat. In 2010, we started using the Puppet Dashboard web interface for 45 sites. As the network continued to expand, in 2014 we switched to PE 3.3, making a complete transition with rewriting the manifest for 75 nodes. This had to be done because Puppet likes to change the rules of the game, and in this case they completely changed the language. A year later, when support for version 3 of Puppet Enterprise stopped, we were forced to migrate to PE 2015.2. I again had to rewrite the manifest for the new servers and purchase a license with a reserve of 100 nodes, although at that time we had only 85 nodes.

Only 2 years passed, and again we had to do a lot of work to switch to the new version of PE 2016.4. We bought a license for 300 nodes, with a total of 130. We again had to make major changes to the manifest, because the new version of the language had a different syntax from the language of the 2015 version. As a result, our SCM switched from SVN version control system to Bitbucket (Git). That was our “relationship” with Puppet.

So, I had to explain to management why we need to switch to another SCM using the following arguments. The first is the high price of the service. I spoke with the RedHat guys and they said that the cost of maintaining a network of 300 nodes using Ansible Tower is half the cost of Puppet Enterprise. If you purchase another Ansible Engine, the cost will be about the same, but you will get much more features than PE. Since we are a state-owned company funded from the federal budget, this is a pretty weighty argument.

The second argument is versatility. Puppet only supports hardware with a Puppet agent. This means that you need to install an agent on all switches, and it must be the latest version. And if part of your switches supports one version, and part – another, you will need to install a new version of the PE agent on them so that all of them can work on the same SCM system.

The Ansible Tower system works differently because it does not have any agents, but there are modules that support Cisco switches and all other switches. This SCM supports Qubes OS, Linux, and 4.NET UTM. Ansible Tower also supports NexentaStore NASs based on the Illumos kernel, an open source Unix-based operating system. This is very little support, but Ansible Tower does it anyway.

The third argument, which is very important both for me and for our administration, is ease of development. I mastered the Puppet manifest modules and code for 10 years, but I studied Ansible for a week, because it is much easier to work with this SCM. If you run executable files, of course, if you do not do it unnecessarily, then reasonable and responsive handlers work with them. YAML-based playbooks scripts are easy to learn and quick to use. Those who have never heard of YAML before can simply read the scripts and easily understand how it works.

Honestly, Puppet greatly complicates your work as a developer, because it is based on the use of Puppet Master. This is the only machine authorized to communicate with Puppet agents. If you made any changes to the manifest and want to test your code, you must rewrite the code for the Puppet Master, that is, configure the Puppet master file / etc / hosts to connect all clients and start the Puppet Server service. Only then can you test the operation of network equipment on one host. This is a rather painful procedure.
In Ansible, everything is much simpler. All that needs to be done is to develop code for a machine that can communicate via SSH with the host under test. This is much easier to work with.

The next big plus of Ansible Tower is the ability to use your existing support system and save your existing hardware configuration. This SCM without any additional actions uses all available information about your infrastructure and equipment, virtual machines, servers, etc. It can communicate with your satellite servers, if any, and provides you with integration that you will never get when working with Puppet.

Another important thing is detailed control. You know that Puppet is a modular system, it is a client-server application, so you must determine the existing aspects of the operation of all your machines in one long manifest. At the same time, the state of each individual element of the system needs to be tested every half hour – this is the default period. This is how Puppet works.

Tower saves you from this. You can perform various processes on a variety of equipment without restrictions, you can do the main work, start other important processes, configure the security system, work with databases. You can do everything that comes with certain difficulties in Puppet Enterprise. So, if you have configured on one host, it will take time for the changes to take effect on the other hosts. In Ansible, all changes take effect simultaneously.

Finally, consider the security module. In Ansible Tower, it is implemented simply amazingly, with great accuracy and thoroughness. You can provide users with access to specific services or to specific hosts. I do this with my employees who are used to working on Windows, restricting them access to the Linux shell. I provide them with access to the Tower so that they can only do the work and run only those services that are within their competence.

Let’s look at the things you need to do in advance to ease the transition to Ansible Tower. First of all, it is necessary to prepare your equipment. If any elements of your infrastructure are not yet in the database, you need to add them there. There are systems that do not change their characteristics and therefore are absent in the Puppet database, but if you do not add them there before moving to Tower, you will lose a number of advantages. This may be a “dirty” preliminary database, but it should contain information about all the equipment you have. Therefore, you should write a dynamic hardware script that will automatically make all infrastructure changes to the database, then Ansible will know which hosts should be in the new system. You will not need to tell this SCM which hosts you have added and which hosts no longer exist, because it will recognize all this automatically. The more data there will be in the database, the more useful and flexible Ansible will be. It works as if it simply reads a barcode of equipment status from the database.

Spend some time getting to know the command line in Ansible. Run some special commands to test the hardware script, write and run some simple but useful playbook scripts, use Jinja2 templates where appropriate. Try writing a role and script for an integrated multi-stage process using a standard, frequently encountered hardware configuration. Play with these things, test how it works. This way you will learn how to work with the tools for creating libraries used in the Tower. I already said that it took me about 3 months to prepare for the transition. I think that based on my experience, you will be able to do this faster. Do not consider this time lost, because later you will feel all the advantages of the work done.

Next, you need to decide what you expect from Ansible Tower, what exactly this system should do for you.

Do you need to deploy the system on empty hardware, on empty virtual machines? Or do you want to keep the original conditions of work and settings of existing equipment? This is a very important aspect for the work of state-owned companies, so you must be sure that you will be able to migrate and deploy Ansible on your existing configuration. Identify the routine administrative processes you want to automate. Find out if you need to deploy specific applications and services on the new system. Make a list of what you want to do and prioritize.

Then start writing code for scripts and roles that will ensure that you complete the tasks you have planned. Combine them in Projects, a logical collection of relevant playbooks scripts. Each Project will relate to a separate Git repository or another repository depending on which code manager you use. You can manage playbook scripts and playbook directories by manually placing them in Project Base Path on the Tower server, or by placing the playbook in any tower source code management (SCM) system, including Git, Subversion, Mercurial, and Red Hat Insights. Within one Project, you can place as many scripts as you want. For example, I created one basic Project, in which I placed a script for the main elements of RedHat, a script for the Linux framework, and scripts for the rest of the basic indicators. Thus, in one project there were a wide variety of roles and scripts that were managed from one Git repository.

Run all these things through the command line, this is a good way to test their health. This way you get ready for the Tower installation.

Let’s talk a little bit about transcoding the Puppet manifest, because I spent a lot of time on it until I figured out what really needed to be done.

As I said before, Puppet stores all the settings and parameters of the equipment in one long manifest, and this manifest contains everything that this SCM should do. When moving on, you don’t need to push all your tasks into one list, instead think about the structure of the new system: roles, scripts, tags, groups and what should go there. Some of the stand-alone network elements should be grouped into groups for which you can create scripts. More complex infrastructure elements that involve a large number of resources, including autonomous classes, can be combined in roles. Before migration, you need to decide on this. If you create voluminous roles or scripts that do not fit on one screen, you should use tags to be able to capture individual parts of the infrastructure.

18:00

The sequel will be very soon …

A bit of advertising 🙂

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99, A unique analogue of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server correctly? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only here 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands! Dell R420 – 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB – from $ 99! Read about How to Build Infrastructure Bldg. class using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *