The Ultimate DevOps Engineer Interview Guide – What to Ask and Prepare for

I started networking at school and have been working for money for over 16 years. I got a lot of jobs, in large companies and small ones, then I opened my own business and regularly hire people myself. Over the years and experience, I, and many others, develop interviewing intuition.

This is when there is no clear algorithm. You just talk to a person and understand something for yourself. You ask what the candidate did in the previous job, you cling to the topic – and now you are just discussing engineering topics, at about the same level as with colleagues. If the conversation goes well and the person likes it, then everything is fine.

Such intuition can hardly be learned from books and texts, it comes on its own with experience. Together with her, phrases like “I don’t care about specific knowledge as much as a general outlook, the ability to seek information, understanding whether we will work together” and so on seep into thinking.

But sometimes, in order not to lose your grip, you still need to remind yourself what knowledge an engineer should have and what questions you can most objectively evaluate the person you see for the first time in your life.

First, I quickly go through all resumes

While I go through the responses, I pay attention to keywords and places of work. I always read the cover letter – a lot of applicants are eliminated at this stage. I post a job posting to find a DevOps engineer, and I get a response from a python developer, a golang developer, or a current courier who is “very interested and would like to try.”

Resumes pass by, which indicate the experience of work in government agencies, something in the spirit of “administrator of the first category in the Central Bank of the Russian Federation.” All these verbose stories about “administration of information systems” I cut off immediately without hesitation. The more officious and bureaucratic the description of the past job, the higher the chance that such an applicant has not seen anything in his life, except for Windows and backups using a graphical interface.

In a quick selection of resumes, the rule helps: the more technologies I meet, the better. If a person’s resume says MySQL, Linux, Postgres, Apache, and so on, chances are there. The person at least heard about technology and, who knows, perhaps even worked with them himself. But let’s be honest – you can write anything on your resume.

At the interview, the first thing I do is check the database

When I become a frail old man and I will be dumbly beaten back, I will start pounding with a stick on the back of everyone who does not know the net! This is a must have for any engineer. We live in a world where everything happens in networks. Even in a closed public sector, there is a local contour. And there sits a developer who writes some kind of proxy service or composes a service that works with an API, and does not understand anything about the API.

I do not require any special knowledge, I do not ask to configure MPLS for me. But what is a subnet mask, what is an IP address – in the 21st century all IT specialists should know. I have no idea what is going on in a person’s head when he does not understand what 127.0.0.1 is. He sits on a local machine, he has a service running on a virtual machine. The service has an endpoint 127.0.0.1, a connection to the database. From ignorance, our hero drives the same thing. As a result: “I can’t connect to the base.” Of course, damn it, it doesn’t connect!

If a person has a CCNA certificate, okay, even if he did not pass it, did not receive it physically, but prepared – this fact is enough for me.

For example, here’s a standard CCNA puzzle to understand how networks work

There are two switches from different networks, there is a router between them. Computer A wants to send data to Computer B.

What happens at this moment?

It is important for me to hear something like: “Yeah, we look at the routing table, we see that you can get to this network through a router. The computer sends a request to the network, tries to find, determine the MAC address from the IP address of this router, and transfers data to it. The router sees that the grid is connected, gives the data to computer B. Computer B receives it, happy end. “

Then I ask across all layers of the OSI model

Has anyone heard of the Spanning Tree Protocol? About the root protocol, about the IP layer? Ok, how does it all work? Does he understand how routing happens? Well, in bulk: routing tables, dynamic routing protocol, TCP transport layer. And so on and so forth.

I want to hear the difference between TCP and UDP. A good specialist will answer me why critical systems (for example, Domain Name System) use the protocol without guaranteed message delivery (UDP).

The answer is simple – it’s faster this way. While arranging a TCP session, you can send a UDP packet 3 times there and receive it back. And no overhead.

I ask questions about DNS

What are the recording types? Does my interlocutor know what an MX record is, how spf works, or how DKIM works.

Yes, this knowledge may not be useful in work. But it is important for me to know whether a person understands the essence of technology, whether he was interested in reading about it. Did he add some records to the DNS and google what an spf record is, or did he not?

Absolutely everywhere now the HTTP protocol is used, and I ask about it

I start by asking the standard questions about the differences between http versions 1.0, 1.1 and version two. I am interested in what headers are and why they are needed. How does the web server understand that it received a request for a specific virtual host, and not for any other. And I always ask a couple of questions about Nginx.

Then I smoothly switch my attention to the TLS protocol

How does SSL TLS work? An engineer needs to understand this at least at a basic level – there is a root certification authority, he signed the certificate, and the certificate is used somewhere.

In TLS, I’m usually interested in the process of establishing a connection. Why do we need private and public keys and how do they interact? If a person fumbles, then I ask a trick question: is it possible to have several different certificates on one IP-shnik?

Answer under the spoiler

The fact is that first a TLS connection is established, and then the HTTP protocol follows, and the web server can determine which site the client is accessing only via the HTTP protocol, by the host header. Nginx does not yet know which site is being accessed, but it should already give the certificate to the client. There are extensions to the TLS protocol that allow the client to pass the hostname to the server that it contacts when trying to establish a TLS connection. And thus choose the required certificate. When this extension was not available, only one site with SSL enabled could be hosted on one IP-schnick.

Moving on to Linux and BASH

You need to know everything Unix, all Unix-like systems. You need to be able to work with Shell and Bash, know the basic commands. ls, mkdir, etc.

Well, if the candidate can script a little in BASH, it means that he tried to somehow automate this story.

On Linux I will ask how to replace lines in a file with other lines. Or how to parse some access.log in Nginx using BASH. For a person to talk about awk, about cat, about sort, about everything that helps fast work.

There are text files everywhere, you need to work with them correctly. If the applicant offers to copy them to Excel and do something there, then I will feel embarrassed.

Then you need to figure out how things are with virtualization.

Conventional virtualization, virtualization through the hypervisor. If a candidate starts talking about paravirtualization, it means that he knows something.

My main question is: what is the difference between container virtualization and normal hypervisor virtualization? It is good if a person compares what is better, what is worse, where it should be used.

Moving on to containers

Find out if the person you are talking to has worked with Docker? Did he compile images, write Docker files, use Docker compose, deploy with it. Why are these containers needed at all and how do they work? Has our applicant brought up the Swarm or Kubernetes container orchestration system? You can ask a whole block of questions, but the main thing is to understand whether the person did the work with his own hands with containers or not?

Asking about CI / CD deployment

I am interested in a huge list of things: how does he set up automatic deployment, how does he set up Continuous Integration? How his applications are assembled, does he use code analysis systems (PVS-Studio, SonarQube). How he writes tests, or how he integrates tests written by developers.

Does it do some kind of integration testing on these assemblies? What happens next with what he collected? Does it somehow add up to artfecates or is it packed into docker containers, pushed to the registry? Let him tell you which systems he used to set up the CI / CD process. It can be GitLab CI, Circle CI, and some cloud solutions. Maybe Jenkins, well, you shouldn’t forget about self-written scripts in PowerShell.

Tell me how a person deploys and I will understand everything. It can be deployed using Helm in Kubernetes, Ansible, scripts, or something else self-written.

About the configuration management system

We speak Ansible most often. Because most people know him. So, why do we need roles, how to encrypt and store secret data, how to upload passwords to a Git repository? And stuff like that.

Learn about the ability to write code

Understanding development is important in order to understand how services interact, why APIs, authentication protocols, authorization are needed. It’s nice if the candidate wrote something himself. Even basic scripts in Python or Shell are enough for me. What matters is not the code, but what tasks the person wanted to solve, what he wanted to achieve.

Coding is needed to support the infrastructure, write local scripts for backups, custom monitoring, to calmly pull out metrics. It often happens in work that there is simply no standard solution for some task.

The final interview questions are about database storage

SQL, NoSQL – what are the differences, what did you work with? More often I meet people with PostgreSQL experience, less often – MySQL. I ask questions about indexes, whether an applicant can set up a replica, whether he can set up logical replication between tables or just data. And what will he monitor in this case? How to speed up the base?

By the way, this is a good question. Let’s say the base is now sitting and resting on the disk. And nothing can be done with it – no one else will buy the server. How do you make it work faster right now?

Just…

Simple: you need to turn off fsync so that the database does not wait for the data to be written to disk, but rather saves it. Linux can give a successful write when it put it in the buffer, and not when it is loaded onto the disk. This is a bit of a crutch mode of operation, even rather dangerous and risky. The food will be cut off at this moment, and everything will start. But this method speeds up the recording by an order of magnitude. But don’t repeat and never do that. And yes, I didn’t tell you that.

To sum up –

• networks
• API level
• transport layer TCP UDP
• DNS and HTTP protocols
• Linux
• container and hypervisor virtualization
• CI / CD
• configuration management system
• containers
• Database

All this allows you to make a complete picture of a person.

The most critical thing on my list is basic knowledge. If you don’t know the base, it’s time to finish your social security assignment.

If a person cannot answer me what a subnet mask is, or does not understand why headers are needed in HTTP, in most cases such an applicant gets the fattest minus in my notebook. Weren’t you curious to know how all these things that you poke a mouse cursor work in?

It’s very hard to work with people who just mechanically repeat what they have learned, instead of figuring out how everything works. A beginner who does not know Docker can easily figure it out if he has an understanding of what HTTP, TLS and networks are.

But finally, the most important point – this entire list is not needed in order to follow it in a real interview. You must go through this list yourself before hiring people.

And if everything works out, call the candidate for an interview and ask him just one question – what was the biggest failure you made at work, how did you fix it and what conclusions did you draw from your mistakes.

When a person has something to tell, such a social security session will pass like a friendly conversation between two engineers with beer.

Well, by tradition, I invite you to join our community in the cart – there we regularly analyze all these topics and many others that will be useful both at the interview and at work.