How we set up the summer internship process in the DS team
An internship is a great way to attract new specialists to your team. But its organization requires a lot of resources. And for these costs to generate value, many factors need to be taken into account.
How to select the best when the number of applications for internships is often tens or hundreds of times greater than the number of places? How to qualitatively evaluate interns in just 2 months? How to set up processes so that the internship prepares the candidate for real work and brings benefits in the tasks performed?
We have been conducting summer internships in the Data Science department for 6 years. Since 2018, 80% of those accepted for internships have stayed with us after completion. And 60% of them are still working and training new generations of trainees.
In this article I’ll tell you what we do to make the internship beneficial to the team.
Purpose of internships
To evaluate the success of an internship, it is necessary to define its goals before the start.
Usually the main goal is to attract new employees. But, even if you don’t have vacancies for permanent junior specialists, an internship can be beneficial: improve your employees, increase the company’s recognition and get solved problems.
Trainee selection and preparation
Drawing up a test task
A high-quality test for beginning Data Science specialists is not an easy task. Therefore, we have formulated the requirements for preparing the task:
Difficulty level. The task is not too difficult so as not to discourage interesting candidates, but at the same time allows you to demonstrate deep knowledge of the topic.
And we try to make the test so that the solution cannot be Googled. Now we have a new challenge: how to make a task that will not take much time, but will be difficult to “GPT”? Who knows how – tell us in the comments 🙂
Data for the task. We offer real tasks as test tasks, so we carefully check whether the data contains sensitive information for the company. And we conduct an assessment for data leakage.
Design and wording. The task is clearly formulated so that candidates do not waste time trying to figure out the requirements. We also pay attention to design and layout.
External review. The test task is validated by team members who do not work on the test project – this way we create conditions similar to the candidate’s situation. They solve the problem and give feedback.
We are experimenting with the format of tasks – last year one of the test options was a kaggle competition. We liked the experience: the public leaderboard encourages candidates to refine the solution and try different approaches. Therefore, this year we did a test only in the format competitions. It turned out to be fun and convenient for both us and the candidates. In the following articles we will talk about this in more detail.
Checking assignments
In the field of Data Science, solutions are non-standard and require a lot of time to verify. They send us several hundred test ones. The time to respond to candidates is limited and the work needs to be checked several days in advance, so we have divided the verification process into two stages:
Automatic calculation of metrics. First, we check all solutions with a script that calculates the metric on the private part of the test sample.
Detailed verification of solutions. We select solutions with a metric above a fixed threshold and check only them in detail. We study how the candidate reasoned during the decision process and what methods he tried. In the works that reached the second round, we evaluate the understandability and interpretability of the research as higher than a slight advantage in terms of the metric.
Automatic verification requires the candidate to strictly adhere to the answer format, which we warn about in the test. This places restrictions on candidates. But we believe that the approach has an additional advantage: it is possible to weed out applicants who do not read the assignment carefully.
It is important for us not only to get new employees, but also to develop applicants. Therefore, we provide feedback on tests and invite candidates to share solutions in the general chat after checking the tasks.
Interview
How do you interview candidates if they don't yet have relevant experience to ask about? We chose to conduct interviews in two stages:
Self-presentation. The guys record videos in which they talk about themselves, education, interests and ambitions in the field of Data Science.
Online interview. We select topics that are familiar to the candidate and select interviewees who are well versed in them.
Training of mentors
The internship lasts 2 months, so mentors prepare in advance:
They are trained in how to set tasks, give feedback, etc.
Prepare tasks for the trainee: create a clear description and check the availability of data for solution.
They formulate the first version of the goals, plan and criteria for completing the internship, which will then be discussed with the intern.
What does the same board look like after the internship?
Advice:
Preparing instructions for mentors and trainees requires resources. But in the future, this will save time on conveying context and ensure that everyone receives the same information.
Internship
Tasks
Our team has many different areas and it is difficult to formulate common criteria for all. But we have formulated recommendations on what the task for an intern should be:
The mentor himself understands how to solve it.
There is no ironclad deadline that other teams or clients depend on.
The lower the level of uncertainty, the better. Tasks must be analyzed deeply enough to be able to achieve results in two months.
Work during an internship is as close to reality as possible. Therefore, interns immediately begin to interact with other roles in the company – from gaining access to data to discussing the details of the solution with the customer.
Internship criteria
Internship criteria are our expectations of the intern when performing tasks and the decision to continue working depends on them. They are discussed and recorded at the first meetings with the mentor. And the team and employee can evaluate intermediate results using them at any time.
The criteria should be clear, measurable and achievable. Let's look at examples of well and poorly formulated criteria:
A good option | Bad option |
Research results are documented: | There is research documentation |
During the code review there are no questions about code style and efficiency | High quality code |
Softskills meet the team's requests: | Trainee joins the team |
Advice:
You can define criteria for each month of work with a newbie. This will show him the expected path of development, and will help you assess the dynamics.
Inclusion in corporate culture
Kontur is a large company with many products and teams. To make it easier for trainees to immerse themselves in processes, we have developed onboarding and online courses for them.
There are also events that help interns integrate into the team. They are carried out both at the company level and at the department level.
Advice:
Invite interns for lunch, tea/coffee, and invite them to team events outside of work – this will make it easier for them to establish communication within the team.
Communications with a mentor
Kickoff meetings
In the early days, the mentor helps the trainee understand the processes, the domain, and the task. Topics of kick-off meetings:
Introductory meeting: about the place of our department in the company, about the team and the project.
Development processes: what categories of tasks there are, their typical work flow, the review process adopted by the team.
Tools: where we write code, documentation and manage tasks.
First task: subject area, description of the task, why the task is important for the company.
Discussion of the goals and criteria for completing the internship.
Advice:
Once you and your trainee have established your goals and objectives, ask them to explain them in their own words. The next day, ask again – this will help make sure that you and the intern see the goals the same way and that they are fully recorded.
Regular communications
We let the trainee know that he can always have the support of a mentor. But the frequency of communications depends on the independence of the trainee. Usually, we first create a chat for the rapid exchange of information and assign daily sessions, and then adjust the frequency of communication depending on needs.
Internship equator
In the middle of the internship, we give the intern detailed feedback so that he can adjust his work until the end of the internship. Feedback should be based on a comprehensive assessment from different angles, including a self-review of the trainee himself.
How we do it:
We collect feedback from employees with whom the intern has worked.
The trainee and mentor separately review progress on the task in writing. They also evaluate the implementation of goals and give a forecast of whether they will be completed on time.
We hold a meeting where the trainee, mentor, manager and technical lead discuss the trainee’s feedback and progress.
We record the criteria for internships that have already been closed and steps to close the remaining ones. We adjust goals if we see objective reasons for not achieving them.
Hiring decision and feedback
At the end of the internship, we evaluate candidates and decide who we invite to work with us.
The difficulty of assessment is that trainees work on very different tasks and each has a different mentor. The goals and criteria that we wrote down at the beginning and updated in the middle of the internship help here. The assessment and comparison process is systematic and predictable for candidates. And so that interns have time to appeal, we inform them of the results a few days before the end of the internship.
Announcing the results is also important – we want to part with employees who were not a good fit for us on a positive note. You need to prepare for this: the one who tells the candidate about the decision must do it correctly and give the necessary information about the reasons.
After the internship
In the end, it is important to compare expectations with the result. And here the goals that we set before the start of the internship help. We hold a retrospective in which all organizers and mentors participate and jointly evaluate our results.
We also collect feedback from all involved persons – interns and company employees.
Based on the retrospective and feedback, we draw conclusions in order to correct shortcomings next year.
And, of course, we praise each other for our successes and share our experiences and best practices with colleagues 🙂
Conclusion
Organizing and conducting an internship requires a lot of effort and time. But in the end, we test the candidate’s skills in practice:
Technical skills – worked on a real problem using the same process and tools as other employees.
Goal setting, planning and self-organization – participated in the formulation of goals and steps to achieve them, trained to predict their work.
Self-reflection – practiced analyzing my progress and processing feedback about myself.
Softskills and the ability to work in a team – independently resolved issues with other roles. And we even checked at parties how he would fit into the team.
And if the candidate stays with us, then we get a colleague who is already immersed in the subject area, familiar with the processes and tools, and will quickly integrate into the overall workflow.
Internship checklist
We determine the goals and criteria of the internship.
We are assembling a team: mentors, test checkers, interviewers.
We are preparing tests, materials and an internship program.
We check test scores, interview and select interns.
We are starting an onboarding program for interns and holding kick-off meetings for immersion.
We provide feedback at the equator of the internship.
We decide to invite interns to the team.
We conduct a retrospective and work with the results.
Internship schedule
