Performance Review and the identification of secret knowledge (review and video report)

Performance Review and the identification of secret knowledge (review and video report)

On April 26, at the conference KnowledgeConf 2019 , the report “Performance Review and Identification of Secret Knowledge” was heard. Usually we talk about technology, however, in order to develop as a company, we are doing far from it. This presentation, dedicated to engineers and their development, is a good example. If you are a team leader or are thinking about how to grow employees in a team, this article (and the report itself) may be useful.

Traditionally, we are pleased to present video with a report (50 minutes, much more informative than the article), and below - his pressing in text form, enriched with some details that were not mentioned in the report itself.

Why Performance Review?

Before Flanta, I was a technical director at a software company. I wanted the team to grow, to be able to solve more and more complex tasks and the staff improved their skills. And for this, two things were required:

  1. Transparent Growth Rules.
  2. Feedback.

It was necessary to get some rules according to which the employee could develop in the company and according to which it is possible to give him timely feedback to help with problems along the way.

Is it easy to build a career ladder?

As it turned out, no. Even if you have job descriptions for a higher level employee, you can't just take a set of keywords from it and say that these are the necessary steps to improve your career. If you tell an employee, “Learn Symfony 4, PostgreSQL, and MySQL,” that won't help him much. Do not teach him all MySQL! Details are important, so you need some kind of tool to measure a person’s skills ... something like a ruler .

Imagine a ruler, but instead of divisions, it has quality skill levels that can naturally be distinguished. If we had such a lineup, we would be able to attach real tasks, real experience and specific training materials to it. So?

Unfortunately, no: in practice, such a ruler turns out to be a separate topic for “holy wars”, since it is impossible to say with mathematical precision whether a person is able to implement complex business logic or not. But there is good news: in the same practice, mathematical accuracy is not so necessary . You can use the method of expert assessment: to give the most experienced and most immersed in the situation of the employee the opportunity to make a decision about belonging to the level of skills, and from this already push off.
We'll have to turn a blind eye to poor accuracy, to the mediocre scalability of the method, and much more, but at the same time, we can admit that this method does its job quite well. Not only that - and to this day it is used a lot. Often a team leader may well act as such an “expert” in his team, and this is ok.

Is it easy to give feedback?

It sounds logical that you need to periodically (for example, once a quarter) to meet with the employee and give him feedback. And try to organize it in such a way that the process is filled with meaning, not with a cargo cult. It seems like a good idea to take our “rulers” with skills:

... organize an N-dimensional space from them and tell the person where he is in this space, and then agree on where to go next:

N-dimensional space is hard to imagine, so we can use a table for this purpose. Rows in it will be our skills, and in the columns - the level of skill development. If we hold two such sessions, we can see something like the following picture:

... which would seem to convince us of the loyalty of the chosen path and the tools chosen - after all, people are developing. And when there is a critical mass of changes, you can raise the salary of the employee and change the label on his post, for example, saying that he is now Middle PHP Developer.

But how does everything happen in practice?

In theory, it sounds cool and transparent. I tried to start the mechanism in this form - and they were ... frankly, not very. The attempts of colleagues in the workshop known to me also did not inspire. The main problems are:

  • Not all employees are equally responsive to feedback . It is easy to find yourself in a situation when the question of lack of authority arises, and in such conditions it is difficult to give an expert assessment.
  • Highlighting key skills , which are used for assessment, turns out to be a nontrivial history in practice - especially if you are a man “from the plow”. It is difficult to ignore details and strike a balance between accuracy and generalization.
  • The presence of such “feedback” and “ladder” did not correlate with the motivation of the staff . If they developed, it is rather contrary to the system than thanks to it.

Flanta's need for a Performance Review

When I came to the "Flant", I saw a number of problems that made me think about the Performance Review, namely:

  • Engineers did not have an understanding of what they should grow in, in which direction to develop. This question was sometimes voiced, but a clear answer could be received quite rarely.
  • Engineers did not feel involved in the company, did not feel the return from it and its participation in their careers.
  • The company began active growth, and in these conditions it is especially necessary to build clear and working mechanisms for turning newcomers into experienced engineers and managers.

Therefore, I really wanted to highlight the skills that are needed for each of the positions, and create conditions for career growth.

But it turned out to be not so obvious ...

Looking for a list of skills

It turned out that no one can highlight the requirements for the skills of DevOps engineers and generally identify any "skill levels". The closest description of this position is in the immortal Robert Heinlein:

Anyone should be able to change diapers, plan invasions, cut pigs, design buildings, manage ships, write sonnets, keep accounts, build walls, set bones & lt; ... & gt ;, program computers, cook deliciously, fight well, worthy to die.

That is, the team leaders expected and raised their people as fullstack engineers who are able and correctly figure out the details of the task, and solve all technical issues, and bring it to production.
Of course, it inspired me. It's great to work in a company where everyone is a universal fighter, a versatile person and an excellent engineer! It was clouded only by the fact that everything looked like a huge dark cave with secret knowledge, which is scary to enter . I didn’t want to condemn engineers for such trips.

As you may have guessed, there were dozens of technologies with hundreds of nuances under the hood.We do not limit our customers in choosing the technology stack for their applications, so our engineer is faced with a wide variety of databases, queue servers, frameworks and languages, diagnostic tools, development and debugging tools, as well as features of various data centers and platforms. < br/>
Over the past 5 years, I have witnessed a fundamental change: if it used to be almost the norm when a technical cadre declared: “We work on platform X with database Y and we will not work on anything else”, now some stations can even release stack control, relying on microservices and ( quite controversial ) slogan: “If it breaks, it’s easier to rewrite everything from scratch. "

Under these conditions, creating a list of technologies for evaluating employees is not a trivial task. And to keep it constantly relevant is too time-consuming and difficult organizationally. Therefore, we tried to find a solution and want to share it.

Rethinking PR

We decided to try a different way, but first to rethink our actions and expectations. By launching a Review, we would:

  • ... tried to measure the performance of employees;
  • ... talked to them;
  • ... expected that in the end they will become happier, better understanding their prospects and their intended career growth.

We invested about one and a half months to rethink these points and below I will tell you what happened.

Performance Evaluation

I happened to look at something like this in an attempt to understand what might be wrong here:

And at some point an interesting thought came: here an excess amount of information is depicted. In fact, concrete “divisions” are not so important - the picture can be simplified to the following:

We are interested in development speed , and not the fact that a person has reached a particular level. Was there any development? Has a person studied a lot or a little? That's what the real figure is!

Investigating the secret of the skill list

Earlier we came to the conclusion that the list of skills is secret knowledge for us: engineers know something, do something every day, but what exactly is difficult to formulate.

One of the ways to reveal secret knowledge is “on the trail,” that is, by recording what is happening in order to analyze it later.

Just as we can follow the trail in the forest to understand that there are squirrels, foxes and bears, we will try to reveal the secret knowledge ...

Pilot Project

Timlid Peter kindly agreed to participate in the pilot project to launch the Review in an updated form. Peter understood ideas, and we planned meetings with employees. As a result of self-training, Peter formulated the following feedback about one of his engineers, Leonid:

This is an example of poor-quality feedback: if you heard about yourself in such formulations, it would hardly help you much. Even if everything is formulated more clearly in the head of a tmlid (which is not a fact at all), it is not very useful to record information in this form, because six months later such “footprints” are of no use - even the author will not remember what it is about.

We would like the feedback to be as objective as possible, polite and more likely to help us learn more about each other than it was a means to “motivate”.

Lead PR training

As a result, we formulated the following rules of preparation for PR for timlids:

  1. Before each review, you need to allocate from half an hour to an hour and a half to prepare. It is better to prepare the day before, and not on the same day.
  2. Go through all the trackers (ticket systems, instant messengers, karma systems, etc.), even if the manager is lazy and he resists, appealing to his beautiful memory. View notes from the previous review.

  3. Write a number of abstracts for discussion. These abstracts must contain:
    • Facts, with justification in the form of links to trackers;
    • The personal relationship of the reviewer: an explanation of why he (a) highlighted this item, and what is so important about it.

By the way, we have posted examples of bad and good formulations here .

PR process

The review itself takes place always in person . Despite the fact that we work remotely, we cannot talk about any review through documents/questionnaires/services. In our case, the communication goes through Google Hangouts and there are a team leader, an engineer, HR and, if necessary, other interested parties.

The review goes through several stages:

  • Talk about distracted to tune in to a conversation with each other.
  • A story about what a person did good , for which I want to thank him and explain why this is important. We want such good things to continue as well?
  • Mark unambiguously bad : breaches of agreement, discipline, and other things that you consider unforgivable and guilty of which is undeniable.
  • Discuss the incomprehensible or controversial according to the following rules:
    • Mark the facts and personal attitude to these facts.
    • Ask a general question (like "What do you think about this?") to get more information from the employee about the problem.
    • Ask the question: does the employee think that there is a problem? It may turn out that for some personal reasons he does not consider this important. We need to make sure that he shares our perceptions.
    • Ask the question: what exactly is the employee going to do to solve the problem?
    • If an employee does not share your problem, offers vague solutions - it is highly likely that he will not do anything with it. And to put pressure on him is pointless - it is better to try to understand why this is the case.
  • Ask feedback from the employee. Did his life in the company correspond to his expectations for this cycle? If you were really able to build a dialogue before, then at this stage you will receive feedback, and not muffled muffling, as often happens.
  • Talk about salary : do you raise it, and if not, what exactly needs to be corrected in order to raise it. The issue of salary is very important and complex and, in general, is the main motivation for an employee to participate in the Performance Review. There is a suspicion that there is not much point in trying to start some review processes, if it doesn’t affect the salary, but this is not 100% ...


Results of the first year

The analysis of the records accumulated over the first year helped us:

  • open the veil of the secrets of the identity of our engineers;
  • fill up the knowledge base with instructions and information that causes the most problems for employees;
  • improve the climate in teams.

Engineers have become much more tangible and grow quickly , they have an understanding of the direction for building their career.

Other significant moments:

  • Beginners' problems were solved more constructively when they were adapted: managers suddenly discovered for themselves that they could communicate with the newcomer about the problems ... and he would improve!
  • It became clearer what kind of employees we are waiting for in the company, so the requirements for hiring were adjusted.
  • It became more obvious to the employees themselves what they might want from them and what kind of development prospects they have.
  • There were sketches for the competency matrix, and there is also a mechanism for its constant updating.

Engineer Portrait

It may seem obvious that an engineer, for example, should really love what he does and enjoy working ... One way or another, these are the four skills that turned out to be most in demand (by a large margin):

Other important skills were:

Let's try to find out in the second year

In the meantime, attentive to details , the reader may have noticed that we didn’t measure performance very well ... Perhaps over time, the list of skills will come up and a suitable competency matrix will appear, or we will introduce some other rating system. < br/>
In addition, right now we are looking for ways to put on stream training on conducting a review among managers - we don’t have a ready-made solution yet.

A separate complex issue that is not so easy to formalize is the link between the evaluation system and the issue of employee salaries. It seems to us that this question is directly determined by the company's business model, and a solution for one business may not be very suitable for another. (About our model already told on the habre: we are built almost like a franchise, and as the efficiency increases and competence of the team she has money for salaries.)

Videos and Slides

Video from the performance (~ 50 minutes):

Presentation of the report:


You may also be interested in the following publications:

Among other reports in our blog:

Source text: Performance Review and the identification of secret knowledge (review and video report)