How can you tell if a football manager is actually good at their job?

An important thing to remember about Andre Villas-Boas is that he had ridiculously good hair.

You don’t spend a record-shattering €15 million (£12.9; $16.3m) fee to sign a rookie manager away from Porto unless you’re pretty sure you know what you’re getting, and one thing Chelsea knew for certain, back in the heady days of 2011, was that the man with a swirling, fox-red side-parting looked impossibly cool getting tossed in the air during trophy celebrations.

Villas-Boas at Porto in 2010 (Dimitar Dilkoff/AFP via Getty Images)

Hair like that had sexy new ideas — a philosophy, perhaps. It had the sort of rakish sweep that could command a press conference, smouldering volcanically above the jagged peaks of an unbuttoned collar. But when the 33-year-old prodigy conducted his first interview as the world’s most expensive manager, all of the glamour quickly drained away.

“Don’t expect something,” Villas-Boas warned gently, “from one man.”

True to his word, he was sacked by March.

Villas-Boas to Chelsea might have gone down as a historic blunder if not for all the other managers teams have squandered transfer fees on in the last few years alone: Marco Rose to Borussia Dortmund (€5 million up front for one lacklustre season); Adi Hutter to Borussia Monchengladbach (€7.5 million, ditto); Julian Nagelsmann to Bayern Munich (€25 million for 19 months); Graham Potter to Chelsea (let’s not talk about it). These were the cream of the crop, the head coaches clubs couldn’t afford to wait around for, yet in their new jobs they had the shelf life of a bunch of bruised bananas.

How do we know if a manager is good? The question sounds almost too obvious to ask — anyone down the pub will be happy to explain it to you loudly over a pint — but professional organisations with millions at stake whiff on it every year. Apparently the answer isn’t great hair. It can’t be trophies, either, since those are pretty much only available to managers already at top clubs. If the study of up-and-coming coaches can be called a science, it remains a largely theoretical one.

“We’ve done work with football clubs and leagues, actually, around what predicts head coach success and it’s very, very hard,” says Omar Chaudhuri of the sports consultancy 21st Group. “There are very few strong predictors.”

Everyone loves a winner, so it makes sense that employers would start by looking for coaching talent toward the top of the table. But we also know that in the grossly unequal world of European football, wage bills are destiny for most teams, no matter who’s in the technical area. The managers we admire most are the ones who find a way to punch above their weight.

To pick out those overachievers, we can start by modelling the relationship between squad strength and success using crowdsourced “market values” from Transfermarkt, which are a decent proxy for player quality when you don’t have wages handy. We’ll average this season’s values with last season’s, where available, to give coaches some credit for player development, then weight the values by minutes played to account for absences.

For the performance side, we’ll use a 70/30 blend of non-penalty expected goal difference and actual goal difference, which captures team strength pretty well and puts more emphasis on the parts of the game coaches are likely to have some influence over (creating and denying chances) than the parts they probably don’t (finishing, saving shots, successfully lobbying for penalties by doing the VAR rectangle thing with their fingers).

The results are striking. Over the last seven seasons across Europe’s top leagues, our simple player quality model can explain around 80 per cent of teams’ success.

But what about the remaining 20 per cent — who should get credit for that?

When we look at the outliers on the chart above, it seems fair to say that Gian Piero Gasperini’s freewheeling style helped elevate Atalanta’s mid-budget squad into a Champions League contender a few years back, and the whole platoon of head coaches and interim guys who oversaw Schalke’s disastrous 2020-21 campaign probably weren’t so hot at their job. Maybe performance over squad value is a fair measure of what a manager brings to the table.

Reassuringly, this season’s list of top teams for adjusted goal difference over expected is a veritable who’s who of coaching legends and the game’s hottest up-and-coming managers.

Xabi Alonso has turned down overtures from Bayern Munich and Liverpool to stay at German champs-in-waiting Bayer Leverkusen, while Brighton’s Roberto De Zerbi, whom no less an authority than Pep Guardiola called “one of the most influential managers of the last 20 years,” remains a strong contender for both jobs.

In Catalonia, Barcelona have been making eyes at Girona’s Michel. Sebastian Hoeness, Paulo Fonseca, Thiago Motta and Will Still have flocks of admirers, and maybe we should all be paying more attention to whatever Eric Roy’s got cooking at Brest.

So is that it — have we cracked the not-so-secret formula to finding Europe’s next top manager?

Well, hang on a second.

One important trait for a good sports stat is stability, or how much it varies from season to season. If last year’s performance can’t predict next year’s because the number is too sensitive to context, you probably don’t want to make it the sole basis for any expensive hiring decisions.

By that standard, our manager metric is a bust. For head coaches who change jobs, there’s no correlation whatsoever between the previous year’s performance above or below expectations at their old club and their first season at their new club. Even though goal difference added seemed pretty good at identifying this season’s hottest managers, it has zero predictive value for new hires.

When Chelsea spent £21.5 million to sign Graham Potter, he was coming off one of the best runs by any head coach in the last seven years: in 2020-21 and 2021-22, Brighton finished 22 and 13 adjusted goals better than expected. His seven months in London went, er, not quite as well.

Brighton, meanwhile, signed Roberto De Zerbi even though his final season at Sassuolo had been about average compared to their squad value. He’d had a pretty good season the year before that, and a respectable stint outside the top five leagues at Shakhtar Donetsk in between, but nothing that might have hinted that his first season at Brighton would be the fourth-best out of hundreds in our dataset.

What can explain the difference between these two very different hiring stories? Maybe there’s a clue in how Brighton’s famously analytical owner Tony Bloom explained his process. “I am confident,” he said of the De Zerbi hire, “his style and tactical approach will suit our existing squad superbly.”

De Zerbi (facing camera) and Potter in 2022 (Bryn Lennon/Getty Images)

Smart clubs don’t just hire successful managers in hopes that they possess some innate knowledge of how to win. They’re careful to match a coach’s tactics to the players they already have, knowing that changing styles will cost them money and time.

“I don’t want to have to replace 15 players or something like that over two years,” says one veteran analytics consultant, who requested anonymity to protect client relationships. “Because then it becomes a project of just kind of cycling through players and hoping things work out.”

Not every club is as careful about this step as Brighton. Chaudhuri explains that searches often start with a “performance piece” to determine whether managers are making the most of their current squad, but “then you have a playing style piece, which clubs generally tend to be quite vague on how they want to play. They say, ‘We want games to be attractive and exciting,’ whatever that means. And then you go, ‘Okay, tell us what you think that looks like.’”

The other consultant agrees. “I had this meeting yesterday, I gave five candidates, like, ‘What do you think of these five?’” he says. “And he was like, ‘Well, I like these four.’ But I said, ‘One of these four is actually not the style you said you want.’”

Figuring out which managers have exceeded expectations is the easy part. You can watch their players flinging them into the air at a trophy celebration and envision your club doing the same next season. But success, on its own, is fickle. It also tends to be expensive. The right question isn’t “How do we know if a manager is good?” but “How do we know if a manager will be good for this group of players?”

The secret ingredient in hiring the right coach is style — and not just the kind that comes with really good hair.

(Header photo: Lars Baron/Getty Images)

The Athletic recently profiled six of European football’s most innovative up-and-coming managers.