The Future of the Perfect Job

The State of the Recruiting Industry

Despite our best efforts, we're still not terribly good at matching talent to opportunity.

There remains a significant gender gap in the worlds' workforce, as measured by both the relative number of women in the workforce and pay equality. This is especially true at the top where women sit in the big chair of only 23 of the Fortune 500. Elsewhere 98% of US venture funding goes to whites and asians, although they only make up 66% of the US population (and nearly all of that goes to males under 30 years old). Meanwhile, the pipeline is broken: universities are five times more expensive than 30 years ago, offer less career benefit, and appear to be firmly stuck in the 19th Century.

You can't tell me, that in a world where such glaring bias is endemic, that we are successfully matching the right people with the right opportunities. (In fact, I would wager that we're doing a lot worse in the areas that we don't track.)

Bad talent matching is frustrating for both employees and employers, and it's not like nobody is working on the problem. Yet, it becomes increasingly imperative as the gig economy rolls forward. Insert here the oft-touted number of jobs you will have in your lifetime vs your grandparents had in theirs: although I'm not sure what's worse, getting the wrong job once for life, or twenty timestwo to three disappointing years at a time.

Big Data to the Rescue?

I see an article almost daily about how big data is reshaping recruiting. That all sounds marvelous. Tremendously exciting. Yet the sceptic in me is still dubious as to how mining my current network and subjecting me to a couple of online assessments can tell you much about my ability to be a rockstar anything.

Let's use a simple example: lets say you're mining LinkedIn for a programmer from a pool of two candidates: A and B. A is more social and has also job-hopped 5 times in the last 5 years, each time moving just before he was fired because, at the end of the day, he's not a great programmer. So A has several times the number of connections as B, and has also received many more endorsements, reviews, and has been tagged 500% more than B for a number of core skills you are looking for. Meanwhile, B is a fantastic programmer at a small company where she has worked unnoticed for the last 5 years, under-employed because she's largely unaware of the opportunities around her. She has generally ignored LinkedIn, has no real network to speak of, and knows zero recruiters.

So the algorithms keep offering up hacker A into the hiring process and ignoring B. That's bad, but this in itself makes A more and more likely to be found next time. In effect, A has quickly risen to page one of recruiting Google, which position ensures they will likely stay there.

As always, the spoils go to those best able to work the system. Not those best suited for the work. Everyone loses (except A).

At the end of the day, the recruiting process is still dominated by human bias: confirmation bias, selection bias, ingroup bias, status-quo bias, bandwagon effect, projection bias, false consensus bias, and the anchoring effect, to name a few.

We Must Be Better At Measuring Performance

So, how do you measure suitability? Well, that's the question the whole business world has been struggling with since eugenicists starting building the perfect human in the 1900s and psychometrists started trying to measure them.

But what are you measuring? Are you measuring my ability now, or my potential? Are you measuring "hard" skills (programming or drawing), "soft" skills (leadership or innovation), or other intangibles (heart or drive)? Which is more important? How about morality (honesty)? Level of personal debt? (Those with a large boat payment are much less likely to quit than those without one.)

In short, the algorithms can only mine the data they have, and the data we have is awfully sparse, inconsistent, and mostly self-selected.

The Future Is Not An Update of the Past

What we need is much better data, and that means much better data collection, from more varied sources, and better sources. We need to work out how to get information on the best passive candidates without forcing them to master the current self advertising system. This may also include breaking up some well established cartels or ways of doing things.

It's an exciting prospect, and it actually means turning the entire recruiting system on its head: not merely giving new tools to the current incumbents. This is not recruiting 2.0, but a completely new field.

I'm looking forward to it. Because until we work out how to do that, we will be data mining late-night infomercials to work out what set of kitchen knives is the best.