It seems like this is measuring algorithms against the disparate impact standard of all demographic groups needing to have the same aggregate results.
Which is extra weird because the samples to this are applications, not humans, so this is subject to bias in how people apply to these positions. So if a demographic group is more likely to apply to some jobs they are not qualified for, this paper would say they are being discriminated against.
On top of all this, there isn't even really a claim that the algorithms are picking up on anything demographic related. One of the vendors they look at pymetrics, which makes players play abstract games and uses that to pre-screen people.
In the abstract, it makes sense that monocultures are problematic since ML bias alone (in the bias vs variance sense) would just randomly harm folks in a fairly persistent way. But it's also not immediately clear that this even applies to the pymetrics example where I think they have a large assortment of games they make people play for different positions?
It's also not clear that these systems breed monocultures if the inputs into them are firm/position-specific, e.g. job descriptions.
Though honestly I would be far more interested in the validity of these measures at predicting actual on the job measures like performance reviews, etc.
> So if a demographic group is more likely to apply to some jobs they are not qualified for, this paper would say they are being discriminated against.
Your understanding appears to be incorrect.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
This seems to strongly support they argument that effectively a single AI makes a single decision for a candidate across "all" positions they apply for rather than independently assessing them for each position.
Essentially it's more or less saying they're is one hiring manager for the entire industry and if they have a random reason they don't like you, you won't be hired for any job in the industry.
There is a single evaluation function for the industry and if it puts you a negative for any reason in the model's distribution, every job that uses it will do so.
Those are somewhat separate concerns. You could have companies making independent hiring decisions while systematically discriminating against demographic groups, and you could also have companies all use the same system that systematically disadvantages certain individuals, but it's unrelated to their demographics and instead based on things like their resume not being easy to OCR.
In this case, the claim is that both are happening: companies aren't making decisions independently and they're doing so in a way that discriminates against certain demographics. But the evidence needed for each half of the claim is different.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
Thanks for this note, I missed this when skimming it. I would love to see their actual analysis here explained more than a single line, but this doesn't say the original study found no adverse impact at the job type level (they seem to say this wasn't analyzed), but rather that firms seemed to look more independent. Which makes sense for the headline, but is not about their notes on harms, which I still think have all the weaknesses I outlined.
A few years ago the same professor did an analysis of racism in police traffic stops. Compared to this current study on job applications, the conclusions from the police study were more convincing because they actually observed the words spoken by the cops in each interaction.
> if a demographic group is more likely to apply to some jobs they are not qualified for
Can you expand on this? How is a whole demographic group not qualified for jobs on wide spectrum? Is this about certain industries? Certain jobs? Certain groups?
My criticism here is that all you need to get this result is that there are some jobs that some groups apply to more than other groups despite being less qualified.
We know from other literature that men are much more likely to apply for jobs they are unqualified for, there could be other group differences, or there could be a bias of more men from a certain demographic applying for a job type than men from other demographics.
I don't know the answer, but there's a lot of ways for this sort of thing to show up when you look at data with a fine tooth comb and ask why its not perfectly even everywhere.
I'm sure we could have a whole separate argument about the disparate impact standard's validity for society (it is a matter of law ofc), but even if you accept that standard, I think you should be skeptical of the harms noted in this paper.
> How is a whole demographic group not qualified for jobs on wide spectrum?
If a demographic groups has, for example, a culture that encourages overconfidence in relation to actual qualifications, then it is reasonable to expect that that demographic group will be relatively over-represented in the application pool relative to its aggregate qualifications.
Similarly, a culture that instills under-confidence relative to actual qualifications can be reasonably expected to be under-represented relative to its aggregate qualification.
The usual flow is that I have a great HR interview, then I'm assigned an online intelligence (what dots should be in the next box) test and a personality test, and then the company wants nothing to do with me.
They manage to screen me out before I have the opportunity to talk about anything computing related.
(The old horror-stories of 'I couldn't reverse a BST on a whiteboard so I didn't get the job' seem wonderful in comparison now. The non-computing people have captured the hiring pipeline into computing companies)
There are dozens - dozens! - of us outside the US.
I drew the opposite conclusion from your link: (Title VII of the Civil Rights Act prohibits employment tests that are not a 'reasonable measure of job performance'). All an employer would need to say is "We've found that people who can't dots-in-box are bad at cody"
Coinbase wanted me to do one before I did the second round of interviews and I'm in the US. Intelligence and personality tests. Wound up telling them that I didn't want to enable the type of discrimination they facilitate and that with the best faith reasoning for using them is at best a sign the company is indexing on the wrong things for hiring.
Anecdotally, I've only seen them done in northern European companies, but every northern European company I've interviewed for had them. It seems to be a regional-ish thing.
Not if they are dressed up in a particular way, and not if intelligence is genuinely relevant. I have done a few of these tests for jobs in the US before. They are just bad.
The 2000 census gave enumerator applicants a small multiple choice test that was similar to an IQ and then they hired foremen and line workers working down from the highest scores. I've never worked with as many smart people at a temp job!
NAL, but have worked in this. Griggs is a bit more complicated than that, and its progeny modify application anyway.
The TLDR is that arbitrary tests are permissible if there's no disparate impact. Tests with disparate impact are permissible iff they are not arbitrary (i.e., "directly" assess job responsibilities).
So, for example, Leetcode may have disparate impact, but it's "direct" enough to be permissible. On the other hand, most "AI Assessments" are actually so badly implemented that they're effectively random - and a coin flip won't have disparate impact.
IIRC the personality tests are mostly designed to assess how a candidate deals with authority and to try and suss out if they’re consistent/honest. Testing for ‘personality disorders’ more or less.
I doubt the tests actually help, due to the reasons you provided (and more).
A lot of corporate "metrics" are pseudoscientific. They have a superficial veneer of being "scientific", because there's, like, math and numbers and stuff, man. The world continues to function despite this foolishness, not because of it.
Part of why this is so popular in the US is because the US is a hyper-individualistic culture. In other countries, relations are more important. This irks Americans who think this necessarily entails nepotism (it can), but I would say that 1) this overestimates the relative effectiveness and objectivity of low-info hiring practices, and 2) ignores the fact that all knowledge of another person occurs only through relationships. We're inherently social animals and organizations are inherently social phenomena. (2) is partly why many companies pay referral bonuses. They're relying on the knowledge of someone you have of someone you know. This makes sense. If I've worked with someone, I am in a much better position to evaluate their qualifications in a meaningful way than some HR person or some random whoever. A sane company doesn't care about satisfying some weird, arbitrary ideological benchmark. They care about assembling a team that can work effectively.
There's a mindset that freaks out over the mere potential for something to be abused or suboptimal or whatever, and categorically decides that it's better not to have that thing at all. (Gov't is a great example. Yes, gov'ts can become abusive, but they're also the only force that can stand between you and, say, abusive corporate power.)
I passed a series of those and since I remembered the questions from a relatives autism diagnosis testing I asked what they do since they are effectively filtering for things like that.
HR rep said those applicants should probably go see a shrink instead (!!???) and that was the end of me interviewing there.
The testing needs to end. The people using these tools don't know how they work, what they are testing and what blanket denials of personality types really means.
About 20 years ago, I remember getting my hands on an answer key for the personality screener used to work at Target. This was just for a $7/hr cashier position, but it had a very low pass rate. To them, the ideal candidate for them was: always positive and optimistic, preferred being around people than being alone, never complained, frequently sought approval from peers and authorities, always followed every rule no matter what.
So it wasn't explicitly designed against people with disabilities, the rule-following aspect may be more present in autistic people - but for a lot of these, I can't see many people passing if they answered honestly.
"I know the rules of the game and what it will take to continue to be employed in this position."
---
One employer I had gave a test that included such questions as "It is ok to get into fights behind the store if you are not on the clock" and "It is ok to take inventory as long as it costs less than $5."
You don't see how a customer facing position in a retail chain would reasonably want all these personality traits in their hires as a matter of operating a good business?
Wanting those traits and asking about those traits in a self-reporting questionnaire are two different things.
If it’s a questionnaire you are functionally just screening for liars or people who don’t know how to use the full spectrum of a distribution and put in 5/5 or 0/5 for everything.
I am miserably bad at soft-skills interviews and never get past this round. Been over a year since I've had somebody actually try to assess my technical competency in any real capacity.
I'm also getting maybe 1 INITIAL interview every 3 months right now because of this AI screening stuff and I just haven't felt like re-writing my resume to game them.
IMO, soft-skills interviews more a test of your storytelling abilities than anything else. At Google, people often used to joke about candidates who cannot even pass the Googleyness interview, which is supposed to be the easiest of all Google interviews.
One thing I discovered years ago was that even if you are pretty good at soft-skills type stuff and also pretty good at technical stuff what I couldn't do is context switch between an hour or so of doing "soft" stuff to a technical question - even though it was a trivial question. I lost a CTO position over that - mind you I think they went out of business a couple of years later...
If you're trying to cultivate a chill workplace with colleagues you enjoy having coffee with, that's a different objective from building software which works correctly.
Right now I'm trying to watch Book of Boba Fett on Disney Plus. When I cast Disney to my TV and hit play, it shows the animated Disney logo and sound for about a second, pauses/buffers for a couple of seconds, and then skips to the start of the next episode (and so on, until it runs out of episodes). I can temporarily fix it by turning everything off and on again, and starting the episode on my tablet before hitting the cast button.
Lots of people have a hard time just freebasing in an abstract conversation about how they work and storytelling “My Journey” type stuff but work just fine in an actual team setting with concrete products, features, and problems to think and talk about.
This is the stuff with which I struggle the most. I'm an introvert, and "my journey" sounds so insufferable and egotistical to me, I physically cringe at the thought of having to talk about this kind of stuff.
At the end of the day, I just want a paycheck and to work on at least marginally interesting problems. I'm not interested in having to lie about how passionate I am about what company X is doing, nor am I a salesperson that feels comfortable hyping myself up. It feels so fake it becomes a distraction during the interview, which causes me to freeze up and start floundering.
I work hard and I take pride in what I produce, I have plenty of hobbies and get along with others well, and I thrive in environments where I get to mentor and be mentored by others. These are the soft skills that are actually important for working on a team, but they're the most difficult to convey in the traditional interview format.
The guy from the carbon fiber + silver tape titanic sub had super people skills. But if you don’t want to be crushed in a submarine by a 10.000 feet water column, you’ll rather have the clumsy/awkward/jerk guy with superb tech skills leading the project.
I suspect the counfounding factor of hundreds of other applicants makes it hard to tell whether you're specifically being discriminated against or just one of the 999 people who didn't get the job.
(There are some extreme measures that you can try like applying under a different name, although that then forces some awkwardness later on when you actually need your government name for tax and bank information)
> The old horror-stories of 'I couldn't reverse a BST on a whiteboard so I didn't get the job' seem wonderful in comparison now
> They manage to screen me out before I have the opportunity to talk about anything computing related
When I was in college about 10 years ago, I was dreaming a company would interview me on actual algorithms, but sadly I rarely had the occasion to do anything above basic coding.
If you want to see clearly what you can do to get hired, the following perspective helped me a lot. From experience, most hiring processes seem to be shaped less by technical signal and more by the interviewer's defensibility strategy in case of a bad hire. What I mean by that should be clearer from the list below:
- informal interview plus experience matching, hires based on how similar candidate prior jobs seem to be for current role <- if candidate is bad, the interviewer can justify the decision by pointing to the candidate's background.
- informal interview and vibe check with the team or personality test check if candidate is compliant if senior or charismatic if junior <- if the hire is bad, responsibility is diffused across the group.
- take-home project with a nominal 1-hour time limit, but an implicit expectation that candidates spend days on it. Since the interviewer cannot verify how long anyone spent, they default to rewarding the most polished submission.
- take-home project with narrow stated requirements, followed by judgment against unstated "best practices" the company follows <- if the hire is bad, the interviewer can point to the candidate's code and show it matched already what the company looked for, since the style is recognisable.
- CV farm, the company is collecting CVs and has no serious intent to hire <- interviewer doesn't exist
- if the interviewer has no skin in the game (is not verified, performance doesn't matter, they're a consultant leaving next month anyway), anything could happen. This is the most dangerous kind of interview because almost anything can happen and it gives you the least actionable data.
- formal interview pipeline, usually found at large corporations or in finance; interviewer has a clearly scoped job and are expected to evaluate one part of the candidate against a rubric, not make a general judgment about overall hireability. Biases will still exist, but they are more constrained because the process uses multiple interviewers, trained evaluators, explicit scoring grids <- if the hire is bad, the decision is defensible because the interviewer followed the assigned process.
So, interview pipelines can be predictable. It is that you should identify what kind of process you are in as early as possible. If it is experience matching, make your background look obviously adjacent to the role. If it is a take-home, assume polish will count more than the stated time limit. If it is a vibe screen, technical skill may not be the primary variable. If it is a formal pipeline, prepare for the rubric. And if it is a CV farm or a low-accountability interview, do not over-update on the rejection.
In your specific case, I wouldn't overindex on on the intelligence or personality assignment. More probable the CV already got deproritised, but they also sent you the test automatically. The rejection may tell you less about your ability than about the kind of pipeline you were in.
> hires based on how similar candidate prior jobs seem to be for current role <- if candidate is bad, the interviewer can justify the decision by pointing to the candidate's background.
I have found that people are often not very good at doing new things, so it is much easier to find someone to do the same job they've already done than to ask people to do even a slightly different job.
Some people are adaptable, but the vast majority are not.
They are smart enough not to say. The usual pipeline is HR then both Alva tests at once (which seems backward to me - why not use the screen as the actual screen, instead of wasting an hour of HR time). All I know is I'm in their pipeline after the HR but not after the Alva tests.
Let me rephrase that. In your own view, with which component of the test do you think you're having trouble getting through? How many times has this happened, and if it's more than twice have you considered some of the various test practice resources?
This sort of thing needs to be illegal. We saw a similar thing wtih RealPage. So many corporate landlords use it that it essentially becomes anticompetitive price-fixing.
I've heard a claim that an issue with these ATS AI Systems is that your CV gets scored and that score is cached for some period from 3 to 12 months. So any application with a completely different company with your name will just yield the exact same score. If true, it means that if you score badly for whatever reason, you're going to get auto-rejected by every company that uses that system before ever being seen by a human.
This seems to fit anecdotal data where people have applied for hundresd of jobs and never gotten anything other than an automated rejection. But obviously that's not proof or confirmation. But if it is, it's almost like being a voncicted felon. It greatly limits your ability to find a job and that's a huge problem.
I don't know what the solution is but I hope these companies get sued for states for issueslike this where actual discrimination occurs.
Recruitment and hiring has been a mess for decades, though, especially in tech.
It's like all the leetcode bullshit. We know that is not a valid measurement for actual performance in a dev job, but that doesn't stop managers from using it.
What they need is a number, a rating, on how much of a fit each candidate is, through some process that can be described as objective and fair. The algorithms provide that.
If we make this illegal, they'll just come up with some other bullshit.
In an ideal world, companies would assess each and every candidate individually and on their merits. But no-one has time or patience for that, so we have these bullshit systems.
From what I seen, the leetcode is self reproducing dysfunction. There is whole subculture that believes being good at leetcode like tests and puzzles makes you inherently superior in all aspects, because you are smart. After all, they were good at leetcode like tests and are superior and smart. It is people who grew up in that subculture becoming managers recreating the culture they grew up in.
That is why the "not a valid measurement for actual performance in a dev job" thing does not matter. Too many people are emotionally invested in this being important measure. Their and their friends self worth is attached to it.
Is it... illegal... to put white characters on your name field?
Just thinking out loud here, but what if we fed the AI (and ONLY the AI) a different name for every role. By adding two invisible characters on the space between your name? Would it then make a new index?
Could join those in the slow life, invest in passive revenue streams, and keep cash burn rates minimal. You will be fine avoiding debt-based indentured 1980 metropolitan cultures.
Learn to play a cheap instrument, garden vegetables, paint miniatures, volunteer at pet shelters, or travel to odd destinations. Play the long game, and remember to have fun.
You owe corporate nothing outside what they paid for... and not a cent more. =3
Passive revenue streams are dependent on someone else not having access to those passive revenue streams.
Yes, it would be great to be free of debt, but for me it would have to mean moving away to somewhere real estate prices are not only low, but dropping for all too understandable reasons. And also a huge distance away from friends and family. There's a reason people mostly don't do this, and it's not that they feel a moral obligation to corporate.
>Passive revenue streams are dependent on someone else not having access to those passive revenue streams.
I think many assume it is some sort of zero-sum-game. This simply isn't how most unique successful product and service based niche businesses operate.
Most firms that directly try to hyper-scale their way into market dominance simply fail within a few years. The smarter bros often cash-out after the IPO these days.
Some people do feel entitled to others free time, and post-unknown-risk capital investments. Those folks can't help anyone succeed at anything except bankruptcy. =3
Highly dependent on ones specific financial situation, risk-tolerance, and employment trajectory.
While not a personal preference, most acquire revenue properties as they build equity over time. Assets secure lower loan rates, qualify mortgage fixed-payment schedules on investments, and require good management-companies to handle leases.
Generally, mitigating tax exposures by investing in small businesses is still popular. Sometimes they work, and sometimes they don't... but it is money people will lose anyway if they do nothing. Specifically, my first business investment was a few vending machine locations as a teenager, after a summer dropping hardwood floors.
Everyone starts somewhere, but blindly cloning what others do is usually unwise. ymmv =3
Some spend their time investing from an early age... classifying assets and liabilities. My point was a few grand invested as a kid seems small, but will turn into a lot several decades later.
Others spend their lives making decisions out of impulsive narcissism. Unless you are a trust-fund kid, life can have very real consequences if things go sideways.
Most will learn the hard way... only lawyers and politicians get paid for excuses in life. =3
I thought this was going to be about how the whole software industry has been cargo-culting FAANG coding interviews that are heavily reliant on algorithms trivia...
This clearly falls under the definition of automated individual decision-making per Article 22 of the GDPR and would be blatantly illegal if it were done in the EU. The GDPR is explicitly designed to outlaw this kind of algorithmic profiling and exclusion in hiring.
"We only attempt to identify adverse impact based on self-reported racial information and do not impute data, noting that we do not have access to features like applicant name. 62.35% of all applicants in the data do not self-report race as belonging to one of the four racial groups we study."
Step one: Decision makers who can change these processes need to be aware of this problem. Many companies fail this simple task.
Step two: These decision makers must be held accountable for the success of the process. Many companies fail this simple task.
Step three: These decision makers must be willing to admit that they made a mistake, and risk loss of prestige and political capital. Guess how likely that is.
And the bigger the company, the worse it gets. It's a good thing we didn't go through 20 years of consolidations and mergers. Oh wait.
There are two problems here - the volume of application spam ( which creates the need for automated filtering in the first place, and is now probably AI aided ), and the fact that there isn't a single decision maker.
You have HR which decides to outsource filtering, and then the outsourced company who decides how it's done.
The line managers actually trying to recruit are no where near this decision - indeed they don't share a common manager till you get to the CEO - who is too busy to care about this sort of stuff.
In my experience the only way to fix this is to tell HR that you want the unfiltered CV list and do it yourself. The problem with that is if you work at a large well known company you'll get 100's if not 1000's of applicants for any job you advertise and most applicants don't appear to have even read the job description. So you are committing to a very large amount of work.
Lots of monocultures exist in hiring even without an algorithmic scoring system. That's roughly how every stupid hiring fad works, and how it's always worked, because most employers have no idea how to identify great potential employees.
Hiring managers and companies choose algorithms and hiring fads because they don't know how to be really certain of who to hire, so they'll settle for either assuming someone else's expertise will save them, or for some rubric that "everyone is doing" so it "can't be that bad".
When I first became a hiring manager, I was working for a public university. Our salaries were limited, being staff rather than faculty and being public servants, to between 1/3 and 1/2 the going salary for equivalent cybersecurity professionals in the private sector. I did not have the option to hire the people everyone else was trying to hire. I also faced one of the key risks of working in a public institution: once you keep someone past their probationary period, it is very, very hard to fire them. So, it's important not to get it wrong. I learned some things that I have carried forward into every hiring manager or senior leadership role since:
1. I base hiring practices on Manager Tools behavioral interviewing systems (https://manager-tools.com). No affiliation, they've just made my work life better.
2. I became really good at understanding what my team or organization really needs. Most hirers focus way too much on "years of experience" and specific technologies than is usually wise. As my favorite former supervisor said, "I can teach a smart person cybersecurity, but I can't teach a dumb [or unmotivated] cybersecurity person to be smart."
3. I became really good at developing people, and ensuring that the managers under me were as well. We couldn't lay someone off just because their technical specialty became irrelevant, so we couldn't afford to hire people who weren't lifelong learners, or to fail as coaches to ensure that learning was always taking place.
4. I cast as wide a net as my HR and regulatory overlords would let me (and now, as a business leader, I cast a huge net). I look for things that aren't just useful at the moment, but are useful long term, in my candidates. I don't care about pedigree.
I end up paying less for good employees due to simple supply and demand: I often go for the diamonds in the rough that don't have 10 competing offers.
I end up having really good employees who generally stay with me long term, because I apply long-term thinking in hiring, and structure my teams around constant learning and development.
I dodge a LOT of bullets... people who have just the right pedigree to look like great hires worth a lot of money, but who'll disappoint me until the day they leave.
When it's a tight labor market -- too few candidates for roles I care about -- I'm tapping a hiring market that other managers aren't aware enough of, and still finding talent while they have roles that sit open for months.
It seems like this is measuring algorithms against the disparate impact standard of all demographic groups needing to have the same aggregate results.
Which is extra weird because the samples to this are applications, not humans, so this is subject to bias in how people apply to these positions. So if a demographic group is more likely to apply to some jobs they are not qualified for, this paper would say they are being discriminated against.
On top of all this, there isn't even really a claim that the algorithms are picking up on anything demographic related. One of the vendors they look at pymetrics, which makes players play abstract games and uses that to pre-screen people.
In the abstract, it makes sense that monocultures are problematic since ML bias alone (in the bias vs variance sense) would just randomly harm folks in a fairly persistent way. But it's also not immediately clear that this even applies to the pymetrics example where I think they have a large assortment of games they make people play for different positions?
It's also not clear that these systems breed monocultures if the inputs into them are firm/position-specific, e.g. job descriptions.
Though honestly I would be far more interested in the validity of these measures at predicting actual on the job measures like performance reviews, etc.
> So if a demographic group is more likely to apply to some jobs they are not qualified for, this paper would say they are being discriminated against.
Your understanding appears to be incorrect.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
This seems to strongly support they argument that effectively a single AI makes a single decision for a candidate across "all" positions they apply for rather than independently assessing them for each position.
Essentially it's more or less saying they're is one hiring manager for the entire industry and if they have a random reason they don't like you, you won't be hired for any job in the industry.
There is a single evaluation function for the industry and if it puts you a negative for any reason in the model's distribution, every job that uses it will do so.
Those are somewhat separate concerns. You could have companies making independent hiring decisions while systematically discriminating against demographic groups, and you could also have companies all use the same system that systematically disadvantages certain individuals, but it's unrelated to their demographics and instead based on things like their resume not being easy to OCR.
In this case, the claim is that both are happening: companies aren't making decisions independently and they're doing so in a way that discriminates against certain demographics. But the evidence needed for each half of the claim is different.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
Thanks for this note, I missed this when skimming it. I would love to see their actual analysis here explained more than a single line, but this doesn't say the original study found no adverse impact at the job type level (they seem to say this wasn't analyzed), but rather that firms seemed to look more independent. Which makes sense for the headline, but is not about their notes on harms, which I still think have all the weaknesses I outlined.
> There is a single evaluation function for the industry
Could this be an opportunity in disguise? Somehow learn what this function wants, maximize it, then the entire industry opens up?
Apparently the key 'hack' is to present white or hispanic.
No idea why reiterating the exact point of the article got downvoted.
A few years ago the same professor did an analysis of racism in police traffic stops. Compared to this current study on job applications, the conclusions from the police study were more convincing because they actually observed the words spoken by the cops in each interaction.
https://www.pnas.org/doi/10.1073/pnas.1702413114
> if a demographic group is more likely to apply to some jobs they are not qualified for
Can you expand on this? How is a whole demographic group not qualified for jobs on wide spectrum? Is this about certain industries? Certain jobs? Certain groups?
This is not about being entirely unqualified.
My criticism here is that all you need to get this result is that there are some jobs that some groups apply to more than other groups despite being less qualified.
We know from other literature that men are much more likely to apply for jobs they are unqualified for, there could be other group differences, or there could be a bias of more men from a certain demographic applying for a job type than men from other demographics.
I don't know the answer, but there's a lot of ways for this sort of thing to show up when you look at data with a fine tooth comb and ask why its not perfectly even everywhere.
I'm sure we could have a whole separate argument about the disparate impact standard's validity for society (it is a matter of law ofc), but even if you accept that standard, I think you should be skeptical of the harms noted in this paper.
> How is a whole demographic group not qualified for jobs on wide spectrum?
If a demographic groups has, for example, a culture that encourages overconfidence in relation to actual qualifications, then it is reasonable to expect that that demographic group will be relatively over-represented in the application pool relative to its aggregate qualifications.
Similarly, a culture that instills under-confidence relative to actual qualifications can be reasonably expected to be under-represented relative to its aggregate qualification.
This is keeping me out of work at the moment.
The usual flow is that I have a great HR interview, then I'm assigned an online intelligence (what dots should be in the next box) test and a personality test, and then the company wants nothing to do with me.
They manage to screen me out before I have the opportunity to talk about anything computing related.
(The old horror-stories of 'I couldn't reverse a BST on a whiteboard so I didn't get the job' seem wonderful in comparison now. The non-computing people have captured the hiring pipeline into computing companies)
I don't have a dog in the race, also I'm not based in the US, but aren't intelligence tests for hiring illegal in the US?
https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.
There are dozens - dozens! - of us outside the US.
I drew the opposite conclusion from your link: (Title VII of the Civil Rights Act prohibits employment tests that are not a 'reasonable measure of job performance'). All an employer would need to say is "We've found that people who can't dots-in-box are bad at cody"
I just dug up the link (https://www.alvalabs.io/hiring-system/assessments/logic-test) to take another look, and sure enough, there's giant text saying "A strong predictor of job performance." Consider HR's arses covered!
They have the nerve to label it is a "logic" test. I bet I'd be the only one on their staff able to write out simple natural deduction proofs.
Coinbase wanted me to do one before I did the second round of interviews and I'm in the US. Intelligence and personality tests. Wound up telling them that I didn't want to enable the type of discrimination they facilitate and that with the best faith reasoning for using them is at best a sign the company is indexing on the wrong things for hiring.
That told me they wanted me to do this and I told them to pound sand. Doesn't help that their online reviews indicate its a sweatshop.
Anyway, they did layoffs like 2 weeks later so I guess I dodged a bullet there.
Anecdotally, I've only seen them done in northern European companies, but every northern European company I've interviewed for had them. It seems to be a regional-ish thing.
Not if they are dressed up in a particular way, and not if intelligence is genuinely relevant. I have done a few of these tests for jobs in the US before. They are just bad.
The 2000 census gave enumerator applicants a small multiple choice test that was similar to an IQ and then they hired foremen and line workers working down from the highest scores. I've never worked with as many smart people at a temp job!
[dead]
apparently not
you can wait for a reply from tptacek later in the day, or use the search at the bottom to find previous replies
here: https://hn.algolia.com/?query=tptacek%20iq%20test&sort=byDat...
NAL, but have worked in this. Griggs is a bit more complicated than that, and its progeny modify application anyway.
The TLDR is that arbitrary tests are permissible if there's no disparate impact. Tests with disparate impact are permissible iff they are not arbitrary (i.e., "directly" assess job responsibilities).
So, for example, Leetcode may have disparate impact, but it's "direct" enough to be permissible. On the other hand, most "AI Assessments" are actually so badly implemented that they're effectively random - and a coin flip won't have disparate impact.
It's quite scary. I think a lot of these HR types are true believers. They think that
1) these tests are valid and objective
2) they are qualified to understand how personalities would interact in a complex system
It's astrology for professionals, and companies have let themselves be captured by what are effectively religious zealots.
IIRC the personality tests are mostly designed to assess how a candidate deals with authority and to try and suss out if they’re consistent/honest. Testing for ‘personality disorders’ more or less.
I doubt the tests actually help, due to the reasons you provided (and more).
> It's astrology for professionals
A lot of corporate "metrics" are pseudoscientific. They have a superficial veneer of being "scientific", because there's, like, math and numbers and stuff, man. The world continues to function despite this foolishness, not because of it.
Part of why this is so popular in the US is because the US is a hyper-individualistic culture. In other countries, relations are more important. This irks Americans who think this necessarily entails nepotism (it can), but I would say that 1) this overestimates the relative effectiveness and objectivity of low-info hiring practices, and 2) ignores the fact that all knowledge of another person occurs only through relationships. We're inherently social animals and organizations are inherently social phenomena. (2) is partly why many companies pay referral bonuses. They're relying on the knowledge of someone you have of someone you know. This makes sense. If I've worked with someone, I am in a much better position to evaluate their qualifications in a meaningful way than some HR person or some random whoever. A sane company doesn't care about satisfying some weird, arbitrary ideological benchmark. They care about assembling a team that can work effectively.
There's a mindset that freaks out over the mere potential for something to be abused or suboptimal or whatever, and categorically decides that it's better not to have that thing at all. (Gov't is a great example. Yes, gov'ts can become abusive, but they're also the only force that can stand between you and, say, abusive corporate power.)
Abusus non tollit usum.
I passed a series of those and since I remembered the questions from a relatives autism diagnosis testing I asked what they do since they are effectively filtering for things like that.
HR rep said those applicants should probably go see a shrink instead (!!???) and that was the end of me interviewing there.
The testing needs to end. The people using these tools don't know how they work, what they are testing and what blanket denials of personality types really means.
Wait, they’re filtering out autistic traits or looking for them in candidates?
Filtering out is my guess.
About 20 years ago, I remember getting my hands on an answer key for the personality screener used to work at Target. This was just for a $7/hr cashier position, but it had a very low pass rate. To them, the ideal candidate for them was: always positive and optimistic, preferred being around people than being alone, never complained, frequently sought approval from peers and authorities, always followed every rule no matter what.
So it wasn't explicitly designed against people with disabilities, the rule-following aspect may be more present in autistic people - but for a lot of these, I can't see many people passing if they answered honestly.
> I can't see many people passing if they answered honestly.
You're not supposed to answer honestly, you're supposed to answer in such a way as to convince them to hire you.
"I know the rules of the game and what it will take to continue to be employed in this position."
---
One employer I had gave a test that included such questions as "It is ok to get into fights behind the store if you are not on the clock" and "It is ok to take inventory as long as it costs less than $5."
There are people who failed that test.
You don't see how a customer facing position in a retail chain would reasonably want all these personality traits in their hires as a matter of operating a good business?
Wanting those traits and asking about those traits in a self-reporting questionnaire are two different things.
If it’s a questionnaire you are functionally just screening for liars or people who don’t know how to use the full spectrum of a distribution and put in 5/5 or 0/5 for everything.
I am miserably bad at soft-skills interviews and never get past this round. Been over a year since I've had somebody actually try to assess my technical competency in any real capacity.
I'm also getting maybe 1 INITIAL interview every 3 months right now because of this AI screening stuff and I just haven't felt like re-writing my resume to game them.
IMO, soft-skills interviews more a test of your storytelling abilities than anything else. At Google, people often used to joke about candidates who cannot even pass the Googleyness interview, which is supposed to be the easiest of all Google interviews.
> miserably bad at soft-skills interviews
Is that because of an actual lack of soft skills or is it because the interviews are bad?
> I just haven't felt like re-writing my resume to game them.
Not defending the AI interview assistance BS, but if you wanted a job bad enough then you'd eventually do this, not the latest after several months?
One thing I discovered years ago was that even if you are pretty good at soft-skills type stuff and also pretty good at technical stuff what I couldn't do is context switch between an hour or so of doing "soft" stuff to a technical question - even though it was a trivial question. I lost a CTO position over that - mind you I think they went out of business a couple of years later...
Aren’t soft skills much more important than hard skills when it comes to building a team?
It depends on the objective.
If you're trying to cultivate a chill workplace with colleagues you enjoy having coffee with, that's a different objective from building software which works correctly.
Right now I'm trying to watch Book of Boba Fett on Disney Plus. When I cast Disney to my TV and hit play, it shows the animated Disney logo and sound for about a second, pauses/buffers for a couple of seconds, and then skips to the start of the next episode (and so on, until it runs out of episodes). I can temporarily fix it by turning everything off and on again, and starting the episode on my tablet before hitting the cast button.
Maybe they have a really strong team, I dunno.
Lots of people have a hard time just freebasing in an abstract conversation about how they work and storytelling “My Journey” type stuff but work just fine in an actual team setting with concrete products, features, and problems to think and talk about.
> storytelling “My Journey” type stuff
This is the stuff with which I struggle the most. I'm an introvert, and "my journey" sounds so insufferable and egotistical to me, I physically cringe at the thought of having to talk about this kind of stuff.
At the end of the day, I just want a paycheck and to work on at least marginally interesting problems. I'm not interested in having to lie about how passionate I am about what company X is doing, nor am I a salesperson that feels comfortable hyping myself up. It feels so fake it becomes a distraction during the interview, which causes me to freeze up and start floundering.
I work hard and I take pride in what I produce, I have plenty of hobbies and get along with others well, and I thrive in environments where I get to mentor and be mentored by others. These are the soft skills that are actually important for working on a team, but they're the most difficult to convey in the traditional interview format.
We need to be careful about those absolutes.
The guy from the carbon fiber + silver tape titanic sub had super people skills. But if you don’t want to be crushed in a submarine by a 10.000 feet water column, you’ll rather have the clumsy/awkward/jerk guy with superb tech skills leading the project.
You might be surprised to hear that there are great engineers who are also good at people skills.
In no place I said that there weren't. I don't understand where you got this idea.
Are the soft skill important for team work the same they test on interviews?
Based on an experience of never seen the relevant skills tested, and never been able to test for them as an interviewer, I really, really doubt that.
The easier the task, the more likely it is for soft skills to matter.
It could be that those HR teams are engaging in some busy work - pretending to be looking for candidates so they/their company looks busier.
I suspect the counfounding factor of hundreds of other applicants makes it hard to tell whether you're specifically being discriminated against or just one of the 999 people who didn't get the job.
(There are some extreme measures that you can try like applying under a different name, although that then forces some awkwardness later on when you actually need your government name for tax and bank information)
> The old horror-stories of 'I couldn't reverse a BST on a whiteboard so I didn't get the job' seem wonderful in comparison now
> They manage to screen me out before I have the opportunity to talk about anything computing related
When I was in college about 10 years ago, I was dreaming a company would interview me on actual algorithms, but sadly I rarely had the occasion to do anything above basic coding.
If you want to see clearly what you can do to get hired, the following perspective helped me a lot. From experience, most hiring processes seem to be shaped less by technical signal and more by the interviewer's defensibility strategy in case of a bad hire. What I mean by that should be clearer from the list below:
- informal interview plus experience matching, hires based on how similar candidate prior jobs seem to be for current role <- if candidate is bad, the interviewer can justify the decision by pointing to the candidate's background.
- informal interview and vibe check with the team or personality test check if candidate is compliant if senior or charismatic if junior <- if the hire is bad, responsibility is diffused across the group.
- take-home project with a nominal 1-hour time limit, but an implicit expectation that candidates spend days on it. Since the interviewer cannot verify how long anyone spent, they default to rewarding the most polished submission.
- take-home project with narrow stated requirements, followed by judgment against unstated "best practices" the company follows <- if the hire is bad, the interviewer can point to the candidate's code and show it matched already what the company looked for, since the style is recognisable.
- CV farm, the company is collecting CVs and has no serious intent to hire <- interviewer doesn't exist
- if the interviewer has no skin in the game (is not verified, performance doesn't matter, they're a consultant leaving next month anyway), anything could happen. This is the most dangerous kind of interview because almost anything can happen and it gives you the least actionable data.
- formal interview pipeline, usually found at large corporations or in finance; interviewer has a clearly scoped job and are expected to evaluate one part of the candidate against a rubric, not make a general judgment about overall hireability. Biases will still exist, but they are more constrained because the process uses multiple interviewers, trained evaluators, explicit scoring grids <- if the hire is bad, the decision is defensible because the interviewer followed the assigned process.
So, interview pipelines can be predictable. It is that you should identify what kind of process you are in as early as possible. If it is experience matching, make your background look obviously adjacent to the role. If it is a take-home, assume polish will count more than the stated time limit. If it is a vibe screen, technical skill may not be the primary variable. If it is a formal pipeline, prepare for the rubric. And if it is a CV farm or a low-accountability interview, do not over-update on the rejection.
In your specific case, I wouldn't overindex on on the intelligence or personality assignment. More probable the CV already got deproritised, but they also sent you the test automatically. The rejection may tell you less about your ability than about the kind of pipeline you were in.
> hires based on how similar candidate prior jobs seem to be for current role <- if candidate is bad, the interviewer can justify the decision by pointing to the candidate's background.
I have found that people are often not very good at doing new things, so it is much easier to find someone to do the same job they've already done than to ask people to do even a slightly different job.
Some people are adaptable, but the vast majority are not.
Are you failing because of the dot test or because of the personality test?
They are smart enough not to say. The usual pipeline is HR then both Alva tests at once (which seems backward to me - why not use the screen as the actual screen, instead of wasting an hour of HR time). All I know is I'm in their pipeline after the HR but not after the Alva tests.
Let me rephrase that. In your own view, with which component of the test do you think you're having trouble getting through? How many times has this happened, and if it's more than twice have you considered some of the various test practice resources?
Time and motion study neurodivergent slave class optimisation.
Have a computer take your personality test is dystopian
This sort of thing needs to be illegal. We saw a similar thing wtih RealPage. So many corporate landlords use it that it essentially becomes anticompetitive price-fixing.
I've heard a claim that an issue with these ATS AI Systems is that your CV gets scored and that score is cached for some period from 3 to 12 months. So any application with a completely different company with your name will just yield the exact same score. If true, it means that if you score badly for whatever reason, you're going to get auto-rejected by every company that uses that system before ever being seen by a human.
This seems to fit anecdotal data where people have applied for hundresd of jobs and never gotten anything other than an automated rejection. But obviously that's not proof or confirmation. But if it is, it's almost like being a voncicted felon. It greatly limits your ability to find a job and that's a huge problem.
I don't know what the solution is but I hope these companies get sued for states for issueslike this where actual discrimination occurs.
Recruitment and hiring has been a mess for decades, though, especially in tech.
It's like all the leetcode bullshit. We know that is not a valid measurement for actual performance in a dev job, but that doesn't stop managers from using it.
What they need is a number, a rating, on how much of a fit each candidate is, through some process that can be described as objective and fair. The algorithms provide that.
If we make this illegal, they'll just come up with some other bullshit.
In an ideal world, companies would assess each and every candidate individually and on their merits. But no-one has time or patience for that, so we have these bullshit systems.
From what I seen, the leetcode is self reproducing dysfunction. There is whole subculture that believes being good at leetcode like tests and puzzles makes you inherently superior in all aspects, because you are smart. After all, they were good at leetcode like tests and are superior and smart. It is people who grew up in that subculture becoming managers recreating the culture they grew up in.
That is why the "not a valid measurement for actual performance in a dev job" thing does not matter. Too many people are emotionally invested in this being important measure. Their and their friends self worth is attached to it.
Is it... illegal... to put white characters on your name field? Just thinking out loud here, but what if we fed the AI (and ONLY the AI) a different name for every role. By adding two invisible characters on the space between your name? Would it then make a new index?
Consider characters that look the same but have different unicode?
This is just one of many reasons why my current job is likely to be my last. I feel like so much of modern life is just irredeemably broken right now.
Could join those in the slow life, invest in passive revenue streams, and keep cash burn rates minimal. You will be fine avoiding debt-based indentured 1980 metropolitan cultures.
Learn to play a cheap instrument, garden vegetables, paint miniatures, volunteer at pet shelters, or travel to odd destinations. Play the long game, and remember to have fun.
You owe corporate nothing outside what they paid for... and not a cent more. =3
https://www.youtube.com/watch?v=bjhKTqdxRdo
Passive revenue streams are dependent on someone else not having access to those passive revenue streams.
Yes, it would be great to be free of debt, but for me it would have to mean moving away to somewhere real estate prices are not only low, but dropping for all too understandable reasons. And also a huge distance away from friends and family. There's a reason people mostly don't do this, and it's not that they feel a moral obligation to corporate.
>Passive revenue streams are dependent on someone else not having access to those passive revenue streams.
I think many assume it is some sort of zero-sum-game. This simply isn't how most unique successful product and service based niche businesses operate.
Most firms that directly try to hyper-scale their way into market dominance simply fail within a few years. The smarter bros often cash-out after the IPO these days.
Some people do feel entitled to others free time, and post-unknown-risk capital investments. Those folks can't help anyone succeed at anything except bankruptcy. =3
> invest in passive revenue streams
Sounds great. So how exactly do you get started with that, then?
Highly dependent on ones specific financial situation, risk-tolerance, and employment trajectory.
While not a personal preference, most acquire revenue properties as they build equity over time. Assets secure lower loan rates, qualify mortgage fixed-payment schedules on investments, and require good management-companies to handle leases.
Generally, mitigating tax exposures by investing in small businesses is still popular. Sometimes they work, and sometimes they don't... but it is money people will lose anyway if they do nothing. Specifically, my first business investment was a few vending machine locations as a teenager, after a summer dropping hardwood floors.
Everyone starts somewhere, but blindly cloning what others do is usually unwise. ymmv =3
So basically, in order to have a steady stream of money coming in, you need to start with having a steady stream of money coming in? :-D
And often your investments will lose you money, so try to avoid the unprofitable ones, especially when starting out.
Indeed, most kids first big investment is in their schooling.
Sometimes folks have help, sometimes its all debt, and sometimes it pays off eventually.
However, living beyond ones means is almost always unwise. =3
Some spend their time investing from an early age... classifying assets and liabilities. My point was a few grand invested as a kid seems small, but will turn into a lot several decades later.
Others spend their lives making decisions out of impulsive narcissism. Unless you are a trust-fund kid, life can have very real consequences if things go sideways.
Most will learn the hard way... only lawyers and politicians get paid for excuses in life. =3
I want my net worth to be dependent on my skills, not the pile of money I happen to have laying around
Differentiating assets and liabilities is a skill-set like any other. =3
I've found that if I apply for any company that uses Workday, it's an immediate rejection, so I don't bother with them anymore.
I would not be surprised if their system breaks somewhere between us clicking submit and some storage system. Their HR stuff is terrible.
I thought this was going to be about how the whole software industry has been cargo-culting FAANG coding interviews that are heavily reliant on algorithms trivia...
If I had to make an algorithm that would correct these injusticies, I would end up just hiring that algorithm as it's way easier than hiring a human.
This clearly falls under the definition of automated individual decision-making per Article 22 of the GDPR and would be blatantly illegal if it were done in the EU. The GDPR is explicitly designed to outlaw this kind of algorithmic profiling and exclusion in hiring.
https://gdpr-info.eu/art-22-gdpr/
https://www.bloomberglaw.com/external/document/X4BBTPFO00000...
I wonder how they determine an applicant's ethnicity. Is it by the name?
From a glance at the paper (https://arxiv.org/pdf/2605.27371) it looks like they're using self-reported ethnicity.
"We only attempt to identify adverse impact based on self-reported racial information and do not impute data, noting that we do not have access to features like applicant name. 62.35% of all applicants in the data do not self-report race as belonging to one of the four racial groups we study."
Waaaait why is it not in the incentives of companies hiring to automatically fix this? They instantly get better candidates for cheaper wages.
Step one: Decision makers who can change these processes need to be aware of this problem. Many companies fail this simple task.
Step two: These decision makers must be held accountable for the success of the process. Many companies fail this simple task.
Step three: These decision makers must be willing to admit that they made a mistake, and risk loss of prestige and political capital. Guess how likely that is.
And the bigger the company, the worse it gets. It's a good thing we didn't go through 20 years of consolidations and mergers. Oh wait.
There are two problems here - the volume of application spam ( which creates the need for automated filtering in the first place, and is now probably AI aided ), and the fact that there isn't a single decision maker.
You have HR which decides to outsource filtering, and then the outsourced company who decides how it's done.
The line managers actually trying to recruit are no where near this decision - indeed they don't share a common manager till you get to the CEO - who is too busy to care about this sort of stuff.
In my experience the only way to fix this is to tell HR that you want the unfiltered CV list and do it yourself. The problem with that is if you work at a large well known company you'll get 100's if not 1000's of applicants for any job you advertise and most applicants don't appear to have even read the job description. So you are committing to a very large amount of work.
True I agree and I think there must be some head of recruiting who is in the details who should be kinda held responsible for this.
its an information and principal agent problem.
We don't know how to measure worker productivity -> its hard to even say what a good hire is.
We don't have good standardisation around whatever measurements we do take -> hard to say anything about hiring at all.
People are more interested in their own prestige than hiring the best option for the company -> too many candidates get hired in the wrong places.
many of these problems do seem solvable given risk taking and statistics. However, culturally hiring managers aren't inclined to do either.
Yes, but they also hiring is so random already they don't care about it being like 3% better or whatever.
Lots of monocultures exist in hiring even without an algorithmic scoring system. That's roughly how every stupid hiring fad works, and how it's always worked, because most employers have no idea how to identify great potential employees.
Hiring managers and companies choose algorithms and hiring fads because they don't know how to be really certain of who to hire, so they'll settle for either assuming someone else's expertise will save them, or for some rubric that "everyone is doing" so it "can't be that bad".
When I first became a hiring manager, I was working for a public university. Our salaries were limited, being staff rather than faculty and being public servants, to between 1/3 and 1/2 the going salary for equivalent cybersecurity professionals in the private sector. I did not have the option to hire the people everyone else was trying to hire. I also faced one of the key risks of working in a public institution: once you keep someone past their probationary period, it is very, very hard to fire them. So, it's important not to get it wrong. I learned some things that I have carried forward into every hiring manager or senior leadership role since:
1. I base hiring practices on Manager Tools behavioral interviewing systems (https://manager-tools.com). No affiliation, they've just made my work life better.
2. I became really good at understanding what my team or organization really needs. Most hirers focus way too much on "years of experience" and specific technologies than is usually wise. As my favorite former supervisor said, "I can teach a smart person cybersecurity, but I can't teach a dumb [or unmotivated] cybersecurity person to be smart."
3. I became really good at developing people, and ensuring that the managers under me were as well. We couldn't lay someone off just because their technical specialty became irrelevant, so we couldn't afford to hire people who weren't lifelong learners, or to fail as coaches to ensure that learning was always taking place.
4. I cast as wide a net as my HR and regulatory overlords would let me (and now, as a business leader, I cast a huge net). I look for things that aren't just useful at the moment, but are useful long term, in my candidates. I don't care about pedigree.
I end up paying less for good employees due to simple supply and demand: I often go for the diamonds in the rough that don't have 10 competing offers.
I end up having really good employees who generally stay with me long term, because I apply long-term thinking in hiring, and structure my teams around constant learning and development.
I dodge a LOT of bullets... people who have just the right pedigree to look like great hires worth a lot of money, but who'll disappoint me until the day they leave.
When it's a tight labor market -- too few candidates for roles I care about -- I'm tapping a hiring market that other managers aren't aware enough of, and still finding talent while they have roles that sit open for months.
[dead]
[dead]
[dead]