IQ Tests, Traditional or Structured Interviews
Do you know which method is best to gauge top candidates?
September 16, 2010
The traditional open-ended interview is at best moderately useful. For the past few decades, research on staffing has shown the advantages of systematic testing and staffing methods over interviews alone. Textbooks like Herbert G. Heneman, III and Timothy Judge’s Staffing Organizations (Herbert G. Heneman, III and Timothy Judge, Staffing Organizations Sixth Edition. McGraw Hill Irwin) recite a list of methods that add predictive power to interviews. Yet, many of the alternative methods are under-utilized, including the least expensive among them, cognitive ability or IQ tests. It is true that some job applicants dislike IQ and other kinds of tests. But inclusion of an objective IQ test, an integrity test or a work sample can significantly improve your firm’s productivity and profitability (F.L. Schmidt and J.E. Hunter. “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings.” Psychological Bulletin 124:2, 1998.). Combination of objective tests with interviewing can substantially increase a firm’s profitability. But doing so remains an unknown ideal.
One approach may be to ask objective, structured questions and then weight and score the answers in the same way across all applicants. But research has not necessarily found that such structured interviews outperform the traditional open-ended interview approach. According to Hunter and Schmidt both structured and unstructured interviews have a validity of 0.35 (Ibid). However, other studies find that unstructured interviews have considerably lower mean validities than structured interviews and other methods like bio-data. Thus, there is a controversy. While 0.35 is a good validity for unstructured interviews — even if it is accurate — it can be significantly improved with more objective measures. If, as other studies find, unstructured interviews have 0.25 validity, then it is even more important to supplement them with objective measures. Interviews should be the icing on the cake of a longer list of selection methods that is weighted in accordance with validity.
The validation of a selection method means its correlation with a measure of job performance. In some jobs such as sales this is easily accomplished with a measure such as sales volume. In service jobs like auditing, consulting and law hours billed; years retained with the firm; whether or not an individual was promoted to partner; performance appraisal ratings; and appraisals by clients are potential criteria or performance measures to correlate with structured interview ratings or other selection instrument scores.
One potential problem with performance appraisal instruments, though, is whether they have been validated against other performance measures. There are considerable threats to the validity of performance appraisal methods such as “just like me” bias and the halo effect, that is, whether good performance in one narrow area, say appearance, provides a halo that is over-generalized to all other areas.
HR departments can easily validate selection instruments in two ways. First, they can correlate employees’ scores on the selection instrument with actual job-performance measures. Second, they can use selection instruments that automatically correlate with important aspects of the job. For example, a structured interview that asks job-related knowledge questions would be “content valid.” A quiz on auditing that is applied uniformly to all applicants might have similar validity to an IQ test, especially if the questions are difficult.
The best methods of hiring involve job tryouts and work samples. Job tryouts have become popular in the form of internships. Rather than rely on first impressions in an interview, firms can set up an internship program and have accounting students from nearby colleges intern for say, 10 hours per week, during school and full-time during the summer. Wages could be set at below-market rates and first-time interns can work for free. The interns would be appraised objectively by several managers using a standardized instrument. A selective firm might keep one out of three, five or 10 interns. Many months of observation and objective measurement are preferable to a one-hour interview.
Work samples would involve a consultant’s showing a report or other work product that they have produced. Like job tryouts, work samples alone have higher validity than unstructured interviews.
A more expensive approach that might be useful for sensitive or higher level jobs is an assessment center. Assessment centers make use of role plays, in-box exercises and case-study exercises, some of which are reviewed by line managers and trained psychologists behind one way mirrors. Properly executed, assessment centers can have good validities. The problem with them is that they are expensive because they require a facility in which the assessment takes place and personnel time to evaluate the applicants.
Cognitive ability or IQ tests are especially useful if there is a spread of intellectual ability across the applicant pool, as is the case with accounting and auditing. If a firm has restricted hiring to elite schools, there may be limited spread or, alternatively, all of the applicants’ mental abilities may be above a threshold whereby job performance is influenced by IQ. However, research consistently shows a healthy correlation between job performance and IQ.
There are a number of additional measures of job performance. Bio-data involves a measurement of the characteristics of successful in-house managers. What are the background traits that have been associated with success in the firm in the past? A psychologist can construct an instrument based on interviews with the firm’s successful employees and then test applicants to see if they have the same traits. Integrity tests measure the extent to which employees have or believe in their own integrity. Content valid tests are like job tryouts but focus on one element of a job, such as a typing test.
High Performance Staffing
Combining several objective measures with traditional informal interviews will improve the quality of new hires and increase productivity. In a 1998 study in Psychological Bulletin (Ibid.), Hunter and Schmidt find that “Overall, the three combinations with the highest multivariate validity and utility for job performance were an IQ test plus a work-sample test (mean validity of 0.63), an IQ test plus an integrity test (mean validity of 0.65) and an IQ test plus a structured interview (mean validity of 0.63).” In contrast, they claim that unstructured interviewing alone has a 0.35 validity, although other studies have found it has a lower validity.
In a 2004 article in Journal of Personality and Social Psychology (F.L. Schmidt and J.E. Hunter, “General Mental Ability in the World of Work: Occupational Attainment and Job Performance.” Journal of Personality and Social Psychology. 86:1, 2004.), Hunter and Schmidt show the range of IQs in a number of professions from the highest (accountant, which beat law) to the lowest (teamster) and show that for the most complex jobs, such as accountant, the correlation between IQ and job performance is nearly 0.6, while for the least complex job it is lower, about 0.2. Hence, IQ measurement is critically important for the accounting field.
But it is also likely that the quality of the accounting program from which the applicant graduated has a good correlation with IQ. Thus, using school quality as a proxy for IQ would be somewhat effective, but not as effective as using actual IQ tests in hiring.
Hunter and Hunter (Hunter, J.E. and Hunter, R.F. “Validity and Utility of Alternative Predictors of Job Performance.” Psychological Bulletin 96:1, 1984.) find that there is a powerful effect on productivity when high quality measures are used in combination. In 1984, they estimated that the federal government could improve the economic value of its productivity by $15.61 billion if they used IQ tests. In their article, which appeared in Psychological Bulletin, Hunter and Hunter show that effects would be greater for high-IQ professions like accounting. It would seem that for some firms the use of additional selection instruments over interviewing is an untapped source of competitive advantage.
Rate this article 5 (excellent) to 1 (poor). Send your responses here.
Mitchell Langbert, PhD, is an Associate Professor at Brooklyn College. Widely published on the subject of human resource management, Langbert has consulted and served as an expert witness.