EBOOK
A Buyer’s Guide to Pre-Employment Assessments
What are pre-employment tests?
Pre-employment assessments are a way for organizations to learn more about their candidates and to make more informed and objective hiring decisions. Assessments provide an additional layer of information about a candidate's skills and abilities beyond what can be gained from resumes and interviews. As a result, 82% of organizations use assessments of some form.
With such a wide range of assessment types and vendors on the market, we created this guide to help you select the assessment provider that best suits the needs of your organization.
Why Use Assessments?
Pre-employment assessments help organizations make better hiring decisions. These decisions ultimately define the very people who make up your organization, which is why making smarter, more data-driven decisions can have major benefits.
Among the many benefits, organizations typically use assessments to:
Raise quality of hire
Increase employee retention
Improve employee performance
Save time and increase hiring efficiency
Improve customer satisfaction
Increase sales revenue
Setting The Stage
Before you start researching assessment providers, take some time to understand your goals, your current hiring process, and the roles for which you are hiring.
Goals
If you’re looking to implement pre-employment assessments, you likely already have a couple of goals in mind. Your goals can inform the assessment solution that makes the most sense for you. Here are just a few examples of goals you might have, and how they might affect your ultimate decision.
Goal: Raise quality of hire across the board
Takeaway: Look for a provider that offers assessments that are scientifically validated to predict job performance. Bonus points for providers that can back up improvements in quality of hire with case studies and validation data.
Goal: Save time in the hiring process
Takeaway: Look for a provider that offers a user-friendly system with simple test administration and results that are easy to interpret. You may also be interested in an off-the-shelf solution with faster implementation.
Goal: Assess high-volume applicant pools
Takeaway: Look for a provider that offers easy test administration in bulk, robust reporting features, and candidate-friendly assessment experiences that are device-agnostic. With a high volume of candidates, you may also prefer a subscription-based service as opposed to a pay-per-test model.
Goal: Standardize an assessment solution across the organization
Takeaway: Look for a provider that offers a flexible, customizable solution with user-access roles that can accommodate many departments. You may also want the ability to create custom benchmarks specific to your organization.
Process
Pre-employment assessments are not an end-all solution that makes the hiring decision for you. They are just one of the multiple steps that make up a best-in-class hiring strategy.
The hiring process is exactly that – a process. It contains certain steps that take you from the initial sourcing stage all the way up to the moment an employee is officially hired. And just as with any process, the order of the steps matter.
Where you choose to place the assessments in the process can have an impact on the outcomes you get from the assessment program, and this decision ultimately depends on the goals you established earlier.
Where in the Hiring Funnel to Place Assessments
Criteria’s Recommendation: At the Beginning
We typically recommend that organizations put assessments up front in the hiring process because it offers two key benefits.
First, assessments give you an effective way to prioritize as you move down the hiring funnel. Many of our clients use assessments prior to the interview, a traditionally time-consuming step in the hiring process, in order to pre-qualify who to invite to the interview. If one of your goals is to save time hiring and to create a more efficient process, this is one of the best ways to achieve this.
Second, the results also provide you with a new lens to see the potential in your candidates by highlighting certain applicants who may not have caught your eye based on their resumes alone. If one of your goals is to broaden your applicant pool and identify untapped talent, administering assessments earlier in the process can help you achieve this.
At the End
Leaving testing to the very end of the process is somewhat rare, and not typically recommended. Organizations that test at the very end typically use the assessment as a final point of validation before extending an offer.
The major con of leaving testing to the end is that it can be psychologically difficult to eliminate a supposedly "great" candidate from the running based on testing conducted at the end. The main benefit of assessments is to provide you with objective, data-driven information about a candidate's qualifications. If you select a shortlist of candidates based on other less predictive, and often misleading, factors such as their experience and an unstructured interview, you risk weighting these less predictive factors more heavily than the assessments. While assessment results should never make the decision for you, they provide the most value when they can be incorporated into your decisions about the candidate in a reliable and consistent way.
If you select a shortlist of candidates based on other less predictive, and often misleading, factors such as their experience and interview, you risk weighing these less predictive factors more heavily than the assessments. While assessment results should never make the decision for you, they provide the most value when they can be incorporated into your decisions about the candidate in a reliable and consistent way.
ATS or HRIS Integrations
One final important consideration when it comes to your process is your current HR technology. If you are using an ATS or HRIS platform, you will likely want to integrate your assessment solution with your existing HR tech stack. However, this isn’t always the case. You should evaluate how important integration is to your process, and whether it will be a must-have or a nice-to-have when selecting your final assessment provider.
Roles
Finally, you will need to evaluate who you are planning to test with the assessments. This requires a consideration of which departments will be using the tests, the job levels you are planning to assess, industries and job types, as well as whether you intend to use the assessments as a talent management tool for internal employees, either for career development or team building.
Organization-Wide or Department-Specific
If you are planning to incorporate assessments across your entire organization, you will need a provider with a robust assortment of assessments that can fit the needs of the many roles for which you will hire. In addition, you will likely need ways to customize user-access roles within your company to ensure access to the many stakeholders who will be using the system. If you are planning to use the assessments for a smaller group of hires or for a specific department, your needs will be simpler.
Job Level
Different job levels may call for different types of assessment solutions. Entry-level candidates will need assessments suited to their level of experience, and they may also require shorter assessments, or ones that are more interactive or game-based. Entry-level jobs also typically tend to have a higher volume of applicants per job opening, meaning that you may want to prioritize assessments earlier in the hiring process.
Industry and Job Type
Your industry, as well as the job types for which you plan to hire, are extremely important when you select an assessment provider. The most obvious consideration is to ensure that the assessment provider offers assessments that are relevant for your industry and role. Ideally, the assessment provider will have experience in your industry, having conducted studies and benchmarks for roles that are relevant to your organization.
Internal Employees
Finally, you may be interested in using assessments not just for pre-hire but for post-hire talent management as well. If this is the case, look for an assessment provider that offers tools for assessing internal candidates, as well as ways for managers to apply the information in an effective way.
Questions to Ask
Once you have established your goals, your ideal process, and who you plan to test, you can start researching assessment providers and narrowing down those that most likely fit your needs.
We gathered a list of 21 key questions to ask providers in order to find the right match.
Assessments
The most critical set of questions you will need to ask are about the assessments themselves.
1. What assessments do they offer?
Different assessment providers offer different types of assessments. Based on your industry and the roles for which you plan to hire, you will need to determine what types of assessments you would like to use. If you aren’t as familiar with pre-employment testing, the provider may be able to provide you with guidance on which assessments are right for you.
There are a wide range of assessment types. Some of the most common types include tests of cognitive aptitude, personality, emotional intelligence, and acquired skills. They vary widely in their predictive ability and their use cases. For example, cognitive aptitude is the most predictive type of measure, and this is true across all job types and job levels. Therefore, cognitive aptitude assessments could be used broadly across an organization. On the other hand, tests that measure specific skills, such as Excel skills or fluency in a programming language, have a more specific use based on the role.
The goal here is to determine what information you would like to collect to make a more informed, and ultimately more predictive, hiring decision. If you are planning to test for just one type of role, a niche provider that is specific to your industry could make sense. If you are planning to test for multiple roles, or to potentially roll out testing across the entire organization, then you may want to select a provider with a larger assessment portfolio. This enables you to take advantage of a greater breadth and depth of assessment types.
2. Are the assessments scientifically validated?
In addition to which assessments a provider offers, it is also critically important to learn more about how valid the assessments are. An assessment program is successful if it helps organizations make more informed hiring decisions that are predictive of the outcomes that matter, such as job performance and employee retention. However, in order to deliver these results reliably and consistently over time, an assessment provider needs to have a robust validation process for its assessments. To get clarity on this topic, ask a provider what their validation process looks like.
Related questions include:
How were the assessments validated?
How reliable are the assessments?
Can they provide case studies or documentation around validation?
If they have a team of industrial and organizational (I-O) psychologists, you can also ask to speak to them about any additional questions you may have.
Even more importantly, you want to make sure that the assessments are validated for use in a pre-employment setting. This has legal implications as well as predictive implications. Some widely used assessments, such as the Myers-Briggs (MBTI) and the DISC assessment, are not validated for use in a pre-employment context, exposing organizations to legal risk if they use them to make hiring decisions. While these types of assessments can be helpful for team building or career development activities post-hire, they are not validated tools for making hiring decisions.
3. Can I take the assessments?
The next logical step is to try out the assessments for yourself. This is an important step because it gives you the opportunity to view the process from the perspective of your candidates. It will also give you a high-fidelity feel for the assessments themselves, helping you understand if the assessment is relevant for the role, if the difficulty level is appropriate, and if it provides an appropriate experience for the candidate.
4. Are the assessments candidate-friendly?
For many organizations, the candidate experience is a major consideration when building out the hiring process. If candidate experience is a big initiative for your organization, you will want to evaluate how well the assessments provide a candidate-friendly experience. When taking the assessments yourself, you can begin to answer the following questions:
Is the experience overly tedious?
Is it engaging and seamless?
Can the assessments be taken on any device?
Are there steep tech requirements that candidates must have in order to take the assessments?
Do the assessments provide an experience that will represent your brand in a positive way to your candidates?
Does the provider offer assessments in different, engaging formats, including game-based or adaptive assessments?
Another major consideration is the length of the assessments. Shorter assessments are typically better for candidates, provided that they still provide high-quality, validated data to your organization. Through our own data that we have collected at Criteria, we determined that the optimal amount of testing to administer to candidates is about 40-45 minutes. Requiring candidates to take testing beyond this limit typically results in gradual drop-off in test completion rates.
5. Can candidates easily cheat or “game” the tests?
This is a question of how well the provider protects its testing IP and how well the assessments are designed to collect valid, reliable data. In order to get an accurate read on a candidate, you need to be confident that the candidate wasn’t able to cheat or game the assessment, thereby invalidating the results.
You may want to ask how the provider ensures that candidates can’t access answers online, fudge their answers on personality tests, or generally game the assessment.
Some providers reduce cheating or “gaming” by using internal validity checks to catch any inconsistent or suspicious activity during the assessment. Some also employ adaptive or dynamic testing to provide each candidate with a unique assessment at the appropriate difficulty level.
Results
Can I interpret the results?
Robust, validated assessments are great, but they are somewhat useless if your team can’t interpret the results and confidently take action based on the findings. What this comes down to is how easy it is to interpret score reports and assessment results.
Assessment providers range widely in how complicated the results might be. Some assessments can be interpreted right off the bat, while other assessments may require formal training and even certification for your team. This can impact how easy it is to roll out the testing to different departments, as well as the amount of time you can expect to implement a new testing program.
Can I customize the results?
Depending on your organization’s needs, you may want a custom solution as opposed to an off-the-shelf solution. In practice, what this might look like is the ability to create custom benchmarks or baselines within your organization. Some providers may be better equipped to provide this type of service offering.
Can I get the reporting I need?
Reporting is a less exciting but nevertheless necessary part of the hiring process. When aggregating and reporting your hiring metrics, you want to be able to pull the reports that you need from the system. Reporting capabilities range widely, and you may need to request a closer look at the provider’s capabilities once you have determined your needs.
Services and Features
What level of customer support do they offer?
When it comes to customer support, every organization’s needs are different. Some organizations may prefer to use a turn-key solution with minimal contact with the provider. Others may prefer a hands-on approach with an assigned account manager. Not all providers offer the same level of support, so it is important to determine what your needs are and to see if they align with what the provider offers.
What level of candidate support do they offer?
Aside from the support being offered to you as a customer, you may also want to learn more about the support that the provider offers to candidates. During any assessment, candidates inevitably have questions. Does the provider offer reliable support for candidates, or would the bulk of that fall on you to answer candidate inquiries? This is an often-overlooked question, but it is an important one to ask because it can impact the candidate experience and ultimately their perception of your brand.
Is my data secure?
This varies in importance from one organization to the next. But if data security is a priority for you, this is an area you will want to probe. Some issues around data security include how the organization collects and uses Personally Identifiable Information (PII), and whether they are compliant with regional guidelines such as GDPR or CCPA. Other issues include security of the provider’s platform, as well as issues relating to server reliability and the possibility of service disruptions if the provider’s technology were to go down.
Do they offer the features I need?
Some assessment providers may solely provide access to the assessments, while other assessment providers may offer a broader range of features and capabilities. The features you may need is highly specific to your specific organization, and it is important to compile a list of desired features that you wish to find in an assessment provider. Dividing them by Must-Haves and Nice-to-Haves can help you prioritize features and ultimately make a final decision between providers.
For example, you may want a tool that provides you with unlimited users so that your large team can scale to use the service. Or you may need a platform that provides Single Sign-On (SSO) capability, enabling your team to use a centralized login to enhance security. You may also be interested in other features related to hiring – for example, you might want to manage your candidate pipeline within a platform, or to use the same tool to send candidate communications.
Fortunately, this question is fairly easy to answer. The assessment provider will likely be able to provide an overview of all the features they can offer, along with demonstrations of how each works.
Process and Implementation
Can I integrate with my existing technology?
Assessments are just one part of your larger hiring process. If you are using an ATS or HRIS provider to conduct much of your hiring process, you may want to integrate an assessment solution into that process. Technological integrations can be complicated, but they can also be simple. Not every assessment provider has the capability to integrate with every other service provider in the HR tech space.
Ask the assessment provider if they do integrate with your ATS or HRIS of choice. It may be necessary to request a demonstration of the integration to ensure that it fits your needs. Some integrations operate quite seamlessly with minimal intervention required. Others can be more labor-intensive to set up. The conversation may ultimately need to include a representative from your ATS or HRIS provider along with a representative from the assessment provider to determine capabilities.
Can everyone on my team get access?
If your organization plans to roll out an assessment program across multiple departments and teams, the question of access becomes increasingly important. Some assessment providers will offer unlimited user access the system, which can help organizations scale their testing program.
Beyond simple access, you may want to determine how well you can control user access roles within the platform. Because candidate assessments involve the collection of sensitive candidate data, you will likely need the ability to control the level of access that is granted to different members of the team, from the HR admin to the data analytics team and all the way up to the CHRO or head of talent acquisition.
How customizable is the solution?
Different organizations have different needs for customizing their solutions. Customization exists on two different levels. First, how customizable are the assessments as well as the benchmarks used to evaluate candidates? While customizable assessments may sound like an attractive concept, more customization of the assessments themselves is not necessarily a positive thing, because it often reduces the validity and ultimately the efficacy of the assessment.
Second, how customizable is the assessment platform? In the last point, we discussed the desire to have customizable user access roles to the platform. You may also want to customize other elements for your organization, such as customizable reporting, customizable candidate pipelines, customizable branded assessment experiences, and more. For many organizations, an out-of-the-box solution is perfectly suitable.
For others, a customizable solution may be required.
How quickly can I expect to get up and running?
When it comes to implementing the assessment program, you will need to know how long it takes before you can start testing candidates.
A number of factors can impact the implementation time. For example, if a technology integration needs to be built, you should expect a longer timeline. If the assessments themselves are complicated to interpret, you may need to factor in the time it will take to train your recruiters and hiring managers to interpret the reports. And the level of customization you require will also impact the timeline. Off-the-shelf providers will likely be able to provide a much faster onboarding process than an entirely custom solution provider.
If you have an urgent need for assessments, then faster implementation will be a requirement. If you have a less pressing timeline, then a longer implementation may work for you. Nevertheless, it is important to ask the assessment provider what the expected timeline looks like so that you can communicate it to other stakeholders within your organization.
Do they follow legal and ethical guidelines?
Legal defensibility is understandably a concern for organizations looking to incorporate assessments into their people management processes. However, when you use assessments that are professionally developed and well-validated, assessments can enhance the objectivity, equitability, and legal defensibility of your decisions.
Simply asking about their validation process for assessments can go a long way. A poorly designed test with algorithmic bias will skew insights and expose you to ethical, legal, and reputational consequences. And while some of the newest assessment technology, like AI and machine learning, can seem attractive, these newer technologies may be less rigorously tested for bias or predictive value. Ask whether their AI based solutions are validated by experts in employee selection, such as IO Psychologists, and whether they comply with laws specifically focused on AI or machine learning, such as the City of New York's law on the use of AI in employee selection. Ultimately you want to identify an assessment provider that produces reliable and consistent assessments that can help you make fair and objective hiring decisions.
Pricing
Is it unlimited or pay-per-test?
The cost of the service will be a critical deciding point no matter what type of product you are researching. However, the actual pricing model may also impact your decision. Two major pricing models exist on the market for assessment providers. First, there is the unlimited testing model, in which the provider offers the service in the form of a subscription, whereby you pay a flat-fee for unlimited use of assessments.
Second, there is the pay-per-test model, in which there is a cost to administer an assessment, and you pay each time a candidate takes the assessment.
These two models have pros and cons. However, at Criteria we believe that an unlimited testing model confers a number of major benefits to organizations. For example:
Predictable costs: One of the biggest benefits is that you will have more predictable costs. You know exactly what the total cost will be before getting started, and you don’t have to factor in additional costs if you decide to administer more tests or hire more people than you expected. This enables your organization to scale at your own pace without worrying about out-of-control recruiting costs.
Lower total costs: Flat-fee pricing almost always helps organizations lower the total costs of administering pre-employment tests. With online tests, the incremental cost of delivering tests is relatively low, and the transactional pay per-test model, which originated in a time when test scoring wasn’t automated, doesn’t make as much sense today.
Enables earlier testing: Flat-fee pricing can help you drastically reduce time spent in the hiring process by encouraging you to test early in the process. As we mentioned earlier in this guide, assessments add the most value when they are administered early in the hiring process because they help you prioritize large applicant pools. However, testing early in the process as opposed to testing a smaller, pre-filtered group of candidates later on requires you to administer a massive number of tests. Flat-fee pricing allows you to administer tests to large groups of applicants in the early stages of hiring without having to worry about the additional fees associated with per-test pricing models.
Other Considerations
We collected a few final questions that aren’t as obvious, but can help inform your decision about which provider you ultimately choose to partner with.
Is the provider investing in innovation?
Pre-employment assessments are designed and validated using a combination of research from psychology, data analytics, and more. If there is anything we know about science and technology, it is that it is continually evolving, progressing, and improving. Many techniques and methodologies that were considered cutting edge 30 years ago are now outdated, while still others have stood the test of time.
The important thing to note is that the validity and efficacy of an assessment is based on how well it fits the hiring needs of the modern organization. In today’s world, this means creating assessments that do their best to:
• Provide highly predictive data
• Deliver assessment experiences that are engaging to candidates
• Create a positive image for the brand
• Build a more diverse, engaged, and high performing workforce
Assessment providers that are invested in innovation are typically investing in building new and innovative assessments that fit these goals. You can seek to learn more about an organization’s investment in innovation by asking about their general product roadmap or asking about what they have released in the last year.
Does the provider use their own testing?
You may not have considered this question, but it is worthwhile to ask a provider if they use their assessments to hire their own employees. If the answer is yes, it’s very likely that the provider truly believes in and advocates for their product, and that they have experienced positive results by using the assessments. It can also help the provider understand the perspective of your organization because they have used the product themselves.
Does the provider have a good hiring track record?
Any assessment provider is essentially selling their perspective, expertise, and best practices as it relates to hiring. As a result, it can be useful to see if the assessment provider has been successful with their hiring and talent management efforts. However, simply asking a sales representative won’t quite answer the question. There are a couple of ways you can uncover this information indirectly:
• Visit the company’s Glassdoor page to see if employees are happy there
• Visit the company’s Careers page on their website to see how they present themselves
• Read a job description for an open position at the company
• Check out the company’s social media profiles
While you don’t want to make too many assumptions about an organization based on this information, it can potentially serve to validate your decision to select an assessment provider in the final stage of your decision process.
Selecting a Pre-Employment Assessment Provider Checklist
Use this checklist to help you select an assessment provider that fits the needs of your organization.
What assessments do they offer?
Are the assessments scientifically validated?
Can I take the assessments?
Are the assessments candidate-friendly?
Can candidates easily cheat or “game” the tests?
Can I interpret the results?
Can I customize the results?
Can I get the reporting I need?
What level of customer support do they offer?
What level of candidate support do they offer?
Is my data secure?
Do they offer the features I need?
Can I integrate with my existing technology?
Can everyone on my team get access?
How customizable is the solution?
How quickly can I expect to get up and running?
Do they follow legal and ethical guidelines?
Is it unlimited or pay-per-test?
Is the provider investing in innovation?
Does the provider use their own testing?
Does the provider have a good hiring track record?