Assessing Cognitive Competencies at Work

An Hr's Handbook for Understanding How Cognitive Skills Evolve with Job Roles and Levels

cognitive-competencies-at-work

THE SCIENCE BEHIND HUMAN INTELLIGENCE

Human intelligence is the mental quality that consists of the abilities to learn from experience, adapt to new situations, understand and handle abstract concepts, and use knowledge to manipulate one’s environment. In simpler words, it is the ability to acquire and apply knowledge and skills.


cognitive-emotional-intelligence


Intelligence =

Cognitive Intelligence + Emotional Intelligence

Cognitive intelligence is defined as the combination of verbal, numerical and spatial abilities which includes visualizing, use of memory, word fluency, verbal relations, perceptual speed, induction and deduction.

Emotional intelligence is a form of intelligence involving the ability to read one’s own and others’ emotions and use this reading to manage one’s own thoughts and behaviors.


cognitive-intelligence-icon

COGNITIVE INTELLIGENCE

Charles Spearman in the early 20th century established that there must be one central factor that influences our cognitive abilities. He termed this central factor as general intelligence.

‘g’ factor: also known as general intelligence factor is a variable that is required in all tasks that require intelligence. ‘g’ is also responsible for achievement of positive correlations among different cognitive tasks, reflecting the fact that an individual's performance on one type of cognitive task tends to be comparable to that person's performance on other kinds of cognitive tasks.

general-intelligence-icon

G is powered by four core cognitive brain processes that are closely associated with and directly impact cognitive intelligence. These are – attention, speed, memory and visualization.


The definition of ‘g’ was further expanded and bifurcated into two by Raymond Cattell, who proposed that ‘g’ consists of two parts: Fluid and crystallized Intelligence.


cognitive-intelligence-core-brain-function


Fluid intelligence can be defined as the ability to solve unfamiliar problems, by making use of logical reasoning. This ability is not dependent on prior learning, life experience or education.


Example: The true use of fluid intelligence is exemplified by early men who used instincts, logical reasoning and problem solving skills to survive. They did not have any education or access to any form of knowledge that would help them to meet their day to day needs of food, shelter and protection from predators.

Fluid-intelligence-example

Crystallized intelligence is defined as the ability to use learned knowledge and experience.


Example: When we are using formulas to solve mathematical problems or applying rules of grammar to make sentences and write, we are using crystallized intelligence.

crystallized-intelligence-example


MEASURING COGNITIVE INTELLIGENCE

In order to measure intelligence, we need to identify the cluster of abilities or skills, in the form of which intelligence manifests itself. These skills and abilities cluster is called a ‘competency’.


We have identified the key competencies that help in accurate measurement of our fluid intelligence, crystallized intelligence and core cognitive brain functions.


Core Competencies to Measure Cognitive Intelligence

icon-64


core-brain-functions(1)

Competencies to Measure Core Brain Functions



fluid-intelligence-icon

Competencies to Measure Fluid Intelligence



crystallized-intelligence-icon

Competencies to Measure Crystallized Intelligence


USING COGNITIVE INTELLIGENCE AT WORKPLACE

Is it required to create a cognitive framework for each job role?


We cannot possibly create a competency framework for each job role in existence. So we decided to explore if there are certain job roles that require the same cognitive abilities and would therefore have the same competency framework.


Our research yielded that yes there in fact are job roles that require the same cognitive abilities.
For example Customer Service, Receptionist and Sales Executive all require basic communication skills and performing the same kind of task day after day.



job-categorization-icon

Mettl’s Job Categorization Framework

Any job in the world involves carrying out a task. The type and level of cognitive competencies required to complete a task in a particular job role, is dependent on the nature of the task.

The task can be:

cognitive-competency-framework

The next question that comes to mind is-
How can we derive the type and level of competencies required in a job role on the basis of the nature of the job?


Mettl’s Framework for job categorization takes into account three core competencies along with two factors pertaining to each core competency.
We identified three core competencies that are required in any job role. These three core competencies are:


job-categorization-framework

Information Processing: Refers to a person’s proficiency is carrying out numerical, analytical and critical reasoning.


Solution Generation: Refers to a person’s ability to think out of the box by making use of creativity and abstract reasoning competencies.


Decision Making: Refers to a person’s ability to solve problems and take well thought out, rational and accurate decisions.


Two factors pertaining to each core competencies are:

Frequency: How many times in a day does an employee need to make use of a particular competency

Level: How difficult are the tasks involving use of a particular competency, which will determine if an employee needs to have basic knowledge of a competency or be proficient in it.


job-families-icon

Cognitive Competency Framework for Job Families

Mettl’s Job role categorization framework has identified five different pools of job roles that differ from each other in the nature of the job and the cognitive competencies required. We call this pool of job role as a ‘job family’. Each job family contains job roles that are similar in nature and require the same set of cognitive competencies to perform them. In other words, there is one cognitive competency framework for each job family.

framework-for-job-categorization-mobile



CALCULATING EFFECTIVENESS OF COGNITIVE ASSESSMENTS

To make a cognitive assessment effective in choosing the right talent for a job role, it needs to accurately identify candidates from the talent pool who are likely to succeed at performing that job role. Once the assessment has been created for a particular job role, its effectiveness in selecting the right talent in an organization needs to be tested.


To have a cognitive assessment adept in identifying the right talent we fine-tune the three parameters identified earlier: the assessment mean score ,difficulty index and discrimination index.


Mean Score is the average score that candidate pool giving the same cognitive assessment achieved.


assessment-mean-score

Low mean score signifies that less number of candidates from the talent pool were able to score high on the aptitude test.


Medium mean score means that moderate number of candidates were able to score high on the aptitude test.


High mean score means that majority of candidates from the talent pool were able to score high on the aptitude test.


The ideal mean score for you depends upon what level of candidates you are looking for. Example, Out of your talent pool, If you want to select candidates with the highest cognitive intelligence, your cognitive assessment should have a very low mean score.


This is how you can adjust your mean score to control the number and intelligence level of candidates selected:


cognitive-mean-score

Note: You can adjust the mean score to adjust the number and IQ level of candidates. For example, if you want to select the top 5 percentile of the talent pool, you should aim for an aptitude test with mean score of 5%. It is a tool used for benchmarking an aptitude test as per an organization’s requirement.


Disclaimer: The kind of talent you end up with depends on the quality of the talent pool in general. If your talent pool quality in general is poor, selecting even the top 1 percentile may not provide you with high IQ candidates. Vise-a -versa, if your talent pool quality is too high, even lower percentile candidates may turn out to be high in IQ as the average IQ of the talent pool is high.


The Difficulty Index is the probability that candidates will answer a test question correctly. Every question that is included in a cognitive assessment is benchmarked via its difficulty index.

For example, if out of 100 students, 20 students were able to answer a particular question correctly, the difficulty index for that question would be: 20/100 = 0.5.

The difficulty level of a question is determined by its index range. Lower the difficulty index higher is the difficulty level of the question. It is a tool used for benchmarking an aptitude test as per an organization’s requirement.


Difficulty level is inversely correlated with mean score. High difficulty index signifies that majority of the candidates will perform poorly/fail the test, whereas high mean score signifies that most candidates were able to score high and hence passed the test.


LEVEL OF DIFFICULTY

INDEX RANGE DIFFICULTY LEVEL
0.00-0.20 VERY DIFFICULT
0.21-0.40 DIFFICULT
0.41-0.60 AVERAGE/MODERATELY DIFFICULT
0.61-0.80 EASY
0.81-1.00 VERY EASY

This is how you can adjust your difficulty level to control the number and intelligence level of candidates selected:


high-difficulty-level

If the average difficulty level of all the questions in a cognitive assessment is high, it means that on an average the questions included in the assessment are difficult. This means that candidates, on average, will fail the test. However those few who pass will have high IQ as they were able to clear a test deemed difficult by most. So a cognitive assessment with high difficulty level questions will churn out a pool of candidates that is low in number and high in intelligence.


low-difficulty-level

If the average difficulty level of all the questions in a cognitive assessment is low, it means that on an average the questions included in the assessment are easy. This means that candidates, on average, will pass the test. However the IQ of these candidates will not necessarily be very high as they were able to clear a test deemed easy by most. So a cognitive assessment with low difficulty level questions will churn out a pool of candidates that is high in number and low in intelligence.


Discrimination is the degree to which candidates with high overall score in a cognitive assessment also answered a particular question correct. It is an index of a question’s effectiveness at discriminating those who performed well in the test from those who did not. It determines if a cognitive assessment is valid or not.

In layman terms, if a question is answered correctly by the same proportion of high intelligence candidates and low intelligence candidates, then that question is not valid as it is not able to discriminate between the two.

Note: Unlike mean score and difficulty index, discrimination index is not a benchmarking tool. It's a validation tool which is critical to determine if an aptitude test is usable or not.

LEVEL OF DIFFICULTY

INDEX RANGE DISCRIMINATION LEVEL ACTION
0.19 AND BELOW POOR ITEM SHOULD BE ELIMINATED OR NEED TO BE REVISED
0.20-0.29 MARGINAL ITEM NEEDS SOME REVISION
0.30-0.39 REASONABLY GOOD ITEM FOR IMPROVEMENT ssa
0.40 AND ABOVE VERY GOOD ITEM RETAIN
 

Ideal discriminatory index: The ideal discriminatory index for a question should be 0.4 and above. A cognitive assessment where each question has a discriminatory index of 0.4 or above, will be able to effectively filter our undesirable talent and select desirable talent accurately suited for a job role.


The Mettl Advantage: Addressing Special Circumstances

How does Mettl customize and benchmark assessments to ensure accurate talent assessment?
By matching the talent pool available with the job role requirement.


Case 1: Overqualified Talent : Step up the game (Quadrant 1)


If the talent pool possess higher intelligence than the job role demands, chances are that the cognitive test ideal for that job role/level will be too easy for the talent pool, which means-Higher number of candidates will ace the test The company will end up with a large pool of screened candidates.

Problem: More number of candidates than available job openings.

Solution: Pick the best, increase the difficulty level of the assessment, so that lesser number of candidates will be able to ace it. The company will go away with the cream-de la-cream of the talent.

cognitive-ability-question

Suggestions: Candidates that lie in this quadrant are ideal participants for succession planning, high potential identification and leadership development.


Case 2: Underqualified talent : Mellow it down (Quadrant 2)


If the talent pool possess lower intelligence than the job role demands, chances are that the cognitive test ideal for that job role/level will be too difficult for the talent pool, which means-
Lower number of candidates will ace the test. The company will end up with a very small, compromised pool of screened candidates.

Problem: Less number of candidates than available job openings.

Solution: Make the best of a bad situation, decrease the difficulty level of the assessment, so that more number of candidates will be able to ace it. The company will still be able to fill its openings, even if with slightly compromised talent.

Suggestions: Candidates that lie in this quadrant are ideal participants for learning and development programs to upskill them.

Case 3: Special Case


In case of Quadrant 2 and 4, the cognitive abilities possessed by the candidate pool coincides with those required by the job role and hence will result in good job performance, higher job satisfaction and better work-life balance.


COGNITIVE INTELLIGENCE ASSESSMENT: USE CASES

The correlation between cognitive intelligence and work performance is 51% (minimum) and it can go as high as 84%.

High intelligence is an indicator of success on the job which is measured on the basis of:

• Evaluation of performance on tasks similar to those encountered on the job

• Performance ratings by supervisors

• Position in the occupational hierarchy


Why can measuring cognitive intelligence help in predicting success at work?

Cognitive intelligence mainly influences performance through the rate at which people learn knowledge relevant to the job – people with higher cognitive intelligence learn faster. But cognitive intelligence predicts success even when you take account of job knowledge. With high cognitive intelligence, people are more able to go beyond existing job knowledge and make judgements in unfamiliar situations.


hiring-icon

HIRING

Find Potential Hires With the Right General Aptitude Tests

leadership-icon

Leadership Development

Assess General Aptitude of Candidates for Leadership and Strategic Roles

learning-and-development-icon

Learning & Development

Identify and develop your high potentials with Mettl Learning Agility Assessment

hiring-icon

Use of cognitive assessments/aptitude tests has increased by 29% overall

Its usage during hiring has increased by 28% and during L&D by 101%. The increased usage during L&D is driven by the fact that high cognitive intelligence equals to high learning agility and as IT sector is constantly evolving, its important that the employees in this field are highly agile in learning to keep up with new technologies

hiring-icon

Use of cognitive has increased by 411% in L&D

Cognitive assessments/aptitude tests are the most commonly used assessments in pharma industry with 40% usage among all assessment tools. They have taken precedence over technical and psychometric with companies focusing more on subject matter knowledge and pharma manufacturing skills, which requires candidates/employees to have high cognitive abilities.

bfsi-test-icon

Usage of Cognitive assessments/aptitude tests has increased by 207% overall

Increasing demand for financial analysts and wealth managers from insurance industry has led to a sudden rise in usage of cognitive assessments/aptitude tests during hiring and L&D. These roles require high

hiring-icon

Usage of cognitive assessments/aptitude tests has increased by 99% overall

The surge is usage is led by increase in demand for consultants that can work across various industries and learn the tricks of the trade fast. This requires high brain processing power which can be easily tested via cognitive assessments.


Case Study

A BLUEPRINT FOR DEVELOPING FUTURE-READY WORKFORCE


How A Telecom Behemoth was Able to Build Better Teams through Mettl’s Cognitive Ability Assessments

Problem

The Client was struggling with inflow of low quality talent, poor learning and development and leadership development programs.

build-better-team-with-cognitive-assessment
Asset 86

Issue with hiring: the hiring process that the client was following was not able to identify the top talent needed for a job role

Asset 87

Issue with Learning and development: The company was struggling with poor ROI yielding training programs

Asset 88

Issue with Leadership development: The company was not able to identify promising high potentials that they could use to create a succession pipeline

Solution

The client company was facing these issues with identifying skill proficiency and skill gaps in potential hires as well as existing employees as it was not making use of effective cognitive assessments.

Mettl devised a 4 step process to create effective cognitive assessments specifically suited to meet the client’s talent needs in order to make their hiring, L&D and Leadership development processes more effective.


employee-cognitive-ability-processt

Impact

The client was able to create a ‘future ready workforce’ that was high performing and learning agile by

✔   Hiring the right quality talent

✔   Accurately identifying skill gaps in them that can be filled with appropriate training

✔   Successfully identifying high potentials that can be used to create the succession pipeline






con   Conclusion

Employee knowledge and skills are the backbone of an organization. The ability to learn and apply knowledge and skills to succeed at any job role is what cognitive assessments measure. The more accurately they are able to measure a candidate’s cognitive intelligence, the better quality talent an organization will end up with.

Top talent is spoiled for choice. They are limited and they are wanted. The faster you identify and on-board them, the better, as they are not available in the market for long.

Effective cognitive assessments provide organizations with the power to quickly and effectively differentiate the real talent from the fake one.



DOWNLOAD THIS COMPLETE REPORT


GET A COPY

facebook 

Mettl is one of the largest and fastest growing online talent measurement solution providers globally that has been at the forefront of online assessment technology since its inception in 2010. We enable organizations to build winning teams by taking credible people's decisions across two key areas: Acquisition and Development.

80+

Countries

2000+

Clients

27+

Industries
Mettl online assessment © 2010-
x
Banner_Webinar