Building continuity with online exams for the National Institute of Securities Markets (NISM)

NiSM has taken significant strides in redefining financial education within the global securities markets. One of its essential purposes is creating an adequate pipeline of new-age, well-rounded and competent professionals in the securities market through a bouquet of uniquely positioned academic programs. Hence, maintaining the education and examination cycle is extremely important for the institution. Since it is an educational initiative by SEBI, every individual must take a SEBI-mandated voluntary written examination at any of NiSM’s 250+ accredited test centers spread across the country.

The academic calendar was on course, but the pandemic altered the plan. The campus and the test centers were closed for operations since the onset of the COVID-19 and the subsequent lockdowns. Resultantly, the institution was uncertain about its critical pen-and-paper exams for various courses. So were countless market participants who could not obtain or renew their credentials despite several months’ preparation.

Initially, NiSM was reasonably skeptical about shifting its assessments online. However, the management eventually decided to go ahead with the online system, considering the candidates’ health and safety and the need for continuity for the professionals and the securities markets.

But that was not enough.

NiSM had to validate, analyze and measure a lengthy value-proposition checklist of functional and non-functional requirements before getting the ball rolling.

Before we detail this intriguing story of this mega-partnership, the following were some NiSM’s requirements, out of the sixty-four parameters it had listed:

A robust examination platform

NiSM’s exams were a global mark of excellence that set the standard in the securities market. Hence, NiSM sought a dynamic end-to-end online examination platform to conduct such high-stakes exams. The platform needed to offer a comprehensive approach, covering all areas: exam creation, scheduling, delivery and evaluation, besides post-exam review and results declaration by certifying eligible candidates.

Finest proctoring supervision with anti-cheating tools

SEBI endorsed and accredited NiSM exams. Thus, every stage of the exam required an enhanced security feature, which mandated an easy-to-use and superior quality proctoring feature. Besides, it also needed experienced and professional proctors to verify and monitor the test-takers throughout the exam, akin to an on-site examination, to ensure academic integrity. Furthermore, the institute sought anti-cheating tools to deter candidates from indulging in any untoward activity and helped it adhere to strict internal policies and protocols.

Highly scalable infrastructure

NiSM had always conducted pen-and-paper-based exams. Therefore, it had established processes to cater to multiple test-takers appearing for the exam in the traditional format. Hence, it necessitated an online system that ensured the same accessibility and the ability to scale up during the examination process without impacting the candidate experience.

High integrity

Data security, academic integrity and exam confidentiality were NiSM’s significant prerequisites for administering any exam online. Furthermore, most of its certifications were job-oriented and directly impacted the candidate’s employability and reputation. So, validating the candidates’ abilities and skills – that they deserved the degree – was equally critical. Hence, it warranted a secure and reliable environment, robust audit trail and viable evidence to handle instances of malpractice.

Easy accessibility on a simple system

NISM understood that not every test-taker was well-informed about modern technology and would not be able to use it to his/her advantage if it was complicated. Similarly, a patchy internet connection across locations could make taking online exams challenging. Thus, it mandated a system with a comprehensive yet straightforward interface, which could provide a seamless experience to candidates appearing from any location with minimal internet bandwidth requirement.

Discovering Mercer | Mettl

NiSM was a SEBI initiative. Therefore, it had an established public procurement policy and followed all norms and guidelines before making any high-value procurement. Hence, it took a similar route when onboarding an online examination platform and proctoring provider.

NiSM set up a technical evaluation committee to investigate and vet every proposal. The task was clear and concise – the committee thoroughly evaluated each prospect’s deliverables against NiSMs laid down requirements to ensure meeting every criterion without compromising on the conditions. Many service providers participated, but soon only eight participants remained in the fray.

The eight vendors offered multiple demonstrations and presentations to appraise the management about their platforms’ prowess and how they would fulfill NiSM’s unique needs. Mercer | Mettl, too, demonstrated a remarkable ability to deliver on each of NiSMs asks.

Every service provider ticked off every parameter on NiSM’s list. However, it eventually boiled down to the solution’s “user acceptance testing.” Mercer | Mettl was unprecedented at offering a seamless user experience to every user that tested the platform. This ease of use won NiSM’s confidence, and Mercer | Mettl bagged the proposal for its large-scale high-stakes exam.

Mercer | Mettl EXAMIN- The online proctoring and examination system

Mercer | Mettl’s Examin was the ideal value proposition that succeeded in meeting NiSM’s online examination needs. This proficient and end-to-end online exam management system was robust enough to match the institution’s scale, simultaneously offering credible, secure and swift exams with no errors or drawbacks. It also encompassed test creation to real-time reporting, complete with a dedicated admin dashboard for detailed analytics and logs with every recording.

Here are some benefits that the institution accrued:

• Mercer | Mettl’s team substituted NiSM’s administrators and coordinators, helping them upload the question papers created by faculty in the required format of MCQs and objective questions.
• It flexibly accommodated a range of other parameters that academia required.
• Continued technical support and assistance on exam days enabled the faculty to schedule glitch-free exams at any given time.
• The faculty also witnessed a considerable reduction in the time needed to set up exams, with increased productivity and lowered staffing tasks.

• NiSM’s test-takers were spread across the country. Thus, Mercer | Mettl’s cutting-edge application ensured that every test-taker could take the exams seamlessly, even with a rudimentary internet speed of 512 kbps.
• All the user instructions and exam guidelines were well documented and made available to candidates at every step of the exam to ensure a seamless experience.
• The platform was user-friendly, easy to navigate and even catered to those with limited technical literacy. It replicated and created a test-center-like exam experience for the candidates.
• The institution could swiftly scale its exams from near-zero to thousands every month, using Mercer | Mettl’s software with a proven track record of conducting a hundred thousand successful proctored assessments in a day.

Mercer | Mettl’s focus on stringent cybersecurity protocols addressed NiSM’s concern regarding data breaches, unauthorized access, impersonation and privacy at every stage of the examination process. Mercer | Mettl implemented the following procedures to ensure utmost fairness and security during each exam session:

• Data security

Mercer | Mettl’s host, Amazon Web Services (AWS), one of the leading cloud storage services providers on the market, proved to be a highly secure cloud-based platform for exam papers and computation storage.

• Data encryption in transit and at rest

HTTPS (256-bit SSL encryption) ensured secure data transmission and that webcam proctored tests ran without downloading the software. Similarly, Mercer | Mettl created an uncompromisable maximum security storage environment for applicants’ data, questions and exam results.

• Access rights management

Mercer | Mettl’s team ensured the safety and encryption of the available data by defining clear access rights, log reports and audit trails of various system resources.

Strict guidelines with multi-factor authentication for handling exam-related material prevented unauthorized access.

Mercer | Mettl validation included GDPR compliance and ISO 27001:2013 certification.

• 3-point authentication

Mercer | Mettl had deployed over a hundred authorized personnel to conduct a 3-point candidate authentication to confirm only the right individuals took the exam. Live authentication began only after the candidate clicked on the exam link. Here is how the process unfolded:

• First, the system asked the candidate to click their photos via a webcam.
• The candidate also had to show their ID proofs to the authorizer via a webcam.
• They had to send across customizable registration details that authorizers verified on the spot against previously collated data.

Mercer | Mettl aligned three proctoring approaches and a dynamic set of anti-cheating features to mitigate any violation that could undermine NiSM’s prestige and legacy. Everything was preconfigured on the system, offering similar accountability levels as in-person invigilation. The only prerequisite for the exam was to enabling permission to share applicants’ screens, video and audio, following which Mercer | Mettl’s best-in-class proctoring solutions effectively monitored the proceedings. Here is how:

AI (Artificial Intelligence) proctoring

• Mercer | Mettl’s futuristic AI algorithm trained with more than 2.8 million proctored assessments efficiently detected up to eighteen distinct digressions during the exam.
• With over 98% accuracy, the intelligent algorithm helped human proctors flag various suspicious cases, such as the non-visibility of face/ presence, detection of mobile phone, presence of an additional person, distracting movements, etc.

Live proctoring with human proctors

• Mercer | Mettl team onboarded about a hundred proctors to monitor the test-takers remotely, from start to finish.
• Initially, the proctor-to-student ratio was 1:20. However, it was later adjusted to 1:10 to ensure a more rigorous check and reduce the administrators’ supervision workload. The revised ratio helped solve concerns about identification and eliminated cheating with strict scrutiny.
• Each proctor was given one-to-one and one-to-many proctoring view features.
• The proctors also had the option to pause /stop /resume the exam in real-time. They also had a live chat option that enabled the invigilators to interact with candidates to allay suspicions or queries.
• The entire session was recorded and could be viewed later by authorized personnel only.

Record-and-review proctoring

• NiSM also utilized the record-and-review proctoring feature for specific exams that did not require continual invigilation. The exam-taking session was recorded and updated on the cloud that could be viewed at any time.
• This automated process saved the management considerable time and effort as the team did not have to supervise the candidates in real-time. Instead, it reviewed every recorded session diligently at its convenience and scanned it for any red flags or suspicious activities.

Audio proctoring

The candidates were taking their exam remotely, so the sanctity of the environment had to be maintained. Hence, Mercer | Mettl enabled its powerful AI-based audio proctoring software- ‘Speech Sense’ – to scan the test-taker’s physical environment for additional sounds or noises using the microphone.

• Audio proctoring helped ascertain the difference between sounds that can and must not be in a testing environment.
• It accurately identified details such as faint sound levels, whispers, mumbling and kept a count of unique speakers in the audio feed while tagging each speech segment to a particular speaker.
• By differentiating between human speech and various background sounds or noises, Speech Sense ensured that every test-taker had a level-laying field without being hauled up because of background noises.
• A warning message flagged and reported the same in the test log if a candidate tried to mute the microphone during the exam. The candidate could not continue with the test until he/she unmuted the mic.

• Mercer | Mettl’s limited navigation control feature was another advanced anti-cheating solution that boosted the management’s confidence in ensuring a highly secure and controlled online environment.
• MSB locked the applicants’ test screens and prevented them from opening new browser tabs/incognito tabs/windows.
• It blocked functions such as copying/pasting and restricted shortcut keys and three-finger swipes.
• During the exam, it also prevented access to secondary devices, such as cables, hard drives, or external devices such as the USB or internet dongle. If applicants synchronized any such device before the test or attempted to synchronize them during the exam, MSB would not let the exam commence and immediately log the applicant out.
• It did not allow exam-takers to move out of the application before finishing the test.

Mercer | Mettl team also activated its proprietary algorithm- the ‘Credibility Index’ – to meet NiSM’s student integrity score requirement. This exclusive feature provided an auto-generated and validated integrity score for every exam-taker after taking inputs from AI flags, recorded details of the proctoring sessions, and the test finish status. Hence:

• With valuations such as “high,” “medium,” and “low,” the faculty could quickly identify and filter out candidates who might have resorted to cheating.
• Documentary evidence of an applicant’s integrity empowered and assured NiSM that only the most deserving candidates got certified.
• NiSM ensured continuity in the securities market, with minimal hiccups and instant actionable insights.
• Candidates got a fair opportunity to prove their skills, with cheating mitigation tools. On the other hand, NiSM got reliable and trustworthy information.

Mercer | Mettl’s avant-garde solution safeguarded NiSM’s academia’s integrity, delivered unprecedented convenience, and enabled continuity at a time when campuses and test centers were off-limits. As a result, not only did NiSM get the same quality and rigor as an exam conducted at a physical test center, but it also did not have to defer the exam or the future of any participant.

The entire video recording and audit trail available after the examination helped NiSM bring credibility to the overall examination process. The Institutes internal full-time courses that went online with classes had its semester exams dithering. But with the solution in place, even those exams went online. Test-takers well-being and safety were also ensured by enabling them to take the examination from the comfort of their home.

The way forward

Mercer | Mettl’s team and NiSM management plan to complete one lakh proctored assessments by the end of the current year. NiSM has even extended its contract, which means even if the test centers opened, and it resumed in-class learning, it would take the hybrid model of online-offline exams. Some of NISM’s low-stake exams would continue to be delivered using Mercer | Mettl’s system.