If one were to ask what is the most important ingredient in making a technology product or executing a technology project, the prompt answer would be great programmers. That said, good programmers are not easy to come by; any clues as to how would you plan to hire all those talented geeks out there when they are not even interested in your job postings on various job portals. Here’s a hint: programmers love challenges; be it in form of hackathons, codeathons or an assessment that truly measures their coding prowess; throw them a good coding challenge and there you have it – a landslide of coders ready to hack a code! Obviously for you, a great pool of skilled coders ready to be hired.

To simplify your “geek-selection” problem, there are several software development/coder assessment platforms out there doing the rounds on the Internet. But, spotting the right online assessment platform is not as easy as it sounds. As recruiters and technology managers, you need a coding assessment platform that simulates real-world coding environment – the way a software developer will code at work. Code simulator platforms available in the market today fail to create an environment that corresponds to certain or all aspects of a real life coding software. If you take example of pilot training, simulators are used to train the trainee pilot, but the plane simulation is built in such a manner that it emulates the cockpit, air controls etc., as it would be available on a real plane. The idea is to bring training as close to reality as possible, so when a trainee/intern, fresher or experienced hire is hired he already knows what platform and tools he has to use and how.

Our product and engineering folks have combined a few business and technical considerations to be kept in mind while selecting your platform for running coding assessments for recruitment or training / learning effectiveness; we have seen that these considerations are typically missed in the hype and would want you to pay due attention to all these subtle points;


Coding Question Support depth


1) Can you create your own questions based on your own requirements

  • Should the question creator be very technical to create questions.
  • Are you free to you use your own data structures while creating questions; for e.g., if only STDIN is available, the question creator is forced to use data structures that are easily converted from a String

2) Can the platform support testing of front-end coding skills

  • Is there scope for languages like HTML5, CSS3, JavaScript etc.?

3) Can real-life problems be created that involve code-snippets integrated with a database

  • Is Read/Write from/to a database supported in the platform using industry standard ways

4) Does the interface have support for multiple classes and files

5) Is integration with standard IDEs like Eclipse and Visual Studio supported

  • This helps the candidate quickly adapt to a new problem solving environment

6) Does the offering have “project” support

  • Can custom source code be attached with a question for reference / use by a candidate
  • Does the system allow multiple classes to be included along with the template from which a candidate starts attempting the problem

7) How many data types are supported as input and output arguments when generating a code snippet for the required function

  • Are common data types like long, float, char, byte and their equivalent wrapper classes available;
  • How many array types are supported
  • Are common collections like Map, Set, List available

8) Is it possible to attach additional resources like a text / xml file with the problem

  • This helps the problem creator to have more leeway in designing problems for the candidates

9) Does the candidate need to write unnecessary boiler plate code to read inputs when a template hasn’t been specified

  • For example, given a map of employee id and their names, how easy is it to sort them on the basis of employee names

10) Is the candidate handicapped or are tools available for a developer under normal circumstances provided to the candidate while attempting a coding assessment

  • Is Auto completion of code context sensitive and does it work in all scenarios

11) How many value added services are provided through the platform for grading

  • Is feedback of the candidate coding style and static code analysis available
  • Are customized grading options available

12) Can the candidate while attempting a question check against any arbitrary input to ensure that the generated code passes checks that she wants it to pass

13) Are standard unit testing frameworks available

  • For eg, jUnit in Java

14) Is support for common frameworks of a programming language available

  • For e.g., is it possible for a candidate to write code involving WCF in C# or Spring enabled solution with Java
  • How many languages are supported by the platform

15) Does the platform support reusability

  • Are coding questions re-usable; ideally they should be outside the scope of an assessment and should be possible to combine a question in different assessments
  • What is the depth of sample content available for a test author to experiment

16) Hygiene elements

  • Support for CTRL+Z to delete the last typed word
  • Line numbers starting from a logical number instead of an arbitrary number because of the hidden template
  • Availability of common shortcuts used in popular IDE like Eclipse or Visual Studio

Creating Assessments from Questions


1) Does the platform have the capability to logically group multiple questions in a section (which may be timed)

  • In addition to coding questions, can the platform help the test author combine other question types like Multiple choice questions (MCQ) and other skills like psychometric skills to get a full picture
  • Does the platform support rich questions with media elements like audio / video

2) Does the platform account for all scenarios of test completion

  • In addition to normal submission or time expiry, are events like Window closure detected

3) Proctoring support – how easy is it to prevent cheating and plagiarism even when the test is being taken remotely

  • Are features like automatic face matching, multiple face detection and browser locking available
  • Does the platform allow a test administrator to remotely finish a particular candidate’s test
  • What are the other tools available to the test administrator when an unfair advantage for a particular candidate is detected – live chat, live video capture, live screen capture

4) Can a test author collect customized information from a candidate before the start of the test in specialized text, yes/no and date input fields.

5) What happens in case of network disruption

  • If the candidate is unaware, the test continues to proceed and all the time spent is eventually lost

6) Access control

  • How strong is access control and what are the rules that you can impose on other team members using the same account
  • Can you restrict access based on various features, modules and data to other people in the organisation

7) Are the assessments editable entirely at any stage of their life cycle

  • This might be double edged, as on one side you get flexibility and on the other side there is potential that reports across candidates might not be totally accurate

8) API

  • How extensive are the APIs; can it pass data to a remote system when the test concludes
  • Can it be integrated in a low cost manner to an existing system


These are the broad technical and business specifications that you as a hiring manager, training manager or a senior developer should keep in mind before investing in a technology that is aimed at hiring the better of the “technical world”.

Hope that you find it useful. Think we have missed something important? Comment and let us know, we will try to help you. If you are looking for a solution on the similar lines do checkout Mettl’s online coding assessment platform.