Highly Anticipated Adaptive Test Unveiled!

At last, as discussed in our column on the same subject which appeared two years ago, the adaptive version of the placement test in English as a Second Language (eTCALS) is now available and ready to replace the previous version.

The story begins on Aug. 26, 2008 when the Ministère de l’Éducation, du Loisir et du Sport (MELS) launched a call for tenders on the modernization of the placement test in English as a Second Language. This call was to grant a mandate to computerize the TCALS 98, a placement test for ESL developed in the late Nineties. A few weeks later, on October 14, MELS informed the CCDMD that it had been selected for this project.

Placement test in English as a Second Language (eTCALS)

The CCDMD then formed a team of specialists to produce an adaptive test that could be renewed from year to year. This project involved a number of challenges. While the online test was in itself relatively simple from the viewpoint of programming, the real challenge was to update the test and to make it adaptive.

What is an adaptive test, you ask! Adaptive tests take the responses of their subjects into account, evaluating them in order to be able to quickly locate skill level by providing precisely targeted questions. Adaptive tests are used primarily to evaluate skills such as the mastery of a language. They can also offer a wide variety of questions each time the test is given. For example, each subject is evaluated by answering questions that are different from those of other subjects tested, and each test has a number of different questions depending on the skill level of the subjects under evaluation.

This feature, the adaptive aspect of the test, required the development of quite complex mathematical and programming algorithms to ensure optimal operation. After two years of reflection, analysis, discussion and planning, the job was done. The adaptive eTCALS will be available at the beginning of May 2011.

Another advantage of the adaptive test is that it will be possible to add questions to the database at any time. This year, for example, we will insert a hundred questions to be categorized by the test in the coming months and progressively integrated into it.

Last but not least, while developing the test, we had to make sure that this new version could classify students the same way as the previous unadaptive version did. We therefore did a dry run at three colleges. Rimouski, Bois de Boulogne and Valleyfield each gave us the names of 25 students from various English courses to participate in a comparative assessment between the former standard test and the new adaptive test, one after the another. To the delight of the entire development team, the students were classified in the same way in both tests! Although we knew we would succeed in the end, we are nonetheless extremely proud of this product and those who designed it.

One can imagine that although complex, the advantages of having produced an adaptive test are numerous. For one thing, the length of the testing process is reduced. In the coming weeks, we will start development of a placement test for French second language that could potentially be used in the anglophone colleges by students who must take French courses. And who knows where this path may lead! It may be possible soon to develop such a test to measure the skills of our students in any number of subjects.