Wednesday, August 3. 2005Automated Grading is Not Automated Assessment
If I find another person claiming their MCQ tester offers fully automated assessment, I will scream. It's similar to the idiots who claim that unit testing is "Fully Automated Testing". Its just wrong. Automate: To convert to automatic operation. The easiest part of multiple choice tests, is marking them. The easiest choice of any limited input/single correct answer exam is the grading. If you don't believe me, here is a challenge: "Write me a multiple choice question that you believe could only be answered by someone who has an indepth knowledge of X" (X can be C#, PHP, The Rolling Stones back catalogue, whatever). Time yourself doing this. Then ask a friend the question and time yourself marking it. There is a massive difference. If you could have a program that would either write the questions, or mark the answers which would you take? (Incidentally, please leave any such questions you think of in the comments below, I'd like to see what stuff people come up with.) Automated grading is pretty handy, don't get me wrong. But its not really impressive. Its just a program that matches input against a list, the technology has been around since 1959. These days a decent automated grader should tell you things like "Which was the hardest question" , "What is the most common misconception/i.e. the most chosen incorrect answer" and the like. If it doesn't do that, then its no better than 1960s tech
Trackbacks
Trackback specific URI for this entry
No Trackbacks
Comments
Display comments as
(Linear | Threaded)
that's brilliant. really true. writing the questions is the hard part -- and yet it is quite automatable.
writing the code snippets would become the tricky part. and hey, writing code snippets is *fun*. good stuff.
Indeed, writing code snippets is good fun. But writing a system that generates them is even more fun
 Thats what I did! |
About:Switch to Dark on Light!
This website is the online diary of me, Des Traynor, a User Experience Researcher in Dublin, Ireland. I work with Contrast. I usually write on 5 topics: I update about 3-4 times per month. Be sure to subscribe so you don't miss this good stuff. If this is your first time here, check out the archives.My official homepage provides more information about who I am, and what I research. You can contact me at destraynor [at] gmail [dot] com Quicksearch |
{document.getElementById(%22switcher%22).style.display=%22none%22;document.body.style.background=%22#fcfcfc%22;document.body.style.color=%22#333%22;document.getElementById(%22blogbanner%22).src=%22http://www.destraynor.com/serendipity/templates/destraynor/header-alt.png%22;function%20changeSizeColor(tag,size,%20color){var%20elements%20=%20document.getElementsByTagName(tag);for%20(var%20i=0;i<elements.length;i++){elements[i].style.fontSize=size;elements[i].style.color=color;}}var%20all%20=%20document.getElementsByTagName(%22*%22);for(var%20i=0;i<all.length;i++){all[i].style.background%20=%20%22white%22;all[i].style.color%20=%20%22#444%22;}changeSizeColor(%22h1%22,%2232px%22,%22#333%22);changeSizeColor(%22h2%22,%2222px%22,%22#222%22);changeSizeColor(%22h3%22,%2220px%22,%22#111%22);changeSizeColor(%22p%22,%2216px%22,%22#444%22);changeSizeColor(%22a%22,%22%22,%22#cc0000%22);var%20paragraphs%20=%20document.getElementsByTagName(%22p%22);for(var%20i=0;i<paragraphs.length;i++){paragraphs[i].style.lineHeight=%22150%%22;paragraphs[i].style.fontFamily=%22Arial,Helvetica,Verdana,sans-serif%22;}})()){kind=link}