Paper-based coding/software engineering test importance
The importance of paper-based coding test.
100% yes. In beginning courses, it’s practically a cognitive requirement. Let me see if I can break down for you why it is so important. It comes down to what we can know that the student knows.
Let’s say we have a student who is a rather mediocre coder and who, to be honest, really has no idea what is going on in class. We’re going to give him a problem in Java, which is to go through an array of ints, add all of the numbers in said array, and return the result. We're also going to let him use an IDE to build it, so that he can find syntax errors and spot-check himself for silly errors by running the code. The first thing he writes is this:
for(z=array[0], z < array, z+1){
z=array[0];
}So, how much can we say that he actually knows? I can forgive that he used commas in his for loop. I might even, depending on his background, forgive that he forgot to declare z. But what can be said about the fact that he set z to array[0] instead of to 0? Or that he is comparing z to array instead of array.length(), or that he is never actually changing z in his z+1 statement? These are mistakes of comprehension, not syntax. What, also, can I say about the fact that he keeps setting z to array[0] over and over again?
The fact is that, if he writes this into an IDE, a few things will happen. A bunch of code will light up red. A red line will remind him that he forgot to make a return. Another red line will tell him that z < array is not possible, because they are of different types. So, he will start typing things until the red lines go away. Does he necessarily understand the things he is changing, or has he been trained, Pavlovian-like, to change code that has been underlined in red?
As he makes some red lines disappear, new red lines pop up. He then turns his attention to those until he makes all the red disappear, and now he has this:
int g = 0;
for(int z=array[0]; z<array.length; z=z+1){
g=array[0];
}
return g;This code is much, much better. It’s still not perfect. But, given the flaws in his first draft, do we have any reason to believe that he really understands this new version and why it is actually better? Or is he just changing code until the red goes away?
Now, he runs it, and it immediately ends by returning the value 0. So, since he still has about 5 minutes left to his exam time, he starts just changing things at random. He changes his int z to be = 0. Now he starts getting a number returned! He tries putting g into g=array[0], then he puts z into it. Now he has g=array[z].
He dithers about a bit, and finally adds a + in there. He now has the correct code:
int g = 0;
for(int z=0; z<array.length; z=z+1){
g+=array[z];
}
return g;He turns it in to the teacher, who says, “Boy, he sure understood this! That’s one student that I don’t need to give any extra attention to, because he is in great shape in my class!”
The takeaway questions then are:
- How well does the student understand the material?
- How well does the student’s grade reflect that?
- How much worse are the odds that the student will now receive the extra help that he so desperately needs?
We need tests that actually show what you really know, which is not always the same thing as what you can produce with prompting help from an IDE.
54
100% yes. In beginning courses, it’s practically a cognitive requirement. Let me see if I can break down for you why it is so important. It comes down to what we can know that the student knows.
Let’s say we have a student who is a rather mediocre coder and who, to be honest, really has no idea what is going on in class. We’re going to give him a problem in Java, which is to go through an array of ints, add all of the numbers in said array, and return the result. We're also going to let him use an IDE to build it, so that he can find syntax errors and spot-check himself for silly errors by running the code. The first thing he writes is this:
for(z=array[0], z < array, z+1){
z=array[0];
}So, how much can we say that he actually knows? I can forgive that he used commas in his for loop. I might even, depending on his background, forgive that he forgot to declare z. But what can be said about the fact that he set z to array[0] instead of to 0? Or that he is comparing z to array instead of array.length(), or that he is never actually changing z in his z+1 statement? These are mistakes of comprehension, not syntax. What, also, can I say about the fact that he keeps setting z to array[0] over and over again?
The fact is that, if he writes this into an IDE, a few things will happen. A bunch of code will light up red. A red line will remind him that he forgot to make a return. Another red line will tell him that z < array is not possible, because they are of different types. So, he will start typing things until the red lines go away. Does he necessarily understand the things he is changing, or has he been trained, Pavlovian-like, to change code that has been underlined in red?
As he makes some red lines disappear, new red lines pop up. He then turns his attention to those until he makes all the red disappear, and now he has this:
int g = 0;
for(int z=array[0]; z<array.length; z=z+1){
g=array[0];
}
return g;This code is much, much better. It’s still not perfect. But, given the flaws in his first draft, do we have any reason to believe that he really understands this new version and why it is actually better? Or is he just changing code until the red goes away?
Now, he runs it, and it immediately ends by returning the value 0. So, since he still has about 5 minutes left to his exam time, he starts just changing things at random. He changes his int z to be = 0. Now he starts getting a number returned! He tries putting g into g=array[0], then he puts z into it. Now he has g=array[z].
He dithers about a bit, and finally adds a + in there. He now has the correct code:
int g = 0;
for(int z=0; z<array.length; z=z+1){
g+=array[z];
}
return g;He turns it in to the teacher, who says, “Boy, he sure understood this! That’s one student that I don’t need to give any extra attention to, because he is in great shape in my class!”
The takeaway questions then are:
- How well does the student understand the material?
- How well does the student’s grade reflect that?
- How much worse are the odds that the student will now receive the extra help that he so desperately needs?
We need tests that actually show what you really know, which is not always the same thing as what you can produce with prompting help from an IDE.
