On Comprehensive Exams

[ad#righthandside-tall]It is undoubtedly the case that most students do not only accept the state of things, but they never even question why things are the way they are. This should perhaps not be loathed, since the answer most students would be likely to get upon questioning is “this is the way it has always been – deal with it.” That’s not much of an answer.

In my program, in contrast to most PhD programs, there has not been a formal and required exam as part of the program. Recently, however, this has changed. The reason for this is that the department has received poor scores in peer reviews and that the dissertations produced by students have not always been of the best quality. Fair enough, poor results should lead to change in order to effectuate improvement.

The reaction by the department is to increase the number of tests students have to take, and this is – as far as I understand – imposed retroactively also on students in the “old” program. Not only will there be a qualifying exam after the student has finished the program’s core course, but there is a paper to be submitted after the second year and a comprehensive exam as part of the dissertation proposal.

This is where I get confused. I fully understand the attempt to “control” that students have indeed understood the materials in the courses before they get to move on and start working on their research. But then, when they have reached the point where they are to defend their dissertation proposal to their committee, does it make sense to have another written test?

I think not.

It is easy to see how the faculty obviously panicked and felt the need to “do something.” Another test is however the wrong way to go. To understand this, let’s have a look at the process leading up to the dissertation defense. This process means the student works alone and/or together with his/her advisor to produce a text introducing the problem, the literature, what value is to be produced, the methods to do so and the expected problems and results.

When one’s advisor finds that the student is ready to defend the proposal and the committee has had a chance to review and comment on the proposal, then the student is invited to orally defend the proposal. The committee will ask questions and make sure the student knows what he’s/she’s doing. That’s fine; after all, the student is supposed to learn how to do research.

But at this point, what does a written test based on the material in the proposal add to the process? This is difficult to understand. It is especially troublesome since the comprehensive exam is a written test with limited time and closed books. How many researchers would you guess study the literature to the point of memorization before they write grant applications and research project plans? Of course they don’t – why would they? It is a waste of time to memorize details of others’ research that may not be relevant. And even to the point it is relevant, the articles and books are always available and need to be reread a thousand times in order to produce a fair analysis.

So “real” researchers don’t do that – but students should?

But maybe there is a reason to test the students – for sake of quality. Sure, this could be the case. Let’s have a look at this argument: students are to write a closed-book, memorization-testing exam on the material on which they are to do research in order for the department/faculty to make sure that the dissertation will be of good quality. Of course, for this to be even close to a real solution we need to assume that memorization of material makes one’s own research better and that the test as constructed by the committee does just this. Also, we need to assume the committee consists of experts in the exact area chosen by the student for research.

But remember the process as discussed briefly above. No student gets to defend his/her dissertation proposal without the consent and approval of the advisor. So if the department has experienced low-quality dissertations the problem is really that advisors don’t take their job seriously and let students defend their dissertation proposals prematurely. (We here ignore the troublesome fact that committees seem to consistently have chosen to approve proposals and dissertations of poor quality.)

So it must be concluded that the comprehensive test, as it is designed in my department, can only be seen as a test of whether the advisor has done his/her job properly. If the student is allowed to defend the dissertation proposal he or she must have already been deemed “ready” to do so by the advisor (and committee). So the student needs to take a comprehensive exam on the basis for his/her own research because the department cannot trust the advisor to do his/her job?

I guess this is where the pseudo-arguments “but everybody else does this” and “this is how it has always been done” are added. After all, without these pseudo-arguments there would be nothing left. The students are forced to literally waste weeks of their lives memorizing details of research they already know in order for the department to know the advisor and committee have done what they are supposed to.

If the problem is that some advisors (I doubt it is even a majority) allow their students to defend their proposals (and dissertations?) prematurely, then it is reasonable to assume the faculty already know who they are. But they are obviously afraid to bring this out in the open, possibly because they are likely to make enemies with people they have to work with for the rest of their lives. After all, academia spells t-e-n-u-r-e and this means life-long service in the same department. So it is a lot easier to have students jump through another hoop for the sake of appearing to assure quality and thereby saving face and avoid making enemies.

So what does this mean? It means, of course, that I will have to spend a lot of time memorizing things I believe my committee members believe is important in the material relevant for my research. It also means a break from my line of thinking and this, in turn, means I will have to start over with the research project – thereby losing even more time. Then I can finally do the research that I am in the department to do in the first place.

For the record, I sincerely doubt my advisor would allow me to defend an unfinished proposal. Instead, he would be frank with me and tell me that my proposal simply doesn’t cut it. After all, I am working with him to produce a document that will be approved by the committee (the advisor usually has the last word) – and if not, then the advisor has obviously not done his job. So this means I am not really affected by the new rules, except for the hoop-jumping part: I will most likely have to take the test. In order for the department to know “for sure” (if there is  such a thing) that my advisor has done his job.

It also means I will have to memorize a bunch of stuff that I am likely to challenge in my own research. I fail to understand how memorizing [at least some] conclusions that I am sure are wrong, or at least drawn on faulty bases, would help me, unless it is the case that I have misunderstood the articles and that another read will help me understand this mistake (both conditions are, of course, necessary for this to not be a complete waste of time).

Does this help my research? I sincerely doubt it. Does it potentially improve the quality of my future dissertation? Not a chance. The reason for this is that what is important in a PhD student’s research is not whether articles have been memorized, but whether (1) he or she can think (which is undoubtedly a scarce quality in academia…) and (2) if he or she has received proper guidance from his/her advisor.

The former is not dependent on a formal test such as the now offered comprehensive exam. In fact, if the student can think intellectually a dumb memorization-based test is likely to either bore the student (hopefully not to such a degree that he/she decides there are more worthwhile activities to spend lifetime on) or dull his/her mind.

The latter is also independent on any formal test of the student, since guidance is rarely testable and what is important is finding and understanding the literature. Both should be easy to identify in a proposal and oral defense, whereas they are both quite impossible to test in a formal written exam.

So why enforce a comprehensive exam, especially if in addition to a qualifying exam and a required two-year research paper? I have, I believe, suggested the true reasons above. But it must be concluded that since there is no logical reason to test whether a student has memorized the existing literature, especially since what is of importance is the judgment of primarily the advisor and secondarily the committee, there is no real reason to have such a test.

Sometimes the world isn’t very rational. Or, perhaps, in a tug-o-war world where one’s turf is everything one has, the only thing one can do is jump through the right hoops and hope for the best.

But please do prove that this disillusioned student is but a little confused.