Carole H. McAllister
The word processor has quickly become a valuable tool not only in the teaching of writing, but in the process of writing itself. More and more students are composing their writing assignments on word processors, in computer labs and at home, and recent research has documented the positive effects on both student attitude towards writing and the quality of writing produced (Daiute, 1983; Schwartz, 1984).
We English teachers fortunate enough to have our students writing on word processors want to believe that the ease of writing the students experience and their accompanying attitude change toward writing and revision will result in improved texts. We are excited by the possibilities of this new technique. But does our enthusiasm generate assumptions about quality that have not yet been established? Does the knowledge that a student's paper was produced on a word processor have any effect on us as evaluators? Is it possible that there is an illusionary effect which occurs in the eye of the beholder?
Simply put, the knowledge that a composition was composed on a word processor might alter the perception of its quality. This phenomenon might be due to inferences we make about the writing of the paper (e.g., it must have been rewritten more than usual), or about the student writer (e.g., the student must have a lot of initiative to have learned how to use a word processor). The purpose of the present experiment was to test whether the instrument of composition (word processor, typewriter, hand), influenced teachers' perception of quality.
Thirty university composition teachers (full-time as well as part-time members of the English faculty) at Southeastern Louisiana University served as subjects.
Each subject received a packet containing instructions, the same student essay, and rating scales. The instructions explained that the essay was one of their own students' papers. The essay, a beginning, first-year level theme, had no major errors and had originally been graded "C" by a fellow faculty member, a non-participant in this experiment.
One third of the subjects received handwritten versions of the essay, one third
received typewritten versions, and one third received word-processed versions. (Subjects were randomly assigned to these different conditions.) Although the method of composing was clear from the physical appearance of the essay, the subject' instructions referred to the essay as handwritten, typewritten, or written on a word processor to underscore the method of composition. Following the grading of the essay, subjects were asked to respond to semantic differential questions concerning their perceptions of both the essay and its composer. Sample items selected to measure subjects' perceptions of the essay included: objective-subjunctive, clear-confusing, successful-unsuccessful, and unique-typical. Items chosen to measure subjects' perceptions of the composer included: dull-sharp, logical-intuitive, usual-unusual, and masculine-feminine. Subjects were asked to rate the student and the essay on the various seven-point semantic differential scales. (See Appendix)
Grades were converted to a fifteen point scale ranging from 1 (F-) to 15 (A+). A one way ANOVA performed on these scores was significant F (2,27) = 3.72, p<.O37. Inspection of the means revealed that graders awarded the highest grades to the paper thought to be composed on the word processor, the next highest to the handwritten paper, and the lowest grades to the typed paper (Table 1).
One way ANOVA's on the semantic differential scales revealed no significant differences.Mean Grades as a Function or Instrument of Composition --------------------------------------- Condition n Grade --------------------------------------- Word processor 10 6.45 Typed 10 4.14 Handwritten 10 5.33 Note Scores range from 1 (F-) to 15 (A+) ---------------------------------------
As predicted, papers thought to be written on word processors received significantly higher grades than did papers thought to be handwritten or typed. Why, then, do we have this illusionary effect of the same paper perceived to be of a higher quality when written on a word processor? The fact that this experiment included the typewritten version of the same essay rules out simple explanations such as the ease of reading the paper. Apparently, the belief that the paper was written on a word processor did, in fact,
alter the perception of its quality. Unfortunately, attempts to explain this effect in terms of inferences the subjects made either about the writing of the paper (e.g., it must have been rewritten more than usual) or about the student composer (e.g., he or she had the initiative to learn to use a word processor) proved unsuccessful. The semantic differential questionnaire showed no significant differences among the three groups (word processed, typewritten, handwritten)--neither in the subjects' perceptions of the composer nor in their perceptions of the essay itself. However, many of the subjects did not choose to use the full range of the seven-point semantic-differential scale. In fact, thirty percent of the subjects did not really use the scale, but chose to invoke extreme values. Their judging both essay and composer as either all black or all white, i.e., their failure to evaluate in shades of gray, could be masking significant differences between the three groups.
If their perceptions revealed no significant differences, then we need to focus on the subjects themselves and their response to both the technique of word processing and their role in the experiment. Did the subjects consciously or unconsciously pick up on the "demand characteristics" and give the investigator what she was looking for, i.e., "this essay was written on a word processor, and I am probably supposed to give this a better grade?" The effects of demand characteristics have been widely demonstrated in psychological research (Orne, 1969).
Another possible explanation might lie in the subjects' concern with the image they were presenting to the investigator. If word processing is being saluted as THE technique of the moment, then perhaps the subjects would want to appear equally accepting of this technique, "with it." Consciously or unconsciously, they might have rated the word-processed paper higher. Again the psychological literature shows such evaluation apprehension effects to occur often in experiments (Rosenberg, 1969). Thus, the subjects' awareness of the investigator, their concern for how the investigator might be judging their reactions, might well play a strong part in the significance or insignificance of the data. While these arguments might be simply an artifact of this experiment and not seem relevant to the actual classroom situation, they still remain important considerations for those conducting research on the effects of word processing.
Even another consideration stemming from subjects' awareness of investigator presence relates to how they graded the essays. If these subjects thought they themselves were being evaluated as paper graders, would they grade any more stringently or carefully than they would normally? I raise this question since there seemed to be a large number of poor grades given the essay, more evidence of the subjects' extreme responses.
Indeed, the subjects' awareness of investigator presence is strongly evident in the number of subjects who felt compelled to comment (unsolicited and defensively) on the validity of the semantic differential questionnaire. Some typical responses, written next to the items,
include: masculine-feminine-"not valid"; logical-intuitive, objective-subjective--"carries neither of these qualities to the extreme," or "illogical, not compatible terms"; active-passive, quick-slow--"unclear" or "not applicable." Further, even though the subjects were asked to grade the essay as though it were a first-year theme, some addressed their comments on the essay itself to the investigator, rather than to the student who supposedly composed it, even addressing the investigator by name.
Finally, in choosing English teachers as subjects for experiments, one must be prepared for attitudes not always favorable to the scientific method. One particular subject's remarks (written in response to the semantic differential scale) reflect this skepticism.
"These are only the misgivings of the poor responder to objective questionnaires, who knows the agony of having to make grids from which the complexity of actual perception and life have evaporated. Blessed is the serene and logical objectivity of the collector of such data who, tranquilly unconscious of such agonies, manipulates numbers as if they were living people . . . Bogged in the mystery of the ineffable and the inscrutable as I return this to you."The present findings are important not only to those who do research on the use of the word processor in the teaching of writing, but to those who use English teachers as subjects. Studies that compare compositions written on word processors with those written by other means should take into account the illusionary effects on quality that come from subjects' knowledge that the writer used a word processor. In future
research, investigators might try keeping subjects blind to the original instrument of composition in order to assess more accurately the impact of word processing on the quality of writing.
CAROLE H. McALLISTER teaches at Southern Louisiana University.
Daiute, C. (1983). The computer as stylus and audience. College Composition and Communication. 34, 134-145.
Orne, M. T. (1969). Demand characteristics and the concept of quasi-control. In R. Rosenthal and R. Rosnow (Eds.), Artifact in behavioral research (pp. 143-179). NY: Academic Press.
Rosenberg, M. J. (1969). The conditions and consequences of evaluation apprehension. In R. Rosenthal & R. Rosnow (Eds.), Artifact in behavioral research (pp. 279-349). NY: Academic Press.
Schwartz, H. J. (1984). Teaching writing with computer aids. College English, 46, 239-247.
Semantic Differential Questionnaire
Rate the student who wrote the essay on the word processor on the following scales. Be sure to circle the appropriate number for each scale.
individualistic 1 2 3 4 5 6 7 regular active 1 2 3 4 5 6 7 passive quick 1 2 3 4 5 6 7 slow busy 1 2 3 4 5 6 7 lazy dull 1 2 3 4 5 6 7 sharp logical 1 2 3 4 5 6 7 intuitive objective 1 2 3 4 5 6 7 subjective rational 1 2 3 4 5 6 7 irrational unique 1 2 3 4 5 6 7 typical usual 1 2 3 4 5 6 7 unusual masculine 1 2 3 4 5 6 7 feminine
Rate the essay written on the word processor on the following scales. Be sure to circle the appropriate number for each scale.
objective 1 2 3 4 5 6 7 subjective usual 1 2 3 4 5 6 7 unusual simple 1 2 3 4 5 6 7 complex clear 1 2 3 4 5 6 7 confusing good 1 2 3 4 5 6 7 bad successful 1 2 3 4 5 6 7 unsuccessful valuable 1 2 3 4 5 6 7 worthless logical 1 2 3 4 5 6 7 intuitive rational 1 2 3 4 5 6 7 irrational individualistic 1 2 3 4 5 6 7 regular unique 1 2 3 4 5 6 7 typical