It seems that I have touched a nerve. Since "Student Writing: Can the Machine Maim the Message?" was published in Academic Computing in January 1990, I have received well over 100 letters, and friends have sent me printouts of the firestorm of electronic mail that flared in response to the piece.
The mix of correspondence has been quite interesting, with heavy
support for my position from cognitive psychologists and human-computer
interaction specialists. But there has also been a steady stream
of criticism of the type represented by the letter from Megabyte
University and the essay from Kaplan and Moulthrop. I am glad
to have an opportunity to respond to this criticism.
My own experience of more than ten years with students in remedial
sections of first-year English, with students in standard first-year
classes, and with international students has shown me that, used
with care, WRITER'S WORKBENCH can be a fair indicator of what
human teachers will like, that is, find acceptable. Granted, if
you want to "trick" WRITER'S WORKBENCH (or any text-analysis
program), you can, knowing that WRITER'S WORKBENCH is stupid,
and that it can only count endlessly and spit out numbers; you
can outsmart it gleefully by giving it a task it was never designed
to do, for example, analyzing narration or description. But as
far as exposition is concerned, in a study published in Computers
and Composition, Reid and Findlay (1986) showed a high
correlation of sentence length, word length, and readability scores
with holistic scores given to expository essays by human graders.
At the college level, teachers are accustomed to seeing average
sentence lengths of more than 20 words and sentence complexity
commensurate with that length. Indeed, at one point in its history,
back in its own Dark Ages, perhaps, the National Council of Teachers
of English announced that the ability to write complex sentences
is a measure of students' readiness to do college-level work.
So I believe that WRITER'S WORKBENCH, even with its limitations,
can give reasonable indications about the level of students' writing.
In fact, as some correspondents have noted, even if some individuals
are reluctant to accept WRITER'S WORKBENCH's statistics as "intelligent,"
it is still statistically significant that the scores produced
by Macintosh writers and those produced by IBM writers were so
different. Such a difference indicates that something is going
on here--something worth looking at.
The answer to the question raised in the Megabyte University letter
about the identical topics used by two Macintosh writers is quite
simple: One of the essays that students read from the textbook
for the course (The Prentice Hall Reader, edited by G.
Miller) suggested an essay topic on a popular-culture phenomenon
such as fast foods: Therefore, several students decided to use
it. Actually, several of the topics used by both groups of students
came from suggestions in The Prentice Hall Reader, including
the ones on nuclear war and drunk driving. The point is, no matter
where they got the ideas for their topics, students writing on
Macintosh computers generally chose different types of topics
than did students writing on IBM computers.
I would also like to explain more about how students selected the computer sections within my original study. A survey I did last semester among students enrolled in computer sections demonstrated that, although the sections had been labelled in the course selection book as either IBM or Macintosh, 75% of the students surveyed said that they had not consciously chosen either one computer or the other: They simply enrolled in a section of computer-supported first-year English because of the time of day that it met or because their friends were enrolled in it. Of the 25% who said they had thought about their selection in terms of computer choice, most gave only the most general reasons for their choice: For example, the Macintosh is user-friendly, or the IBM is used in business and will, therefore, be more useful to them when they graduate.
So the comments that the Megabyte University writers make about
a need for details about race, ethnicity, class affiliation, and
gender do not seem particularly pertinent here. The students in
all sections were a typical University of Delaware mix--55% female,
45% male, mostly white, middle class, with mid-range SAT scores.
Unfortunately, in spite of efforts to change the situation, our
University does not enroll a particularly diverse student body
from many different backgrounds.
In our writing, we do indeed teach writing as a process. We stress
the need for conferencing, peer editing, pre-writing, and revision.
But the papers produced at these various stages in the process
showed such noticeable differences that all of the instructors
queried were disturbed at what they saw. Although it is true that
the examples I used in my article represented Macintosh users'
early drafts, I quoted from them because they were so different
from the IBM users' early drafts. The statistical analysis done
of WRITER'S WORKBENCH, however, was performed on the final drafts
of the essays.
When the course was given (Spring, 1987), students in Writing
Program courses had to use public sites around the campus and
work mostly on their own because the English Department did not,
at that time, have its own site. Fortunately, we now have one,
so teachers are far more involved in coaching computer writing
than in the past. But the point is, students in all sections of
computer-supported first-year English received the same treatment--approximately
90 minutes of in-class training and handouts that they could take
with them for reference. Because all students received the same
amount of training, it seems fair to compare the products of their
labors.
The writers of the responses are correct in stating that I made qualitative observations about students' writing; but, then again, those of us who teach writing do that all the time. That is what we are paid to do! We evaluate, we judge, we grade, and we make judgments about what constitutes "good," that is, acceptable writing for whatever course we are teaching, and what does not constitute acceptable writing. So, in making qualitative observations, I was doing something that I have done in many courses for many years and that all of us do as composition teachers all the time.
Moreover, I train teachers of writing at the University of Delaware.
I run workshops for them on grading, and I, along with others
in the program, have developed grading standards for first-year
English and for various other writing courses. How have we done
that? By reading many essays, by reading articles on holistic
scoring, and by talking at length about what constitutes excellence,
ordinariness, and mediocrity in writing. We cannot claim absolute
objectivity all the time, but then, who can?
Moulthrop and Kaplan make many excellent suggestions about research design, and I will certainly keep them in mind as I move ahead . My published essay, however, never claimed to be a tightly structured study. It certainly is "descriptive" research. But I think it is important for teachers to report what they see happening to writing in their computer classrooms.
When the alphabet came into use, few people were aware of the changes it would bring in thinking (see McLuhan and Powers, 1989; Ong, 1977, 1982; Heim, 1987, 1990; and Costanzo, 1989). But we are in a position to be aware, to be cognizant of changes wrought by technology, to initiate discussion in the profession, and to alert those who design the hardware and software. Not everyone will agree about the future directions that writing and the teaching of writing should take, but the dialogue must be free and open to all to participate in.
Judging from many of the letters I have received from teachers
and researchers who generally agree with my findings, I believe
we can anticipate a continuing, lively debate. "Can the Machine
Maim the Message?" focused on concerns many teachers up to
then had voiced privately; now, those issues have become the subject
of vigorous and open discussion--exactly what I hoped the article
would accomplish.
Marcia Peoples Halio is the Assistant Director
of The Writing Program at the University of Delaware.
Costanzo, W. V. (1989). The electronic text: Learning to write,
read, and reason with computers. Englewood Cliffs, NJ: Education
Technology Publications.
Heim, M. (1987). Electronic language: A philosophical study
of word processing. New Haven: Yale University Press.
Heim, M. (1990). Infomania. In C. Ricks (Ed.), The state of
the language (pp. 300-306). Berkeley: University of California
Press.
McLuhan, M., & Powers, R. (1989). The global village: Transformations
in world life and media in the 21st century. New York: Oxford
University Press.
Ong, W. (1977). Interfaces of the word: Studies in the evolution
of consciousness and culture. Ithaca: Cornell University Press.
Ong, W. (1982). Orality and literacy: The technologizing of
the word. New York: Methuen.
Reid, S., & Findlay, G. (1986). WRITER'S WORKBENCH analysis
of holistically scored essays. Computers and Composition,
3, 6-32.