Carel J. M. Jansen, Michael F. Steehouder, Albert Pilot, Dick
Schrauwen, and Paul J. M. Looijmans
"They write so badly; they are barely capable of putting their thoughts down on paper; even letters of application contain grammatical and spelling errors": these are some of the complaints one can frequently hear about Dutch students' language skills. Writing instruction should be made much more effective, as is pointed out to Dutch writing teachers in Postsecondary Education--not only by government and industry, but also by fellow teachers. This request is clear and justified enough, but how can any hard-working language teacher comply with it?
The classical approach of composition instruction is simple: teachers explain roughly what features characterize a good text, emphasizing the importance of correct formulation and faultless spelling, and once or twice a year they make the students write a short essay. This essay is then assigned a grade and a simple commentary, and both teachers and students trust that the next essay will automatically be better than this one.
p. 33
It becomes more and more evident, though, that this approach is not a satisfactory one. Writing is a complex process, and teachers who want students to improve on this process cannot confine themselves to product-oriented instructions and the didactics of the red pencil. Good writing instruction is process oriented. Students should be given a realistic idea of the distinct phases that have to be passed through iteratively (see, among others, Frederiksen and Dominic, 1981; Flower, 1981), and they have to be instructed exactly what to do in order to arrive at an acceptable product, starting from task specification.
Fortunately, in Dutch Postsecondary Education a similar form of writing instruction is becoming increasingly popular. In the majority of schools, the handbook Leren Communiceren (Learning to Communicate, Steehouder et al.., 1984) is being used; in this text, heuristics are presented for students who will perform communicative tasks. Yet, the process-oriented instructions are in themselves not sufficient. Students can only learn how to write well if they are given the chance to practice writing. In doing so, they should also be informed precisely about the progress they are making. As in all other forms of proper skills education, effective writing instruction presupposes not only adequate process-oriented instruction and useful exercises, but also high-quality feedback.
An important problem Dutch teachers share with their counterparts in other countries is
p. 34
the lack of time for teachers to supply high-quality feedback. However good their intentions, these teachers are often forced to confine themselves to some general comment, which each student suspects is meant for someone else. The number of students that have to be assisted is too large to provide everyone with the comments that would most adequately fit their writing achievements.
We have explored the possibility of using existing text-analysis programs such as EPISTLE (recently renamed CRITIQUE) and WRITER'S WORKBENCH to provide the solution to this problem. These programs generate "automatic" information on text characteristics such as mean word and sentence length, percentage of passive sentences, and number of nominalizations. They also point out mistakes such as those in "The Harrison contract was written by Bob, Lee, and I," and "We will accept the funds, send the receipts to the payers and crediting their accounts" (see, among others, Macdonald et al.., 1982; Heidorn et al.., 1982).
Yet, like many teachers, we have concluded that this is not the most effective approach to the feedback problem; programs like EPISTLE and WRITER'S WORKBENCH have a fundamental shortcoming: they do not understand the meaning of the text. Consequently, they lack the intelligence to take over the most important feedback tasks of the teacher. In the comments of most teachers, notions like document structure, selection and handling of information, and tone play an important part. The text feedback generated by a modern computer program, on the other hand, does not exceed the level of a elemental and superficial style
p. 35
analysis. In view of the current state of the art in artificial intelligence, it is unrealistic to expect that text-control programs for writing instruction will have much more to offer (see Wresch, 1984).
Even though we have to admit that computer programs have limitations, we are also aware of the help computers can give in commenting on students' texts. The key to a successful relationship between instructor, student, and program lies in the proper division of labor: the teacher takes care of intelligent tasks, the computer takes care of menial ones. The teacher determines which student will be given which comment on which text. The computer's task is to print feedback on texts and organize students' performances. However obvious this division of tasks may be, we know of only a few programs structured this way: RSVP, CAMELOT (see, among others, Anandam et al.., 1980; Anandam, 1983; Camelot, 1982), WRITER/GRADER/READER (see Marling, 1983), and REPORT (see Marshall, 1985). In the same period, a feedback-supporting program package has been developed in the Netherlands as well, in joint participation between Nijenrode, the Netherlands School of Business, and Twente University of Technology, by the name of ALEXIS (also spelled ALECSYS: an All-purpose Learner-oriented Efficiency-increasing Commentary SYStem).
ALEXIS (2) helps teachers comment on the products of their students. The system translates simple codes introduced by the teacher into feedback texts which inform the
p. 36
student what is wrong with the text, and/or what should be done to improve the next paper. ALEXIS consists of four subprograms. With "Prolex" (the production program), feedback text can be created, and if desired, changed. "Dislex" (the distribution program) takes care of printing an orderly copy for the student. "Curlex" (the course-member's program) administrates the activities and achievements of the students. The central element of ALEXIS, though, is "Sellex" (the feedback-selection program).
"Sellex" allows the teacher to assign commentary to each student, along with the line numbers in the student's text to which the commentary applies. It is very important that the teachers, just as in an individual teacher-student discussion, can differentiate their feedback. Depending on a variety of factors (graveness of the error, frequency, feedback supplied earlier), instructors should be able to choose from a great diversity of commentaries. "Sellex" provides for this choice, as it allows for broad and in-depth differentiation of feedback.
Whenever the teacher finds a shortcoming in a student's text, he or she has the choice from five possible commentaries (and combinations of such commentaries):
p. 37
0.1.4 In this passage you describe an instruction for a series of actions. Your description is too concise: the reader is not clearly informed about what exactly he or she should do. If you want to make sure that your public, with the help of your text, is able to carry out the required action(s) faultlessly, you must give all the necessary information.
0.1.4 Give more about the series of actions described in this passage. A good starting point is the standard action structure: What is the purpose of the action? What conditions need to be satisfied? What is the broad outline of the proceeding? How are the sub-actions carried out? How is the proceeding checked?
0.1.4 In chapter 4 of Leren Communiceren, six standard structures are discussed which can serve as starting points in text production. It would be a good idea if you would read again section 4.1.4, which deals in more detail with the standard action structure.
0.1.4 In section 7.6 of Leren Communiceren you will find three exercises in constructing a plan on the basis of standard structure. Find out how these exercises will improve your skills in handling standard structures.
"Sellex" enables the instructor to make "in-depth" differentiations as well. The feedback he or she gives may have different levels of specificity. When instructors find a student trying to deal with two questions in a single paragraph, they can give a highly specific commentary at the level S.2.3.1 ("arrangement error: more than one question to the paragraph"). They can also select the somewhat less-specific feedback level S.2.3 ("arrangement error: inadequate paragraph arrangement"). Even less specific is feedback level S.2 ("arrangement error"). The least specific commentary is generated when the
p. 39
instructor merely selects S ("structure is faulty").
Some simple calculations tell us that the broad and in-depth differentiations in "Sellex" enable the teacher to react in dozens of different ways to the same error. But which way is best? An error description at the most specific level, an error description and corrective advice at a somewhat less specific level, or merely rough study advice? To answer these questions, no unambiguous, detailed prescriptions can be given, no more than in the "classical" individual teacher-student discussion. It is clear, though, that the feedback decisions of teachers are better accounted for as their insight into the structure of their feedback files and the educational history of their students increases.
The commentary texts in ALEXIS have been hierarchically arranged in order to give the teacher the greatest possible insight into the feedback file. The arrangement of the subject matter in Leren Communiceren has served as a starting point, and, whenever practice is required, the file has been supplemented with error categories that did not figure in Leren Communiceren, but did turn up in students' products.
At the moment, ALEXIS contains around 1,100 commentary texts. To enable efficient searching in this large database, several search and selection facilities are provided. The teachers can type the index number of a text, or may ask for a part of the list of error names, from which they can make a choice. Another possibility is to type a substring of the error name the teacher is looking for. If more error
p. 40
names match such a substring, they all are displayed on the screen, and the teacher can select the right one.
In order to inform teachers adequately about the educational history of their students, ALEXIS contains a feature enabling the teacher to request this history at any stage of the text commenting. ALEXIS then supplies a survey of the errors the student made in earlier assignments, along with the relevant commentary given on those occasions.
After obtaining positive results at Nijenrode with a predecessor two years ago, the current version of the program was applied for the first time at both Nijenrode and Twente University last year. Again, both staff and students reacted in a positive way. The students mentioned as strong points the selectivity and the informative value of the feedback; whereas the staff appreciated the promptness of the system and the ample differentiation possibilities. There has not yet been a detailed study of the effects of ALEXIS feedback on the learning achievements of the students, but it is beyond doubt that both parties have a high opinion of its learning efficacy.
This by no means implies that ALEXIS is not susceptible to improvement. In the first place, the wording of the feedback texts needs more attention. An error description which appeared utterly unambiguous during the development of ALEXIS sometimes turns out to be misunderstood by students, and what seemed to be obvious corrective advice in the preparation stage
p. 41
turned out to be hard to execute in practice. Also, the need has turned up for a modest facility to supply some extra feedback that is not (and does not need to be) integrated in the system, to individual students in incidental cases.
Furthermore, a manual is needed to assist the instructors in determining the quantity and class of commentary they wish to supply. Anandam et al. (1979) suggest that no more than five commentaries be given at a time, so students aren't overwhelmed. This seems sound advice. We would like to add that in supplying feedback, the final objective of the course should be kept in mind. Ideally, students learn to determine for themselves what the strong and weak points of their texts are; ideally, instructors make their advice redundant as the course proceeds. We have already asserted that detailed prescriptions for feedback supply are hard to give. Generally speaking, though, instructors can be advised to make the feedback broader and deeper in the beginning of the course than at the end. However, more research is needed--and will be conducted--to develop useful guidelines for teachers to supply feedback using ALEXIS.
In the fall of 1986 a new version of ALEXIS, improved as described above, will be installed in a number of institutions for Postsecondary Education in the Netherlands that are prepared to experiment with ALEXIS. In these institutions, research will be carried out to evaluate the effects of ALEXIS. After the results of this research have been analyzed and
p. 42
the software, where necessary, has been adapted, a commercial edition of ALEXIS can be released. (3)
Finally, we do not mean to suggest that ALEXIS is merely suitable as a feedback program for postsecondary Dutch-language writing instruction. ALEXIS may be applied in any curriculum in which the teachers wish to comment on their students' achievements. "Prolex" allows any conceivable modification and/or supplementation to the feedback files, providing the instructors submit their curriculum to a deliberate, systematic analysis, explicating what the precise learning goals are, what they regard as shortcomings in the achievements of their students, what relevant corrective and study advices are, etc. Good teachers will consider this an advantage rather than a drawback; the quality of their teaching will certainly be enhanced by such a systematic analysis of the curriculum.
1. The text of this essay was prepared from a paper given by the authors at EURIT 86: European Conference on Information Technology in Education, held May 20-23, 1986, Twente University on Technology, The Netherlands. The present article is a reproduction of the original paper, which will be published by Pergamon Press (Moonen and Plomp, 1986).
2. In 1984, Looijmans and Schrauwen wrote the first draft of ALEXIS at Nijenrode, in Basic-Plus, implemented on a PDP 11/44
p. 43
minicomputer. A larger team, which included the authors of the paper, has worked since 1985 on the development of a greatly augmented and improved version for MS-DOS microcomputers. This version, written in MS PASCAL, is discussed in this paper.
3. Readers wishing to request more information on the ALEXIS project may write to:
Anandam, K. (1983). Computer-based feedback on writing. Computers, Reading and Language Arts, 1(2), 30-34.
Anandam K., et al.. (1979). RSVP: Feedback program for individual analysis of writing. New York: Exxon Education Foundation Research Report.
Anandam K., et al. (1980). Effectiveness of a computer-based feedback system for writing. Journal of Computer-based Instruction, 6(4), 125-133.
Camelot: A microcomputer-based instructional system (1982). Paper available from Miami-Dade Community College, Miami, FL.
p. 44
Flower, L. (1981). Problem-solving strategies for writing. San Diego: Harcourt, Brace, Jovanovich, Inc.
Frederiksen, C. H. & Dominic, J. F. (1981). Writing: The nature, development and teaching of written communication (Vol. 2). Hillsdale, NJ: Lawrence Erlbaum Associates.
Heidorn, G. E. et al. (1982). The EPISTLE text-critiquing system. IBM Systems Journal, 21(3), 305-326.
Macdonald, N. et al. (1982). The WRITER'S WORKBENCH: Computer aids for text analysis. IEEE Transactions on Communication, 30, 105-110.
Marling, W. (1983). What do you do with your computer when you get it? Focus: Teaching English Language Arts, 9(3), 48-53.
Marshall, S. (1985). Computer assisted feedback on written reports. Computers and Education, 9(4), 213-19.
Moonen, Y. and Tj. Plomp (Eds.) (in press). Development of educational software. EURIT 86. London: Pergamon Press.
Steehouder, M. F. et al. (1984). Leren communiceren: Procedures voor mondelinge en schriftelijke communicatie (2nd ed). Groningen, The Netherlands.
p. 45
Wresch, W. (Ed.) (1984). The computer composition instruction: A writer's tool. Urbana, IL: National Council of Teachers of English.