In 1981, the National Council of Teachers of English established the Committee on Instructional Technology (CIT) to investigate the potential of using technology, particularly computers, in the teaching of English language arts. A major priority of the Committee was to develop a set of criteria that could be used by educators at all levels to evaluate educational software. The Committee set about this task with the full recognition that it would be difficult to develop a single evaluation form that could be applied to all software. The approach to the task, therefore, was to try to develop criteria that embraced the most dynamic capabilities of the computer and, at the same time, to take into account the various instructional strategies that could be included in the design of a software program.

The Committee on Instructional Technology hopes that these Guidelines will help you with the difficult task of evaluating software. We welcome your comments about how you used them and about the kind of criteria that should be included in future revisions.

Step 1: Familiarize Yourself with the Guidelines

Read through the Guidelines to acquaint yourself with the format, the criteria, and the general procedures for using them. You will notice that the Guidelines are divided into five sections, or categories. The criteria in four of these sections can be used to evaluate almost any type of instructional software. Section III, however, includes variable criteria which can be applied to a software package depending upon the instructional strategy used in the program. The criteria listed in Section III within the body of the Guidelines address programs that use a tutorial/drill and practice strategy. However, if the program you are reviewing uses an instructional game or a simulation, simply turn to the Guidelines Addendum section and use the criteria listed there to evaluate the strategy being used. This section lists criteria that for evaluating Simulations and Problem-solving, Educational Games, Teacher Utilities (a program that allows teachers to alter the content or format of the program), and Word Processing. Not all of the sections included in the Addendum are truly instructional strategies in the strictest sense. However, the Addendum section does allow you to evaluate programs that might deviate from a typical drill and/or tutorial program.

Step 2: Use the Software Yourself

Proceed through the software program and use it as if you were a student. Occasionally, make an error deliberately in the same way that one of your students might. By doing this you will be able to see how the program handles errors or prompts incorrect responses.

Step 3: Let Your Students Use the Software

Allowing students to use the program is the ultimate evaluation. Observe their responses to the program. Try to determine whether they are learning anything from it. Does it motivate them and involve them in learning or do they look bored and uninvolved?

Step 4: Use the Guidelines to Evaluate the Software

When you have thoroughly familiarized yourself with the software program, go through each section of the Guidelines and respond to each item. You might want to write marginal notes by some items for reference later when you write your summary comments.

Step 5: Complete the Overall Evaluation and Write Summary Comments

The Overall Evaluation section of the Guidelines provides for four types of information:

1. A summary evaluation of each of the five sections.

   In this section you have the opportunity to make a qualitative judgment about the program you are reviewing. Rate each section using the scale of 1 to 5 with 1 (poor) representing the lowest rating and 5 (excellent) representing the highest. A rating of NA has also been provided in case the program lacks a management system, supplementary materials, or some other feature that does not detract from the programs effectiveness.

2. Summary Comments

   Here you can write anecdotal comments about the program's strengths and weaknesses. This is an important section because it provides an opportunity to include comments about qualities in the program which are not included in the criteria for each section. This section allows you to provide both a description and an evaluation of the software for yourself and your colleagues.

3. Students Responses to the Program

   The true measure of any instructional program is whether it teaches what it was intended to teach. In this section you can summarize the observations you made when you allowed your students to use the program.

4. Recommendations

   This section could be the most important of all, particularly if you review software for your district or school building. Try to summarize the reasons why you are not recommending the software or how you think it might fit into your language arts program. Try to recommend a specific use for the piece of software and how it might fit in with software programs you already have.
   

Students Responses to Program (briefly indicate number of students, grade levels, etc.):______________________

Recommendations:__________________________

Choose the description that best fits the software you are reviewing and respond to each criterion under that heading with a yes or no.

1. Simulation/Problem-Solving (A program in which students learn through discovery and decision making.)
   
   1. Problem-solving situation is realistic.
   2. Design is motivating.
   3. Procedural tasks are clearly sequenced.
   4. Feedback about user's decision is helpful.
   5. Suggestions are given for optimum performance.
   6. Outcomes or choices are explained.
   7. The program is relevant to the acquisition of English language arts skills.
      

   (Go on to Section IV.)
   

2. Educational Game
   
   1. Format is motivating.
   2. Graphics are appropriate to presentation.
   3. Content is relevant to English language arts skills.
   4. Learner has access to help or review.
   5. There is an appropriate reward for success and no reward for incorrect responses.
   6. Additional information or clues are provided by error feedback.
      

   (Go on to Section IV.)
   

3. Teacher Utility (A program that allows teacher to "author" the content within a programmed format.)
   
   1. Directions to teacher are clear.
   2. Items are easy to enter.
   3. Editing is possible.
   4. Format for presenting items to student is appropriate.
   5. Student directions are clear.
   6. The type of interaction is appropriate to the skills taught.
   7. Record-keeping capability is provided.
      

   (Go on to Section IV.)
   

4. Word Processing/Text Editing
   
   1. Management Features
      1. Utility functions (e.g., cataloging, renaming, protecting, deleting, copying files) are adequate.
      2. Supplemental materials (such as summary command cards, worksheets, spell-checkers) are provided.
      3. Editing utilities (e.g., capacity for adding words to spell-checkers) are adequate.
      4. Management options (such as printing files) are available to and easy to use by students.
         

   2. Safeguards
      1. Directions or warnings at critical decision points (e.g., "Are you sure you want to delete this paragraph?") are given.
      2. Back-up disks are available.
      3. User can undo a previous action (e.g., return paragraph to its original position).
         

   3. Editing
      1. Formatting features (e.g., upper/lower case, centering, underlining, tabs, subscripts) are adequate for intended audience and use.
      2. Editing features (such as deletion and insertion of characters, words, paragraphs) are adequate for intended audience and use.
      3. Command keys are logical and relatively easy to use.
         

   4. Visual Presentation
      1. Displayed characters are sufficiently readable for intended audience (e.g., 25, 40, 80 characters per line).
      2. There is adequate space between lines or print.
      3. Lines terminate at word boundaries.
         

   5. Printing
      1. Print formats (e.g., page width, page length, spacing) are flexible and adequate for intended audience and use.
      2. The selected print format can be displayed on the screen prior to printing
         

   (Go on to Section IV.)
   

5. Other Types of Software
   

   If the software you are reviewing does not fall into any of the above categories, you may wish to give a brief description of it below, followed by your evaluative comments.

   _________________________________
   

   (Go on to Section IV.)
   

Compared with other educational media, instructional software is still in its very earliest stages of development. Developers and publishers of software are still learning how to best use the computer as a teaching medium. For this reason, the evaluation of software remains an imprecise process. Therefore, perhaps the best guidelines for evaluating software is to judge it against its producer's claims. That is compare the product's performance against what the publisher claims it will do. Also compare it against other products that attempt to teach the same concepts and skills. Finally, try to review a wide range of software and note how each uses the computer to present the instruction.

Program:
Producer:
Required Equipment:
Grade Level:
Cost:
Overall Program Objectives:

Single Lesson Objectives (if you are reviewing one lesson only):

Answer yes or no for the following criteria in Section I through V.

1. Management Features
   
   1. Program provides teacher with a management system. (If no, go to Section II.)
   2. Program has record-keeping system that is useful and efficient.
   3. Records are easily retrievable.
   4. Teacher can assign or change performance levels and otherwise modify or add to records.
      

2. Content
   
   1. Content is accurate.
   2. Content is appropriate to grade levels for which it is intended.
   3. Content can be modified by student or teacher.
   4. Possible content modifications are appropriate to the subject matter.
   5. Program can contribute integrally to the total English language arts curriculum.
   6. Program achieves its purpose.
   7. Program is likely to be motivating to students.
      

3. Instructional Strategy
   
   Note: If the software you are reviewing uses an instructional strategy other than tutorial or drill and practice, go to the Guidelines Addendum (Alternate Section III) to complete this section. Then continue with Section IV.
   1. Program is attractive.
   2. Program provides opportunity for practice.
   3. Practice is sufficient to help ensure mastery.
   4. Examples are provided.
   5. Examples are clear.
   6. Presentation is logical and well organized.
   7. Student has control over rate of presentation.
   8. Feedback for incorrect responses is helpful for discovering correct answers.
   9. Program allows learner to review, repeat, or advance according to performance.
   10. Program reports student performance periodically.
   11. Program provides an appropriate balance between content presentation and student interactions or responses.
   12. Program offers a variety of interactions, varying keys pressed or responses required.
   13. Program stimulates cognitive growth (or promotes thinking skills beyond recall of information).
   14. Program complements (or enhances) other English language arts materials.
   15. Program calls for meaningful application of English language arts skills.
       
   
   

4. Ease of Operation
   
   1. Directions to student are clear.
   2. Directions are accessible when needed.
   3. Student can operate program independently.
   4. Student is prevented from getting lost in the program, with no way out.
   5. Student is provided with option to quit or continue at any time.
      

5. Supplementary Materials
   
   1. Program provides teacher's guide.
   2. Program provides supplementary student materials.
   3. If yes, materials are appropriate and useful.
   4. Program provides pre- and posttests.
   5. Replacement print materials are available from producer.

Overall Evaluation

Use the scale at the right to rate this program. (1 is lowest; NA means Not Applicable.)

I.	Management Features	1 2 3 4 5 NA
II.	Content	1 2 3 4 5 NA
III.	Instructional Strategy	1 2 3 4 5 NA
IV.	Ease of Operation	1 2 3 4 5 NA
V.	Supplementary Materials	1 2 3 4 5 NA

Summary Comments (Continue on back of sheet, if necessary.)

Program's Strengths and Weaknesses: _________