(If you have trouble printing this page, you can save the document and print from any word processor:   Under the "File" Menu, select "Save As," "Format:Text," and a location on your computer for the document."

How The 1998 Judging Was Conducted



The review by both Provocateurs and Judges was blind. Each of the seven participating teams was given a referent that was randomly determined, such as "Team G" or "Team N."

Provoking Thought

Each Provocateur read the case responses from all seven teams. Then the four Provocateurs generated a common question intended for all teams and a specific question for each team.

Judging

For the judging, blind responses from all teams were reviewed by all Judges. Each Judge completed written feedback for each team. In addition, a 15-item scale was used by the judges to rate each team's work.

The rating scale was based on that used in previous years. The Judges reviewed the previous scale, added three new items they felt were appropriate for the case, and made a few minor edits in other items. Ratings were made on a four-point scale [from Strongly Disagree [1] to Strongly Agree [4]):

  1. The team *identified* all of the important issues in the case.
  2. The team effectively *addressed* all of the important issues in the case.
  3. The team defined the perspectives of all the relevant stakeholders in this case. (e.g., artists, technicians, supervisors, administration, trainers, organization.)
  4. The team effectively responded to all of the relevant perspectives in this case.
  5. The team effectively analyzed the needs identified.
  6. The team identified appropriate alternative solutions for each need.
  7. The team developed an instructional goal that was appropriate for the case.
  8. The team recommended an appropriate overall solution.
  9. The team's specifications for personnel to be involved in the solution were appropriate.
  10. The team effectively integrated relevant professional knowledge (theories and practices) into their response.
  11. Overall, the team's Needs Assessment and Preliminary Design were appropriate for the case.
  12. Overall, the team's Needs Assessment and Preliminary Design demonstrated excellence.
  13. The team was proactive in making recommendations and/or modifying the environment, as opposed to being only reactive and giving the client what they said they wanted.
  14. The team identified all major project risks and developed plans to manage them.
  15. The team presented an appropriate quality management plan.

For each team, the ratings received for each item were averaged across all six judges, and the team responses ranked accordingly. A first-place winner was named--this team was clearly thought to be the best by the Judges. However it is important to note that each team made unique and valuable contributions to the case solution.


Posting of Responses & Feedback

The case responses have been posted to this web site, along with the Provocateur questions and responses. The Judges' feedback is also included.



Return 		Click here to
return to the top
of this page