(If you have trouble printing this page, you can save the document and print from any word processor:   Under the "File" Menu, select "Save As," "Format:Text," and a location on your computer for the document."

How The 1997 Judging Was Conducted



The review by both Provocateurs and Judges was blind. Each of the five participating teams was given a referent that was randomly determined, such as "Team G" or "Team N."

Provoking Thought

Each Provocateur read the case responses from all five teams. Then the three Provocateurs generated two questions apiece. One of the resulting six questions was a common question, intended for all five teams. The other five questions were tailored specifically for a particular team.

COMMON QUESTION for ALL TEAMS, by Allison Rossett

Will was surprised when he got an email requesting his presence in Marcia's office. When he gave her the document, Marcia had thumbed through it, read the exec summary and turned right to the recommendations. She'd nodded with satisfaction.

So Will figured he was finito with that project. No problem if she wants to chat, he thought to himself, noting that he and Marcia were real tight.

Marcia wasn't alone when he was eventually shown into her office. Sitting at her large conference table was a woman who looked vaguely familiar, but he couldn't place her. She had a copy of his assessment in front of her.

Marcia said, "This is Dr. Gwen von Hinghis. Gwen heads up Education, Training, Technology and Performance Systems for headquarters in Chicago. You've probably heard about their brilliant work on EPSS for customer service. Gwen made all that happen and the CEO has come to depend upon her. Maybe you've seen her articles in professional journals? I asked her take a peek at your needs assessment. I knew you wouldn't mind.

Gwen got right to the point: "Well, Will, I can see what you were driving at here and I appreciate the array of sources you tapped, but I think you kind of let the people at Cadaceus run the needs assessment and dump what they wanted on you. They expressed joy and sorrow about Roberta, but I don't think you got all the information you need. And if you're going to do much training, did you get at what it is that different employees need to know and be concerned about? To create all that training, bet you'd have to do still another needs assessment.

Marcia tells me you're a recent PhD so I'm surprised that your inquiry doesn't reflect more cognitive and constructivist perspectives. Surely that makes a difference in needs assessment, don't you think? Could you take another whack at it with some of this in mind? I know that Marcia will appreciate it, and I will too.

This thing has to be right, you know. We're eager to see where you get to. Right, Marcia?

Will was stunned. He'd enjoyed nothing but kudos in his career, and now this. Well, he was going to think about it and tackle it. Where could he turn for help? He had a little money for consultation...

Help Will. Strengthen his study in ways that reflect your opinions of what he did and didn't do PLUS Gwen's concerns.


Judging

For the judging, blind responses from all teams were reviewed by all Judges. Each Judge completed written feedback for each team. In addition, a 15-item scale was used by the judges to rate each team's work. Ratings on the first 14 items were made on a four-point scale [from Strongly Disagree [1] to Strongly Agree [4]). The last item had eight possible points, and so weighed double:
  1. The team *identified* the important issues in the case.
  2. The team effectively *addressed* the important issues in the case.
  3. The team *defined* the perspectives of relevant stakeholders in this case (e.g., administration, doctors, nurses, nursing assistants, lab staff, patients, public, etc.)
  4. The team effectively *responded to* the relevant perspectives in this case.
  5. The team distinguished between the training and non-training components of the problem.
  6. The team effectively analyzed the needs identified.
  7. The team identified appropriate alternative solutions for each need.
  8. The team developed an instructional goal that was appropriate for the case.
  9. The team recommended an appropriate overall solution.
  10. The team identified all major project risks and allocated project resources appropriately to manage the risks, while avoiding over-resourcing of low-risk project components.
  11. The team's specifications for personnel to be involved in the solution were appropriate.
  12. The team effectively integrated relevant professional knowledge into their response.
  13. Overall, the team's Needs Assessment was appropriate for the case.
  14. The team effectively addressed any additional issues raised by the Provocateurs.
  15. On a scale of 1 to 8, with 1 being 'poor' and 8 'outstanding,' rate the overall quality of this team's response to the case study and provocateur questions. (This item weighted double, with an eight-point response scale.)

 For each team, the ratings received from the judges were averaged over all items and the team responses ranked accordingly. The team responses were all very good, and the ratings bore this out. One winner was named, with three runners-up closely following. The fifth team was disqualified due to a case response that was over the word limit of 1,800 words.


Posting of Responses & Feedback

The case responses have been posted to this web site, along with the Provocateur questions and responses. The Judges' feedback is also included.


Return 		Click here to
return to the top
of this page