Comments from Judge Rob Foshay:

The team did a good job of picking up on the organizational and non-training issues, and many of the solutions were appropriate.

Suggestions for improvement:

1. The team generally missed the specific issue of correct analysis of the RBA by all users. In particular, an analysis of error causes, or a fault tree analysis, should have been called for. Instead, the team took at face value the lab's assertion that the level of rejected results and recalibrations was acceptable.

   In addition, a possible issue in correct operation of the machine is fatigue and stress from emergency situations. If this is found to be an issue, than the certification needs to address automaticity (fluency).

2. The use of the work teams to analyze and correct performance problems is very appropriate. However, since management is used to the language of TQM, the concept should have been explained using that vocabulary rather than "action learning." Similarly, management would probably be more receptive if the issues and interventions were discussed within the framework of change management.

3. There is an inappropriate dependency on training, which is a high-cost and difficult to implement solution. A goal of the assessment should have been to minimize training through non-training interventions such as embedded training (EPSS or job aids), job redesign, etc.

4. The importance of certification of correct operation was not adequately addressed, and didn't even turn up in the summary. This is a key unstated goal of senior management because of the legal defensibility issue. It could be addressed through some combination of periodic recertification testing (probably on a simulator) and automatic checking of user login ID's against certification status. No training, as such, required.

5. Misuse of STAT orders is NOT a training problem! I doubt seriously that anyone lacks knowledge of the policy on when and when not to order STAT. The problem lies in competing and inappropriate feedback cues surrounding the performance, and represents a reluctance to change old performance to maximize use of the RBA's and other technologies.

6. There is a general lack of ranking of importance/cost of problem vs. costs of alternative solutions. For example, lack of interpersonal communications skills by physicians with nurses is certainly a problem, but will be very costly and difficult to deal with. Any other solution strategy would be cheaper and more likely to succeed, however outrageous it would seem in other circumstances.

7. The team missed that the root cause of Robomedics behavior is their focus on the machine rather than its benefits to the client. Hospital management needs to drive home this point and get them to give higher priority to issues of integration of the technology into the hospital -- at their cost.

8. The criticism that the staff ran the needs assessment is at least partially valid, since there was virtually no call for problem analysis or even data collection independent of the self-reports of the staff.

This team had me going there for a minute. Their initial suggestion that there were two sets of issues (human performance and technology/process) agreed with my own assessment of what was happening. Please see my comments for the other teams RE the importance of the social and attitudinal issues at work in this case. Given my strong feeling that these ARE important, I would ask the team whether the Action Team concept would work. Is it too late? Has too much damage been occurred within and among the various staffs of the hospital? The team IS correct in suggesting that the Action Teams "require the full support of the Administration." If the Action Team concept would start with and include a heavy emphasis on teamwork and the importance of the various staffs to patients, this process may work.

The team way wish to reconsider it's response to the common question. The memo to Marcia and Gwen is rather self-serving in that "Will" spends far too much time reacting to what he had done and defending his actions. Will might have taken a different approach here by suggesting alternatives and modifying his original methodology somewhat. The approach Will took here MIGHT work once at this level of management - but Will would probably be seeking alternative employment if he persists in defending his methodology as strongly as he did here.

Your responses to the request for the needs assessment are well reasoned and fairly well presented. I did have a problem relating some of your recommendations to the problems, but this may have been as much a problem with formatting as with anything else. You did a fine job of identifying and distinquishing between instructional problems and environmental problems. My one concern is that the Instructional Goals you have listed are for RBA are goals of the instruction, and not goals for the learners. In your response to the Provocateur, your responses at times seemed defensive. Overall, a good job.

I thought this report was right on the mark. I particularly liked the teambuilding recommendations and the adjustments based on the provocateur questions. Specific comments are provided below.

Background
You used more data than interviews--document analysis, observations, etc.

Key Issues
I liked the summation with reference to Theory Y mission versus Theory X management style!

Intervention
The action learning teams form the heart of your recommendations. This is precisely where the heart of the organizations problems lie.

The description of action learning team is good but fairly general. A reference to professional literature would have helped.

Alternative Intervention
This backing down was unnecessary. Make your recommendation and stick with it--It's the heart of your report!

Instructional Goals
I would add a third goal: Facilitate effective and timely change and adaptation in order to provide more effective patient care.

RBA Intervention
The "teams would be the best change agents" to effect change. AMEN!

"If not Action Learning teams, then recommendations for administrative consideration for implementation of appropriate solutions should come directly from the stakeholders." Wordy but I couldn't agree more!

Hospital certification of all patient-care personnel. Yes!

Technology Intervention
I endorse the recommendation of a strategic plan that includes technology integration. Along those lines--Help define processes for determining which technologies are timely and appropriate to mission and goals, and which are not. Beward the tendency to consider every new technology indispensable. This is particularly true at early stages, before the teams begin functioning properly.

"It is recommended..." is too indirect and pseudo-objective. Use first-person--Be direct and emphatic!

The communication discussion related mostly to top-down communications. I consider the problem to be two-way.

The recommendation of workshops on communication seems unnecessary. Couldn't this function be handled by the teams? I was glad to see your amendation to that effect in response to the provocateur's questions.

Provocateur's Questions
I thought the responses to the provateur questions were, in general, somewhat wordy and indirect. I know your time was limited, but work on style.

You seemed too deferrent to the EPSS, like it was some kind of miracle that was going to come in and solve people's problems. It wasn't even clear to me that you understood what EPSS entailed. For example, an effective EPSS for an RBA would have to be on the RBA itself, or else on some kind of palm- or laptop people are carrying around with them.

I'm also wondering whether EPSS would be overkill for the technical problem being addressed. If you certify personnel in the use of the RBA, is an extended RBA needed, and is it likely to be accessed and used in that environment? I could see how EPSS-like functions could be built into the RBA itself, but that's a product-design issue that should be negotiated with the nerds at Robomedics--who, by the way, if they had any sense, should engage you as true partners in developing any kind of disseminable user-supports products.

Overall, excellent analysis and report. I can see why the Robomedics people would want to hire you on the spot. I definitely would hire you based upon your response to this case study!

Judges' Ratings Note: These item ratings have been averaged across all four Judges.
Item #	Item	Average Rating
1	The team *identified* the important issues in the case.	3.50
2	The team effectively *addressed* the important issues in the case.	2.75
3	The team *defined* the perspectives of relevant stakeholders in this case (e.g., administration, doctors, nurses, nursing assistants, lab staff, patients, public, etc.)	3.00
4	The team effectively *responded to* the relevant perspectives in this case.	3.00
5	The team distinguished between the training and non-training components of the problem.	3.50
6	The team effectively analyzed the needs identified.	3.25
7	The team identified appropriate alternative solutions for each need.	3.00
8	The team developed an instructional goal that was appropriate for the case.	3.00
9	The team recommended an appropriate overall solution.	3.50
10	The team identified all major project risks and allocated project resources appropriately to manage the risks, while avoiding over-resourcing of low-risk project components.	2.75
11	The team's specifications for personnel to be involved in the solution were appropriate.	3.25
12	The team effectively integrated relevant professional knowledge into their response.	3.75
13	Overall, the team's Needs Assessment was appropriate for the case.	3.25
14	The team effectively addressed any additional issues raised by the Provocateurs.	3.50
15	On a scale of 1 to 8, with 1 being 'poor' and 8 'outstanding,' rate the overall quality of this team's response to the case study and provocateur questions. (This item weighted double, with an eight-point response scale.)	6.00
	Overall Average. Note that item #15 has been treated as two items to maintain a 4-point scale referent.	3.19