Thanks again for the opportunity to judge this year's competition. As in the past, it was a great learning experience for me, as well as for the teams.

It was great to see that all the teams did a pretty good job of picking up on the organizational issues as well as the knowledge management issues, and the interventions proposed were generally appropriate. Among the oversights which seemed to be recurring themes were:

1. Many of the teams underestimated the sheer size of the EPSS development project. For a work environment such as this, the system would be huge when fully developed, and will probably represent a number of person-years of effort. A small scale pilot or phased approach would have the advantage of showing early results and reducing size and technology risk.

2. Most of the teams did not ask the question, "what is the performance problem?" precisely enough, though most did identify lots of potential causes. I suggest that the specific performance problem is the high number of takes (rework). I would use Tom Gilbert's concept of the Performance Improvement Point (PIP) to analyze the circumstances under which high retake shots occur, and then target interventions specifically to reduce those classes of shots.

3. A critical factor missing from many of the proposals is feedback. Without feedback on performance, there is no way the teams could possibly know if their performance is improving. This, in turn, is almost certainly likely to lead to underuse of the EPSS and failure of the intervention. In fact, simply providing feedback on performance, by itself with no other intervention, would probably be a powerful intervention in this case, since workers complained that they had too little (a major source of anxiety).

4. Another key intervention, related to feedback, is performance standards.

   These would probably be handled in the context of the Rocket Boy project, rather than as a general rule book, and would be in the form of a style guide. Analysis of the high-rework points would lead directly to guidelines for what standards need to be in the style guide. The style guide should then be used to drive the retrieval system for the EPSS and the performance evaluation/feedback system. This is especially important here, because the knowledge base is mostly visual, not verbal, and won't be well supported by off-the-shelf (verbal) retrieval system solutions. Visual terminology tends to be idiosyncratic and context/team/project-specific.

5. Note that feedback and performance standards are two of the cornerstones of a quality management system (stakeholder buy-in is the third, which some of the teams did address). All of the proposals were weak in their recommendations for a QMS.

6. Very few of the teams picked up on the work flow management problems-"the right hand not knowing what the left hand is doing," etc. However, the use of cross-functional teams, which many of the teams proposed, would be one way to deal with the issue.

7. Some, but not all, of the teams picked up on the need to put in place a requirements definition and upgrade management system for software. The system should be driven by the users, not the programmers, and should cover both internally built and vendor-supplied software. It would be synchronized to the project requirements.

8. Some of the teams seemed to underestimate the issues of obtaining stakeholder buy-in. The EPSS strategy and the team structure interventions, in particular, are likely to be perceived as high risk and in competition with resources needed for production. It will not make sense to jeopardize production rate in order to make these interventions, especially given the priority to become profitable and increase productivity. The standard way to address the issue is by breaking up the interventions into bite-size chunks (with pilots, or by task or team, perhaps using PIPs to prioritize).

9. Finally, use of ID/OD/HPT jargon in this environment is a significant risk identified by only some of the teams. The training department has to change in order to pull off the proposed interventions. But the rest of the company should not be expected to adopt our jargon; doing so will only create the perception that another bunch of non-artists and non-managers are on the loose, muddying up the waters. The entire training department must be careful to use the vocabulary of media production to talk about what is proposed. This will require Jason to be a verbal chameleon-a basic job skill for any ID specialist.

Rob