Instructional design (ID) case studies, which pose authentic ill-defined design problems in realistic environments, can help instructional design students bridge the gap between novice and expert practice. Over the past three years, we have explored aspects of this educational approach through the development of Web-based instructional design cases at the Curry School of Education, University of Virginia. This paper begins with a look at case study methods and how they are used to help students gain experience in ID practice. We describe the design and development of our Web-based ID Case Competition, the 1998 IT Case Event, and follow with a report on the competition involving seven universities. Officials and students felt that the Case experience was valuable for developing ID expertise and preparing students for professional practice. These advantages, and the opportunity for team collaboration, were noted as motivators for participation. Most of the students were enthusiastic about the inclusion of emergent ID issues in the case and felt the experience expanded their knowledge of ID practice and application. We close with a discussion of implications for the preparation of instructional designers and recommendations for future development activities. Introduction The professional practice of instructional design (ID) requires high level problem-solving, critical thinking, and interpersonal skills; design problems are complex and multi-dimensional. An effective designer must be capable of identifying critical issues in a situation, and more importantly, select those issue(s) that can or should be addressed . In a comparative study of novice and expert designers, Rowland (1992) observed that experts tend to interpret an instructional design problem as ill-defined and immediately link the given information to previous experiences addressing similar problems. In this way, the expert is able to infer additional information in order to develop a preliminary concept of what the problem is and then apply templates for examining the situation such as the needs analysis model. As a result, experts reflect on a wide range of factors and spend considerable time interpreting the design problem . Novices, on the other hand, do not possess this body of knowledge and, therefore, accept the given information and move on to generation of solutions without developing an in depth understanding of the problem itself. Instructional design is commonly taught as a set of procedures with clear cut examples that do not match the complexities of real-world situations . Ideally, students are given the opportunity to apply what they learn to an actual design project so they can discover the strengths and limitations of the process and theory. These projects require considerable time, a finite commodity in IT programs. Students, therefore, are exposed to a limited number of professional practice settings . Consequently, novice designers frequently enter the workforce with an understanding of the ID process but without the knowledge base that can help them solve instructional design problems and develop solutions. Case studies, which pose authentic ill-defined design problems can help bridge this gap. In case analysis, instructional design students draw connections between their emerging knowledge of ID and the complex demands of actual practice. Cases can supplement student design projects, allowing further opportunity to reflect on relevant theory and methods as students explore a greater number of design issues in a broader array of environments. Furthermore, students are provided an opportunity to broaden their knowledge base as they collaborate with colleagues to identify effective design solutions . Over the past three years, we have explored aspects of this educational approach through the development of Web-based instructional design cases at the Curry School of Education, University of Virginia. The World-Wide Web enables students at any institution or any interested individuals to access our multimedia case studies and we use this venue to sponsor an annual competition, the IT Case Event. (http://curry.edschool.virginia.edu/go/Itcases) Each year, Teams from up to seven participating universities analyze an instructional design case, while expert officials pose probing questions, evaluate case responses, and contribute their own perspectives on the case. Evaluations of our first two Case Events suggest that case analysis can serve as a valuable supplement and may help expand the depth and breadth of novice designers' expertise . In this paper, we built on this past research, with the intent of describing the case event experiences of students and officials during the 1998 event. We identify skills used during the case analysis, and offer some observations on perceptions of preparation and challenge. We also examine the perceived value of integrating an emerging ID practice: the design of performance support. Our primary research question was, "Is an instructional design case competition a worthwhile medium for expanding professional knowledge and exploring emerging issues in instructional design?" We were also interested in whether developments in our case study and approach to interface design would enhance the sense of realism, thereby encouraging a deeper exploration into the ID issues. We begin this paper by providing background information on case study methods and how they are used to help students gain experience in instructional design practice. We follow with a description of the 1998 IT Case Competition, focusing on the developments in case design and methods we implemented. Evaluative data based on the results of the competition, follow-up surveys and individual interviews serve as the focal point of this study. We close with a discussion of implications for the preparation of instructional designers and recommendations for future development activities. Case Methods and Instructional Design Case methods have long been used extensively in professions such as law, business, and medicine and more recently have gained popularity in other professions such as teacher education, engineering, nursing, and instructional design . The growing interest in this method of instruction reflects recent advances in the cognitive sciences, which suggest that knowledge and skills are best learned in contexts that reflect the way they will be useful in real life. Because of this situated cognition, the individual is able to apply this knowledge to new situations through recall of the experience and the environment in which it occurred . Constructivism builds on this theory by acknowledging a belief in multiple realities; each learner creates his or her own perception of reality based on previous knowledge and experience. Instruction that is based on this paradigm, such as case methods, focuses on student-centered, active learning experiences where learners are encouraged to use prior knowledge to explore and create new interpretations of the world around them. . As the use of cases has grown, case methods have continued to evolve, and can now be found in a variety of forms. In teacher-education and instructional design, case methods have taken the form of complex fictional narratives which are grounded in actual problems and challenges that occur in the real world . Grabinger (1996) has identified these methods as one way of providing rich environments for active learning. They allow students to construct knowledge in an authentic environment where they assume personal responsibility for learning and work cooperatively to produce something of real value. Effective cases are those that are immediately realistic and meaningful to the learners and, like professional practice, are ambiguous and messy, offering opportunity for analysis and consideration of multiple solutions. Presentation within specific contexts makes the case more believable and makes what is learned more readily retrieved from memory when the learner is placed in similar circumstances (L. Shulman (1992) as cited in Ertmer, 1995; . The cases we have developed for the IT Case Event are intended to serve as a useful supplement for instructional design students by exposing them to various professional practice settings in a context that offers "enough depth and complexity to provide realistic challenges" . Since cases can be analyzed in much less time than a design project, there is often the opportunity for students to analyze multiple cases. As a result, students can gain exposure to ID practice in industry, education, museums, and the military and the unique problems that may exist in each type of environment . We recommend that students follow a collaborative problem solving process based on the work of McNergney, Herbert, and Ford (1994):
The need to solve problems through collaboration, frequently with others who have different skill sets and perspectives, is a common occurrence for most instructional designers. Students have limited opportunities to learn how to effectively deal with the conflicting values and multiple points of view that are inherent in the group work. Likewise, they have few occasions to advance and develop support for their own perspectives. Case scenarios encourage professional collaboration, providing students opportunities to improve their oral communication and interpersonal skills as they discuss, debate, justify, and integrate their ideas into what they agree is the best possible solution . Ellsworth found that collaborating students take on a more active role in the learning process, becoming problem solvers, contributors, and discussants. Competition can be a useful adjunct to collaboration, allowing this experience to reflect the real world, where design Teams must often compete with others to identify the best possible solution . IT Case Event: Web-based Case Media Our Web-based cases are designed to emulate the complexities of real-world events within a selected area of professional practice in ID. The design and development of each case is a team effort, with selection of the case environment and characters based on the previous experience of the primary author. The design dilemmas emerge from this experience, the ID experience of the team members, and from problems alumni have encountered in professional practice in particular, those that are not addressed by standard ID theory and methods alone. We design cases to enhance the realism necessary to compel exploration of the ID issues. Starting in 1998, we began conducting needs assessments in the selected case environment. We conduct multiple interviews of stakeholders at various levels across several organizations. Our aim is to obtain stakeholder perspectives about the environment and the ID issues and to identify typical characteristics of individuals working at various levels within this industry. Next, the primary author begins construction of the case with an outline of potential events and critical issues. The team uses this outline to brainstorm potential case materials and possible thematic formats. The author then creates a draft of the entire case, attempting to build a realistic environment and believable interplay between the characters and the issues presented. Production of the case materials involves coding the files that present the case documents in HTML (HyperText Markup language), recruiting talent for the character representations, and creating the media for the case (graphics and audio and video clips to illustrate the characters and events). Once production begins, we start prototyping the case with one-to-one and small group evaluations, followed by revisions. Final modifications and revisions are made once the case has been released and used by a large group of students. For the 1998 case, we also solicited case reviews by several of the industry professionals we had interviewed in the needs assessment to check for consistency and accuracy in the content. We have found that this combination of text and media on the Web can foster both inductive and deductive strategies as the students navigate through the hyperlinked ideas, events, and artifacts. For instance, students can analyze ID methods that have been integrated into the case study or test ID models and processes as they map out what is known or unknown about a given design problem or solution. This process of reasoning is in line with the analytical processes instructional designers must engage in as they create order and coherence out of disparate sources of information . In this study, we explore student and official perceptions of reasoning employed by Student Teams in the 1998 IT Case Event. We also consider whether the changes imparted in our design and of the Web-based cases enhance the learning experience of the learning participants. We ask the following primary question:
And the following secondary questions:
Methods Participants A total of 42 students, 28 female and 14 male, participated in the 1998 IT Case Event. The seven Student Teams, ranging in size from four to seven members, were from the following academic universities: Colorado at Colorado Springs, Colorado at Denver, Northern Illinois, Purdue, San Diego State, Virginia Polytechnic Institute, and Virginia. The students were enrolled in both master's and doctoral programs and six of the Teams had received training in instructional design as part of their programs. The seventh team was enrolled in an introductory design course that had met only twice before the beginning of the competition and its members were participating for course credit. Four of the Teams received course credit for their participation and three Teams participated as an extra-curricular activity. As in previous Case Events, the majority of the students (72%) had more than four years of full-time work experience in a variety of professions from K-12 and adult education to engineering, the military, and business. The seventeen officials included Team Sponsors, Provocateurs and Judges. The Team Sponsors nominated Student Teams and relayed case event communications to the students. They were also encouraged to analyze a case with their Teams prior to the event. The Team Sponsors each nominated one or more professionals to participate as Judges or Provocateurs. Additional officials volunteered or were recruited by the development team. Materials The case developed for the 1998 IT Case Event, "The Chronicles of RocketBoy," takes place in the digital animation sector of the feature film industry . This fictional narrative is based on real issues advanced by the case authors, industry experiences of the authors, and the perspectives shared by the individuals from the digital animation industry who were interviewed. This case also advances ID problems selected in advance by the case authors, including the emerging practice of performance support. The case is written in the first person, through the eyes of an instructional designer, Jason, (Figure 1) who is brought into a large digital animation house to assist in designing an electronic performance-support system (EPSS). With five years instructional design experience, Jason is charged with the task of identifying performance-support measures to increase productivity on the production line and help the company turn a profit. Through several interviews, Jason peels back the complex layers of this organization, identifying the key issues that will help him design the best possible solution. The case concludes as Jason begins to analyze the data from his needs assessment and assures the training department team that a performance-support solution can be developed within the nine-month time frame that has been budgeted for this endeavor. Navigation through the site is guided by a hyperlinked index designed to look like a movie marquee listing the events in the case (Figure 2). The user can also navigate through hyperlinks provided at the bottom each page. Five case ancillaries (letters, charts, and descriptions) are presented as part of an orientation packet for the instructional designer (Figure 3). These documents help depict the structure of the company. Photographs of characters in action and graphics dispersed throughout the case add depth to the case events (Figure 4). Links to articles defining EPSS and its applications in ID were also embedded in the case. The case may be examined at the following URL: http://curry.edschool.virginia.edu/go/ITcases. Figure 1: Introduction to Jason in the Web-based case, "The Chronicles of RocketBoy" Figure 2: Movie marquee as index for the Web-based case, "The Chronicles of RocketBoy"  Figure 3: Ancillaries from the Web-based case, "The Chronicles of RocketBoy"  Figure 4: Interview Screens from the Web-based case, "The Chronicles of RocketBoy"  Procedures Teams were encouraged to prepare for the event by analyzing a case from previous events, and by reviewing case responses and the Judges comments. Participants were also informed that EPSS would be an issue embedded in the case and were encouraged to review the recommended articles that would be linked into the case documents. Once the case for the 1988 event was available, a three-week time-period was provided for analysis response development. Teams were limited to meeting for a maximum of six hours with team members. This time limit applied to face-to-face meetings and to asynchronous discussions (such as e-mail communications) between as few as two team members or with the entire team. No limits were placed on individual review, analysis, or writing, and Teams were encouraged to reference any resource materials they desired. Teams were instructed, however, to respond to the case without participation from others. The entire case response was limited to 2000 or fewer words to encourage concise expression of ideas and to keep the review work of event officials manageable. In developing their responses, the Teams were directed to complete the needs assessment, including analysis of needs and alternative performance support solutions. They were asked to include in their final recommendations an outline of a preliminary design based on the needs assessment, specifying goals, strategies, and target groups where appropriate and to base their case responses on each of the following tasks:
Following their review of the submitted case responses, the four Provocateurs met asynchronously via e-mail to compose a general question for all of the Teams and a specific question for each team in reaction to their analysis. The Teams were allowed up to two hours working together on their response to the Provocateur questions. There was no length limit, however, to this phase of the case activity and the Teams had one week to produce a response to the questions. The case and question responses were then forwarded to the Judges who reviewed the materials individually, evaluated the Teams' responses using a rating scale, and wrote evaluative feedback for each team. The ratings were tabulated and a winner was announced. All of the team responses were posted along with the Provocateur questions and Judges' comments and the participants were encouraged to review and discuss the various solutions presented by the Teams. These materials can be reviewed at the URL: http://curry.edschool.Virginia.EDU/go/ITcases/Site/98Teams/Outcomes. Measures While the Teams were responding to the Provocateur questions, the panel of six Judges met via email to review the rating scale from the previous Case Event. They adapted it from the 1997 Case Event to reflect the challenges presented in The Chronicles of RocketBoy, adding three new items pertaining to emerging issues in ID (see items 13-15 in Table 1). Ratings were made using a four-point scale from Strongly Disagree [1] to Strongly Agree [4]. The following table contains a listing of the response rating items. (Table 1): Table 1: Judge Rating Scale for Team Case Responses
Once the students had submitted their responses to the Provocateur questions, we sent an e-mail request asking them to evaluate their Case Event experience by completing a Web-based survey. We sent similar requests to Provocateurs, Team Sponsors, and Judges, as they completed their contributions to the Case Event. Both students and officials were asked nine common questions with slight variations displayed in the table below (Table 2). In addition, the students were asked whether they had participated as part of a class or some other form of academic credit. The students were also asked if there were other factors that had prompted their decision to participate. This survey is closely aligned with the survey used in the previous study, with the addition of questions reflecting our focus on emergent issues and our interest in the application of the case experience to actual practice. Responses to the surveys were submitted electronically through the survey Web-forms. They were analyzed using simple descriptive statistics (means and standard deviations) and simple qualitative analysis for the open ended questions. Table 2: Survey Questions Answered by All Participants (Text in brackets represent variation in student survey)
Participants who completed the surveys received an email request asking if they would participate in a telephone interview. The interviews with the participant volunteers took between 25 and 50 minutes, and were tape recorded and later transcribed for analysis. The interview questions were adapted from the previous study to reflect our foci. A common set of questions was posed to all participants (see Table 3) and specific questions were directed to students and officials, respectively. Interviewees were encouraged to elaborate on their ideas and add additional comments at will during the course of the interview. Additional questions emerged during the interviews in response to participant comments. Table 4 presents questions that were addressed to students only, with several questions focusing on team process and preparation. Table 5 presents questions posed to the officials. In addition, Team Sponsors were asked if they had a perspective on how the Teams discussed the materials. Provocateurs were asked how they organized their approach to developing the questions, and the Judges were asked how they coordinated the evaluation and feedback of the case. Table 3: Interview Questions Answered by All Participants
Table 4: Interview Questions posed to Students
Table 5: Interview questions posed to All Officials
Data Analysis Quantitative data were analyzed with simple descriptive statistics (means, standard deviations, percentages). The qualitative data included responses to the open-ended survey questions and all of the interview comments. The survey and interview data from each individual were labeled for cross-referencing before creating categories to avoid duplication of individual perspectives in the data analysis. A conventional mode of qualitative inquiry was employed: The corpus of data was read repeatedly until tentative categories emerged to describe the issues reflected in the data. Through an inductive process, the categories were revised and a final coding of the data was completed. Matrices were used to compare individual and team responses to case questions. All data within each category was then studied to determine the strength of the sentiments reflected. Where useful, direct (blind) quotes are included to better describe the outcomes. In addition, results of the previous study are reviewed to identify whether developments in the design and structure of the case have produced anticipated outcomes. Results In the following section, we present our findings, organized according to the categories emerging in the qualitative analysis. Response Rates Sixty-seven percent of the participating students responded to the surveys. We received 28 surveys out of the 42 student participants. An average of four members per team responded with one team having the minimum response of three students. We received survey responses from 14 of the 17 event officials yielding an 82% response rate. Following completion of the surveys, 43% (19 of the 42 students) agreed to be interviewed. Each team was represented in the interviews by at least two students. Seventy-one percent (12 of the 17) event officials agreed to participate in the interviews as well. Reasons for Participation Four of the seven Teams participated as a part of their class, yet the majority of the respondents who commented on their reasons for participation (85%) felt that their participation in the case competition would be beneficial whether they received course credit or not. The students described several motivations for participating in the Case Event, ranging from opportunities to bond with peers to engaging in a healthy competition. Most of the students felt it would help them gain experience and exposure to current ID problems. One student saw this as an opportunity to "gain insights from the professionals in Instructional Design who do this everyday for a living." Several students were attracted to the challenge of the problem, "I think the challenge, the intrigue, the puzzle, and the thrill of working with a great group of people were the most motivating factors for me." One of the Teams had been promised substantial credit and opportunity to interact with doctoral students (they were in their first semester of graduate studies at the master's level) and were disappointed that this had not been realized due to lack of follow through from the Team Sponsor. Preparation for the Event Cases had been previously used as a learning tool by 54% (15) of the students and 79% (11) (of the event officials. Of this group, 67% (10) of the students and 45% (5) of the officials had used cases as an instructional design learning tool. All but one student read at least one of the IT cases, Prescription ID, Harvesting Cooperation, Trials of Terry Kirkland, in preparation for the event. Most of the students reviewed other resources on the Web site: 85% read previous case responses, 71% read expert responses, and 39% of the students indicated they read some of the publications linked into the Web site. Of the officials, 68% indicated they read case responses and 44% reviewed the expert responses and publications, while 86% of the officials indicated they had read at least one of the IT cases. Prior to the release of the competition case, we announced that EPSS would be an issue embedded in the case study and provided the URLs to the articles that would be linked into the case documents. Six of the students indicated they reviewed the recommended articles and conducted research on the topic. Other preparation included research on learning theory and review of instructional design literature, while other students analyzed ID cases in their class. Three of the Teams met with their sponsors prior to the release of the case to discuss Event rules, the nature of previous IT Cases, and performance support. One of the Teams indicated that they had reviewed basic procedures for collaboration as well. Consideration of the Case The interviews with the participants suggest that several different approaches were used to analyze the case, yet there were some similarities among the Teams' coordination of the analysis: All but one team read the case before their first meeting and used that session to share individual perspectives; the exception was from a team that had been formed just prior to the release of the case. One of the Teams went a step further, completing individual analyses and conducting research on related topics before their initial meeting. At this team's first meeting, they compiled their perspectives in a database, grouping similar issues and removing 'non-issues' and created a single list which served as their analysis: "We all came in with our issues list ...and of course there was some redundancy, but there were a lot of things that came out from each individual person that we wouldn't have picked up if we had all met as a team to do that; It gave us a clear focus at the beginning of the what the issues were. We also did quite a great deal of research on EPSS and performance support systems. Which again lended itself to having focus." All but one participant commenting on how the case was accessed, worked from a printed copy and made sense of the issues by reading through the case at least two times, recording key issues in the margins or highlighting them. One individual printed the case's organizational charts and used them to read the case on-line. She subsequently printed only those pages she found particularly relevant to the issues she had identified. All of the Teams met two or three times as a whole group, although two divided into two subgroups during the first meeting to complete the case analysis. They used the second meeting to integrate the analyses produced by the subgroups and formulated a solution. Three Teams met as a whole group to discuss the key issues and then chunked the tasks according to the competition guidelines (needs assessment, performance support, goals, strategies, etc.). Individuals or pairs were responsible for specific tasks according to their perceived strengths. In two Teams, completed tasks were accepted without evaluation or consideration of alternative perspectives. Two Teams commented that they used the second meeting to evaluate and integrate individual contributions with their selected approach to the case analysis and generation of the solution. This was a session where they engaged in debate, voting "yea or nay" and offering suggestions for editing. One of the Teams relied on the more experienced team members to guide this process: "The second session was purely challenge and defense. Even though you were presenting your ideas and trying to be informative, people were like, no, that can't work because of this and you've got to remember we are Jason. We kept trying to anchor ourselves as Jason...Some of the people had been in the working world and they were explaining to us why our ideas just wouldn't work in the real world. Everybody was more grounded in the personalities and they were explaining why this person wouldn't accept that or that person wouldn't accept that, reminding each other. It was really stimulating. I enjoyed it." Although there were various approaches to the generation of the case response, each team designated one or two individuals to compile the ideas into a single document, which was then sent to the entire team for review. The team that had conducted research before their first meeting had a unique approach to the generation and formatting of their solution: After completing the analysis, the team members continued researching topics related to the issues identified in the case and spent their second meeting formulating a solution together. They created a chart with the key issues on one side and possible solutions on the other side. The team leader used this chart to compose a solution in two or three different presentation formats: "I think that probably was the most organizing we did as far as the data goes... I wrote one and then read over it and waited a day and went back and read it again. Then, I wrote another one that was all the same content, but organized and presented entirely differently. So, when we met to decide what we were going to send or submit to you, we looked at the two or three different versions that I had written...and decided on the one that we sent which is kind of an introduction. Then we outlined what we thought the key issues were and then the outline of the strategies and the next steps..." Collaboration Prior experience, group size, and the responsibilities adopted by team members are some of the factors influencing team dynamics. All of the Teams perceived collaboration as an important factor in their productivity and several participants described ways their teams met the challenges of team process. Most of the Teams felt the degree of their prior experience contributed to their effectiveness as a team. One Team commented that their previous experience as a group contributed to their positive team dynamic. "It was the right chemical mix. ...We had been in class together, we had studied together, and we had had open discussions in class, heated discussions in class. Academic discussions. We were prepared, we were a good team, and we wanted to work together. Another team felt prior experience working together resulted in a team that was "homogenous" with very few disagreements. Two of the Teams commented that it took a while for them to gel as a group. A third team noted that they never achieved a sense of team cohesiveness and consequently did not feel able to "adequately flesh out all of the issues as a group." "Maybe of importance was the fact that we had not actually gelled as a group because it was very near the end of the semester. We didn't get an opportunity to go through all of the steps that groups working together go through." Six of the seven Teams had six or more members and three of these Teams commented that the lack of prior experience working together in a large group diminished the effectiveness of their meetings. "Our discussions were out of control with so many different perspectives." Two Teams felt that dividing the larger group into two sub-groups affected their productivity. One team found that forming subgroups to edit sections resolved the challenges of compiling the multiple perspectives of the whole group. The second team had arranged to work in subgroups prior to the event. Members of this team felt there was some redundancy in the work produced by the small groups. However, as one team member observed, "We said that perhaps it was a mistake for us to divide into two separate Teams. We had thought, that had we been in the one group from the beginning, then repetition wouldn't have occurred. However, I think we actually accomplished more in the smaller groups than we did in the bigger group. I think having just one large group would have brought about other problems that would have been just as significant or even more so." One team felt their smaller size contributed positively to the process because "you have enough differences of opinion, but you have also got a small enough group where you are not taking forever to express your ideas." This team identified additional factors that led to their successful collaboration. Prior to the competition, the team set ground rules and designated a team leader who had full autonomy to control discussions. They also created a specific agenda for whole group meetings; they were strictly about decision making and all brainstorming and research was done independently. They used a stopwatch at meetings to guard time which resulted in rapid fire discussions "that were focused and to the point." "I think that was one of the most important things that we had to keep us on time, on track, and to the issue. She was the referee and the lead. So, if we discussed an issue, she was sitting there entering it, asking questions when something wasn't being explained or was going in the wrong direction. She would stop and say, 'This is a dead issue, let's go to the next one...'" Leadership, or lack thereof, emerged as one of the important facets of team collaboration. Five of the Teams began the event without designated leaders. Roles were defined as the need arose and team members took responsibility for completing various tasks. One of these Teams felt that without designated roles, there were too many leaders resulting in, "just a lot of information...and it was hard agreeing and disagreeing with each other." Two others remarked that they were able to share leadership roles, and one of them felt this resulted in varying levels of commitment to the project. A fourth team felt that one or two people dominated the group and some of the team members were consequently withdrawn during discussions. Another team began with designated roles, but found they shifted once the analysis began and different personalities of the team members began to emerge. The locus of control shifted as well, to the person who took on the responsibility of compiling the results and writing the response. "Once the document was created, the writer had control and power over the document because he had written it in a voice that was comfortable for him. Trying to get any changes in that document was sort of like a lobbying effort. You had to make a case. He was like the judge. It was like a lawyer going up for the judge." Two of the Teams expressed frustration from the lack of commitment exhibited by team members. They felt this might have resulted from the way their Teams had been recruited; they did not volunteer and participated as a part of their coursework in ID. One or two of the members felt coerced into participating and did not invest in the process...That they had no choice. So, they were participating, but on a limited level. Whereas there were others that went ahead and accepted it and worked very hard. "Although most of the students interviewed reflected some challenges experienced with team dynamics, 79% (15) felt that these challenges were one of the most valuable aspects of the case experience." Competition The competition and prospects of being judged were considered by over half of the participants (11 out of 19, or 58%) to be incentives for participation in the Case Event. When asked about the effects of the competition on the Case Event experience, there were differing perspectives, however, between students who volunteered and students who were required to participate as a function of their class. Students who volunteered (five Teams) felt the competition was an important component in the case event. It was an opportunity to represent their university and learn from students in other schools: "Whether you win or lose, the competition gives you a measuring stick on where you are at and what you need to learn. I think it is good for a university too, because they can find out what they are doing right and maybe what they need to improve on. It is a very good tool. I would like to see two of these a year, three a year." ...and increase the sense of challenge, motivating them to participate: "I don't know if it influenced our initial approach, but I know it influenced our motivation level. Without a doubt, we wanted to represent the university to the best of our ability. Knowing that these things are going to be published on the Internet, we didn't want to go out and make fools of ourselves, so we tried to do the best job that we could." Students from the Teams who had been required to participate regarded the event as another assignment; "We were able to get some credit for it in the class that we're doing so it could have just been a group project." The students' we interviewed were asked to comment on whether they felt that knowledge of the judging had affected their performance. Their perceptions ranged from those who felt they did not take notice to those who focused the generation of the response on the Judges. Respondents from two of the Teams felt the judging added to the 'real-world' experience: "Having to sit down and seriously look at our case response that was going to be judged. That made a very big difference. The Judges are almost like your employers in that sense; the people who have contracted you to do the instructional design." Other respondents felt motivated to search beyond the information at hand and study the topic... "I think the fact that it was either judged or graded forced us to really do our homework. We went on the web and for instance, did searches on EPSS and we looked through our textbooks. ...it made us really look at things in detail and try and back them up with research." And draw from previous experiences... "I think it made us more aware of the importance of our answers, trying to incorporate everything we've ever learned and to certainly cover all bases. But, I don't think it hindered us in any way or produced any negative affects." Two respondents from one Team described a negative influence on team performance that resulted from knowing the response would be judged: "We made the mistake of trying to second-guess the judging, because it was very conscious and talked about in our team meetings when we were talking about ideas. The team-sponsors for the aforementioned Teams felt their students became stressed as they tried to match their responses to what they anticipated the Judges wanted. They remarked that the students might have felt this way because they are high achievers and expect to be "100% successful." "They definitely saw the value in it, but for some reason, got awfully stressed about it. Maybe they were very competitive and that was just really critical. They really wanted to have the winning response." Five officials commented on the value of the competition aspect of the case event. Three felt it motivated the students: "The competition is really the thing gets them cognitively and emotionally engaged." Two officials expressed concern as to the effectiveness of the competition aspect of the case event: "It's hard to be in a competitive situation where there's winners and losers. Trying to compete with doctoral students and other experienced people can be intimidating for master's level or beginning students." Provocateur Questions and Case Responses The provocateurs reviewed the case responses, posing a common question addressing an issue they felt all of the Teams should probe further and a specific question for each team based on the focus of their individual case response. In formulating the questions, the four Provocateurs took the role of the company manager responding to the instructional designer's recommendations. The common question was posed to all of the Teams: "Can we slow this down a little, Jason? You've only been here for what, a month and you've talked with a few people. And based on that,you're making an assortment of recommendations that resemble re-constructive surgery on our company. I don't think the boss will be all that impressed and, frankly, it seems like overkill to me, too. We're looking to add an EPSS, not restructure the company. What elements of your proposal can be implemented, realistically, within the existing corporate structure and timelines to get the EPSS going and support its operation?" A specific question was developed for each individual team in reaction to the team's case response. For example, one team was asked to evaluate the effectiveness a proposed action would have in the in the immediate future: "One of your suggestions is to establish a debriefing period after each project is completed. That's an intriguing idea and I can see how it might help us identify successes and failures, but only after the fact. What about now? How do you suggest we use this idea to help us develop our EPSS?" Another team was asked to consider the effects their proposed solution would have on the successful collaboration already in place on the production line: "You've recommended a formalized pipeline-partnering program as part of your OPSS. There is already a lot of "mentoring" going on. Employees frequently consult with one another and this seems to be a very positive part of the pipeline. My concern is that a formal partnering program will be more disruptive than supportive. How do you propose to structure a partnering program so that it will support, rather than disrupt, the existing collaboration?" The specific and common questions and responses can be viewed on-line at: Case Event '98 "Outcomes" http://curry.edschool.virginia.edu/go/Itcases. The case response and response to the Provocateur questions were reviewed (blind) by the six Judges over a two-week period. The quantitative results (Table 1) were compiled from the Judges' ratings of the criteria they had identified prior to the event and followed up with comments reflecting on each team's performance. Each team's ratings were totaled and averaged across the fifteen items (Low:1 to High:4). The outcome of the average team ratings were fairly high, ranging from a low of 2.82 (SD=0.36) to a high of 3.73 (SD=0.29). In general, the Judges felt positive about the Teams' performance in their application of ID theory and demonstrated insight into professional practice (see Table 6). The ratings suggest the Teams' weakest performance was in their identification of project risks and development of project management plans (see Table 6, 14 & 15) which influenced their preliminary design (Table 6, 12). Table 6: Judges Ratings for Case Response
In line with our previous study, the officials we interviewed presented a range of perceptions about the relationship between the Teams' performance in the case competition and their design expertise. Two officials felt the Teams' ability to analyze the case reflected the philosophy of the University IT program the student team represented: "I think some might be in a more theoretical program so that's why in my eyes they may have missed some of the practical, real world things that were going to happen. Others seemed like they had a feel for both, the theory and practice together." Five of the officials agreed that, although the Teams reflected a working knowledge of the ID process and systematically conducted the needs assessment, they demonstrated a wide range of real world ID experience: "Every single team used traditional analytical ID approach to try to break the problem down... Yet, I personally felt that we saw along a continuum from the winning response to the weakest response, an increasing distance from experience in the real corporate world." The students felt it was their experience applying the ID process that affected performance. Six of the seven Teams felt they had a strong background in ID theory and the seventh team noted additional confidence in applying the ID process through their prior work experience. Several respondents felt prior experience enhanced their performance and continued to develop during the case analysis: "I felt our team achieved a very high level of interpersonal relations and the use of instructional design theory. And, the use of relevant experience... All those were well applied in this case and I think we developed it even further during the process of working on this case." One official felt that, in addition to their understanding of the theory behind instructional design, most of the participants understood what an EPSS is. However, their treatment of EPSS was limited to the surface features and did not consider the potential risks commenting, "which is not surprising, because that's not in the literature yet." Three of the officials felt the Teams approached the needs assessment as a novice would, accepting the information presented at "face value." They commented that only two Teams went beyond the general needs assessment and had an awareness of the techniques they might use to obtain additional information so their needs assessment would be more thorough. One official reflected that an expert would not automatically pursue the solution requested by the client as many of the Teams did: "I think they did a really good job as beginning people. The solution to the client should have been to go back and do more of a needs assessment and not go into training at this point. But, I think that is experience. In my first five years, I would have pushed for training. It took a lot more experience, sophistication, and maturity to say to a client, 'Hey, I know what you want to have, but we are not ready to do this.' One official felt the students exhibited a degree of inexperience in their proposed solutions as well, suggesting that novices tend to rush and throw many solutions at the problem simultaneously: "They were throwing so many solutions at the clients at one time that the client would have said this is just too much. This is not us...Whereas going in and fixing a couple of things and explaining to the client that this is a life-long change here and will take time to implement. A sign of youth is wanting to go in and fix the world...It is just a matter of learning and patience that comes with more experience. Four of the officials thought the Teams exhibited inexperience with the treatment of performance support. One official felt most of the Teams neglected consideration of quality management issues, an important factor when designing an EPSS: "Not surprising for novice performers versus expert performers. They weren't taking into account the risk management issues involved in the case... They kind-of assumed they would wave a magic wand and the EPSS would happen and that it would happen well. They tended to underrate the large degree of re-do found in the organization as a quality management issue..." Respondents from most of the Teams felt they were challenged, as well, addressing the EPSS issues. "I was definitely challenged in the EPSS area never having ever worked on an EPSS and only doing the research to prepare for the case. So, I wasn't sure how those really worked. I've never seen one." Quality of the Case When we spoke with the students about the degree of realism in the case, we asked if they thought it was sufficiently compelling so they could enter the case and explore the ID issues. Most of them commented that it gave them the impression that it was real enough for them to take on the roles of the characters: "It was realistic. In fact, I must say that while I was doing it, I was convinced that there had actually been someone who had been in a similar experience and type of industry. That is actually the way I thought about the case." One student commented that the case was not realistic because the animation industry is foreign to her. She did find, however, the personalities of the characters to be believable: "There aren't very many people who create animation and certainly none in my world, so, only like reading science fiction. That can be compelling, but, in terms of real, no. What I thought was good was there were a lot of different human elements in there; the different motivations and perspectives. I thought that was realistic." Most of the officials found the work environment in the case to be extremely realistic: "The organizational context issues I thought were very reflective of the kinds of things you would see in a real organization. The details concerning the description of the work flow and the description of the tasks as far as they went were very plausible..." One official felt it helped fill a gap in instructional design training: "There was not just an obvious find, one or two answers. There was a provision for alternate agendas and politics and this kind of stuff is exactly what I have felt is lacking in an awful lot of instructional design training..." This official also commented that he found one aspect of the case only mildly realistic, and that was the amount of responsibility given to a junior hire. A second official began to feel the case had been contrived as he observed the team moving further into it, but this did not affect the students engagement in the process and they became a part of the story: "There were times when all of a sudden the thing seemed like a real-life project and the team was so involved in it that you wouldn't know they weren't working for the company..." Materials and Management of the Case: When we spoke to the students about the interface of the Web-based case, we were interested in identifying whether the navigational design was efficient. We also wanted to know whether our increased focus on textual character development, in place of audio and video clips, was effective in providing an environment that was easy to access, yet deep enough to present the complexities of the real world. Most of the students interviewed felt the materials at the site were easy to access and reflected that the writing style helped them to analyze the case: "I found that it was easier to process the case because of the way it was not only written, but the way you could access things. For example, the fact that each interview was a separate component on the web... you could read what Carl Durer said. You could chronologically look at what happened to Jason on the day he was hired. So, for content purposes...and for chronological purposes, it was easy to process and you could do things simultaneously by clicking back and forth." Respondents who had read previous Case Event cases thought the shift in focus from multimedia to character development improved the quality of the case: "I think it was a very professionally done job. Your web site looks better and better every year. The case responses are getting easier to read and more understandable every year. I think your learning curve has really gone up and you are putting together a classy act here." A student, who was new to instructional design theory, found it difficult to identify the problem within the complex layers of the story: "It was a huge packet for us to digest in the short period of time. I would highly recommend that it be more compact and not as technical, or else make sure that first-year students are not allowed in. It was a real mismatch to try to put it all together and learn everything as we go." Another student from this team commented: "While this was frustrating, I think it mirrors the actual process in a real-life situation." The graphic interface for the Web pages was designed so the case could be effectively viewed in versions of browsers that had been released within the past year. We found, however, that a few of the respondents were using older browsers that did not recognize colored table cells. Consequently, these students had trouble reading the case on-line because the light colored background of the table cells did not display. And they were viewing dark text on a dark background. A few issues emerged concerning the coordination of the case event involving the limitations placed on the students and the format of the response. The time limit for group meetings was seen as difficult by some of the respondents. One student felt the word limit was enough of a time constraint: "Why the need for the time constraints? Isn't the 2000 word limit for the initial response enough of a constraint?" Another respondent suggested increasing the time limit to match a typical workday: "I understand the need for time constraints, it might be helpful if the six hour limit could be expanded upwards by a small amount such as 8 hours, a usual work day." A third respondent felt that, although a challenge, this is reflective of real life: "You could always use more time, but, I think given the case and the situation, ...it was probably proportionally appropriate to real life." Two of the Teams had contrasting views concerning the format of the case response. A member from one team felt that the restriction to develop a text-only response enabled them to focus on the case analysis and design. Another team felt that the text-only restriction diminished the quality of the response they could produce. They suggested that it would be more reflective of actual practice and allow for greater creativity if Teams could place their responses in the form of a report on the Web, including tables, graphics, and links to relevant Web resources. Relevance to Actual Practice Eighty-nine percent (25) of the student survey respondents and 95% (18) of those interviewed commented on issues presented in the case they found to be relevant to actual practice and several issues immerged. The limited time frame for producing results was seen as a challenge for "managing multiple priorities simultaneously" and this, in turn, fed into the organizational dynamics an instructional designer must contend with in a corporation: "The issues surrounding the great gaps that exist between the younger, "techie" types and the older, more experienced employees were particularly important. Being able to provide solutions to bridge this gap was especially relevant. Also, the fast-paced working environment in this particular case is probably very reflective generally of work environments today where deadlines are too short for the amount of work required. " Perspectives on needs assessment ranged from those who had limited prior experience applying the ID process, and learned "the importance of the environmental assessment and how very complicated that can be as part of the overall picture..." "It gives you a whole different perspective on front-end analysis in a real-world type setting as opposed to a classroom situational-type project where you're just going through the motions." ...To students who felt they were well grounded in design theory and application, yet saw new challenges they would face when conducting a needs assessment in environments similar to the case: "I think the other part that's relevant to the practice of ID is the Jason Tilmans, which we all may be in the future... Where, you're told, 'This is what the need is.' You do the assessment and, albeit, it's a need, but combined with a whole bunch of other stuff... Then, how do you choose to link the need that you've been hired to address, but still be realistic? I felt this was very germane to ID practice because it's rare that you get, 'Here's the need, meet it,' and everything's hunky dory." Most of the students interviewed commented on emergent issues such as EPSS and how they must be integrated with the ID process. Some of the students went on to express how their experience addressing these issues during the case has motivated them to continue learning about the topic: "We thought it was very interesting. In fact, the whole team was really excited about it. We thought that the challenge of EPSS was relevant to things that are going on right now in a lot of industries..." Although most of the students commented on team dynamics when reflecting on the case analysis process, seven of them remarked upon their heightened awareness of the need for these skills in actual practice. They compared this need to the challenges of team dynamics presented in the case: "I mean, just taking it from Jason's perspective, I think it is really very important to have a good understanding of what will gain support and develop a team that you can work with. Not that he necessarily did that, or we necessarily suggested it, but this is what I learned from the experience." Six of the students commented on the need to consider multiple perspectives and solutions as well: "In the process of identifying key issues from a complex array of concerns and issues, it is important to quickly identify what are the biggest causes or factors. Also, being able to identify training versus non-training issues and, of course, working with people is always a useful skill in the real world." The officials were asked to reflect on how this experience contributes to the development of ID students professional skills. Four officials commented on the benefit of reading all of the case responses and officials' comments at the conclusion of the event: "If you read someone else's perspective and response, it remains in the back of your mind where you can call on it when you encounter that type of situation or challenge in your career." One official agreed that the case experience provides an opportunity to practice knowledge and skills and get feedback on the results from several resources. However, he reflected on the difficulty in determining how much the case experience contributed to the students' development of their professional skills: "I am hesitating only because I think it is going to be difficult to tell exactly how much or in what way. They got to compare what they came up with against other students, against Judges, against expert responses, so they got a lot of feedback from several different sources that they could use to do a self-assessment... I've just got to believe that is a growth experience, although we don't know anything about how they use that feedback." Another official noted that his students were interested in learning more about the issues that were identified in the case: "They wished they knew more about some of the issues that came up in the case study, for example, different ways to conduct task analyses. Now they want to learn that in the class. The students involved in this project would jump at the chance to get some more instructional experiences over the things they realized they didn't have much experience in; task analysis, EPSS, and project management." Value of Participation The students and officials were asked to share their opinion on the value of the case study method and the case competition using Likert response options from 1 (strongly disagree) to 4 (strongly agree). The students overwhelmingly felt that the use of the case study method is valuable for developing ID expertise (M = 3.61, SD = 0.50) and that the case event was of value in preparing them for future ID projects (M = 3.57, SD = 0.50). This perception was shared by the officials in regard to the value of the case study method (M = 3.93, SD = 0.27) and the value of the case competition in preparing ID students for future projects (M = 3.64, SD = 0.50). Several of the students and officials share their perspectives on the value of the case experience. Most of the students were positive, describing how the experience enhanced their preparation for professional practice: "The opportunity to solve a problem in an area that I knew so little about was very valuable. I was able to make the connection that instructional design isn't always for stand-up training or CBT... This is by far the best learning experience I have had in my entire educational career." and "I just read in the newspaper ads last Sunday about trainers needed with skills in knowledge management. I thought, this is kind-of neat that this case study addressed all these things that we don't really address in a classroom setting or in some work settings, like the one I'm in. Normally, or before this case study, when I thought of instructional design, I thought of designing instruction for stand-up training or computer-based training or self-based training. I never really had the context to put it in a real-life production environment and it just kind-of opened my eyes to a whole new use for this skill set." All of the officials who responded were positive and shared their perspectives on what was most valuable. There were several enthusiastic comments about the value of the case competition experience: "I think this is one of the most significant events that has occurred in their ID education..." and... "All the theory in the world is useless unless practitioners know their roles as change agents. This case placed the students in a situation where understanding varying perspectives and agendas was imperative. They also had to be sensitive to the repercussions of their recommendations. The case pivoted on this, and, as such, was a great experience for the students. The mere fact that this was event competitive was powerful as well. Of the six officials who are not yet using ID cases in their classes, three commented that they are considering integrating cases into their courses as a result of this experience: "We have actually talked about restructuring the two-semester instructional design sequence to actually take advantage of the case study...because it contextualizes. I mean it is just good instructional design... It comes about as close to situated cognition as you can get over the web... Discussion Sixty-seven percent of the participating students and 82% of the officials responded to the follow-up survey and 43% of the participating students and 71% of the officials agreed to be interviewed. They shared their perspectives on the value of the Case Event as a tool for the preparation for professional practice in instructional design. The students also described the processes in which they engaged to analyze the case and create a design solution, and several (8 of the 14, or 57%) of the officials commented on their observations of this process. In the following section, we discuss the implications of our findings as they pertain to our primary question guiding this study and follow with discussion of our secondary questions. Is a Web-based case competition a worthwhile medium for expanding professional knowledge and exploring emerging issues in instructional design? The potential to expand their preparation for professional practice in ID was noted by the majority of the students who commented (11 of 13, or 85%) on their reasons for participating in the Case Event. Students from two of the Teams that were constructed as a part of a course observed a lack of commitment from team members who remarked they had been required to participate. At the conclusion of the Case Event, however, all of the respondents noted the value of the case experience as a tool for developing ID expertise. This suggests that, although most participants value the ID experience gained during the case event, those who participate by choice are more likely to engage in the process and more fully benefit from the experience. Consistent with our previous study, the participants were enthusiastic about the competition and their engagement in the process began prior to the event. All but one student read at least one of the previous IT cases and the majority of them read the accompanying case responses (85%) and expert (71%) critiques. Prior experience using ID cases as a learning tool had increased since our previous study from only one student to almost one third of the students (28%). In addition to reviewing the materials available on the Web site, other preparation included research of learning theory, review of instructional design literature, and procedures for collaboration. Just under one third of the students we interviewed indicated they had reviewed the recommended articles on EPSS as well. The case analysis and response generation described in this paper reinforces the findings from our previous study. There were several common aspects of coordination exhibited by most of the Teams. This included:
Some Teams divided the case response among team members according to the competition guidelines while others completed all of the sections together or in sub-groups. One Team was unique in its approach, completing individual analyses and conducting research before their first meeting. Some significant new aspects that emerged from the analysis and response generation pertained to collaboration. All of the Teams perceived collaboration as important to productivity and there were various factors influencing team dynamics. Several participants felt prior experience (or lack thereof) working together contributed to their effectiveness as a team. Those with prior experience had few disagreements, while others felt that lack of experience working in large groups diminished the effectiveness of their meetings, in particular sharing multiple perspectives and coming to a consensus as a whole group. Some Teams addressed this challenge by splitting into sub-groups, which was seen by some participants as productive and others as a poor use of time due to the redundancy in the work produced in the sub-groups. Leadership was another factor that emerged as an important aspect of team collaboration. Some participants felt that shared responsibility was effective, while others felt it resulted in varying levels of commitment to the project. In spite of these challenges in collaboration, the majority of the respondents (79%) felt that the team process was one of the most valuable learning experiences in this activity. Slavin's research on group goals and individual accountability is reflected in the team that successfully collaborated due to a combination of factors. These included: small group size, a designated leader and action plan, individual generation of responses, and meetings reserved for the integration of ideas and decision making. Students who volunteered to participate in the case event felt the competition was an important component in the case experience, increasing the sense of challenge and motivating them to participate. Most of these students also viewed the prospect of being judged as motivating. They were compelled to research beyond the given information in the case. One team who volunteered, however, described a negative influence on their performance. They reflected their efforts to "second-guess" the Judges throughout the generation of the case response and commented that it added stress to their learning experience. The majority of the officials who commented on the value of competition agreed that it enhances the learning experience by engaging students cognitively and emotionally. There were concerns however, that the challenge to compete with doctoral and experienced ID students could be intimidating for beginning master's level students. This sentiment was voiced by the team composed of novice ID students who felt they were not prepared to compete without a foundation in the ID process. As in our previous study, the Teams performed well in their case responses. According to the ratings on the overall performance item, the Judges indicated agreement or strong agreement that team performance was excellent with the exception of one team that received a lower rating from two Judges. Ratings on the specific criteria suggest that, in general, the Judges felt positive about the Teams' application of ID theory and demonstration of insight into ID practice. Their weakest performance was in the identification of project risks and the development of project management plans. The Judges related several observations on team performance and design expertise to the differences between novice and expert designers, similar to those identified by Rowland (1992). For instance, the case responses reflected the approach taken by novice designers who accept information at face value and move immediately to the generation of solutions. Most of the Teams pursued the solution requested by the client in the case and were observed to throw many solutions at the problem simultaneously. In contrast, experts will draw from previous experience and reflect on a wide range of factors as they interpret the design problem . This was evident in two Teams who went beyond the case's general needs assessment and demonstrated an awareness of techniques they might use to obtain additional information. Students recognized the effects of prior experience on their performance as well, noting that their expertise applying the ID process and interacting with team members was evident in the quality of their case response. Some students commented that this expertise continued to develop during the case experience. Embedding an emergent issue in ID practice, EPSS, into the case study was seen as one of the most challenging and valuable aspects of the case experience. Most of the participants remarked that the case study introduced them to EPSS and to the necessity of learning how to integrate emergent issues into the ID process. They learned that instructional design goes beyond the scope of training and CBT (computer based design). Some of the officials felt that, all of the Teams performed well as beginners; they were able to address the surface issues in their treatment of EPSS. Most of them neglected consideration of important components of EPSS design such as quality management issues, including risk management. This was considered to be another example of novice versus expert performers. Some of the students commented that they were encouraged to continue learning about the topic while others noted they had become aware of its prominence in the field of ID and were pleased they had been introduced to the topic. All of the responding officials and students agreed or strongly agreed that the Case Event experience had been valuable for developing ID expertise and enhanced their preparation for professional practice. One official noted that the Case Event helped the students recognize their role as change agents by placing them in a situation where they were forced to consider multiple perspectives and the consequences of their recommendations. Officials and students alike felt that the case competition was one of the best experiences that had occurred in their ID education. Finally, one official noted the value of the case competition for IT programs, providing them a means to measure what design skills graduates are bringing into the workplace. Did this case experience alter student perceptions of skills they need for professional practice in ID? When asked what they had learned relevant to actual practice, students noted that their awareness of what knowledge and skills they will need for actual practice goes beyond the theory and models of ID they practice in the classroom. The degree of knowledge and skills recognized correlated to the students' prior experience in the field of ID. Those with limited experience applying the ID process gained a deeper understanding of components within the needs such as the value of the environmental assessment and the necessity for obtaining multiple perspectives through multiple interviews. One of the Teams inquired into studying the instructional design issues they had identified in the case with their team sponsor. Students who entered the competition with some prior experience practicing ID drew a correlation between the case experience and the new challenges they would face in actual projects that might occur in environments similar to the case. Most notable was the necessity to support all assumptions and design solutions with evidence. Both beginning and advanced students recognized the need to continue developing interpersonal skills which are essential to successful practice in ID. Officials saw a valuable opportunity for the students to continue to expanding their professional knowledge, as well, by exploring the collection of IT cases, expert responses, and feedback from the Judges. Did the design of our Web-based case enhance the sense of realism and complexity necessary to compel exploration of ID issues and practice? We focused the design of our case on character development and designed the interface to match its theme. Our results suggest our emphasis on character development and the supporting interface design helped enhance the sense of realism: All but one respondent felt the case was realistic and sufficiently compelling for exploration into the ID issues. The disagreeing student (who was from the beginning ID class) felt limited by her lack of knowledge about the animation industry and found it too complex for entry level students in ID. She did, however, find the characters to be believable and representative of personalities she has interacted with in other work environments. A facilitator-guided analysis of the cases may be a better suited learning experience for beginning ID students who do not yet have the background knowledge of ID theory necessary to solve the case problems on their own. According to Vygotsky (1978), expert facilitators can build on the learners' existing knowledge by providing guidance and feedback, which will help them move beyond their current level of competence. As a result, they can begin to solve problems as does someone with expert experience. Those who commented on the sense of realism reflected in the case study felt that character development and case format helped them to focus on the issues in the case. Some noted they had become immersed in the story, taking on the role of each character, and were able to imagine how the personalities might answer their questions. Most of the officials found the case extremely realistic as well, commenting that the key issues represented the types of problems one would find in any setting. Other aspects of the case contributing to the sense of realism included the context of the company, the design of the organization charts, and the presentation of the interviews in dialogue form. Several officials and students commented that these elements helped the reader experience the case through the eyes of the characters. Limitations of these findings It is possible that students who did not respond may have had different perceptions of the value of the case experience as a learning tool in instructional design. Participants received three email notices about the survey and their Team Sponsors were asked to announce it to the team as well. Survey respondents were sent individual e-mail requests for interviews. We feel that all who wanted to respond had the opportunity to do so. This paper presents the experience of participants in a case competition. Therefore, we cannot assume the level of enthusiasm and motivation exhibited during this activity are representative of case methods employed during a regular ID class. We know that the participants feel they gained skills for actual practice from this experience. We do not know however, whether the students will retain their enthusiasm and apply what they learned in other settings. As one official noted, with the wealth of feedback the participants get through the multiple responses from the participants, experts, and judges, we can assume this was a growth experience. We do not know, however, how the students will use this feedback. Research of long-term outcomes can help us identify whether students successfully integrate what they learned during this experience into instructional design projects, teamwork or project management design. Future research and development directions As the use of cases in instructional design classrooms increases, so will the number of students participating in the case competition who have prior experience analyzing ID cases. Future research of Case Event outcomes can help us identify whether these students are transferring what they learned in prior cases to the new environments represented in the competition cases. This may provide further insight into the effectiveness of case studies in narrowing the gap between novice and expert practice of instructional design. While most of the respondents considered the competition and judging valuable aspects of the case event, some of the students were intimidated by the prospect of being judged. Some students tried to emulate what they thought the Judges would consider the "correct" response rather than using their own perceptions to design what they believed to be the best possible solution. This may have diminished the effectiveness of the learning experience and we would like to know what factors contribute to this reaction to the competition environment. Future research can also tell us whether the judging process diminishes the real world experience for these students. Volunteers? For the past three years, we have facilitated the IT Case Event, writing and producing Web-based case studies and coordinating the competition. To encourage collaboration between IT programs across the country, we would like to extend an invitation to other universities to take on this role or coordinate with some other institutions in facilitating the competition. Each university has unique perspectives on application and theory in ID and we feel facilitation of the event from multiple universities can enhance the case experience for all of the participants. Implications for the training of instructional designers We continue to find that the cases are a useful tool for encouraging students to explore ID issues. Students who participate out of choice will gain more from the experience than those who are required to participate. Limiting group size to a maximum of 4 or 5 persons can provide the best dynamics for discussion and consensus. Web-based case methods; students enthusiastically engage in the case analysis and feel that it helps prepare students for professional practice in instructional design. Cases cannot replace actual experience, yet they can expand the learners' depth of understanding applying the design process. The experience in solving problems in the multiple environments presented in the case studies can increase the novice designer's arsenal of design tools. The competition is motivating and, when implemented in a collaboration environment, the case experience provides a valuable venue for expanding professional knowledge. Marti F. Julian is Doctoral Student and Instructor and Mable B. Kinzie is Associate Professor at the University of Virginia. Valerie A. Larsen is Assistant Professor at Indiana University-South Bend. This paper was presented at the annual meeting of the Association for Educational Communications & Technology (AECT), Houston, TX, February 1999. The authors may be contacted at Department of Leadership, Foundations, and Policy (287 Ruffner), Curry School of Education, University of Virginia, 405 Emmet Street, Charlottesville, VA 22903-2495. Phone (804) 924-7471; fax (804) 924-0747; e-mail: mfj
References Bednar, A. K., Cunningham, D., Duffy, T. M., & Perry, J. D. (1991). Theory into practice: How do we link? In G. J. Anglin (Ed.), Instructional Technology: Past, present, and future (2nd ed., pp. 100-112). Englewood, CO: Libraries Unlimited. Ellsworth, J. H. (1994). Education on the Internet : A Hands-On Book of Ideas, Resources, Projects, and Advice. (1st ed.). Indianapolis, Ind.: SAMS Publishing. Ertmer, P. A., & Russell, J. D. (1995). Using case studies to enhance instructional design education. Educational Technology, 35(4), 23-31. Grabinger, R. S. (1996). Rich environments for active learning. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 665-692). New York: Simon and Schuster Macmillan. Julian, M. F., Kinzie, M. B., & Larsen, V. A. (1998, 4/20/98). The Chronicles of RocketBoy, [Instructional Design case presented via the World-Wide Web in a multimedia format]. University of Virginia. Available: http://teach.virginia.edu/go/ITcases [1998, 12/23/98]. Kent, T. W., Herbert, J. M., & McNergney, R. F. (1995). Telecommunications in teacher education: reflections of the first Virtual Team Case Competition. Journal of Information Technology for Teacher Education, 4(2), 137-147. Kinzie, M. B., Hrabe, M. E., & Larsen, V. A. (1998). An instructional design case event: exploring issues in professional practice. Educational Technology Research & Development, 46(1), 53-71. Kovalchick, A., Hrabe, M. E., Kinzie, M. B., & Julian, M. F. (1998). ID Case Studies via the World Wide Web. In P. A. Ertmer & J. Quinn (Eds.), The ID Casebook (pp. 141-148). Upper Saddle River, NJ: Merrill. Perez, R. S., & Emery, C. D. (1995). Designer Thinking: How novices and experts think about instructional design. Performance Improvement Quarterly, 8(3), 80-95. Perkins, D. N. (1992). Technology meets constructivism: Do they make a marriage? In T. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 18-33). Hillsdale, NJ: Erlbaum. Rowland, G. (1992). What do instructional designers actually do? An initial investigation of expert practice. Performance Improvement Quarterly, 5(2), 65-86. Slavin, R. E. (1995a). Cooperative learning: Theory, research, and practice. (2nd ed.). Boston: Allyn & Bacon. Slavin, R. E. (1995b, 1995). Research on Cooperative Learning and Achievement: What We Know, What We Need to Know, [Web site]. Center for Research on the Education of Students Placed at Risk Johns Hopkins University. Available: http://scov.csos.jhu.edu/sfa/cooplear.html [1999, January 14]. Stepien, W., & Gallagher, S. (1993). Problem-based learning: As authentic as it gets. Educational Leadership, 50(7), 25-28. Teslow, L. T., Carleson, L. E., & Miller, R. L. (1994). Constructivism in Colorado: Applications of Recent Trends in Cognitive Science. Paper presented at the Annual Conference of the American Society for Engineering Education, Edmonton, Alberta, Canada. Vygotsky, L. S. (1978). Mind in Society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. |