ASSOCIATE EDITOR
Linda Howard................Nova Southeastern University
EDITORIAL BOARD
H. K. (Morris) Baskett……. The University of Calgary
Dale Cook.....................…... Kent State University
Karen Garver..................…. University of Nebraska
Jan Jackson....................….. California State University
Janice Johnson..........……... University of British Columbia
Kathleen King.................…. Fordham University
Mary Klinger.................….. SUNY Empire State College
Patricia Lawler................…. Widener University
Norma Long.................…… College of Notre Dame of Maryland
Robert Preziosi..........…….. Nova Southeastern University
Mark Rossman.............…... Capella University
Burt Sisco........................… University of Wyoming
Sue Slusarski..................… Kansas State University
To correspond with New Horizons in Adult Education send email
to
horizons@nova.edu or send postal
mail to the following:
New
Horizons in Adult Education
Nova Southeastern University
Programs for Higher Education
1750 N.E. 167th Street
North Miami Beach, FL 33162-3017
NEW HORIZONS IN ADULT EDUCATION
Volume 15, Number 2, Summer 2001
CONTENTS
On Your Mark: Faculty Development
and Student Evaluation
by Barbara A. Frey and Karen Overfield
The
Effective use of Learning Groups in Online Education
by Cheryl L. Doran
FOR YOUR INFORMATION
HOW TO RESPOND TO ARTICLES ON AEDNET
How to Obtain Back Issues and the Cumulative Index to New Horizons
NEW HORIZONS IN ADULT EDUCATION
Volume 15, Number 2, Summer 2001
ON YOUR MARK: FACULTY DEVELOPMENT AND STUDENT EVALUATION
Barbara A. Frey, D.Ed.
Cranberry Training and Development and
Community College of Allegheny County, Pennsylvania
Karen Overfield, Ed.D.
Education Management Corporation
Pittsburgh, Pennsylvania
Abstract
Evaluation of student learning
represents a crucial, time-consuming part of an educator’s job. In
our years of teaching, we observed many faculty members who, while they
have extensive content knowledge, lack skills in assessing student achievement.
As practitioners in a system of 21 career-focused, post-secondary schools,
we sought to change this situation in our organization. The faculty
in our system consists of over 1,500 instructors teaching in 15 different
majors in the areas of applied arts, design, and culinary arts. Many
of our faculty members come to the school with a great deal of professional
expertise but limited background in teaching methodology. This
article addresses the challenges of developing a faculty professional development
workshop on assessment, measurement, and evaluation of achievement in adult
learners. The workshop represented a developmental experience for faculty
based on their needs.
How do I know if my students really learned? Are my tests valid? What are authentic assessments? How do I assess whether students really achieved the learning objectives?
As educators, we ask these and other questions related to student evaluation. For most educators, evaluation represents not only the biggest challenge in their teaching but also the part of the job they least like. Over the years, we have found this to be true based on personal experience, observation, and discussion with colleagues. Moreover, it does not seem to matter what area of education you are in -- K-12, higher education, public, private, or career-focused; in general, evaluation presents the biggest obstacle for faculty.
Evaluation of student learning represents a crucial, time-consuming part of an instructor’s job. Evaluation, moreover, presents a key element in the educational experience. In fact, many people feel evaluation is what sets education apart from the other types of learning experiences we encounter. This is true particularly of the adult learner. Adults have many avenues for learning both in and out of the classroom. What defines the educational learning experience from the others is evaluation. This critical success factor represents one of the most emotionally intense phases in any learning event for both instructor and student. Competent adults performing effectively in their “day jobs,” successful in managing their lives, often can be reduced to fear and trembling on the prospect of being evaluated. For evaluation to be effective and meaningful, it needs to be done right by people skilled in its application.
Steven Brookfield (1992) states “The only educational justification for evaluation is to assist learning” (p. 22). If we take this approach, evaluation takes on a different dimension. Instead of serving to sort students, evaluation becomes a tool faculty use to enhance student learning, self- assessment ability, and reflection.
But how do we do this? And, how do we help faculty accomplish this? In our role in faculty development, we provide support for faculty in a system of career-focused, postsecondary schools. Our students come to us right out of high school, as adults who have decided they want to gain skill, and as career changers. All of them have a dream. And as faculty our role is to help them attain it. One way to accomplish this is to use evaluation as a tool to enhance student learning. When we do this, we eliminate the use of evaluation as a way to punish students or as a control mechanism to assert power over them.
Our system consists of career-focused, post-secondary institutions, 19 schools spread coast-to-coast. Over 1000 faculty members teach in 15 different majors including business, culinary, technology, and creative fields. Associate and bachelor's degree programs are granted; enrollment consists of 14,000 traditional and nontraditional students.
Our organization recruits faculty from industry. The system of schools takes pride in hiring faculty to teach who are successful and recognized in the field. Faculty members are practitioners. They come to the school with a great deal of professional expertise but little background in education or teaching. The area of assessment and evaluation presents a challenge to our faculty. How do we provide them with the knowledge they need to be skilled educators?
What Are Evaluation, Assessment, Measurement, Grading????
To add to the dilemma, educational terminology contains several closely related, and often confused, words. These terms include assessment, grading, measurement, and evaluation. The terms are often confused and used interchangeably by educators. If these terms create confusion for educators, what about people who entered the field through another route? What is the difference between them? One way to define the terms is as follows:
Assessment Analyzes student accomplishment, careful judging of the quality and range of achievement.
Evaluation The
systematic process of collecting, analyzing, and interpreting information
to determine the extent to which pupils
Measurement The process of obtaining a numerical
description of the degree to which an individual posses a particular characteristic.
Today, educators find themselves in an era in education of increasing concern for accountability. From this perspective, evaluation takes on even greater significance. Grade inflation, social promotion, grade retention, and the standards movement signaled educational reform related to evaluation. Parents sue schools because their children cannot read. Students sue schools because they graduated and cannot get jobs. Employers criticize public education because they cannot find employees who possess the skills needed in today’s workforce. It is a vicious cycle. Educators are no longer protected from personal liability related to their judgements about and evaluation of students.
Grades impact students’ personal and professional goals as well as their self-esteem. Professional schools use grades in making decisions on whom to admit, employers use grades to make decisions on whom to offer a job to; financial aid agencies use grades to qualify people for aid. Valid grades support learning through feedback on student achievement and progress. But what if the grades are not valid? And, how can we be sure they are? What grade do we as educators get for the marks we give?
But how could we address
this critical success factor in education? Developing a policy, defining
a procedure, sending out a memo did not seem appropriate. Since we
are a school, one way to get at these questions was to provide a learning
opportunity for our faculty. We wanted to enhance institution effectiveness
by addressing the root cause and attacking the problem. We felt that
we could all get on the same level and at the problem through creating
a community of scholars and learning together and from each other.
Training On Evaluation For Educators
Since we are responsible for the faculty development function, the authors were faced with the challenge of developing a faculty-training workshop on assessment of student achievement. The goal of the program was to design a faculty development workshop for the system that addressed the area of assessment, measurement, grading, and evaluation. The intent was that the program could be implemented at all of the schools. The objective of the program was to provide faculty with the skills they needed to assess student learning. Implementation thus represented a major challenge.
Because we wanted to involve faculty, as well as instructional supervisors in the overall evaluation process, we formed a task force to develop it. Since the task force was geographically dispersed and limited resources of time and money excluded face-to-face meetings, we relied on technology for our communications. We chose the lowest level of technology to accomplish this. Task force meetings thus took place via audio conference call.
Curriculum Design Model
As the task force began to discuss the program and consider alternatives
for curriculum design, we discovered Donald Kirkpatrick. Kirkpatrick
(1994) identified a ten-step process for curriculum development.
We based our decision to choose this model on several criteria, including
the following:
Kirkpatrick’s model contains these components:
1. Determining the needs
2. Setting objectives
3. Determining subject content
4. Selecting participants
5. Determining the best schedule
6. Selecting appropriate facilities
7. Selecting appropriate instructors
8. Selecting and preparing audiovisual
aids
9. Coordinating the program
10. Evaluating the program
Kirkpatrick’s model, moreover,
served as a project plan for the task force. It provided a step-by-step
process to follow to take the idea from concept to reality. The model
helped us manage the project and turn a plan into a reality. A travesty
in education is that we often have wonderful plans that never get implemented.
Determining The Needs
Since the audience for the
workshop was faculty within the entire system, the need for training in
assessment of student achievement was identified by using several different
methods. The use of these different methods provided the opportunity
to include a wider sample of the intended audience.
The results of the needs
assessment reflected a need for training in alternatives in measurement
and evaluation. The areas of high interest consisted of the following:
learning contracts, class critiques, checklists, rating scales, pretests,
case studies, and self-evaluations. Most faculty members demonstrated
high interest and open minds toward the topic. It was obvious that
faculty had many styles of grading that included both norm-referenced and
criterion-referenced standards, point and letter grade systems, and various
levels of organization and criteria. Not surprisingly, the majority
of assessments were conducted in the cognitive domain.
Students expressed an overall satisfaction with the assessment techniques already in place. Their primary interest was in as much feedback as possible in as many formats as possible, other than cognitive-type tests.
Nine administrators completed our needs assessment survey. Administration consisted of deans of academic affairs and department chairpersons. Their areas of focus were establishing criteria and developing a variety of criterion referenced assessment techniques. They expressed concern over grade inflation throughout the system. Administrators were also looking for communication and documentation to reduce legal and ethical challenges of the education system.
Setting Objectives
The following objectives resulted from a thorough analysis of the needs assessment data:
Determining Subject Content
The instructional objectives developed into the following evaluation workshop content outline:
To role model what we were teaching, we used various types of assessment
techniques and aligned the objectives with the content and activities.
Workshop Operations
For us, workshop operations represented an extensive, time consuming step. For convenience, we included steps four through nine in Kirkpatrick’s (1994) ten-step planning and implementing process: selecting participants, determining the best schedule, selecting appropriate facilities, selecting appropriate instructors, selecting and preparing audiovisual aids, and coordinating the program. Because of the close relationship of all of these steps, much of the work on them overlapped and thus was done simultaneously.
The task force debated much over the audience for the workshop. Some felt schools should deliver it as part of their in-service program. Others felt it should be part of an orientation/induction program. Still others felt it should be delivered on an as needed basis. Another issue was whether to make the training voluntary or mandatory. To ensure quality dialogue and transfer of learning, the faculty training group size was limited to 20 - 22 participants.
Another related administrative issue was the length of time for the workshop. The module’s length was projected at 3 hours. Because of time limitations on faculty, one option was to hold the session as a half-day workshop. Another was to hold it during the lunch hour or preparation time for 45 minutes every week for 3 weeks. Still another was to pay faculty to come in and hire substitutes to teach their classes.
Where to hold the session represented another challenge. Should it be at the school or off site? If it were at the school, faculty would likely be interrupted. If it were off-site, we would have transportation and facilities costs to consider.
Because of the unique needs of the schools, we left these coordination choices up to the schools. As a job aid for the program planner, however, we included administrative checklists in the leader’s guide.
The design of the workshop itself was based on theories of adult learning. The task force agreed that we needed to model the principles of adult learning in the workshop if we wanted our faculty to. We therefore wanted to design a highly interactive format making use of individual, partner, and small group experiential activities.
A related design issue was consistency and duplication of delivery. As a task force, we reached consensus that we wanted the workshop to “belong” to the schools. Since we had 19 schools and did not have someone dedicated as a “system” trainer, we needed a way to create a package someone could deliver on demand at the school. We agreed we wanted a format for the leader’s guide that would serve as a resource for the facilitator. The format should be flexible enough for the facilitators to “make it their own,” yet structured enough so they would not have to duplicate effort. We, therefore, devoted much time and effort to the design of the leader’s guide, support materials, and packaging. The leader’s guide contained both an outline of each section of the module as well as a “scripted” version to help the facilitator prepare to teach the module. Transparencies and participant handouts were created as well as answer keys and explanations. Participant manuals provided the course and content overview, activities, assessment examples, related articles, and a resource list. We provided disks for the facilitators to modify and duplicate the documentation.
Once the design was completed, we conducted a pilot. The pilot provided a way for us to conduct formative evaluation. The pilot, moreover, gave use a way to evaluate workshop effectiveness in a clear, timely manner. One of the schools volunteered to conduct the pilot as part of their regular faculty in-service program. Minor modifications were made to the program resulting from feedback from the pilot study.
To build a community of learning,
we decided to certify people in the system as facilitators. This
process served as a form of reward and recognition for faculty. Schools
could nominate strong faculty to go through the certification process.
The process consisted of participating in the workshop, team teaching it
with a “certified” facilitator, and then teaching it on their own.
Once “certified” they could teach the module at their own school or perhaps
be called upon by other schools to deliver the workshop at their location
to their faculty.
Evaluating The Program
According to Kirkpatrick's ten-step process for planning and implementing programs, evaluation constitutes the last step. How can you tell if the program worked? What value did you place on the event?
Evaluation represents a critical success factor in program planning. In fact, evaluation plans can comprise an entire plan within the program plan. Unfortunately program planners quite often leave this step out. In their haste to implement the program and get it out of the door, they neglect to evaluate it or try to retrofit something after the fact.
In reality, program planners should select indicators they plan to use for the evaluation as part of the up-front planning process. If you do not have a clear objective of what you want to measure, it is difficult if not impossible to measure it. When benchmarks are not identified, organizations have no way to identify the value added the program provides. This practice in many cases leads to the reality that the first area organizations cut when times are tough are the training programs, simply because they cannot prove whether or not they did what they said they would. We as educators need to become more proficient at identifying both the qualitative and quantitative benefits our programs provide.
Kirkpatrick developed a 4-level
evaluation model that correlates to his ten-step process for planning and
implementing programs. This evaluation model is widely used in business,
augments the ten-step process model, and correlates well for education.
Kirkpatrick's evaluation model contains these levels:
1. Learner reaction
Were the participants satisfied with the program
2. Learning
Did the participants change their attitudes, enhance their knowledge, or
increase skill levels?
3. Application
Did the participants use what they learned?
4. Results
What was the benefit to the organization?
Following this evaluation model, the program contained several levels of evaluation. Four measures were used to establish the effectiveness of the program:
Logic tells us that a
positive reaction is more likely to enhance a transfer of learning.
One way, therefore to measure the effectiveness of the workshop was whether
the participants reacted favorably to it. Reaction surveys included
ratings on facilities, content, presentation, and activities. Space
for additional comments was also provided; and signatures were optional.
Each time the workshop was delivered, the same evaluation was used.
In this way we could analyze, monitor and compare the results.
During the workshop, faculty had opportunity to revise and modify several of their course records. The review of instructor records provided a basis of comparison between before and after the training session. The records that were reviewed included course syllabi, project descriptions, and faculty grade books. Through assessment techniques and strategies designed for the workshop, faculty increased their awareness of the concepts. Greater attention seemed to transfer to more specific, carefully worded objectives. Objectives were more closely linked to evaluation techniques.
In addition, instructional supervisors periodically review course records generated by faculty. After faculty attended the workshop, instructional supervisors commented that tests reflected more multiple choice items at various levels of Bloom's Taxonomy and less true/false and essay questions.
Evaluation of behavior is Kirkpatrick's third level of evaluation (1994). Classroom observations reflected the level of communication of grades and grading criteria. Classes targeted were first course meetings, class critique sessions, and sessions in which major projects or exams were returned to students. Moreover, in the focus groups conducted as follow up to the workshop, faculty reported more time and attention devoted to writing meaningful comments. They also stressed their goal to return assignments in a more timely fashion. Overall, the workshop seemed to enhance instructors' confidence in their grading system.
Focus groups with stakeholders
were a part of the initial needs assessment. Following the training,
focus group meetings with students, faculty, and academic department chairpersons
compared the same issues to compare the productivity of the training workshop.
Conclusion
A review of the literature written on faculty development noted a resistance of faculty members to participate in training. Generally, faculty are overwhelmed with time constraints and view training as necessary for others, but not for themselves (Maxwell and Kazlauskas, 1992 and Millis, 1994).
In this case, workshop designers recognized the importance of faculty participation in the planning of the faculty development effort. There was a successful balance of faculty, student, and administration participation in planning. The design of the program was not aimed at ineffective faculty, but offered opportunities for all. Having a faculty member co-present with an administrator aided in achieving faculty support for the program. The message to faculty was they were valued and respected. Excellence in higher education is inseparable from the excellence of its faculty.
Conducting a pilot workshop with a small group of faculty was essential to the ultimate success of this project. The piloted study allowed for reinvestigation of activities and time allocation. Feedback allowed trainers to evaluate the readiness of faculty for the program content. Design modifications were made to allow for a more effective transfer of learning.
It appears well planned faculty
development programs can make a difference. In the future, faculty
will be faced with the continued challenges of under-prepared students,
technological innovations, and increased accountability. Faculty
must be helped to meet these and many more challenges through lifelong
learning and development.
References
Angelo, Thomas A. (1993). Classroom assessment techniques. San Francisco: Jossey-Bass.
Brookfield, S. (1992). Giving helpful evaluations to learners. Adult Learning: 22-24.
Kirkpatrick, D.L. (1994). Evaluating training programs. San Francisco: Berrett-Koehler..
Maxwell, W.E. & Kazlauskas, E.J. (1992) Which faculty development methods really work in community college? A review of literature. Community/Junior College Quarterly, 16, 351-360.
Millis, B.J. (1994). Faculty
development in the 1990's: What it is and why we can’t wait. Journal
of Counseling and Development, 72, 454-464.
Appendix A
Faculty Needs Assessment Questionnaire
Assessment, Measurement, and Evaluation
Please answer the following questions on techniques to measure or assess student achievement. We will use your feedback to guide us in future faculty development initiatives. We are interested in how you are evaluating your students and establishing grading criteria, as well as your interest in developing additional techniques.
| GRADING METHODS | CURRENTLY USING |
INFO |
|
| A. Pretests | |||
| B. Learning Contract | |||
| C. Quizzes | |||
| D. Tests (unit, midterm, final) | |||
| E. Papers (research, term, essay) | |||
| F. Assignments | |||
|
G. Projects |
|||
|
H. Journal |
|||
|
I. Case Studies |
|||
| J. Checklists / Rating Scales | |||
| K. Self-Report Techniques | |||
| L. Student Presentation / Demonstration | |||
| M. Peer Appraisals | |||
| N. Other:_________________________ |
2. Indicate with a check mark which of the following criteria
you include in determining students’ grades.
| CRITERIA | YES | NO | N/A |
| A. Effort | |||
| B. Class Participation | |||
| C. Attendance | |||
| D. Attitude | |||
| E. Observations | |||
| F. Spelling | |||
| G. Grammar | |||
| H. Handwriting | |||
| I. Other _________________ |
3. If you give paper and pencil tests, please indicate with a check
mark which types of questions you currently use or those you would like
to consider or learn more about.
| TEST STYLE | CURRENTLY USING | MORE INFO | N/A |
| A. Multiple Choice | |||
| B. Matching | |||
| C. Essay | |||
| D. Case Study | |||
| E. Completion | |||
| F. Rating Scale/Checklist | |||
| G. True/False | |||
| H. Matching |
4. If you teach a studio course, please indicate with a check mark those
techniques you use to assess student achievement. Also check those
techniques on which you would like more information for future consideration.
| TECHNIQUE |
CURRENTLY USING |
MORE INFO | N/A |
| A. Critiques - Individual or Group | |||
| B. Meet Objective / Solve Problem | |||
| C. Use of Type | |||
| D. Color Scheme | |||
|
E. Rendering Techniques |
|||
| F. Overall Presentation | |||
| G. Neatness / Craftsmanship | |||
| H. Construction Skills | |||
|
I. Thumbnail Sketches |
|||
|
J. Finished Comprehensives |
5. Please answer the following questions based upon the grading procedures
of your classes.
| QUESTION | YES | NO | N/A |
|
|
|||
|
|
|||
| C. Are grades students earn in your class an accurate indication of what they have learned? | |||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
|
Appendix B
Faculty Focus Group Questions
A. Do you grade a finished product, idea, or procedures? Please
explain.
B. Does the amount of time spent on a product/project influence the
grade?
C. On what criteria do you evaluate student projects? How did
you determine this criteria?
D. On what areas of student measurement or grading would you like more
information?
E. Are you confident of your current grading procedures?
NEW HORIZONS IN ADULT EDUCATION
Volume 15, Number 2, Summer 2001
THE EFFECTIVE USE OF LEARNING GROUPS
IN ONLINE EDUCATION
Cheryl L. Doran
Capella University
Minneapolis, Minnesota
Abstract
The process and effectiveness of small group collaborative activity in an asynchronous online graduate course offered by Capella University was investigated using a qualitative case study design. Research questions addressed the structure and support of small group collaborative activity within the online setting. Participants were working adults (teachers at the K-12 and post-secondary level, instructional designers, and corporate trainers) pursuing graduate degrees in education; necessary proactive and reactive means of nurturing group processes were addressed. The nature of assignments posed to groups, the issues surrounding group size, and the use of a systematic approach for enabling feedback between students were investigated. The development of group charters requiring all group members to agree formally, at the beginning, on group processes and individual responsibilities within groups was found to be an essential component of successful groups. This paper consists of the background and context of the study, followed by the researcher’s conclusions and recommendations for improved course design, group structure, and instructor support.
Introduction
Distance educators continue to struggle with attempts to find ways to personalize the learning experience and promote active, engaged student involvement. A challenge facing distance educators is to “search out means of reducing structure and increase dialogue so that learners may move from being simply recipients of knowledge to actively embracing and working with objective knowledge to make it their own” (Lauzon, 1992, p. 34). As noted by Garrison, in an educational setting, “information must be shared, critically analyzed, and applied in order for it to become knowledge” (Garrison, 1990, p. 13).
Collaborative learning consists of educational approaches that are used to develop and engage interdependence among individuals involved in learning situations. One means of instructional design practiced in the traditional face-to-face setting to enhance collaboration among students is the use of small group activities and assignments. Small collaborative groups provide a setting in which students “use shared mental models to solve problems jointly, receive interpersonal feedback, and receive social support and encouragement to take risks in increasing one’s competencies” (Johnson & Johnson, 1993, p. 2).
There are few guides for implementing collaborative learning strategies online. Can collaborative learning, through the use of small group activity, be successfully implemented in the online distance learning environment? Using a qualitative case study design, the researcher investigated how small group activities can be structured and supported in order to enhance collaborative learning in the online environment. The study provided data regarding how groups function within an online graduate-level course, as well as data reflecting learner attitudes toward group learning in that environment.
Background
Capella University, founded in 1993, is a distance education institution offering graduate degree and certificate programs in the fields of Business, Human Services, Psychology, and Education. The University, located in Minneapolis, Minnesota, strives to be a leader in the development of new strategies applicable to the distance learning environment.
Capella presents courses in two formats: a directed study format (similar in nature to independent study) and online using an asynchronous discussion group format. Online courses at Capella are offered on a quarterly basis and typically include eight units (one unit per week) consisting of short lectures and assigned reading, upon which discussion questions are developed and posed to students. Student responses to the discussion questions are developed individually and posted to the discussion area associated with the course.
The researcher conducted a study using an online course, The Future of Educational Institutions, as the context. Participants were generally working adults (teachers at the K-12 level and post-secondary level, instructional designers, and trainers in corporate settings) pursuing graduate degrees in education. Based on assigned readings, the instructor posed unit questions that were to be addressed in the asynchronous discussion group by individual students and questions that were to be addressed by dyads or groups of students. Students were expected to complete a course project by the end of the quarter, as well. The group activity aspect of the course was the research focus of the study.
Two sections of the course were developed, differing only in terms of one variable, group size. Students were randomly assigned to one of two sections of the course by Capella University employees working in the school’s Office of the Registrar. Instructors assigned students to groups of 3 or 4 in one section and to groups of 2 in the second section of the course, based on introductory biographies posted in the first week of the course.
The assigned reading material was identical for both sections, as were the assigned “individual” and “group” discussion questions. Groups in both sections were required to write and submit a “group charter” to the course instructors by email. The charter served as an internal “contract” that detailed how members would proceed with group activities. The charters were developed during the second week of the course in order to help group members clarify such logistical issues as member roles, responsibilities, means of communication, and timelines to be applied in completing group assignments. The charters were submitted to the instructors, who reviewed them for completeness and assisted with revisions as necessary.
The researcher sought to gather data on the following research questions:
Data
Data collection procedures
used in the study included semi-structured phone
interviews, a mid-course evaluation (via email), a follow-up
questionnaire, and analysis of the generic course evaluation offered to
all Capella’s online students upon completion of a course.
Results of phone interviews and evaluations reflect that working with others in a small group setting leads to perceptions of higher quality, more reflective responses than if respondents had worked alone. One student states, “I believe this experience can open me up in my learning by causing me to be more attentive or open to the opinions of others in my group. I’m forced to take seriously the responses of [fellow group members]”. Another student reports, “I learned a great deal from the group work. The advantages are plentiful. Collaborating enhances learning; sharing allows partners to appreciate different points of view as they work together to combine responses”.
Positive comments from students also reflect an appreciation for the opportunity to connect with others socially and emotionally. Responses include, “The most important part of the group process for me has been connecting with other individuals in our group. I feel a stronger bond with these people than to some of the colleagues I work with and see every day. This is a great way to build relationships and it broadens my whole world”. Another student responds, “It is really fun sharing with partners living in various parts of the country, sharing the unique perspectives of people from different locations and how these perspectives impact education.”
When asked to report negative aspects associated with the group component of the course, the most glaring concerns involved restrictions caused by time and technology. For example, a student comments, “True collaborative work requires a clashing of opinions; you can’t do this with email. Communication and collaborative are an exchange of strong opinions on a topic so that all parties can analyze these opinions. With email, it’s too easy to be misunderstood. The greatest drawback is the ability to truly communicate by email!” Another student reports, “The constraints of email communication is a challenge. We’re somewhat limited by the media we’re dealing with. When you’re trying to dialogue, email is difficult. You can’t see a person and it’s harder to ask clarifying questions. Also, the process takes so much time!”
Data concerning group charters indicated that, although the process was time intensive, the charters were essential to successful group functioning. Charters varied in detail, but all addressed roles, responsibilities, and time frames. A student reports during a phone interview, “As a beginning exercise, it’s important to set group rules for operation and especially when bringing online learners together from all parts of the world.” Another student reports, “The charter really did help and it kept everything straight – roles, requirements, timelines. It gave us a chance to think about how we wanted to communicate and work together.”
Respondents, however, indicated the need for more supports in the development of group charters. A sample of a group charter was posted online for groups to use in developing their own charters. Each group was asked to email their charter to the course instructor. One student suggests that “it would have been helpful to see all the charters posted online and to have gotten feedback from the instructors about our charter.” Another student states, “I would have loved to read the other charters. I think this would have been really informative as to how the groups are working and could serve as models for us if we were to run into snags. I would have liked feedback on our charter from both the instructor and other students.”
Students reported at least an adequate access to the course instructor; however, there was a perceived need for greater support for the group process. Direct training of group functions were not structured into the curriculum. Several students made comments similar to this: “I would like to experience some sort of team building exercise and hints for how we can incorporate our individual points of view.”
Data concerning the nature of assignments posed to groups indicated that the “best” questions were those that were complex, somewhat ambiguous, and difficult to answer. One student states that the most challenging questions posed to the groups were those that “encouraged creativity in our thinking. There were no easy answers and because of this we had to really stretch our minds to come up with something we could all get behind.” As another student comments, “Some of the questions posed to our group required a lot of thought and synthesis of information. They enhanced our interaction because they required us to correspond quite a bit in order to collaborate in coming up with an answer.”
Conclusions
Small group work within an online course environment can enhance student satisfaction, the depth of instruction, and breadth of learner perspectives. However, time and technological constraints can become difficult barriers to overcome for students attempting to successfully engage in the dialogue necessary for collaboration to occur (Palloff & Pratt, 1999). Individual differences in learning style and preference require the consideration of alternative strategies when constructing an online learning environment that includes small group work.
The group charter was confirmed by this study to be one of the most powerful ingredients of successful group work in an online environment, where students have little opportunity for face-to-face interaction. All students, whether they liked group work or not, reported that the development of a group charter was integral for group evolution and collaborative processes. The group charter established for members a structure within which to work together. Additionally, communications involved in writing the charter appeared to nurture a sense of interdependence among members. Group collaboration occurs when “a group of individuals learn in concert with and through each other” and “come to know, trust, and depend on each other” (Armstrong & Yarbrough, 1996, p. 35).
The study confirmed that proactive and reactive supports are necessary in relation to group processes. The psycho-emotional needs of students must be considered and supported when implementing collaborative learning in an online course. Students will vary in their willingness and initial ability to function within a collaborative learning experience, thus requiring various amounts of supportive coaching and instruction regarding how to operate in such an endeavor. Team-building exercises and class discussion regarding the group process should be included during the initial phase of the course. Group and dyad members need to be presented with a systematic approach that they can use in providing supportive feedback to one another.
Although effects of group size on learner satisfaction and perceptions of learning enhancement were unclear, groups of two and three do appear to be well suited in the online learning environment. A group of three is likely to represent greater diversity of perspective and, thus, a greater need for dialogue and collaboration in the development of responses to instructor-posed questions. However, students new to online learning environments may be best served by placement in dyads, which may be somewhat easier to negotiate than a threesome.
The most effective questions posed to groups were those that were complex and not easily answered, required active problem solving, and required collaborative efforts of members in arriving at solutions. Moreover, these questions need to be relevant to the diversity of backgrounds and interests reflected by students taking a particular course.
Implications
Results of the study indicate that small group work within an online course environment can enhance student satisfaction and learning. The word “can” is key. Many students reported that small group interactions led to responses that were more critically reflective than if responses had been developed individually. Those interactions encouraged students to openly share their unique experiences and critically examine the lens through which they interpreted both the material read and the questions posed by the instructor. In addition, many students enjoyed the opportunity for intense interaction with other students afforded by small group work. For these students, collaboration with a small number of individuals negated the feeling of isolation that can occur in distance education.
However, some students questioned if true collaboration could take place given the time and technological constraints they faced. Deep, interactive, and sustained discussion was challenging within the course timeframe, utilizing e-mail (mainly) as the vehicle of interaction. This concern surfaced in comments made both by students that were satisfied with the group component of the course and by students who were not satisfied with the collaborative group requirement.
Based on the research, several implications consistent were offered for the modification of that course format in future offerings:
The instructor can model
collaboration by providing the overall framework for a course and then
encouraging students to collaborate with the instructor when developing
group assignments as they progress throughout the course. Such assignments
would fit into the instructor’s overall instructional goals for the course,
but would be individualized to meet the needs of a specific group or dyad
of learners. Students might identify real-life problems to work on that
reflect the set of objectives developed by the instructor for a particular
unit of study. In this way, a group of students can create problems that
mirror the significant challenges occurring in their professional situations.
Summary
This study was undertaken
as an exploration of the use of collaborative, small group activity within
an online learning environment. Small group collaboration can be an effective
instructional strategy for enhancing learner satisfaction and learning.
However, many variables must be considered when designing and implementing
this strategy in order for it to be successful and enriching. A great deal
of serious consideration must go into course design, group structure, and
necessary instructor supports. This study touched on several variables
that can be fully understood only through future study by distance educators
as they continue to explore and confirm effective practice in this exciting
and most promising learning environment.
References
NEW HORIZONS IN ADULT EDUCATION
Volume 15, Number 2, Summer 2001
HOW TO RESPOND TO ARTICLES ON AEDNET
To respond to the articles in this issue please send your comments to AEDNET identifying the subject as the title of the article.
HOW TO OBTAIN BACK ISSUES
AND CUMULATIVE
INDEX OF NEW HORIZONS
It is now possible to search the AEDNET
archives
via a web browser.
All the archived issues of New Horizons in Adult Education can be located
at the following: http://www.nova.edu/~aed/newhorizons.html
CALL FOR MANUSCRIPTS
New
Horizons in Adult Education, founded in 1987, is a refereed
electronic journal that provides faculty, graduate students, researchers,
and
practitioners with a means for publishing their current thinking and
research within adult
education and related fields. The journal is published two or three
times a year and is
available electronically on the Adult Education Network (AEDNET) web
page
http://www.nova.edu/~aed/newhorizons.html
New Horizons publishes research, thought pieces, book reviews, point-counter-point articles, conceptual analysis, case studies, interactive articles, and invitational columns. The editorial staff welcomes articles for review submitted either electronically or in a variety of disk formats through regular mail. Guidelines for manuscript submission are available on the AEDNET and New Horizons in Adult Education web page at http://www.nova.edu/~aed/newhorizons.html
If you would like to submit an article, you may contact New Horizons in Adult Education by e-mail or mail.
E-mail address: horizons@nova.edu
Mailing address:
Nancy Gadbow, Editor, New Horizons in Adult Education
Nova Southeastern University
Programs for Higher Education/FCAE
1750 N.E. 167th Street
North Miami Beach, FL 33162-3017
