STUDENT
EVALUATION OF COOPERATIVE LEARNING:
THE INTERPERSONAL INTELLIGENCE INVENTORY
Paris S. Strom, Auburn University
Robert D. Strom, Arizona State University
Presentation to the Annual Meeting of the
American Educational Research Association
Chicago, April 2003
Abstract
One of the most perplexing tasks of teachers using cooperative learning
is to identify teamwork skills exhibited by students during group work.
The Interpersonal Intelligence Inventory is designed for students to evaluate
peer and self-performance. Responses inform teachers about team interaction
from the students’ point of view. The purposes of this instrument
are to identify team skills that individual students demonstrate, provide
anonymous feedback from peers, compare self-impressions with peer observations,
detect learning needs, credit hard workers for initiatives, detect slackers,
and yield an easily understood record of social skills. Outcomes can be
used with student portfolios, and to establish a school data bank on teamwork
skills. Field-testing with 303 high school students determined that psychometric
indicators are in the good range.
* * *
Students should practice the skills that will be required of them as adults.
It is impossible to predict all of the competencies that should be emphasized
so children are well prepared for their future. However, by examining
changing conditions in the labor market, projections for the knowledge
explosion, and shifts in lifestyles, some crucial lessons can be identified.
There is general agreement that functioning as a team member warrants
greater attention. Working alone is a long-standing tradition that is
beginning to fade in favor of students working together in groups (Johnson,
Johnson, & Holubec, 1999; Kagan, 2001). In this context, teachers
are expected to arrange teamwork activities that can facilitate the development
of interpersonal intelligence (Gardner, 2000).
Team Assessment of Individual Performance
Some practical alternatives for evaluating teamwork activities are provided
by corporations, such as Bank of America, Disney, Ford, General Motors,
Hewlett-Packard, Intel, Microsoft, and Motorola. They rely upon group
perspective approaches for evaluating teamwork of individual employees.
Many employees believe that the customary job performance review in which
one person judges the work quality of subordinates on a random schedule
is unfair and obsolete. Workers may ignore suggestions from a supervisor
by rationalizing that ‘the boss does not see me enough to be aware
of all the things I do or how well I perform.’ Evaluation by peers
on the job has gained credibility in the past decade. Criticism and suggestions
from them are taken seriously. Peer evaluations are preferred because
co-workers observe one another daily, know how well they perform tasks,
and can detect limitations. Management also supports team assessment because
this approach has been found to increase productivity (Edwards & Ewan,
1996).
Because group evaluation is becoming prevalent in the employment sector,
adolescents and college students need to learn how to evaluate the performance
of their peers, judge personal competence, and benefit from the criticism
of teammates. Yet, a perplexing task of teachers who use cooperative learning
is to identify teamwork skills demonstrated by each student. Teachers
are uncertain about how to conduct assessment that is accurate and fair.
Students also express disappointment regarding group work evaluation.
Suitable instrumentation can help overcome this limitation (Antil, Jenkins,
Wayne, & Vadasy, 1998).
Student Accountability for Group Work
There are three ways in which accountability for students working in groups
should be redefined. First, accountability should emphasize how each person
contributes to their team. Faculties favor this condition for evaluation
of their own competence. That is, teachers attempt to facilitate achievement
by helping all class members but, in the final analysis, each student
is responsible for his or her own performance. When this view of teachers
is applied to group work, no student can reasonably be held responsible
for the behavior of peers. There is greater motivation to cooperate because
individuals are judged on the basis of recognized teamwork skills and
personal test performance.
Secondly, faculty should share some responsibility with students for evaluation
of group learning. Teachers are experts in their subject so they are the
best qualified to judge work products submitted by teams. However, faculties
are not the best evaluators of what happens in groups because they are
seldom around to see the interaction process. Even when present, they
cannot tell how the initiatives of some students affect the thinking of
others. Students are disappointed when faculty ignore them as a source
of information about how peers contribute to their learning and the outcomes
of team effort. Team members are in the best position to identify the
teammates who influence them and describe the nature of help given.
A third element of this strategy involves comparing self-evaluation with
the observation of peers. Teachers frequently evaluate students by testing.
Learners also need self-evaluation skills in order to make wise decisions
in an environment characterized by over choice and a work place where
knowing how to collaborate is essential. Cooperative learning provides
an ideal condition for a comparison of self-impressions with the observations
of peers. In this way, students can gain the ability to judge themselves.
Guided practice in self-appraisal is necessary to become self critical,
which is vital for performing well in groups. The ability to self- evaluate
enables us to know when to think well of ourselves and when it is appropriate
to change our behavior so it more closely resembles the person we wish
to become.
Student accountability is more likely when all parties understand in advance
the criteria for assessment and processes that teammates will use for
evaluating them. Faculties are implicated too—they must trust students
to participate in evaluation of team learning, and convey the importance
of authentic reporting. The credible way to find out if desired competencies
are learned is to solicit perceptions of teammates. Students observe what
happens in groups, know how each teammate influences their thinking, and
can evaluate self initiatives to enhance productivity. Frequent opportunities
to apply peer and self-evaluation allows students to practice the skills
needed to become better judges of productive self and group functioning
in class, on the job, and at home.
Concepts of Multiple Intelligences
Most of the early test designers accepted the concept of multiple intelligences.
Alfred Binet and Theophile Simon (1905), credited with developing the
first mental tests, saw intelligence as an aggregate of separate cognitive
functions including comprehension, judgment, direction, and invention.
Edward Thorndike (1927) believed that intelligence is demonstrated by
a capacity to respond effectively to novel situations. Just as there are
different kinds of situations, there are different patterns of intelligence
- abstract, mechanical, and social. For example, the skilled leader of
people may be quite inept in mechanical matters while the skilled mathematician
may be notably poor in social intelligence.
Louis Thurstone (1938) described intelligence as composed of seven primary
mental abilities -- verbal comprehension, number ability, word fluency,
spatial visualization, associative memory, reasoning, and perceptual speed.
The Stanford-Binet test, first produced by Lewis Terman and Maud Merrill
(1937, 1960), includes verbal and nonverbal mental tasks. David Wechsler
(1949, 1997) published separate intelligence scales for children and adults.
Joy Guilford (1967, 1981), in his theoretical model known as the Structure
of Intellect, offered the most inclusive concept suggesting the existence
of 120 intelligences –- most of which can be assessed somewhat acceptably
with current measures.
During the past decade, memorization in classrooms has been joined by
greater attention to other mental processes. Much of what students memorize
is current for less time than ever before because information obsolescence
occurs more quickly. Accordingly, creative and critical thinking abilities
needed for solving problems have emerged as primary goals. This shift
has led to increasing interest in measures of creative thinking. Paul
Torrance’s (2000, 2002) figural and verbal tasks reveal originality,
elaboration, fluency, flexibility, and redefinition abilities. Daniel
Goleman (1999) has documented the need to nurture emotional intelligence
to improve the quality of life in a hurried and stressful environment.
Emotional intelligence consists of self-awareness, impulse control, zeal,
persistence, self-motivation, empathy, and social deftness. Robert Sternberg
(1988, 2000) contends that personal success is related to three separate
types of intelligence –- analytical, creative, and practical.
Howard Gardner (2000) proposes that schools recognize eight frames of
mind: (1) verbal – linguistic abilities which implicate speaking,
reading and writing skills;(2) logical – mathematical abilities
which enable deductive and inductive reasoning; (3) visual-spatial abilities
which allow a person to create representations of the world and to think
in pictures; (4) musical – rhythmic abilities which present sensitivity
to pitch and rhythms of sounds; (5) bodily – kinesthetic abilities
which contribute to motor skills and graceful movement; (6) naturalist
abilities which allow someone to observe patterns in nature and understand
natural and human-made systems; (7) intrapersonal-introspective abilities
which permit someone to be deeply aware of personal feelings and purposes;
and (8) interpersonal abilities – the social capacity which makes
it possible to work effectively with others. This latter set of capacities,
sometimes referred to as “social intelligence” is the focus
for the Interpersonal Intelligence Inventory (III) (Strom & Strom,
2002).
Measurement of Interpersonal Intelligence
What conditions should be met by instruments in this context? Should students
be presented with hypothetical situations and their projections of how
they would expect to act be used as indicators of interpersonal intelligence?
Reflections about how one might respond in a given set of circumstances
may differ from the way a person would actually behave in these conditions.
Consequently, it makes more sense to hold students accountable for actual
behavior than for hypotheses of how they suppose they would behave if
they encountered particular situations. One set of responses relates to
behavior that can be corroborated while the other involves imagined behavior
that no one else could confirm. A more appropriate strategy is to rely
upon multiple observers who have sufficient experience to confirm or refute
self-impressions of students regarding their performance as a teammate.
This design relies on the perceptions of peers to identify social skills
which a person demonstrates in repeated interactions with others. By aggregating
the views provided by separate observers in a team, greater reliability
is attained.
The Interpersonal Intelligence Inventory (III) informs teachers about
what takes place during group activities from the students’ point
of view (Strom & Strom, 2002a; 2002b). Figure 1 (see Appendix A) shows
the sequence of tasks for the III formative and summative assessments.
The purposes are to: (1) identify the teamwork skills individual students
demonstrate in cooperative learning groups;(2) provide anonymous feedback
from teammates about competencies;(3) compare self and peer- evaluations;(4)
detect individual and group learning deficits to guide instruction;(5)
credit conscientious students for their initiatives to help others;(6)
detect slackers who expect peers to do their work for them; (7) yield
easily understood documentation of social skills for student portfolios;
and (8) establish a school data bank of teamwork skills.
Students evaluate teammates on 25 team skills covering a wide range of
behaviors that are easily detected when interaction takes place for a
reasonable duration. The team skills were drawn from literature on teaching
and learning, creative thinking and critical thinking, group dynamics,
and interpersonal communication (Strom & Strom, 2002a). Figure 2 (see
Appendix B) illustrates the five clusters of teamwork skills which are
assessed by the Interpersonal Intelligence Inventory. These conceptually
convenient clusters include ways in which a student: (1) attends to teamwork
(for example, this peer does a fair share of work expected of everyone),
(2) seeks and shares information (for example, this peer brings reading
materials for the team to examine), (3) communicates with teammates (for
example, this peer encourages and recognizes the contributions of others),
(4) thinks critically and creatively (for example, this peer uses logic
to challenge group thinking or work methods), and (5) gets along in the
team (for example, this peer avoids using put-downs or blaming others
for problems).
Students identify only those specific skills which have been “very
well demonstrated by each of their teammates,” without estimating
how often each behavior occurred. Requiring the use of frequency type
responses for observation of 25 team skills as they apply to sustained
interaction with 4 or 5 teammates would be unreasonable. But, students
are able to report their overall impressions about behavior of teammates,
whether someone consistently behaves in certain ways as defined by criteria.
The proportion of peers who confirm behaviors were fulfilled is a more
meaningful indicator than frequency of occurrence, which is impossible
to recall. In effect, the response format coincides with how most people
make decisions about the behavior of others in daily life.
Scoring and Feedback
Students complete self and peer assessments by circling only those skills
on the form that have been very well demonstrated by each person during
the group work. Tallies are converted to percentage scores based upon
the number of observers (e.g., 4 teammates would each count 25%; 5 peers
in a group would each count for 20%). Scoring can be by hand, scanning,
or Excel (can be sent via email). The Excel-email scoring procedure offers
the advantage of saving class time for administering and scoring the instrument,
enabling makeup evaluations for those who are absent, responding in a
confidential setting, and receiving private individual feedback outside
class.
Each student receives a Profile showing peer and self–impressions
of his/her teamwork skills demonstrated during group work. By considering
both sets of views, personal achievements and limitations are identified.
A formative evaluation column lets students know how they are doing while
plenty of sessions in a course remain to make changes in their behavior.
The summative column records the second battery of perceptions at the
end of the class, followed by a column identifying improvement, lack of
change, or regression.
Psychometric Indicators
The Interpersonal Intelligence Inventory was used extensively with college
students before being field-tested with adolescents in grades nine through
twelve. A high school was selected where cooperative learning was the
norm for instruction. Ten teachers represented a wide range of subject
matters. One class of each teacher was oriented to the III. The orientation
consisted of five lessons described in the manual, and team skill definitions
they kept and took home to show parents. The 303 students then practiced
team skills for a month before completing the formative assessment portion
of the inventory. For each of the two times that the inventory is administered,
formatively and summatively, students require about 15 minutes to complete
the instrument.
Teachers served as the judgment source for determination of content validity.
Every teacher in the sample reported that the cooperative skills necessary
for acceptable performance at the high school level are included and well
defined by the inventory. No other instrument assessing teamwork skills
was found that offered evidence of psychometric indicators. Therefore,
correlation with an existing measure of the same function could not be
used. Instead, a more stringent procedure was applied. Student-expressed
behaviors were compared with their observed behaviors (Campbell, 1996).
Self-reports from (N = 303) students and their collective reports (N =
1,136) were compared to ascertain agreement levels for each of the 25
teamwork skills.
To use adolescents’ self and peer judgments as predictors, the level
of agreement between the two sources must exceed 50%. When agreement reaches
70%, there is ample justification to conclude that specific teamwork skills
have been demonstrated. Table 1 (see Appendix C) displays the Level of
Performance for 303 Self and 1,136 Peer Reports with Level of Agreement
and Disagreement by Percentage. The level of agreement regarding the demonstration
of team skills ranged from 87% to 99%. On 23 of 25 skills agreement between
self-report and perception of peers was at least 90%. These results suggest
that, when using the III, students can be trusted to honestly report their
perceptions about self and peer fulfillment of specific teamwork skills.
This view was corroborated by a survey indicating that 90% of the teachers
and 88% of the students felt comfortable with overall truthfulness of
student evaluation of teamwork.
Internal consistency was evaluated by an analysis of student team scoring
forms. Cronbach’s alpha coefficient offers a floor estimate for
reliability on the overall instrument. The total alpha coefficient for
303 self-reports was 0.79 and corresponding alpha for 1,136 collective
peer observations was 0.87. These estimates are within the high range
for group analyses. Application of the Flesch–Kincaid Index found
that the reading level for the III is 6.85 indicating that a sixth or
seventh grade reader can understand it.
Implications
Student Portfolios. The purpose of portfolios is to help
students acquire self-evaluation skills by monitoring their own progress,
recognize accomplishments, and detect skills where they must work harder
or seek tutorial help. Ninety percent of the field-test teachers reported
that the Team Skills Profile Form describes student behavior better than
a letter grade or conduct mark and should be placed in student portfolios.
If Profiles are added to the portfolios of grade 9 through grade 12 students,
teachers can know more about individuals when they enter a class and strive
to focus on specific skills that have not been achieved. Instead of having
to discover anew the assets and limits of each student, teachers can benefit
from observations of fellow faculty interacting with the same student
in other classes. Recordkeeping is considered an essential way to monitor
basic academic skills and should be applied to development of social skills.
Parent Involvement. Another benefit of team skills records
is to support involvement of parents in the education of adolescents.
Sometimes parents excuse themselves from a guidance role by claiming that
teenagers should become independent and manage their own affairs. Other
parents believe that, when their children are in secondary school, it
is impossible to get to know and relate to all of their child’s
teachers. Still, regardless of their occupation, parents know that team
skills are necessary for success at work and in the home. In this connection,
80% of field-test teachers felt that parents should be expected to join
them in teaching students how to attain some teamwork skills. Many parents
have employment experience that could be relevant to structured discussions
about teamwork with their children. Orienting parents to teamwork skills
could allow faculty and family to cooperate more closely in teaching and
reinforcing teamwork skills in both environments. Because parents are
obliged to guide sons and daughters over a lengthy time, team skills records
would further legitimize their efforts to monitor progress based on portfolio
data.
Parents and teachers can be defensive during their meetings. However,
when outcomes of the III are discussed as the basis for mutual effort,
neither party is likely to blame the other for their inaccurate perceptions
of a student. It is not a matter of whether a parent or teacher has the
correct perspective. Instead, the adults examine self-impressions of one
adolescent and the way that person is viewed by teammates. In this context,
teachers and parents can unite to help adolescents reconcile the disparity
between self and peer views.
Special Education Students. Current Individualized Education
Plan (IEP) mechanisms cannot fully provide needed information on how well
inclusion is working and ways to improve integration. Are disabled students
acquiring the teamwork skills they were placed in regular classrooms to
learn? How do disabled students see themselves in contrast with the way
non-disabled students view them? What are the team skills where they lag
behind peers? What team skills are readily gained and which ones appear
to be most difficult for this group? By compiling group profiles for learning
disabled, attention-deficit/hyperactivity disordered, mentally challenged,
and physically handicapped, the faculty can determine ways programs predicated
on providing social relationships benefits are succeeding and failing
in the estimate of special education students and their non-disabled peers.
Profiles can also reveal how special education students perceive acceptance
and support received from their classmates.
Gender Considerations. An examination was made of student
observations pertaining to gender in the high school field test of 303
students. It was found that self-impression scores for the 151 female
students were greater than the scores 152 males gave themselves on 17
of 25 team skills. Further, when students evaluated all members of their
team, the girls got more favorable scores than boys on 23 of 25 skills.
Results suggest that such information, when disseminated, can counteract
influences that deflate self-esteem of adolescent females. It also supports
heterogeneous grouping by gender because greater competence of girls in
teamwork skills offers readily available models for boys to learn these
skills.
Teachers are the main recommendation source for student leadership opportunities
and awards. When the Interpersonal Intelligence Inventory outcomes are
used as one of the bases for nomination, girls may be chosen for leadership
recognition more often than when systematic observation of peers is unavailable.
Males might be disproportionately nominated if social and physical dominance
are excessively relied on for defining leadership which is likely to occur
in school and the work place.
Conclusions
Learning to cooperate with peers must become a priority in an interdependent
environment. When students are aware of the importance of teamwork in
the future, cooperative skills are added to their definition of what it
takes to become successful. Teachers and students agree that the evaluation
of group work must be improved. One way that prevailing limitations can
be overcome is by administering the Interpersonal Intelligence Inventory.
Profiles of the III indicating improvements in teamwork skills can become
a part of students’ portfolios. In addition, the school can establish
a data bank of student teamwork skills for use in determining curriculum
needs and recognizing achievements.
References
Antil, L., Jenkins, J., Wayne, S., & Vadasy, P. (1998). Cooperative
learning: Prevalence, conceptualizations and the relation between research
and practice. American Educational Research Journal, 35 (3), 419-454.
Binet, A., & Simon, T. (1905). Méthodes nouvelles pour le diagnostic
du niveau intellectual des anormaux. L’Année Psychologique,
11, 191-244.
Campbell, D. (1996). Unresolved issues in measurement validation. Psychological
Assessment, 8 (4), 363-368.
Edwards, M., & Ewan, A. (1996). 360 Degree feedback: The powerful
new model for employee assessment and performance improvement. New York:
Amacon.
Gardner, H. (2000). Intelligence reframed: Multiple intelligences for
the 21st century. New York: Basic Books.
Goleman, D. (1999). Emotional intelligence: Why it can matter more than
IQ. New York: Bantam.
Guilford, J. (1967). The nature of human intelligence. New York: McGraw-Hill.
Guilford, J. (1981). The structure of intelligence. New York: McGraw-Hill.
Johnson, D., Johnson, R., & Holubec, E. (1999). The new circles of
learning: Cooperation in the classroom and school. Alexandria, VA: Association
for Supervision and Curriculum Development.
Kagan, S. (2001). Cooperative learning. San Clemente, CA: Kagan Cooperative
Learning.
Sternberg, R. (1988). The triarchic mind: A New theory of human intelligence.
New York: Viking Penguin.
Sternberg, R. (Ed.) (2000). Practical intelligence in everyday life. New
York: Cambridge University Press.
Strom, P., & Strom, R. (2002a). Interpersonal Intelligence Inventory.
Bensenville, IL: Scholastic Testing Service.
Strom, P., & Strom, R. (2002b). Overcoming limitations of cooperative
learning among community college students. Community College Journal of
Research and Practice, 26: 315-331.
Terman, L., & Merrill, M. (1937). Measurement of intelligence. Boston:
Houghton Mifflin.
Terman, L., & Merrill, M. (1960). Stanford-Binet intelligence scale.
Boston: Houghton Mifflin.
Thorndike, E. (1927).The measurement of intelligence. New York: Teachers
College Columbia University.
Thurstone, L. (1938). Primary mental abilities. Psychometric Monograph,
1, 2-10. Chicago: University of Chicago.
Torrance, E. P. (2000). On the edge and keeping on the edge. Westport,
CT: Greenwood Publishing Group.
Torrance, E. P. (2002). Torrance Tests of Creative Thinking. Bensenville,
IL: Scholastic Testing Service.
Wechsler, D. (1949). Wechsler Intelligence Scale for Children. New York:
Psychological Corporation.
Wechsler, D. (1997). WAIS-III/Wechsler Memory Scale, Third Edition. San
Antonio, TX: Psychological Corporation.
|