OFFICE of PARENT DEVELOPMENT INTERNATIONAL

HOME

| PURPOSES | TEAM | PARTNERS | THEORIES | MEASUREMENTS | PUBLICATIONS


INTERPERSONAL INTELLIGENCE ASSESSMENT THEORY

STUDENT EVALUATION OF COOPERATIVE LEARNING:
THE INTERPERSONAL INTELLIGENCE INVENTORY


Paris S. Strom, Auburn University
Robert D. Strom, Arizona State University

Presentation to the Annual Meeting of the
American Educational Research Association
Chicago, April 2003

Abstract


One of the most perplexing tasks of teachers using cooperative learning is to identify teamwork skills exhibited by students during group work. The Interpersonal Intelligence Inventory is designed for students to evaluate peer and self-performance. Responses inform teachers about team interaction from the students’ point of view. The purposes of this instrument are to identify team skills that individual students demonstrate, provide anonymous feedback from peers, compare self-impressions with peer observations, detect learning needs, credit hard workers for initiatives, detect slackers, and yield an easily understood record of social skills. Outcomes can be used with student portfolios, and to establish a school data bank on teamwork skills. Field-testing with 303 high school students determined that psychometric indicators are in the good range.

* * *
Students should practice the skills that will be required of them as adults. It is impossible to predict all of the competencies that should be emphasized so children are well prepared for their future. However, by examining changing conditions in the labor market, projections for the knowledge explosion, and shifts in lifestyles, some crucial lessons can be identified. There is general agreement that functioning as a team member warrants greater attention. Working alone is a long-standing tradition that is beginning to fade in favor of students working together in groups (Johnson, Johnson, & Holubec, 1999; Kagan, 2001). In this context, teachers are expected to arrange teamwork activities that can facilitate the development of interpersonal intelligence (Gardner, 2000).


Team Assessment of Individual Performance
Some practical alternatives for evaluating teamwork activities are provided by corporations, such as Bank of America, Disney, Ford, General Motors, Hewlett-Packard, Intel, Microsoft, and Motorola. They rely upon group perspective approaches for evaluating teamwork of individual employees. Many employees believe that the customary job performance review in which one person judges the work quality of subordinates on a random schedule is unfair and obsolete. Workers may ignore suggestions from a supervisor by rationalizing that ‘the boss does not see me enough to be aware of all the things I do or how well I perform.’ Evaluation by peers on the job has gained credibility in the past decade. Criticism and suggestions from them are taken seriously. Peer evaluations are preferred because co-workers observe one another daily, know how well they perform tasks, and can detect limitations. Management also supports team assessment because this approach has been found to increase productivity (Edwards & Ewan, 1996).


Because group evaluation is becoming prevalent in the employment sector, adolescents and college students need to learn how to evaluate the performance of their peers, judge personal competence, and benefit from the criticism of teammates. Yet, a perplexing task of teachers who use cooperative learning is to identify teamwork skills demonstrated by each student. Teachers are uncertain about how to conduct assessment that is accurate and fair. Students also express disappointment regarding group work evaluation. Suitable instrumentation can help overcome this limitation (Antil, Jenkins, Wayne, & Vadasy, 1998).


Student Accountability for Group Work
There are three ways in which accountability for students working in groups should be redefined. First, accountability should emphasize how each person contributes to their team. Faculties favor this condition for evaluation of their own competence. That is, teachers attempt to facilitate achievement by helping all class members but, in the final analysis, each student is responsible for his or her own performance. When this view of teachers is applied to group work, no student can reasonably be held responsible for the behavior of peers. There is greater motivation to cooperate because individuals are judged on the basis of recognized teamwork skills and personal test performance.


Secondly, faculty should share some responsibility with students for evaluation of group learning. Teachers are experts in their subject so they are the best qualified to judge work products submitted by teams. However, faculties are not the best evaluators of what happens in groups because they are seldom around to see the interaction process. Even when present, they cannot tell how the initiatives of some students affect the thinking of others. Students are disappointed when faculty ignore them as a source of information about how peers contribute to their learning and the outcomes of team effort. Team members are in the best position to identify the teammates who influence them and describe the nature of help given.


A third element of this strategy involves comparing self-evaluation with the observation of peers. Teachers frequently evaluate students by testing. Learners also need self-evaluation skills in order to make wise decisions in an environment characterized by over choice and a work place where knowing how to collaborate is essential. Cooperative learning provides an ideal condition for a comparison of self-impressions with the observations of peers. In this way, students can gain the ability to judge themselves. Guided practice in self-appraisal is necessary to become self critical, which is vital for performing well in groups. The ability to self- evaluate enables us to know when to think well of ourselves and when it is appropriate to change our behavior so it more closely resembles the person we wish to become.


Student accountability is more likely when all parties understand in advance the criteria for assessment and processes that teammates will use for evaluating them. Faculties are implicated too—they must trust students to participate in evaluation of team learning, and convey the importance of authentic reporting. The credible way to find out if desired competencies are learned is to solicit perceptions of teammates. Students observe what happens in groups, know how each teammate influences their thinking, and can evaluate self initiatives to enhance productivity. Frequent opportunities to apply peer and self-evaluation allows students to practice the skills needed to become better judges of productive self and group functioning in class, on the job, and at home.

Concepts of Multiple Intelligences
Most of the early test designers accepted the concept of multiple intelligences. Alfred Binet and Theophile Simon (1905), credited with developing the first mental tests, saw intelligence as an aggregate of separate cognitive functions including comprehension, judgment, direction, and invention. Edward Thorndike (1927) believed that intelligence is demonstrated by a capacity to respond effectively to novel situations. Just as there are different kinds of situations, there are different patterns of intelligence - abstract, mechanical, and social. For example, the skilled leader of people may be quite inept in mechanical matters while the skilled mathematician may be notably poor in social intelligence.


Louis Thurstone (1938) described intelligence as composed of seven primary mental abilities -- verbal comprehension, number ability, word fluency, spatial visualization, associative memory, reasoning, and perceptual speed. The Stanford-Binet test, first produced by Lewis Terman and Maud Merrill (1937, 1960), includes verbal and nonverbal mental tasks. David Wechsler (1949, 1997) published separate intelligence scales for children and adults. Joy Guilford (1967, 1981), in his theoretical model known as the Structure of Intellect, offered the most inclusive concept suggesting the existence of 120 intelligences –- most of which can be assessed somewhat acceptably with current measures.


During the past decade, memorization in classrooms has been joined by greater attention to other mental processes. Much of what students memorize is current for less time than ever before because information obsolescence occurs more quickly. Accordingly, creative and critical thinking abilities needed for solving problems have emerged as primary goals. This shift has led to increasing interest in measures of creative thinking. Paul Torrance’s (2000, 2002) figural and verbal tasks reveal originality, elaboration, fluency, flexibility, and redefinition abilities. Daniel Goleman (1999) has documented the need to nurture emotional intelligence to improve the quality of life in a hurried and stressful environment. Emotional intelligence consists of self-awareness, impulse control, zeal, persistence, self-motivation, empathy, and social deftness. Robert Sternberg (1988, 2000) contends that personal success is related to three separate types of intelligence –- analytical, creative, and practical.


Howard Gardner (2000) proposes that schools recognize eight frames of mind: (1) verbal – linguistic abilities which implicate speaking, reading and writing skills;(2) logical – mathematical abilities which enable deductive and inductive reasoning; (3) visual-spatial abilities which allow a person to create representations of the world and to think in pictures; (4) musical – rhythmic abilities which present sensitivity to pitch and rhythms of sounds; (5) bodily – kinesthetic abilities which contribute to motor skills and graceful movement; (6) naturalist abilities which allow someone to observe patterns in nature and understand natural and human-made systems; (7) intrapersonal-introspective abilities which permit someone to be deeply aware of personal feelings and purposes; and (8) interpersonal abilities – the social capacity which makes it possible to work effectively with others. This latter set of capacities, sometimes referred to as “social intelligence” is the focus for the Interpersonal Intelligence Inventory (III) (Strom & Strom, 2002).


Measurement of Interpersonal Intelligence

What conditions should be met by instruments in this context? Should students be presented with hypothetical situations and their projections of how they would expect to act be used as indicators of interpersonal intelligence? Reflections about how one might respond in a given set of circumstances may differ from the way a person would actually behave in these conditions. Consequently, it makes more sense to hold students accountable for actual behavior than for hypotheses of how they suppose they would behave if they encountered particular situations. One set of responses relates to behavior that can be corroborated while the other involves imagined behavior that no one else could confirm. A more appropriate strategy is to rely upon multiple observers who have sufficient experience to confirm or refute self-impressions of students regarding their performance as a teammate. This design relies on the perceptions of peers to identify social skills which a person demonstrates in repeated interactions with others. By aggregating the views provided by separate observers in a team, greater reliability is attained.


The Interpersonal Intelligence Inventory (III) informs teachers about what takes place during group activities from the students’ point of view (Strom & Strom, 2002a; 2002b). Figure 1 (see Appendix A) shows the sequence of tasks for the III formative and summative assessments. The purposes are to: (1) identify the teamwork skills individual students demonstrate in cooperative learning groups;(2) provide anonymous feedback from teammates about competencies;(3) compare self and peer- evaluations;(4) detect individual and group learning deficits to guide instruction;(5) credit conscientious students for their initiatives to help others;(6) detect slackers who expect peers to do their work for them; (7) yield easily understood documentation of social skills for student portfolios; and (8) establish a school data bank of teamwork skills.

Students evaluate teammates on 25 team skills covering a wide range of behaviors that are easily detected when interaction takes place for a reasonable duration. The team skills were drawn from literature on teaching and learning, creative thinking and critical thinking, group dynamics, and interpersonal communication (Strom & Strom, 2002a). Figure 2 (see Appendix B) illustrates the five clusters of teamwork skills which are assessed by the Interpersonal Intelligence Inventory. These conceptually convenient clusters include ways in which a student: (1) attends to teamwork (for example, this peer does a fair share of work expected of everyone), (2) seeks and shares information (for example, this peer brings reading materials for the team to examine), (3) communicates with teammates (for example, this peer encourages and recognizes the contributions of others), (4) thinks critically and creatively (for example, this peer uses logic to challenge group thinking or work methods), and (5) gets along in the team (for example, this peer avoids using put-downs or blaming others for problems).


Students identify only those specific skills which have been “very well demonstrated by each of their teammates,” without estimating how often each behavior occurred. Requiring the use of frequency type responses for observation of 25 team skills as they apply to sustained interaction with 4 or 5 teammates would be unreasonable. But, students are able to report their overall impressions about behavior of teammates, whether someone consistently behaves in certain ways as defined by criteria. The proportion of peers who confirm behaviors were fulfilled is a more meaningful indicator than frequency of occurrence, which is impossible to recall. In effect, the response format coincides with how most people make decisions about the behavior of others in daily life.

Scoring and Feedback
Students complete self and peer assessments by circling only those skills on the form that have been very well demonstrated by each person during the group work. Tallies are converted to percentage scores based upon the number of observers (e.g., 4 teammates would each count 25%; 5 peers in a group would each count for 20%). Scoring can be by hand, scanning, or Excel (can be sent via email). The Excel-email scoring procedure offers the advantage of saving class time for administering and scoring the instrument, enabling makeup evaluations for those who are absent, responding in a confidential setting, and receiving private individual feedback outside class.
Each student receives a Profile showing peer and self–impressions of his/her teamwork skills demonstrated during group work. By considering both sets of views, personal achievements and limitations are identified. A formative evaluation column lets students know how they are doing while plenty of sessions in a course remain to make changes in their behavior. The summative column records the second battery of perceptions at the end of the class, followed by a column identifying improvement, lack of change, or regression.


Psychometric Indicators
The Interpersonal Intelligence Inventory was used extensively with college students before being field-tested with adolescents in grades nine through twelve. A high school was selected where cooperative learning was the norm for instruction. Ten teachers represented a wide range of subject matters. One class of each teacher was oriented to the III. The orientation consisted of five lessons described in the manual, and team skill definitions they kept and took home to show parents. The 303 students then practiced team skills for a month before completing the formative assessment portion of the inventory. For each of the two times that the inventory is administered, formatively and summatively, students require about 15 minutes to complete the instrument.


Teachers served as the judgment source for determination of content validity. Every teacher in the sample reported that the cooperative skills necessary for acceptable performance at the high school level are included and well defined by the inventory. No other instrument assessing teamwork skills was found that offered evidence of psychometric indicators. Therefore, correlation with an existing measure of the same function could not be used. Instead, a more stringent procedure was applied. Student-expressed behaviors were compared with their observed behaviors (Campbell, 1996). Self-reports from (N = 303) students and their collective reports (N = 1,136) were compared to ascertain agreement levels for each of the 25 teamwork skills.


To use adolescents’ self and peer judgments as predictors, the level of agreement between the two sources must exceed 50%. When agreement reaches 70%, there is ample justification to conclude that specific teamwork skills have been demonstrated. Table 1 (see Appendix C) displays the Level of Performance for 303 Self and 1,136 Peer Reports with Level of Agreement and Disagreement by Percentage. The level of agreement regarding the demonstration of team skills ranged from 87% to 99%. On 23 of 25 skills agreement between self-report and perception of peers was at least 90%. These results suggest that, when using the III, students can be trusted to honestly report their perceptions about self and peer fulfillment of specific teamwork skills. This view was corroborated by a survey indicating that 90% of the teachers and 88% of the students felt comfortable with overall truthfulness of student evaluation of teamwork.

Internal consistency was evaluated by an analysis of student team scoring forms. Cronbach’s alpha coefficient offers a floor estimate for reliability on the overall instrument. The total alpha coefficient for 303 self-reports was 0.79 and corresponding alpha for 1,136 collective peer observations was 0.87. These estimates are within the high range for group analyses. Application of the Flesch–Kincaid Index found that the reading level for the III is 6.85 indicating that a sixth or seventh grade reader can understand it.


Implications
Student Portfolios. The purpose of portfolios is to help students acquire self-evaluation skills by monitoring their own progress, recognize accomplishments, and detect skills where they must work harder or seek tutorial help. Ninety percent of the field-test teachers reported that the Team Skills Profile Form describes student behavior better than a letter grade or conduct mark and should be placed in student portfolios. If Profiles are added to the portfolios of grade 9 through grade 12 students, teachers can know more about individuals when they enter a class and strive to focus on specific skills that have not been achieved. Instead of having to discover anew the assets and limits of each student, teachers can benefit from observations of fellow faculty interacting with the same student in other classes. Recordkeeping is considered an essential way to monitor basic academic skills and should be applied to development of social skills.


Parent Involvement. Another benefit of team skills records is to support involvement of parents in the education of adolescents. Sometimes parents excuse themselves from a guidance role by claiming that teenagers should become independent and manage their own affairs. Other parents believe that, when their children are in secondary school, it is impossible to get to know and relate to all of their child’s teachers. Still, regardless of their occupation, parents know that team skills are necessary for success at work and in the home. In this connection, 80% of field-test teachers felt that parents should be expected to join them in teaching students how to attain some teamwork skills. Many parents have employment experience that could be relevant to structured discussions about teamwork with their children. Orienting parents to teamwork skills could allow faculty and family to cooperate more closely in teaching and reinforcing teamwork skills in both environments. Because parents are obliged to guide sons and daughters over a lengthy time, team skills records would further legitimize their efforts to monitor progress based on portfolio data.


Parents and teachers can be defensive during their meetings. However, when outcomes of the III are discussed as the basis for mutual effort, neither party is likely to blame the other for their inaccurate perceptions of a student. It is not a matter of whether a parent or teacher has the correct perspective. Instead, the adults examine self-impressions of one adolescent and the way that person is viewed by teammates. In this context, teachers and parents can unite to help adolescents reconcile the disparity between self and peer views.


Special Education Students. Current Individualized Education Plan (IEP) mechanisms cannot fully provide needed information on how well inclusion is working and ways to improve integration. Are disabled students acquiring the teamwork skills they were placed in regular classrooms to learn? How do disabled students see themselves in contrast with the way non-disabled students view them? What are the team skills where they lag behind peers? What team skills are readily gained and which ones appear to be most difficult for this group? By compiling group profiles for learning disabled, attention-deficit/hyperactivity disordered, mentally challenged, and physically handicapped, the faculty can determine ways programs predicated on providing social relationships benefits are succeeding and failing in the estimate of special education students and their non-disabled peers. Profiles can also reveal how special education students perceive acceptance and support received from their classmates.


Gender Considerations. An examination was made of student observations pertaining to gender in the high school field test of 303 students. It was found that self-impression scores for the 151 female students were greater than the scores 152 males gave themselves on 17 of 25 team skills. Further, when students evaluated all members of their team, the girls got more favorable scores than boys on 23 of 25 skills. Results suggest that such information, when disseminated, can counteract influences that deflate self-esteem of adolescent females. It also supports heterogeneous grouping by gender because greater competence of girls in teamwork skills offers readily available models for boys to learn these skills.


Teachers are the main recommendation source for student leadership opportunities and awards. When the Interpersonal Intelligence Inventory outcomes are used as one of the bases for nomination, girls may be chosen for leadership recognition more often than when systematic observation of peers is unavailable. Males might be disproportionately nominated if social and physical dominance are excessively relied on for defining leadership which is likely to occur in school and the work place.


Conclusions
Learning to cooperate with peers must become a priority in an interdependent environment. When students are aware of the importance of teamwork in the future, cooperative skills are added to their definition of what it takes to become successful. Teachers and students agree that the evaluation of group work must be improved. One way that prevailing limitations can be overcome is by administering the Interpersonal Intelligence Inventory. Profiles of the III indicating improvements in teamwork skills can become a part of students’ portfolios. In addition, the school can establish a data bank of student teamwork skills for use in determining curriculum needs and recognizing achievements.

References
Antil, L., Jenkins, J., Wayne, S., & Vadasy, P. (1998). Cooperative learning: Prevalence, conceptualizations and the relation between research and practice. American Educational Research Journal, 35 (3), 419-454.

Binet, A., & Simon, T. (1905). Méthodes nouvelles pour le diagnostic du niveau intellectual des anormaux. L’Année Psychologique, 11, 191-244.
Campbell, D. (1996). Unresolved issues in measurement validation. Psychological Assessment, 8 (4), 363-368.
Edwards, M., & Ewan, A. (1996). 360 Degree feedback: The powerful new model for employee assessment and performance improvement. New York: Amacon.
Gardner, H. (2000). Intelligence reframed: Multiple intelligences for the 21st century. New York: Basic Books.
Goleman, D. (1999). Emotional intelligence: Why it can matter more than IQ. New York: Bantam.
Guilford, J. (1967). The nature of human intelligence. New York: McGraw-Hill.
Guilford, J. (1981). The structure of intelligence. New York: McGraw-Hill.
Johnson, D., Johnson, R., & Holubec, E. (1999). The new circles of learning: Cooperation in the classroom and school. Alexandria, VA: Association for Supervision and Curriculum Development.
Kagan, S. (2001). Cooperative learning. San Clemente, CA: Kagan Cooperative Learning.
Sternberg, R. (1988). The triarchic mind: A New theory of human intelligence. New York: Viking Penguin.
Sternberg, R. (Ed.) (2000). Practical intelligence in everyday life. New York: Cambridge University Press.
Strom, P., & Strom, R. (2002a). Interpersonal Intelligence Inventory. Bensenville, IL: Scholastic Testing Service.
Strom, P., & Strom, R. (2002b). Overcoming limitations of cooperative learning among community college students. Community College Journal of Research and Practice, 26: 315-331.
Terman, L., & Merrill, M. (1937). Measurement of intelligence. Boston: Houghton Mifflin.
Terman, L., & Merrill, M. (1960). Stanford-Binet intelligence scale. Boston: Houghton Mifflin.
Thorndike, E. (1927).The measurement of intelligence. New York: Teachers College Columbia University.
Thurstone, L. (1938). Primary mental abilities. Psychometric Monograph, 1, 2-10. Chicago: University of Chicago.
Torrance, E. P. (2000). On the edge and keeping on the edge. Westport, CT: Greenwood Publishing Group.
Torrance, E. P. (2002). Torrance Tests of Creative Thinking. Bensenville, IL: Scholastic Testing Service.
Wechsler, D. (1949). Wechsler Intelligence Scale for Children. New York: Psychological Corporation.
Wechsler, D. (1997). WAIS-III/Wechsler Memory Scale, Third Edition. San Antonio, TX: Psychological Corporation.

 

 

APPENDIX A

Figure 1. Interpersonal Intelligence Inventory: Sequence of Tasks

 

 

 

 

 

******

 

 

APPENDIX B

 

Figure 2. Teamwork Skills Evaluated by the Interpersonal Intelligence Inventory

 

 

 

 

 

 

 

 

******

 

 

 

 

APPENDIX C

 

Table 1.

Level of Performance for 303 Self and 1,136 Peer Reports

with Level of Agreement and Disagreement by Percentage

 

 

Interpersonal Intelligence Inventory

Team Skills

 Performance

Self    Peer

%       %

Dis-

agree

%

 

Agree

 %

1

This peer shows acceptable attendance for team meetings.

67

54

13

87

2

This peer arrives on time for team meetings.

90

87

3

97

3

This peer stays focused on the task during class.

83

77

6

94

4

This peer fulfills individual role such as organizer or reporter.

82

77

5

95

5

This peer does a fair share of the work expected of everyone.

94

85

9

91

6

This peer admits uncertainty when in doubt about what to do.

68

60

8

92

7

This peer asks questions that help the group to understand lessons.

77

71

6

94

8

This peer helps others by explaining or reviewing lessons.

71

60

11

89

9

This peer brings reading materials for the group to examine.

37

33

4

96

10

This peer refers to reading materials during discussions.

49

42

7

93

11

This peer shares experiences, feelings, ideas or opinions.

84

74

10

90

12

This peer speaks clearly and uses easily understood vocabulary.

90

87

3

97

13

This peer limits the length of comments so others get to talk.

80

79

1

99

14

This peer listens to everyone and respects their views.

92

84

8

92

15

This peer encourages and recognizes the contributions of others.

73

65

8

92

16

This peer considers views that differ from their opinions.

72

62

10

90

17

This peer uses logic to challenge group thinking or work methods.

64

55

9

91

18

This peer carefully thinks about ideas before reaching conclusions.

70

65

5

95

19

This peer builds on the ideas of others.

80

72

8

92

20

This peer offers new ways of looking at ideas or problems.

78

68

10

90

21

This peer takes suggestions for improvement in a friendly way.

83

75

8

92

22

This peer avoids using put-downs or blaming others for problems.

84

79

5

95

23

This peer accepts compromise as a way to deal with conflict.

84

77

7

93

24

This peer keeps trying even when the task becomes hard.

82

72

10

90

25

This peer expresses hope about group success.

75

72

3

97

 

 

******

 

Copyright © 2003 by Paris S. Strom and Robert D. Strom

For permission to use a whole or parts of this paper, write to [email protected] or [email protected]