a.      Robert Jones—Mathematics

                                                              i.      Introduction

                                                            ii.      Background

                                                          iii.      Implementation

                                                          iv.      Comments

                                                            v.      Statistics

b.      Fari Towfiq—Mathematics

                                                              i.      Introduction

                                                            ii.      Benchmarks

                                                          iii.      Implementation and Observations

                                                          iv.      Explaining Core Skills and Assessment

                                                            v.      Team Presentations

                                                          vi.      Skills Assessment Form

                                                        vii.      Self-Assessment and Team Self Assessment Form

                                                      viii.      Statistics

c.       Cindy Anfinson—Mathematics

                                                              i.      Introduction

                                                            ii.      Hypothesis and Expectations

                                                          iii.      Procedure

                                                          iv.      Conclusions

                                                            v.      Statistics

                                                          vi.      Future

d.      Matthews Chakkanakuzhi—Mathematics

                                                              i.      Introduction

                                                            ii.      Expectations

                                                          iii.      Observations

                                                          iv.      Concerns and Comments

                                                            v.      Conclusion

e.      Michael Mufson—Performing Arts

                                                              i.      Prologue

                                                            ii.      Preliminaries

                                                          iii.      Procedures/Process

                                                          iv.      Outcomes

f.        Lee Chen—English as a Second Language

                                                              i.      Introduction

                                                            ii.      Two Team Projects

                                                          iii.      Project #1

                                                          iv.      Project #2

                                                            v.      Lessons Learned

g.      Cynthia Watson—English as a Second Language

Introduction: Description of the Departmental Project Team

The purpose of the departmental project team is to work in conjunction with the Assessment of Learning Project (ALP) to apply Palomar College’s list of Core Skills to the content and activities of courses in three selective disciplines.  The five members of the Departmental Project Team represent three disciplines at Palomar College.  Fari Towfiq and Cynthia Watson are the Team’s coordinators and they also serve on the ALP team.  Robert Jones represents the discipline of mathematics.  Michael Mufson represents the discipline of the performing arts. Lee Chen represents the discipline of English as a Second Language.  Recently, Lee Kerckhove has joined the team as a co-coordinator of the departmental project team, and Matthews Chakkanakuzhi and Cynthia Anfinson have joined the mathematics portion of the project.  A brief overview of this project follows.

MATHEMATICS

Robert Jones, Assistant Professor, is following his new assessment-inclusive syllabus in his Pre-Algebra class.  Fari Towfiq, Associate Professor, is implementing the pilot assessments in her Intermediate Algebra classes.  Towfiq and Jones worked together to formulate the benchmarks for Mathematics and are using the same assessment methods and feedback forms in their respective classes. They have been very careful to make their assessments convenient enough to fit into mathematics courses that must cover a prescribed number of chapters in a standardized text in 16 weeks. Towfiq and Jones have selected two Core Skill areas to assess in their respective algebra classes. In the first area of Cognition, they provide for the assessment of students' use of the two Core Skills of Problem-Solving, and Transfer of Knowledge. The "Skills Assessment Cover Sheet" is a feedback form they developed to be filled out by the instructor and attached to students' homework or exam papers after a grade has been assigned. Towfiq and Jones do not intend for this Cover Sheet to determine any part of their students' grades. In their classes it is for on-going feedback only. They also developed four forms (Peer Assessment, Team Assessment, Self-Assessment, and Instructor Evaluation of Teamwork) that single out Teamwork in the Core Skill area of Social Interaction. Each provides for feedback/assessment from various perspectives on team participation and team presentations.  Recently, Chakkanakuzhi and Anfinson contributed to the revision and modification of these forms.

PERFORMING ARTS

Michael Mufson, Associate Professor, Theater Arts, is applying the benchmarks that he formulated to his Introduction to Theater class, which includes improvisation, workshop scripting, and performance of the students' own pieces. Mufson chose the general Core Skills (ALP website) categories of Social Interaction, Aesthetic Responsiveness, and Cognition as the basis for his Introduction to Theater Arts assessment pilot. His first two feedback forms, "Self Assessment" and "Peer Assessment," focus on the Social Interaction abilities of "Teamwork" and "Effective Citizenship." Students use them to assess their three collaborative projects. His third feedback form, "Visual Representation Assessment," applies to individual students' Visual Representation projects. It focuses on the Cognitive category of Analysis and Synthesis.

ENGLISH AS A SECOND LANGUAGE

Lee Chen, Assistant Professor, has included his writing assessment benchmarks in the syllabus of his on-line ESL Advanced Writing class. His benchmarks are just as useful in the "real-space" classroom as they are in the virtual classroom. Lee Chen has designed an on-line ESL Advanced Writing course to be offered by Palomar College in the 2000-2001 academic year.  He developed two forms.  The first, "Instructor Feedback," applies to the sub-categories of Speaking and Listening.  It provides information to the student from the instructor about the quality of his/her participation in the virtual class discussions. All students in the online class receive e-mailed feedback twice during the semester--in the third week and at the end.  A second form, "Individual Assessment," appraises the student's writing, which is the fourth sub-category under the Core Skill of Communication. Students receive feedback via this Individual Assessment at the beginning and end of the semester, at about the same times they receive feedback on the quality of their online discussion.

Cynthia Watson, Associate Professor, is teaching a Literacy Level English as a Second Language class using videotape assessment as a source for feedback and evaluation in the two language modes of speaking and listening.  She has developed a video assessment for her Beginning ESL class, which focuses on the Communication Core Skills of Speaking and Listening (ALP website). She developed assessment benchmarks for student video presentations at four proficiency levels, and a form that she and her students use when they rate videotaped presentations. This form closely (but not exactly) follows a primary trait rating scale design. At the beginning of the semester, it is used for feedback only; at semester's end it is used both for feedback and to assign a grade.  In Beginning ESL, Watson videotapes students twice--at the beginning and end of the semester. She then tracks these students' developing speaking proficiencies by videotaping them at the end of each semester as they move upward through the levels of the ESL program.

Individual Reports

Robert Jones – Mathematics Department

Introduction

For the Spring 2000 semester, I piloted assessment materials in my Prealgebra class (Math 15) that Fari Towfiq and I had developed the previous Fall.  We both agreed that we wanted to emphasize the critical thinking component of the Core Skills document (as identified by ALP) within the context of our math classes, and we also wanted to make teamwork & communication a more significant part of the work that the students did in class.  Since Fari and I realized that we’d have difficulty getting other members on the math faculty to try our materials--especially if they generated a significant amount of extra work--our goal with these materials was to make assessment a more ‘seamless’ part of the day-to-day activities in our classes.

Background

At Palomar College, Math 15 is the only math class (along with Math 10 – Arithmetic) without a prerequisite.  Typically, the majority of the students taking this class are re-entry students who’ve been out of school for a long time; others find themselves in Math 15 after making an unsuccessful attempt to pass a placement exam that would enable them to take Math 50 (Algebra).   Many of these students have a tremendous amount of anxiety towards math and a very low opinion of their abilities – attitudes that have been developed through previous experiences with mathematics that, for the most part, have been characterized by frustration & failure.

Hence, aside from trying to get students ready to take Math 50, one of my main tasks in teaching any Math 15 class is simply to get students to have a more positive attitude toward their work and toward mathematics in general.  Even prior to my participation with ALP, I’ve found that this can be facilitated by structuring the class so that students are active participants in the work that takes place there.  As opposed to having students merely take notes (as passive learners), I try to give them meaningful tasks and daily objectives that they meet by working independently in small groups.  (Students are told on the first day that ‘groupwork’ is a significant learning style in my classroom, and it’s part of the syllabus that I give to them.  I don’t employ this method in all of my classes, but I’ve found that this approach is well-suited for students at this level.)  In this environment, students learn that mathematics isn’t just a bunch of skills that are practiced in isolation, but an active process that requires communication – much like most work in the ‘real’ world.  Groups are encouraged to be self-sufficient and, by not having to spend the majority of the class lecturing at the chalkboard, I’m free to walk around the class and to provide help as needed.  Hence, I can quickly assess my students’ understanding of concepts while they’re learning them, either by asking students to explain ideas to me or to other members of their groups.

Within this context, then, I believed that the assessment benchmarks that Fari and I developed would be less ‘invasive’ than they might be in other classes.  It didn’t seem to be much of a ‘step’ for students to move from explaining concepts within their groups to explaining them to the class.  Also, I wanted to know if the critical thinking forms that Fari and I had generated could have a positive effect on student learning in a developmental class.

Implementation

My Math 15 class for Spring 2000 met from 4:00 p.m. to 5:20 on Mondays and Wednesdays.  I began the semester with about 30 students.  They were told on the first day that, in addition to the traditional evaluation tools--3 tests, daily quizzes, homework checks, and a final exam--they would also be evaluated by their ‘participation.’   I didn’t go into any details as to what this meant (i.e., using group presentations to review for the tests & the final exam), choosing instead to provide them as the class developed.   None of the materials that Fari and I developed were used until after the first exam; I wanted my students to feel comfortable within the class and to have a minimum level of mathematics to communicate with.

Here’s the timeline I followed:

Weeks 1 to 4:  Groups formed & class routine established; Test #1 given at the end of Week #4.

            Weeks 5 to 10:  Group presentations begin (see Comments section); Skills Assessment (critical thinking) forms completed & returned with Test #2 (Week 10).

Weeks 11 to 15:  ‘Presentations’ continue in modified form (see Comments section); Skills Assessment (critical thinking) forms not used with Test #3 (see Comments section).

Week 16:  New material completed (as specified in the Course Outline of Record); review for Final (one class period).

Comments

1. Group Presentations: To prepare for Test #2, four groups (sixteen students total) were randomly picked from the existing teams and assigned topics – order of operations, adding and subtracting signed numbers, length and perimeter, and the ‘meaning’ of multiplication - to review for the class.  In addition to demonstrating examples, each group provided three to five review problems related to their presentations.  Because of time constraints, I wasn’t able to give the groups time in class to work on their presentations – they had to work on them outside of class and be ready to ‘go’ the next time we met.   As a result, these initial presentations weren’t really ‘creative’; for each, one to two members wrote problems on the board while the remaining members asked the other students in the class how to solve them.  None of the groups used the overhead projector or any other ‘visuals’ (posters, etc.).  Some groups and group members were a little nervous and ‘stiff’ (and they told me so!), but this was to be expected.  Overall, I can say that I was generally pleased with the level of effort and participation shown, and I felt that a positive standard had been set for future presentations.  The class completed the peer review forms, which were given to the group members after each presentation. Also, I evaluated each group and gave them a grade; realizing that I wouldn’t have time outside of class to look at the self and team assessment forms, I didn’t have the groups fill these out.

2.  Initially, Fari and I intended our students to make two group presentations – a portion of the review for one test and part of the review for the Final.  However, due to time constraints, I modified the presentation process, making it much more informal.  Groups were now sent to the board or called upon--usually once or twice each period (toward the end of class)--to demonstrate selected solutions.  In this way, communicating mathematics simply became a regular part of our daily routine, not just something that was done before a test. Team members were still expected to justify their results – both verbally and in writing.  Without a lot of time to prepare, these presentations were more spontaneous than those associated with reviewing material for tests, and  they were invaluable in helping me to determine my students’ conceptions of the topics they were learning.  I quickly scored these presentations using a four point rubric; the peer review and associated paperwork were eliminated.  By using this new format, I was able to ensure that each student had at least two opportunities to speak to the class.      

3. Skills Assessment Form: Although my students appreciated the comments and feedback this form provided, filling these out for the students who took Test #2 turned out to be a very time-consuming exercise.  Hence, I had to abandon using the form for Test #3.  (Fari and I subsequently modified this form, making it easier to fill out.)  Regardless of using the form or not, I do believe that, as critical thinking continued to be emphasized in the work students did both inside and outside of class, many of them became more aware of mathematics as a process – a way of thinking – in which the explanation was as important as the answer.

Statistics

Having piloted these new materials in just one class, I believe it’s difficult to make statistically significant comparisons with other Math 15 classes that I’ve taught.  At the very least, more work should be done in our department to ascertain the effect these approaches have on learning.  (I am currently using these materials in my Prealgebra class for the Spring 2001 semester.)

            Having said this, here are some numbers for the class I taught in Spring ‘00 and the two previous sections of Math 15 that I taught in the fall of 1998.  I used the same textbook for both semesters; Tests 2 & 3 and the final were similar in terms of content & length.  (‘Median score’ - 50% of the class scored above this mark, and 50% scored below;  - the arithmetic average for the given class – i.e., the sum of the scores for the given class divided by the number of scores;  represents the ‘standard’ deviation – it’s a measure of how far an ‘average’ score in a given class falls from.)

Test #2 –

Test #3 –

Final –

Fari Towfiq--Mathematics Department

Introduction

In the Fall 1999 Semester, my Math colleague Robert Jones and I, decided to modify the way we teach two of our basic mathematics courses--Prealgebra (Math 15) and Intermediate Algebra (Math 60).  Under the guidance of the ALP Departmental-Project Team, we prepared to depart from our traditional ways of presenting the courses.  

During this Fall Semester, we selected three Core Skill areas for which we wanted to create performance benchmarks.  We hoped to use our new benchmarks to assess the quality of our students' learning via their performance in those three areas.  In the Spring 2000 Semester, we did indeed apply our new means of assessment and our modified instructional methods to our respective math classes.  The three areas on which we focused our benchmarks follow here:

                        Area 1: Cognition

                                                Analysis and Synthesis

                                                Problem Solving

                                                Creative Thinking

                                                Quantitative Reasoning

Transfer of Knowledge and Skills to a New Context

                        Area 2: Communication

                                                Speaking

                                                Listening

                                                Reading

                                                Writing

            Area 3: Social Interaction

                                                Teamwork

During this process of creating and applying our benchmarks, we realized that, if we were not careful, Robert and I could generate a lot of extra work for ourselves and for other faculty who would like to use our material.  Thus, we added a new goal to our benchmark project:  keep it as user-friendly and efficient as possible.  We did not want to make the assessment process so time-consuming that it would overburden our already very busy Math Department colleagues.

Benchmarks

Robert and I created five different feedback forms for assessing in the three Core Skill Areas of Cognition, Communication and Social Interaction.[CW1]  The “Skills Assessment Cover Sheet” allows the instructor to give feedback to students in the Core Skill area of Cognition. The instructor attaches the Cover Sheet to students’ homework, project, or exam papers after a grade has been assigned. The other four forms (Peer Assessment of Team Presentations, Team Self-Assessment Form, Self-Assessment Form, Instructor Evaluation of Teamwork) single out Communication and Teamwork.  Each provides feedback/assessment from various perspectives on team participation and team presentation.

Implementation & Observations

For the Spring 2000 Semester, I piloted our new assessment materials in my two Intermediate Algebra (Math 60)classes.  One class met from 2:00 to 3:50 p.m., Tuesdays and Thursdays.  The other met from 4:00 to 5:50 p.m. on the same days.

Explaining Core  Skills & Assessment

            In addition to my routine first-day explanation of the course objectives, I talked about the importance of the Core Skills and emphasized the Core Skills this class would focus on.  I also explained the benchmarks that we would use to assess these Core Skills.  I told them what their grade for the course would be based on:  5 tests, 5 homework assignments, 4 projects, participation/attendance and a final exam.

Team presentations

The second day of the class, I told the students that by the end of the second week of classes they needed to form teams of 3 to 4 members each.   Each team would be responsible for one or two presentations that would serve as review sessions for the tests and the final exam. Teams would choose their own topics. By the end of the second week of the classes the teams were formed and some of the teams had selected the chapters that they wanted to give presentations on. Throughout the semester before each test, at least two groups gave presentations, using the chalkboard, overhead projectors and computers to review for the test.

After I gave the team assignment in both classes, I noticed a very different classroom dynamic developing—a dynamic that I was not used to seeing in my more traditionally taught Intermediate Algebra classes of the past. Presenters were quite resourceful.  Many of them created handouts on the key topics. Several of them shared the handy learning techniques they had discovered during preparation for their presentation.  In these two new classes students were communicating with each other.  They were excited, and they accepted new responsibilities.  I had not observed these activities and attitudes in the traditional classes I had taught just one semester earlier (Fall 1999)--classes which did not include teamwork, presentations, or self-assessment.

These group presentations gave me a couple of important insights into the teaching-learning process.  I improved my teaching when I learned that the group presentations gave me an opportunity to assess students' knowledge and to correct their misunderstandings before they actually took the test.  One very interesting insight about student learning came to me through one of my best "A" students.  She came to me before her presentation and asked me to teach her the pronunciation of the algebraic expressions.  I assumed that these presentations are the only technical oral presentations that most of my students would be exposed to before they graduate or transfer to a four-year college.

At the end of each presentation, the class completed the Peer Review forms and gave them to the presenting.  I also assessed each team and gave oral feedback to the team and to the class.  I did not use the Instructor Evaluation of Team Work form.  I realized that it was more important to give immediate feedback to the small group--rather than delayed written comments--and to give that feedback so that the whole class could benefit from it, rather than just the team only.  

At first, Robert and I intended our students to make two team presentations – a segment of the review for one test and a piece of the review for the final.  However, because of the time constraints every team gave only one presentation, except for one team that did two reviews.

Skills Assessment Form

I filled out this form for three different exams (#1, #3, #4).  Even though it provided valuable comments and feedback to the students, it was very time-consuming to fill out these forms.  It doubled my grading time.  Robert and I have talked about making changes in this form that we think will make it easier to fill out.

Self-Assessment Form and Team Self-Assessment Form

At the end of each presentation, the team members completed these two forms and gave them to me.  I reviewed them outside of class and gave my oral commentary on them in the next class.  I gave my feedback without naming teams or individual presenters.

Statistics

The table above shows Average Exam Scores, Final Grade results, and Retention percentages for four Intermediate Algebra (Math 60) classes. I taught two of these classes in the Fall 1999 Semester, and two in the Spring 2000 Semester.  In the '99 classes, I used traditional presentation and testing methods.  In the '00 classes, I used the assessment innovations Robert Jones and I  had designed especially for this ALP pilot semester.

I set up the table so that it would be easy to compare the Fall 1999 classes to the Spring 2000 classes, as well as to compare the Afternoon Classes to the Late Afternoon Classes.

What the table reveals about the effect of teamwork, timely assessment, and feedback lets me see a trend make an educated guess.  At this point I am a very long way from being able to make any statistically supported claims.  There are two topics I would like to comment on, however.  The most consistent trend I see is in Student Retention;  in both the Afternoon and Late Afternoon classes, the retention percentage is better in the 2000 classes than in the 1999 classes. 

I also see something very interesting in the comparison of the two Late Afternoon classes.  Not only is the 2000 retention percentage higher than that for 1999, but the Exam scores and Final Grades are consistently higher in 2000 than in 1999.  My educated guess is that I was the variable associated with this impressive difference between the two classes.  I think I did a much better job introducing and implementing the new assessment-inclusive activities in the Spring 2000 Late Afternoon Class. Things went more smoothly, my instructions and feedback were clearer, and students understood my goals better in that class because I had the earlier Afternoon Class to practice and make my mistakes in.

I look forward to the coming semesters in which I'll continue using our ALP inspired methods and materials.  That work to come will give clearer definition to what I'm only glimpsing now.

Cindy Anfinson--Mathematics Department

Introduction

In the Fall 2000 semester I participated in the Pilot Test of the ALP feedback forms developed by Robert Jones and Fari Towfiq for the Mathematics Department.  The feedback forms covered the three Core Skills areas of Cognition, Communication, and Social Interaction.  Five feedback forms had been prepared by Robert and Fari; I used four extensively (Peer Assessment of Team Presentations, Team Self-Assessment Form, Self-Assessment Form, and Instructor Evaluation of Teamwork Form) and one briefly (Skills Assessment Cover Sheet Form).  I also prepared the draft of the Transfer of Knowledge and Skills to a New Context for the Benchmarks for Core Skills.

For the Fall 2000 semester I piloted the assessment materials in my Math 15 (Prealgebra) class that met from 12 – 12:50 pm on Mondays, Wednesdays, and Fridays.  I selected this class to pilot the materials in for several reasons.  First, in Math 15 I always review before every exam, so I knew I had ample time in my schedule for the students to present review materials.  Second, I had taught the same course in the Fall 1999 semester.  The two courses met at the same time and on the same days.  I felt that I would be getting a very similar student population, and by controlling the variability, I could have more confidence in the results of my study.  The only major difference I noticed between the two Math 15 courses was the Fall 2000 course had fewer non-native speakers of English than the Fall 1999 course.

Hypothesis and Expectations

My expectations of the pilot test program were very positive for two major reasons.  In a past Math 15 course I had taught I had students prepare a review for the final exam.  The students in that course did an outstanding job and seemed to enjoy the experience.  One group created a "Fraction Action" game show, while another discussed halving or doubling a cookie recipe to illustrate why we need to multiply or divide fractions (and they brought the cookies in for the class to enjoy!).  I felt that the students were excited and empowered by the teamwork they participated in.  The second reason I expected good results from the pilot test program came from my discussions with Fari Towfiq.  She had good results in her courses and I was hopeful that the program would create a similar positive and effective learning environment in my course.

I expected the following outcomes from the pilot test program.  First, I anticipated that the Fall 2000 class would perform as well or better than the Fall 1999 class.  Since the two classes were so similar in student composition I was expecting the Fall 2000 class to outperform the Fall 1999 class.  I was very hopeful that the class would show positive group interactions from the teamwork and have more enthusiasm than my previous Math 15 class. I feel that good group dynamics are very important in my classes, especially in developmental math courses.  I have observed that students who get to know and form friendships with their fellow students tend to “stick it out” better. I hope that by comparing retention rates I may be able to justify this observation.

Procedure

On the first day of class I handed out my syllabus and explained how they would be graded.  I gave 4 exams, 4 homework assignments, 8 projects (one of which would be the classroom presentation), weekly quizzes, and a final exam.  I did not go into great detail about the classroom presentation on the first day, but I did say they would be responsible for being a member of a group that would present one review for an exam. I had decided to grade their presentations so that the students would take them seriously.  I decided to weight their review the same as a project for the final grade.

At the start of the second week of the semester I told the students they needed to get into a group of 3-4 students by the end of the week.  I let them self-select their own groups.  I do not feel comfortable placing students in groups by assignment.  The students then wrote their group members’ names on a 3” x 5” card and handed it to me.  A week before each exam, I randomly selected two 3” x 5” cards and read out the teams.  I meet with the selected teams after class, and randomly assigned each team to present a review on half of the material that was to be covered on the exam.  I told them the basic requirements for their review was that they give a 5 to 10 minutes classroom presentation, give a handout to everyone in the class, and all participate equally on the presentation.  I made available overhead transparencies, overhead markers, and copied their handouts if they so requested. 

I used the forms developed by Fari and Robert during their presentations.  Each student watching the presentation received a Peer Assessment of Team Presentation Form.  After the presentation I collected those forms, later read over them myself, and handed them back to the presenting team during the next class period to review.  I handed each team member a Team Self-Assessment Form and a Self-Assessment Form and collected those at the end of their presentation.  I kept these for my files.  Finally, I used the Instructor Evaluation of Teamwork Form to assess their presentation and to assign a group grade.  I made copies of this form for each team member with their grade on it and handed it back to them at the next class meeting.  I graded very generously on a scale of 1 to 10.  If a team met my basic requirements for the presentation and put a reasonable amount of effort into the presentation I gave them a 9.  Most groups received a grade of 9 or 10.

I had misplaced the Skills Assessment Cover Sheet and forgot to use it on the first 3 exams.  This is my mistake.  I did use it on the fourth exam, and on the final exam.  I followed Fari Towfiq’s suggestion of selected one problem, working out the answer in detail and using that as a master copy.  Then, after grading the exam, I looked at a student’s response to the question I had selected, and marked the steps that they needed to improve on.  I stapled this to their exam as a cover sheet.  I did not find this very time consuming at all, and I will pilot this technique in two Math 50 classes in the Spring 2001 semester and will be able to report in more detail on its effectiveness.

Conclusions

I observed some positive outcomes due to the class presentations.  The students reported that since they had to speak in front of the class, they studied much harder than they would have without a presentation.  No one wants to look foolish in front of their peers, and the students felt the pressure to do a good job. The first group to present did an outstanding job and really set the tone for the rest of the semester.  They made a wonderful, computer-generated review handout, had well written overheads and did an overall excellent presentation.  Mathematical symbols are hard to generate on many word processors, so I appreciated the effort they took in making that handout.  Most of the groups thereafter also made up handouts on the computer.  This was not something I was expected from Prealgebra students!  I was very impressed by the depth and quality of the review sheets they handed out to the class.  Another group did a fun game show type format, where they had the guys sit on one half of the class, and the girls on the other half.  They would write a problem up on the overhead, and one guy and one girl would race to the board and try to work out the problem correctly first! 

The classroom environment exhibited overall good large group dynamics.  Most of the students in that particular class were on the quiet side.  Having to work in a team, observing fellow students give presentations, and learning the names of their fellow students seemed to break the ice in that class.  I was also pleased to note that most students seemed very compassionate towards each other.  They gave mainly good marks on their Peer Assessment review forms, and the criticism was constructive overall. 

Statistics

I would now like to present a chart outlining the statistics comparing the Fall 1999 Math 15 class with the Fall 2000 Math 15 class.  I computed each exam average by totaling the scores on the exam and dividing by the number of students who took the exam.  I computed the retention rate by taking the number of people who took the final and dividing by the number of people who took the first exam.

Looking at the statistics I noticed the following.  First, the retention rate for Fall 2000 (ALP class) was much higher than the retention rate for Fall 1999.  The average grades for exam #1, #2, and the final exam were higher for the Fall 2000 class than the Fall 1999 class.  The exam average grades for the #2 and #3 were higher in the Fall 1999 class than the Fall 2000 class (ALP), but not by much.  I'm not sure why that dip occurred.  However, the ALP class (Fall 2000) overall did much better than the Fall 1999 class.  I find these results very encouraging.  I would like to comment on the difference between the number of students who took the final and the number of students who received a final grade.  To compute the average of the final exam, I only calculated using the number of the students who actually took the exam.  To compute the average final grade I included all students who received a letter grade.  Of course, some students "disappeared" on me and received a grade even though they did not take all the exams.

Future

Fari Towfiq, Robert Jones, Mathews Chakkanakuzhi and I met at the end of the Fall 2000 semester to discuss the forms themselves.  We came up with some modifications that I anticipate will make the forms easier to use for the instructor.  I will be using these new forms in two Math 50 (Beginning Algebra) courses in the Spring 2001 semester. I will also use the Skills Assessment Cover Sheet on each exam, which will allow me to more effectively discuss its effectiveness.  I am excited about continuing in this pilot program.  Assessing students before they take they exam, improving group dynamics, and allowing students to exhibit skills they do not normally get to exercise in a math course (e.g., speaking in front of a group and writing mathematics for their peers) are all effective methods to determine if the students are performing effectively in the three Core Skills areas of Cognition, Communication, and Social Interaction. 

Matthews Chakkanakuzhi—Mathematics Department

Introduction

I am an Associate Professor in the Department of Mathematics. After completing my high school and first three years of college in India, I came to the U.S. as a junior in the Department of Mathematics at the California State University in Los Angeles where I completed my B.S. and M.S degree programs in Mathematics. While at CSULA, I had the opportunity to work with many students from students from various academic, ethnic and socio-economic backgrounds. These include working as tutor and/or discussion leader in the Minority Engineering Program (MEP), MESA program and Educational Opportunity Program. I have never studied in any community colleges. Also, at CSULA, I came to know the benefits of study-groups. However, as a student I was never forced to work with any students or groups in any of my classes by any instructor. Instead, often in my classes we formed our own groups to study and discuss the materials covered in class. These groups were made of volunteers of self-motivated students and we all benefited from each other. Most my classes were traditionally lecture classes and there were never any group-work done in any of my classes.

Only when I started teaching that I realized that not every student learns from self-study or traditional lectures. I always noticed (and many research supports) that students who are goal-oriented and who have good study habits tend to do better in classes than others. These students often seek help and voluntarily form study-groups. However, I never thought that I would have to motivate students to study or do their homework. Also, many times when I had my students do group-work, I find a group of students working very hard and another group who hesitate to work at all. By keeping track of these students, I came to know that the same group of students, who willingly do the required work, produces good results in the exams and good grades in the class. Despite of these observations, I was always in the look out for new ideas, motivational techniques and teaching methods to improve the student learning in my classes. I tried techniques such as collecting homework, giving pop-quizzes, having students prepare for classes prior to the lecture etc. I found out that each new technique benefits a few more students and never found a unique technique that works for all.

In Spring 2000, while being a member of the Peer Evaluation Team of Fari Towfiq, I came to know in detail the work done by the Assessment Learning Project Team here at Palomar College. I was very pleased with the multiple means of assessment that Fari used and the group presentations done by students in class when they prepare for the exams. I really liked the idea of having students prepare and present the review for the exams. This will allow the instructor to fill in any gaps or correct any mistakes, and give constructive feedbacks to the students before they were tested. I decided to try this out in my classes.

Expectations

There were many expectations. I wanted more students to do well in exams and hence improve student retention. Also, by working in groups, I expected all the students to participate more in the learning process and hence create a mutually encouraging and helping atmosphere in class. I envisioned many small communities of students becoming life-long friends by helping each other to reach their educational and career goals. I also

I mainly focused on the following benchmarks created by the ALP Team:

Cognition:

Problem Solving

Critical Thinking

Quantitative Reasoning

Transfer of Knowledge and Skills to a New Context

Social Interaction:  Teamwork

During Summer 2000, I had my Calculus I (Math 140) students do the group-presentations to prepare the class for the exams. This was done very informally. Other than verbal feedback and some points towards their homework I did not give any additional incentives or instructor evaluation of the presentation. Also, no peer evaluation (by the team itself or groups) was done. The results were encouraging.

In Fall 2000, I implemented these semi-formally. With the evaluation forms Fari and the department ALP Team created, I had the students in my Beginning Algebra (Math 050) class evaluate each team and their presentation. Also, I gave them written feedbacks on each team’s presentation for its contents, completeness and accuracy. Results were more encouraging.

Observations

I found many similarities and differences in students between the two classes.

·                    Students in the higher class (Math 140) were more open and willing to do these presentations, while some of the students in the remedial class (Math 050) show unwillingness to participate and cooperate.

·                    It was obvious from the presentations that each team in Math 140 participated more or less equally, while in Math 050 only 2 out 4 members of each team (team leaders) actively participated in the preparation phase of their presentation.

·                    In Math 140, the average exam scores of the team that did the presentation for that was generally higher than that of the class. In Math 050, the average grades of the team leaders were consistently higher than that of the class in every exam.

·                    Quality of the presentations was better in Math 140 compared to that of Math 050.

·                    The peer evaluations by students or team members gleaned very little information on their knowledge of the subject in Math 050. Most comments were “feel good notes”. This may have been due to my inexperience in having students do teamwork and my not coaching the students properly on this matter. I did not have my students do peer evaluations in Math 140.

·                    Many of the Math 140 students are continuing with the same study-groups in their Math 141 and Math 205 classes, while in Math 050 I see only team leaders are continuing to study with any group.

Concerns and Comments

Time---It takes an enormous amount of time to implement such a procedure in any class. I found myself working a lot more hours just to write informal feedbacks on their presentations. A lot more time will be needed to implement it formally.

Class size---The class size seems very large to do such a task. Between 20 and 25 per class will be an ideal size, as the students can get more personalized feedback on their work.

Motivation---Many students learned to work with the groups and in the process improved their grades, presentation skills and communication skills.

Retention---I noticed a better retention rate in my remedial class. However, the distribution of grades did not change significantly.

Conclusion

The implementation of these assessment techniques was a wonderful experience for me. I learned many things about my teaching and how my students learn. This definitely has helped many of my students in their mathematics classes and other classes. I was able to find out what the students learned prior to their tests and was able to help them better by knowing each student’s strengths and weaknesses. I look forward to implement these in more and more of my classes. I definitely recommend these assessment and learning techniques to instructors of any subject. I appreciate the help and support that I received from the Assessment Learning Team and I am looking forward to working with the team in the coming years.

Michael Mufson—Performing Arts

Prologue

The purpose of this project is to develop formats for assessing student progress in the core skill of group interaction.   The students in my introduction to theatre class use a collaborative creation process to respond to essays and play-scripts.  Each student participates collaboratively creating two original theatre performances.  The value of these projects has always been clear to me, however, it has been difficult to assess the individual student's contributions to the project and to apply concrete measurement to the student’s progress.

Preliminaries

            I developed two forms to assess the student's group interaction skills  (See attached).   On the self- assessment form the student identifies her own contributions to the project, her strengths, her weaknesses and ranks her contributions in relation to the rest of the group.  On the peer assessment form all the other group members rate various aspects the student’s participation on a 1-5 scale and write individual comments to the student. The course included five collaborative presentations.  Each presentation spanned a progressively longer development time and involved a progressively more complex task.

            The first difficulty I encountered was allotting the time to administer the assessment forms.  The collaborative creation process is very time intensive and for the first two projects I did not administer the assessments.  In retrospect I could have just done the self-evaluation on these projects.

Procedures/Process

The task for Project Three was to create a five-minute theatre composition that captured the essence of Oedipus and followed the Climactic Model of Theatre.  For this project the class divided into five groups.  The smallest group has six students and the largest had eleven.  The presentations of these projects spanned two days.  After discussing the projects and wrapping up the unit I administered the forms.  Several students from the groups that presented on the first day were not in attendance on the second day.  For the absent students, I had them complete the forms during subsequent classes.  Eventually I acquired a complete set of forms.

The students were excited to give and receive input in this manner.  It took longer than I anticipated to complete the process.  I had each student fill out her own self-assessment first and then pass the peer assessment (copied on the reverse side) to the other group members.  When the group finished filling out the peer-assessments, each student retrieved her own form to read her peers' comments.  At the end of class I collected all the completed forms.

To my surprise, I found the self-assessments to be much harsher than the peer assessments.  This reflects the general problem of low self-esteem among community college students.  Most of the peer comments were light, humorous and encouraging.  However, in the case of those students who missed one or more classes or did not participate fully, the comments tended to be frank but gentle.  A small percentage of the comments were quite severe. The process for the remaining two projects was similar to the first.

The task for Project Four was to create a ten-minute theatre composition about a “culture different than your own.” Each student wrote a short statement on an index card about a “culture that fascinates” her.  The entire class engaged in a consensus process of sorting the statements into groups based on commonalities.  This process resulted in four groups. 

The final project involved creating a ten-minute theatre composition that would utilize the theatre’s unique qualities to reflect the students’ generational-cultural aesthetic in such a way that it surpasses the appeal of mass media and entertainment.  Each student created a visual representation (collage) of her response to the task. The entire class engaged in a consensus process of sorting the visual representations to determine the groups. This resulted in four groups.

For both projects I had intended to administer the assessments at the end of class after the performances.  In both cases the performances used the whole class time and I administered the forms at the beginning of the next class period.  Several students were absent and this resulted in an incomplete process again.

Despite the logistical difficulties, the assessments were very useful for the students and for my ability to gauge the impact of the projects.

Outcomes

The qualities we are assessing here are difficult to quantify or observe in a statistical fashion, so I will rely on empirical data to measure the outcomes.

1.                  There was a significant shift in the quantity of comments from the first set of assessments to the third.

For project 1 61% of the class received fewer than 4 comments from peers.

For project 2 79 % of the class received fewer than 4 comments from peers.

For project 3 20 % of the class received fewer than 4 comments from peers.

2.                  There was also a significant improvement in the substance of the comments between the first and third projects.

a. Some representative comments from the first project include:

                                                               i.      Did a good job.

                                                             ii.      You are so great

                                                            iii.      You did a good job putting you ideas in

                                                           iv.      Nominated best actress in a leading role

                                                             v.      Awesome

                                                           vi.      Best acting since Nichalson

                                                          vii.      Brilliant

                                                        viii.      Tubular

                                                           ix.      At times I thought you were unnecessarily critical.

                                                             x.      Mike worked hard creating our ideas but didn’t seem comfortable with his own.

b. The comments from the second project were roughly similar to the first.  Even fewer comments of real substance.

c. Some representative comments from the third project include:

                                                               i.      You did a great job laying low and listening to others’ ideas.  You rock.

                                                             ii.      Good go-getter, brilliant ideas, good acting skills, over all experienced and smart.

                                                            iii.      Non-status Quo ideas.  Good insight into immigration problems.

                                                           iv.      Very focused on what the main idea should be.  Very easy to work with.

                                                             v.      She missed a few of the important rehearsals.  This really hurt us.  She had lots of ideas that seemed to help the play.

                                                           vi.      When you spoke up and gave an idea it was always good.  Do it more often.

                                                          vii.      Need to work on your problem with superiority.  Gave quality ideas and insight.

                                                        viii.      Whoa! Freek.  Show up on time will ya.

3. Below are case studies of four individual students’ progress indicated on the assessment forms.

1. Student A.

b.     Student A.

The data on Student A’s forms clearly indicates an increase in contribution and in self-confidence.  Notice particularly the shift in attitude indicated by the self-assessment. The comment for the first project is expressed in the negative and her best claim to contribution is “never distracting.”  In the second project she laid claim to one significant idea.  By the third project she expressed pride in making daily contributions to the project.

b.     Student B.

The student in this case is working on the opposite problem from student A.  By the final project he received no negative comments about his ability to accept others’ ideas. Student A’s description of his own compromise indicates a conscious effort to give control to others.  On project 2 he simply gave control away.  On project three he engaged in the process and was willing to compromise (although perhaps not happy about it.)   I would characterize the progression in the “contribution comments as 1. Claiming the bulk of the responsibility, 2. Dismissing his contribution by making it into a joke, 3. Expressing pleasure in the project even though he was not in control of the whole thing.

c.     Student C.

Lee Chen—English as a Second Language

Introduction

During this past Spring Semester, I was, regrettably, unable to implement my Virtual Communication benchmarks because my online ESL 103 class (Advanced Academic Writing) was canceled due to low enrollment.  As a result, I used my ESL 1, Level 5 class (Advanced Non-Credit ESL) as my pilot course.  Instead of applying my own Virtual Communication benchmarks, which would have been inappropriate, I used Michael Mufson's (Theater Arts) Social Interaction benchmarks.

Two Team Projects

In my on-ground ESL class, I required two team projects, one for each half of the semester. Students were put into 5 teams of 4 to 5 members each. There were new teams formed for the second project.  I designed both multiple-week projects as parts of my thematic units for the course in such a way that my Level 5 students would have ample opportunity to acquire and practice the teamwork skills subsumed under the Core Skill of Social Interaction.  I planned that they would use these skills as means to acquire English, as well as content knowledge and research techniques.

Project #1

The first project was entitled “A Famous First-Generation Mexican Immigrant.” Each team drew one of the five names I provided: Salma Hayek, Ricardo Montalban, Anthony Quinn, Paul Rodriguez, and Carlos Santana. Over a two-week period, students were to do instructor-assisted  and self-initiated research on the Internet and/or with print materials and get ready to make a team presentation utilizing a project display board.

The day after the project was assigned, I gave students a two-sided form with Michael Mufson's "Self-Assessment" form on one side and his "Peer Assessment" on the other. The idea was for them to start paying attention to all aspects of their own and their peers’ teamwork performance while they worked on their team project.

Because of the complex nature of the project, the actual student presentation date was postponed once. On the big day, the teams took turns presenting their famous Mexican. At the end of the class, everybody completed their Self-Assessment. They completed their Peer Assessment the following day.

Later, I assigned a percentage grade to each student, based on their Self- and Peer Assessments and on my own observations. I returned the forms to the students with some low points [CW8] [LC9] highlighted. I explained that I had pointed out those areas for them to work on and to do a better job for their next team project.  All during the project I, of course, had supported them with profuse positive oral feedback.  I just saved the corrective feedback till the final assessment.

Project #2

The second project was part of a thematic unit entitled “The Brain.”   I gave the new teams suggested topics for presentation. This time I made the two assessment forms separate instead of on a sheet back-to-back. Because of the demanding academic nature of the topics, I gave some additional assistance to some teams by bringing in copies of related articles and newspaper source information.

Teams settled on the following topics:

·                    Can TV harm the brain?

·                    Factors that disturb one’s state of mind

·                    How to have a healthy brain?

·                    The anatomy of the brain

·                    The corpus callosum

This project took teams much longer to prepare than the previous one. As with the first project, I made sure that I gave some class time for teams to work on their projects. And as with the first project, I had students complete their assessments and assigned a percentage grade for each student.

Lessons Learned

Prior to the semester, I had always done some teamwork in class, but nothing had been done in such a systematic and conscious way. I learned the following lessons thanks to the projects and their assessments. These lessons, both good and bad, are listed in no particular order.

·                    Group dynamics were much better, thanks to the formalized teamwork requirement. Although the teamwork system had always been in effect in my classes--for small group work and classroom management-- the added team presentations really contributed to how students organized and mobilized themselves. The use of the Mufson's Social Interaction Self- and Peer Assessment Benchmarks provided the necessary tools for my students to become reflective learners, particularly with regard to their teamwork skills.

·                    The team projects can be very time-consuming. I ended up changing the number of team presentation projects from an ambitious three to two. Fortunately, because the projects were so tied with the content learning at hand and with the four language skills, the time was well spent. On the day of presentations, even the audience were given some concurrent tasks to do that were focused on the presentation content.

·                    Assigning grades based on the assessment forms can be problematic. For the first presentation, the two-sided forms contained both the Self- and the Peer Assessments. Students doing the Peer Assessment got to see how an assessee had evaluated himself/herself on the other side of the form. As a result, students tended to be more face-saving for each other than the second time around, when they didn’t get to see others’ Self-Assessments. A common-sense approach has to be adopted if assigning a grade is to be desired.

·                     Team projects need to be related to students’ interests, needs, and comfort levels. A case in point was how no teams chose to present on a brain researcher for their second project, presumably because they didn’t know much about those scientists. The solution seems to lie in providing an array of choices for the teams to pick from. Another case in point: during a follow-up discussion, students talked passionately about more practical topics like “Can TV harm the brain?” Again, I let my students engage in this natural-selection process.

·                    Where to find information and how to evaluate the information found were two great side benefits from doing the team presentation projects. In addition to the Social Interaction skills, students honed their critical thinking skills to some extent. This classroom assessment of teamwork skills was formative, not summative. Even though I opted to provide a grade, I did so to encourage my students to focus on how they can do a better job for a future team project. I wanted them to concentrate on doing a good job working in teams and presenting their projects. No tests resulted from these projects.

·                    Most students liked to work in teams to do such projects. I found this feeling surfacing time and again in their journal entries and also in their end-of-semester reflections on their last project. Here are five students’ testimonies:

Cynthia Watson—English as a Second Language

            It is with great sadness that we note that Cynthia died before completing her final report.  She was a dear friend and a wonderful colleague and her spirit and energy remain a permanent contribution to the students of Palomar College and the work of the ALP Team.