a.      Robert Jones—Mathematics

                                                              i.      Introduction

                                                            ii.      Background

                                                          iii.      Implementation

                                                          iv.      Comments

                                                            v.      Statistics

b.      Fari Towfiq—Mathematics

                                                              i.      Introduction

                                                            ii.      Benchmarks

                                                          iii.      Implementation and Observations

                                                          iv.      Explaining Core Skills and Assessment

                                                            v.      Team Presentations

                                                          vi.      Skills Assessment Form

                                                        vii.      Self-Assessment and Team Self Assessment Form

                                                      viii.      Statistics

c.       Cindy Anfinson—Mathematics

                                                              i.      Introduction

                                                            ii.      Hypothesis and Expectations

                                                          iii.      Procedure

                                                          iv.      Conclusions

                                                            v.      Statistics

                                                          vi.      Future

d.      Matthews Chakkanakuzhi—Mathematics

                                                              i.      Introduction

                                                            ii.      Expectations

                                                          iii.      Observations

                                                          iv.      Concerns and Comments

                                                            v.      Conclusion

e.      Michael Mufson—Performing Arts

                                                              i.      Prologue

                                                            ii.      Preliminaries

                                                          iii.      Procedures/Process

                                                          iv.      Outcomes

f.        Lee Chen—English as a Second Language

                                                              i.      Introduction

                                                            ii.      Two Team Projects

                                                          iii.      Project #1

                                                          iv.      Project #2

                                                            v.      Lessons Learned

g.      Cynthia Watson—English as a Second Language

Introduction: Description of the Departmental Project Team

The purpose of the departmental project team is to work in conjunction with the Assessment of Learning Project (ALP) to apply Palomar College’s list of Core Skills to the content and activities of courses in three selective disciplines.  The five members of the Departmental Project Team represent three disciplines at Palomar College.  Fari Towfiq and Cynthia Watson are the Team’s coordinators and they also serve on the ALP team.  Robert Jones represents the discipline of mathematics.  Michael Mufson represents the discipline of the performing arts. Lee Chen represents the discipline of English as a Second Language.  Recently, Lee Kerckhove has joined the team as a co-coordinator of the departmental project team, and Matthews Chakkanakuzhi and Cynthia Anfinson have joined the mathematics portion of the project.  A brief overview of this project follows.

MATHEMATICS

Robert Jones, Assistant Professor, is following his new assessment-inclusive syllabus in his Pre-Algebra class.  Fari Towfiq, Associate Professor, is implementing the pilot assessments in her Intermediate Algebra classes.  Towfiq and Jones worked together to formulate the benchmarks for Mathematics and are using the same assessment methods and feedback forms in their respective classes. They have been very careful to make their assessments convenient enough to fit into mathematics courses that must cover a prescribed number of chapters in a standardized text in 16 weeks. Towfiq and Jones have selected two Core Skill areas to assess in their respective algebra classes. In the first area of Cognition, they provide for the assessment of students' use of the two Core Skills of Problem-Solving, and Transfer of Knowledge. The "Skills Assessment Cover Sheet" is a feedback form they developed to be filled out by the instructor and attached to students' homework or exam papers after a grade has been assigned. Towfiq and Jones do not intend for this Cover Sheet to determine any part of their students' grades. In their classes it is for on-going feedback only. They also developed four forms (Peer Assessment, Team Assessment, Self-Assessment, and Instructor Evaluation of Teamwork) that single out Teamwork in the Core Skill area of Social Interaction. Each provides for feedback/assessment from various perspectives on team participation and team presentations.  Recently, Chakkanakuzhi and Anfinson contributed to the revision and modification of these forms.

PERFORMING ARTS

Michael Mufson, Associate Professor, Theater Arts, is applying the benchmarks that he formulated to his Introduction to Theater class, which includes improvisation, workshop scripting, and performance of the students' own pieces. Mufson chose the general Core Skills (ALP website) categories of Social Interaction, Aesthetic Responsiveness, and Cognition as the basis for his Introduction to Theater Arts assessment pilot. His first two feedback forms, "Self Assessment" and "Peer Assessment," focus on the Social Interaction abilities of "Teamwork" and "Effective Citizenship." Students use them to assess their three collaborative projects. His third feedback form, "Visual Representation Assessment," applies to individual students' Visual Representation projects. It focuses on the Cognitive category of Analysis and Synthesis.

ENGLISH AS A SECOND LANGUAGE

Lee Chen, Assistant Professor, has included his writing assessment benchmarks in the syllabus of his on-line ESL Advanced Writing class. His benchmarks are just as useful in the "real-space" classroom as they are in the virtual classroom. Lee Chen has designed an on-line ESL Advanced Writing course to be offered by Palomar College in the 2000-2001 academic year.  He developed two forms.  The first, "Instructor Feedback," applies to the sub-categories of Speaking and Listening.  It provides information to the student from the instructor about the quality of his/her participation in the virtual class discussions. All students in the online class receive e-mailed feedback twice during the semester--in the third week and at the end.  A second form, "Individual Assessment," appraises the student's writing, which is the fourth sub-category under the Core Skill of Communication. Students receive feedback via this Individual Assessment at the beginning and end of the semester, at about the same times they receive feedback on the quality of their online discussion.

Cynthia Watson, Associate Professor, is teaching a Literacy Level English as a Second Language class using videotape assessment as a source for feedback and evaluation in the two language modes of speaking and listening.  She has developed a video assessment for her Beginning ESL class, which focuses on the Communication Core Skills of Speaking and Listening (ALP website). She developed assessment benchmarks for student video presentations at four proficiency levels, and a form that she and her students use when they rate videotaped presentations. This form closely (but not exactly) follows a primary trait rating scale design. At the beginning of the semester, it is used for feedback only; at semester's end it is used both for feedback and to assign a grade.  In Beginning ESL, Watson videotapes students twice--at the beginning and end of the semester. She then tracks these students' developing speaking proficiencies by videotaping them at the end of each semester as they move upward through the levels of the ESL program.

Individual Reports

Robert Jones – Mathematics Department

Introduction

For the Spring 2000 semester, I piloted assessment materials in my Prealgebra class (Math 15) that Fari Towfiq and I had developed the previous Fall.  We both agreed that we wanted to emphasize the critical thinking component of the Core Skills document (as identified by ALP) within the context of our math classes, and we also wanted to make teamwork & communication a more significant part of the work that the students did in class.  Since Fari and I realized that we’d have difficulty getting other members on the math faculty to try our materials--especially if they generated a significant amount of extra work--our goal with these materials was to make assessment a more ‘seamless’ part of the day-to-day activities in our classes.

Background

At Palomar College, Math 15 is the only math class (along with Math 10 – Arithmetic) without a prerequisite.  Typically, the majority of the students taking this class are re-entry students who’ve been out of school for a long time; others find themselves in Math 15 after making an unsuccessful attempt to pass a placement exam that would enable them to take Math 50 (Algebra).   Many of these students have a tremendous amount of anxiety towards math and a very low opinion of their abilities – attitudes that have been developed through previous experiences with mathematics that, for the most part, have been characterized by frustration & failure.

Hence, aside from trying to get students ready to take Math 50, one of my main tasks in teaching any Math 15 class is simply to get students to have a more positive attitude toward their work and toward mathematics in general.  Even prior to my participation with ALP, I’ve found that this can be facilitated by structuring the class so that students are active participants in the work that takes place there.  As opposed to having students merely take notes (as passive learners), I try to give them meaningful tasks and daily objectives that they meet by working independently in small groups.  (Students are told on the first day that ‘groupwork’ is a significant learning style in my classroom, and it’s part of the syllabus that I give to them.  I don’t employ this method in all of my classes, but I’ve found that this approach is well-suited for students at this level.)  In this environment, students learn that mathematics isn’t just a bunch of skills that are practiced in isolation, but an active process that requires communication – much like most work in the ‘real’ world.  Groups are encouraged to be self-sufficient and, by not having to spend the majority of the class lecturing at the chalkboard, I’m free to walk around the class and to provide help as needed.  Hence, I can quickly assess my students’ understanding of concepts while they’re learning them, either by asking students to explain ideas to me or to other members of their groups.

Within this context, then, I believed that the assessment benchmarks that Fari and I developed would be less ‘invasive’ than they might be in other classes.  It didn’t seem to be much of a ‘step’ for students to move from explaining concepts within their groups to explaining them to the class.  Also, I wanted to know if the critical thinking forms that Fari and I had generated could have a positive effect on student learning in a developmental class.

Implementation

My Math 15 class for Spring 2000 met from 4:00 p.m. to 5:20 on Mondays and Wednesdays.  I began the semester with about 30 students.  They were told on the first day that, in addition to the traditional evaluation tools--3 tests, daily quizzes, homework checks, and a final exam--they would also be evaluated by their ‘participation.’   I didn’t go into any details as to what this meant (i.e., using group presentations to review for the tests & the final exam), choosing instead to provide them as the class developed.   None of the materials that Fari and I developed were used until after the first exam; I wanted my students to feel comfortable within the class and to have a minimum level of mathematics to communicate with.

Here’s the timeline I followed:

Weeks 1 to 4:  Groups formed & class routine established; Test #1 given at the end of Week #4.

            Weeks 5 to 10:  Group presentations begin (see Comments section); Skills Assessment (critical thinking) forms completed & returned with Test #2 (Week 10).

Weeks 11 to 15:  ‘Presentations’ continue in modified form (see Comments section); Skills Assessment (critical thinking) forms not used with Test #3 (see Comments section).

Week 16:  New material completed (as specified in the Course Outline of Record); review for Final (one class period).

Comments

1. Group Presentations: To prepare for Test #2, four groups (sixteen students total) were randomly picked from the existing teams and assigned topics – order of operations, adding and subtracting signed numbers, length and perimeter, and the ‘meaning’ of multiplication - to review for the class.  In addition to demonstrating examples, each group provided three to five review problems related to their presentations.  Because of time constraints, I wasn’t able to give the groups time in class to work on their presentations – they had to work on them outside of class and be ready to ‘go’ the next time we met.   As a result, these initial presentations weren’t really ‘creative’; for each, one to two members wrote problems on the board while the remaining members asked the other students in the class how to solve them.  None of the groups used the overhead projector or any other ‘visuals’ (posters, etc.).  Some groups and group members were a little nervous and ‘stiff’ (and they told me so!), but this was to be expected.  Overall, I can say that I was generally pleased with the level of effort and participation shown, and I felt that a positive standard had been set for future presentations.  The class completed the peer review forms, which were given to the group members after each presentation. Also, I evaluated each group and gave them a grade; realizing that I wouldn’t have time outside of class to look at the self and team assessment forms, I didn’t have the groups fill these out.

2.  Initially, Fari and I intended our students to make two group presentations – a portion of the review for one test and part of the review for the Final.  However, due to time constraints, I modified the presentation process, making it much more informal.  Groups were now sent to the board or called upon--usually once or twice each period (toward the end of class)--to demonstrate selected solutions.  In this way, communicating mathematics simply became a regular part of our daily routine, not just something that was done before a test. Team members were still expected to justify their results – both verbally and in writing.  Without a lot of time to prepare, these presentations were more spontaneous than those associated with reviewing material for tests, and  they were invaluable in helping me to determine my students’ conceptions of the topics they were learning.  I quickly scored these presentations using a four point rubric; the peer review and associated paperwork were eliminated.  By using this new format, I was able to ensure that each student had at least two opportunities to speak to the class.      

3. Skills Assessment Form: Although my students appreciated the comments and feedback this form provided, filling these out for the students who took Test #2 turned out to be a very time-consuming exercise.  Hence, I had to abandon using the form for Test #3.  (Fari and I subsequently modified this form, making it easier to fill out.)  Regardless of using the form or not, I do believe that, as critical thinking continued to be emphasized in the work students did both inside and outside of class, many of them became more aware of mathematics as a process – a way of thinking – in which the explanation was as important as the answer.

Statistics

Having piloted these new materials in just one class, I believe it’s difficult to make statistically significant comparisons with other Math 15 classes that I’ve taught.  At the very least, more work should be done in our department to ascertain the effect these approaches have on learning.  (I am currently using these materials in my Prealgebra class for the Spring 2001 semester.)

            Having said this, here are some numbers for the class I taught in Spring ‘00 and the two previous sections of Math 15 that I taught in the fall of 1998.  I used the same textbook for both semesters; Tests 2 & 3 and the final were similar in terms of content & length.  (‘Median score’ - 50% of the class scored above this mark, and 50% scored below;  - the arithmetic average for the given class – i.e., the sum of the scores for the given class divided by the number of scores;  represents the ‘standard’ deviation – it’s a measure of how far an ‘average’ score in a given class falls from.)

Test #2 –

Test #3 –

Final –

Fari Towfiq--Mathematics Department

Introduction

In the Fall 1999 Semester, my Math colleague Robert Jones and I, decided to modify the way we teach two of our basic mathematics courses--Prealgebra (Math 15) and Intermediate Algebra (Math 60).  Under the guidance of the ALP Departmental-Project Team, we prepared to depart from our traditional ways of presenting the courses.  

During this Fall Semester, we selected three Core Skill areas for which we wanted to create performance benchmarks.  We hoped to use our new benchmarks to assess the quality of our students' learning via their performance in those three areas.  In the Spring 2000 Semester, we did indeed apply our new means of assessment and our modified instructional methods to our respective math classes.  The three areas on which we focused our benchmarks follow here:

                        Area 1: Cognition

                                                Analysis and Synthesis

                                                Problem Solving

                                                Creative Thinking

                                                Quantitative Reasoning

Transfer of Knowledge and Skills to a New Context

                        Area 2: Communication

                                                Speaking

                                                Listening

                                                Reading

                                                Writing

            Area 3: Social Interaction

                                                Teamwork

During this process of creating and applying our benchmarks, we realized that, if we were not careful, Robert and I could generate a lot of extra work for ourselves and for other faculty who would like to use our material.  Thus, we added a new goal to our benchmark project:  keep it as user-friendly and efficient as possible.  We did not want to make the assessment process so time-consuming that it would overburden our already very busy Math Department colleagues.

Benchmarks

Robert and I created five different feedback forms for assessing in the three Core Skill Areas of Cognition, Communication and Social Interaction.[CW1]  The “Skills Assessment Cover Sheet” allows the instructor to give feedback to students in the Core Skill area of Cognition. The instructor attaches the Cover Sheet to students’ homework, project, or exam papers after a grade has been assigned. The other four forms (Peer Assessment of Team Presentations, Team Self-Assessment Form, Self-Assessment Form, Instructor Evaluation of Teamwork) single out Communication and Teamwork.  Each provides feedback/assessment from various perspectives on team participation and team presentation.

Implementation & Observations

For the Spring 2000 Semester, I piloted our new assessment materials in my two Intermediate Algebra (Math 60)classes.  One class met from 2:00 to 3:50 p.m., Tuesdays and Thursdays.  The other met from 4:00 to 5:50 p.m. on the same days.

Explaining Core  Skills & Assessment

            In addition to my routine first-day explanation of the course objectives, I talked about the importance of the Core Skills and emphasized the Core Skills this class would focus on.  I also explained the benchmarks that we would use to assess these Core Skills.  I told them what their grade for the course would be based on:  5 tests, 5 homework assignments, 4 projects, participation/attendance and a final exam.

Team presentations

The second day of the class, I told the students that by the end of the second week of classes they needed to form teams of 3 to 4 members each.   Each team would be responsible for one or two presentations that would serve as review sessions for the tests and the final exam. Teams would choose their own topics. By the end of the second week of the classes the teams were formed and some of the teams had selected the chapters that they wanted to give presentations on. Throughout the semester before each test, at least two groups gave presentations, using the chalkboard, overhead projectors and computers to review for the test.

After I gave the team assignment in both classes, I noticed a very different classroom dynamic developing—a dynamic that I was not used to seeing in my more traditionally taught Intermediate Algebra classes of the past. Presenters were quite resourceful.  Many of them created handouts on the key topics. Several of them shared the handy learning techniques they had discovered during preparation for their presentation.  In these two new classes students were communicating with each other.  They were excited, and they accepted new responsibilities.  I had not observed these activities and attitudes in the traditional classes I had taught just one semester earlier (Fall 1999)--classes which did not include teamwork, presentations, or self-assessment.

These group presentations gave me a couple of important insights into the teaching-learning process.  I improved my teaching when I learned that the group presentations gave me an opportunity to assess students' knowledge and to correct their misunderstandings before they actually took the test.  One very interesting insight about student learning came to me through one of my best "A" students.  She came to me before her presentation and asked me to teach her the pronunciation of the algebraic expressions.  I assumed that these presentations are the only technical oral presentations that most of my students would be exposed to before they graduate or transfer to a four-year college.

At the end of each presentation, the class completed the Peer Review forms and gave them to the presenting.  I also assessed each team and gave oral feedback to the team and to the class.  I did not use the Instructor Evaluation of Team Work form.  I realized that it was more important to give immediate feedback to the small group--rather than delayed written comments--and to give that feedback so that the whole class could benefit from it, rather than just the team only.  

At first, Robert and I intended our students to make two team presentations – a segment of the review for one test and a piece of the review for the final.  However, because of the time constraints every team gave only one presentation, except for one team that did two reviews.

Skills Assessment Form

I filled out this form for three different exams (#1, #3, #4).  Even though it provided valuable comments and feedback to the students, it was very time-consuming to fill out these forms.  It doubled my grading time.  Robert and I have talked about making changes in this form that we think will make it easier to fill out.

Self-Assessment Form and Team Self-Assessment Form

At the end of each presentation, the team members completed these two forms and gave them to me.  I reviewed them outside of class and gave my oral commentary on them in the next class.  I gave my feedback without naming teams or individual presenters.

Statistics