ASSESSMENT OF LEARNING PROJECT

PALOMAR COLLEGE, SAN MARCOS, CA

DEPARTMENTAL PROJECT TEAM

FIRST REPORT

CREATING BENCHMARKS IN THREE DISCIPLINES

February 24, 2000

This document is a collection of educational assessment benchmarks, presented in the form of efficient and easily applied rating scales, comment sheets, and criteria lists. Under the guidance of Palomar College's Assessment of Learning team (ALP), five Palomar faculty created these benchmarks by applying the college's list of Core Skills to the content and activities in one of the their current courses.

This first report records the outcome of the Departmental Project Team's one-semester planning phase (August-December, 1999). It is divided into four sections: 1."Mathematics" by Fari Towfiq and Robert Jones, 2. "Performing Arts" by Michael Mufson, "English as a Second Language" by Lee Chen, and 4. "English as a Second Language" by Cynthia Watson.

A second report due in August 2000, will summarize the Team's applications of their plans during the Spring 2000 semester. Its purpose will be to recap the team members' experiences during their implementation semester and to share their reflections on the results of their work.

THE DEPARTMENTAL PROJECT TEAM

The five members of the Departmental Project Team represent three disciplines at Palomar College. Two of its members, Fari Towfiq and Cynthia Watson, the Team's coordinators, also serve on the ALP team. A brief overview of Pilot Team membership follows.

MATHEMATICS

Robert Jones, Assistant Professor, is following his new assessment-inclusive syllabus in his Pre-Algebra class.

Fari Towfiq, Associate Professor, is implementing the pilot assessments in her Intermediate Algebra classes.

Towfiq and Jones worked together to formulate the benchmarks for Mathematics and are using the same assessment methods and feedback forms in their respective classes. They have been very careful to make their assessments

convenient enough to fit into mathematics courses that must cover a prescribed number of chapters in a standardized text in 16 weeks.

PERFORMING ARTS

Michael Mufson, Associate Professor, Theater Arts, is applying the benchmarks that he formulated to his Introduction to Theater class, which includes improvisation, workshop scripting, and performance of the students' own pieces.

ENGLISH AS A SECOND LANGUAGE

Lee Chen, Assistant Professor, has included his writing assessment benchmarks in the syllabus of his on-line ESL Advanced Writing class. His benchmarks are just as useful in the "real-space" classroom as they are in the virtual classroom.

Cynthia Watson, Associate Professor, is teaching a Literacy Level English as a Second Language class using videotape assessment as a source for feedback and evaluation in the two language modes of speaking and listening.

SECTION 1

MATHEMATICS BENCHMARKS

Fari Towfiq and Robert Jones

Fari Towfiq and Robert Jones have selected two Core Skill areas to assess in their respective algebra classes. In the first area of Cognition, they provide for the assessment of students' use of the five Core Skills of Analysis and Synthesis, Problem-solving, Creative Thinking, Quantitative Reasoning, and Transfer of Knowledge. The "Skills Assessment Cover Sheet" (#1 below) is a feedback form to be filled out by the instructor and attached to students' homework or exam papers after a grade has been assigned. Towfiq and Jones do not intend for this Cover Sheet to determine any part of their students' grades. In their classes it is for on-going feedback only.

The next four forms single out Teamwork in the Core Skill area of Social Interaction. Each provides for feedback/assessment from various perspectives on team participation and team presentations.

SKILLS ASSESSMENT COVER SHEET

Five Core Skills in the area of Cognition

(to be attached to the front page of work turned in)

Skills Assessment for

Test # / Homework #:

1. Analysis & Synthesis Understanding & building upon concepts to establish connections between them & advance to valid conclusions.

•Look at question/problem

Needs

Improvement

Outstanding

Comments

2. Problem Solving Analyzing problems and developing, evaluating & testing solutions.

•Look at question/problem

Needs

Improvement

Outstanding

Comments:

3. Creative Thinking Formulating ideas and concepts in addition to using those of others.

•Look at question/problem

Needs

Improvement

Outstanding

Comments:

4. Quantitative Reasoning Using mathematical concepts and methods to understand, analyze, and explain concepts in quantitative terms.

•Look at question/problem

Needs

Improvement

Outstanding

Comments:

5. Transfer Applying knowledge & skills to new & varied situations.

•Look at question/problem

Needs

Improvement

Outstanding

Comments

2. PEER ASSESSMENT OF TEAM PRESENTATIONS

Core Skill: Teamwork

• Complete and return to the presenting team.

Review for Test #

Team Members:

Topic:

1. Speaking Members speak in an understandable and organized fashion to explain their ideas:

Needs

Improvement

Outstanding

Comments

2. Writing Members write in an understandable and organized fashion to explain their ideas:

Needs

Improvement

Outstanding

Comments:

3. TEAM SELF ASSESSMENT FORM

Core Skill: Teamwork

• Complete & return with all self-assessment forms to your instructor.

• The purpose of this form is to provide feedback on each member’s participation. Answer each question honestly:

Questions (1 = lowest; 5 = highest)	Member 1	Member 2	Member 3
Did member attend all group meetings?
Did member work well with others?
Did member contribute quality ideas to the project?
Did member explain ideas clearly to group?
Did member listen carefully to ideas raised by group?

How does your group rate the overall effectiveness of working together?

(1 = lowest; 5 = highest)_____________________________________________

Comments:

4. SELF ASSESSMENT FORM

Core Skill: Teamwork

Self-Assessment for

Team Review for Test #

• Complete & return with team assessment form to your instructor.

Questions (1 = lowest; 5 = highest)	Score
Did you attend all group meetings?
Did you work well with others?
Did you contribute quality ideas to the project?
Did you explain your ideas clearly to your group?
Did you listen carefully to ideas raised by your group?

Describe your most significant contribution to your team’s presentation:

How would you rate the amount of your participation? Questions (1 = lowest; 5 = highest.)

How would you rate the quality of your participation? Questions (1 = lowest; 5 = highest.)

5. INSTRUCTOR EVALUATION OF TEAMWORK

Core Skill: Teamwork (Social Interaction area)

Review for Test #

Team Members:

Topic:

1. Cohesiveness & Organization:

Comments:

2. Speaking:

Comments:

3. Writing:

Comments:

Overall Score: (#1 + #2 + #3)

SECTION 2

PERFORMING ARTS BENCHMARKS

Michael Mufson

Michael Mufson chose the general Core Skills (ALP website) categories of Social Interaction, Aesthetic Responsiveness, and Cognition as the basis for his Introduction to Theater Arts assessment pilot. His first two feedback forms--"Self Assessment" and "Peer Assessment"--focus on the Social Interaction abilities of "Teamwork" and "Effective Citizenship." Students use them to assess their three collaborative projects. His third feedback form, "Visual Representation Assessment," applies to individual students' Visual Representation projects. It focuses on the Cognitive category of Analysis and Synthesis.

(Core Skill: Social Interaction)

Self Assessment
Name:		# Of participants in your group:
# Of class sessions devoted to this project.	# You missed:		# You were late:
Describe the project in one sentence:
On a scale from 1 to the # of participants in your group, rank the e amount of f your participation in relation to the rest of the group. (1 = the top)
On a scale from 1 to the # of participants in your group, rank the quality of f your participation in relation to the rest of the group. (1 = the top)
Describe your most significant contribution to the project:
On a scale of 1 to 5 rate your ability to sway others to your idea or opinion (5=highest).
On a scale of 1 to 5 rate your ability to consider and accept others’ ideas (5=highest).
Describe one ocassion in the planning process on which you had to compromise:

Peer Assessment Form

(Have each member of your group use one column to assess your participation)

Your Name:

Project:

The purpose of this form is to provide feedback on your peers' group participation skills. It is essential that you answer each question honestly. Use the comment boxes below for additional feedback.

Questions (5-highest; 1=lowest)

Participant 1

Participant

Participant 3

Participant 4

Participant 5

Participant 6

Participant 7

Did student contribute often to the project?

Did student contribute quality ideas to the project?

Did student listen carefully to all the ideas raised by the group.

Did student attend all the group meetings?

Did student work well with others?

Did student compromise when necessary?

Did student explain ideas clearly to the group?

COMMENTS:

Visual Representation (VR) Guidelines: This is a creative format for communicating your understanding and analysis of the material. Sources for the representation might include: cut and paste photographs from newspapers and magazines; photo-copies of images from text book or other research sources; illustrations created or commissioned by you; fields of color, line or texture using paint, marker, crayon, food coloring or other sources. Your representation should include some words. You may decide on the balance of words to images. Your representation may be kinesthetic—it may be folded or opened or turned or experienced in some hands-on way to communicate your idea. Consider the elements of: color, shape, line, movement, balance, weight as you compose the images and text. The successful representation will create in the viewer an immediate and overall impact. A bunch of loosely connected images pasted randomly on the page will not communicate effectively.

For Oedipus and the Climactic model find two sources of images and information about Greek Theatre. Find a way to include the bibliography in the visual representation.

Be prepared to present your VR to class. Approximately 3 minutes.

Aesthetic Responsiveness, Analysis and Synthesis

TA 100: Introduction to Theatre

Visual Representation of the Climactic Model & Oedipus

Name:
CONTENT:
Variety/Quality of source material
Relation of visual images demonstrates relations between ideas covered in the material
Point of view/Conclusion is clear
Effective use of text
Use of shape
Immediate, overall impact
Cohesion/unity of images
Clarity of analysis
VISUAL ELEMENTS:
Use of symmetry and balance
Kinesthetic elements
Interactive elements
Use of movement
Use of texture
Use of color
Use of line
COMMENTS:	POINTS

SECTION 3

ENGLISH AS A SECOND LANGUAGE

Lee Chen

Lee Chen has designed an on-line ESL Advanced Writing course to be offered by Palomar College in the 2000-2001 academic year. The two forms below apply to the major Core Skill (ALP website) category of Communication. The first "Instructor Feedback . . . ," applies to the subcategories of Speaking and Listening. It provides information to the student from the instructor about the quality of his/her participation in the virtual class discussions. All students in the online class receive emailed feedback twice during the semester--in the third week and at the end.

The second form, "Individual Assessment . . . ," appraises the student's writing, which is the fourth subcategory under the Core Skills of Communication. Students receive feedback via this Individual Assessment at the beginning and end of the semester, at about the same times they receive feedback on the quality of their online discussion (Form #1)

1. Instructor Feedback for Online Class Discussions (Virtual Speaking and Listening)

Level	Beginning (=Grade of “C”)	Developing (=Grade of “B”)	Accomplished (=Grade of “A”)
Frequency	A lurker who reads all or most postings but contributes only about 1/3 of the required number of postings per week.	Student reads all postings, posts 2/3 of the required number of postings per week.	Student reads all postings and posts the required or more than the required number of postings per week. Postings tend to be spread throughout the week rather than clustered together, indicating frequent access to the discussions.
Quality	Postings tend to address peripheral issues, are generally accurate, but with some omissions and/or errors. There is a tendency to recite facts rather than address issues.	Postings tend to provide good general answers but may not always directly address discussion questions. Postings are dominated by personal opinions rather than analysis and creative thought. Assertions are not supported by evidence.	Postings are characterized by clarity of argument, depth of insight into theoretical issues, originality of treatment, and relevancy, and sometimes include unusual insights and flashes of brilliance. Arguments are well supported.
Interaction	Student never includes questions that stimulate discussion and seldom responds to questions raised by others.	Sometimes postings include questions that stimulate discussion or respond to questions raised by others.	Postings frequently include good questions that stimulate discussion. Student frequently responds to questions from others.
Length	Postings may frequently miss or exceed the instructor’s length requirement, if any.	A few postings may miss or exceed the instructor’s length requirement, if any.	Any single message never misses or exceeds the instructor’s length requirement, if any.
Etiquette	Postings are never rude and never reflect hostility.	Postings are never rude and never reflect hostility.	Postings are never rude and never reflect hostility.
Mechanics	Postings may contain numerous errors in spelling and grammar, indicating lack of proofreading.	Few, if any, errors in spelling and/or grammar, indicating proofreading.	Few, if any, errors in spelling and/or grammar, indicating proofreading.

2. INDIVIDUAL ASSESSMENT OF WRITING

Writing: Students will write in an understandable and organized fashion to explain their ideas, express their feelings, or support a conclusion.

Beginning Writers

Ö	Demonstrated Competence
	A high level of generality causes the writing to seem thin.
	Writers need to include supporting detail rather than simple statements.
	Repetition rather than development signals this level of writing.
	Although form is formulaic, paragraphing is present.
	Papers exhibit weaknesses in focus, structure, syntax, diction, or mechanics.

Developing Writers

Ö	Demonstrated Competence
	Writers seem to be in control of the writing.
	The writing shows independent thought and the writer’s active engagement.
	Writers show a clear sense of order.
	Words are expected that are absorbed from reading rather than listening.
	Writers tend to turn in clean, correct papers.
	Writers are aware of rhetorical strategies.
	Writers use parallelism, repetition, contrast, and the rhetorical question.
	Writers show mature use of subordination.
	Flaws are not so numerous/serious as to compromise the writer’s proficiency.

Accomplished Writers

Ö	Demonstrated Competence
	The writing invites reading, is unusual.
	Writers have something worthwhile to say, shows a sense of self.
	What writers see is unexpected; what they write is fresh.
	Writers risk more despite occasional impatience with detail.
	Writers bring intellectual/imaginative/intuitive resources to the writing.
	Writers can read with understanding and penetrate beneath the obvious.
	Writers have a capacity to observe and reflect.
	Writers are fluent.

SECTION 4

ENGLISH AS A SECOND LANGUAGE

Cynthia Watson

Cynthia Watson has developed a video assessment for her Beginning ESL class, which focuses on the Communication Core Skills of Speaking and Listening (ALP website). What follow are (1) the assessment benchmarks for student video presentations at four proficiency levels, and (2) the form she and her students use when they rate videotaped presentations. This form closely (but not exactly) follows a primary trait rating scale design. At the beginning of the semester, it is used for feedback only; at semester's end it's used both for feedback and to assign a grade.

In Beginning ESL, Watson videotapes students twice--at the beginning and end of the semester. She then tracks these students' developing speaking proficiencies by videotaping them at the end of each semester as they move upward through the levels of the ESL program.

(Core Skill: Communication--Speaking Ability)

VIDEO TAPE BENCHMARKS FOR FOUR LEVELS

LEVEL 1

Criterion Possible Points

1. Time (.5 minute maximum) 0-1

2. Pronunciation, Fluency, Understandability 0-5

3. Physical Characteristics: 0-5

a. eye contact (with camera lens)

b. facial expression

c. body language

4. Grammar 0-5

5. Opening & Closing 0-2

6. Required Topics: 0-8

a. name (first and last)

b. home country

c. date of birth

d. age

e. marital status

f. number of children

g. occupation

h. plans for the future _____

Total Possible Points 26

LEVEL 2

Criterion Possible Points

1. Time (1 minute maximum) 0-1

2, 3, 4, & 5 Same as Level 1 0-17

6. Required Topics: 0-9

a. name (first and last)

b. home country

c. marital status

d. number of children

e. how long in the U.S. and California

f. education in home country and U.S.

g.work experience in home country and U.S.

h. present occupation

i. plans for the future ______

Total Possible Points 27

LEVEL 3

Criterion Possible Points

1. Time (1.5 minute maximum) 0-1

2, 3, 4, & 5 Same as Levels 1 & 2 0-17

6. Required Topics: 0-10

a. name (first and last)

b. home country

c. marital status

d. number of children

e. how long in the U.S. and California

f. education in home country and U.S.

g.work experience in home country and U.S

h. present occupation

i. Show and explain one item that represents one of the speaker's

j. favorite activities.

k. plans for the future ______

Total Possible Points 28

LEVEL 4

Criterion Possible Points

1. Time (2.5 minute maximum) 0-1

2, 3, 4, & 5 Same as Levels 1, 2, & 3 0-17

6. Required Topics: 0-10

a. name (first and last)

b. home country

c. how long in the U.S. and California

d. marital status

e. number of children

f. education in home country and U.S.

g. work experience in home country and U.S.

h. present occupation

i. Show and explain one item that represents a significant

event in the speaker's life

j. plans for the future ______

Total Possible Points 28

VIDEO ASSESSMENT FORM Speaker____________________ Level________ Date__________ Viewer_____________________ Semester________
TIME _____	WORDS pronunciation fluency understanability	PHYSICAL eyes face body		GRAMMAR		RHETORIC opening & closing remarks	TOPICS __name __home __d. o. b. __age __marital __childr __occup __plans
Circle the appropriate numbers in the 6 category rating boxes below,
1 right length 0 too short OR too long	5 4 3 2 1 0		5 4 3 2 1 0		5 4 3 2 1 0	2 open & close 1 open OR close 0		8 7 6 5 4 3 2 1 0
COMMENTS	COMMENTS		COMMENTS		COMMENTS	COMMENTS		COMMENTS
								TOTAL [perfect = 26]