Summative Assessment of Problemsolving and Skills
Outcomes
PART I: Problemsolving
Problemsolving Outcome: Students will use
mathematical reasoning to solve problems and a
generalized problem solving process to work word problems.
Sampling design: From nine Math 25 sections taught
by instructors participating in the Teaching
Community, we chose a random sample of three students from each section. The
sample was taken
from students who were passing the course prior to taking the final exam. The
sample contained 18
students. (Two of the eight instructors did not submit student work, so the
sample did not contain
students from three sections.)
Method: We wrote four problems that met the
criteria for problemsolving in the DE Problemsolving
Outcome. The four problems required students to apply concepts in a context,
demonstrate conceptual
understanding, explain their reasoning, and use multiple approaches. The four
problems were also
designed so that multiple strategies could lead to a solution; in this way, the
problems were more than
standard word problems. Every instructor put these four problems on his/her
final exam. (#2, 3, 7, and
17 from the common final exam)
Technique: All four problems from each student
paper were assessed holistically using a rubric
written by the Teaching Community earlier in the semester. We began with a
benchmarking exercise in
which each instructor graded the same three papers. We then discussed the scores
on these three papers
and reached consensus. Next, each final was assessed independently by three
instructors. If at least one
of the three scores differed from the mean of the three scores by more than 0.5,
that student’s work was
discussed by the whole group until consensus was reached. Otherwise, scores were
averaged.
Summary: See rubric for description of scores
Stemplot of rubric scores
2 5 represents an average score of 2.5
Mean 3.6 Standard deviation .8
Quartiles: 2.5 3 3.5 4 5
Student Profile Based On The Mean Rubric Score:
A mean score of 3.6 indicates that the average student in this sample can
complete some important
components of the problemsolving task and communicate with sufficient clarity
to get his/her idea
across. Portions of work may be unorganized or difficult to follow. Some
strategies may be unclear or
inappropriate. At times, mathematical reasoning is not clear. There is evidence
of applying relevant
concepts and appropriate processes throughout, though implementation is only
partially successful in
some cases.
Observations: Instructors noted that nearly all
students in the sample attempted every problem.
Uniformly, the students did a good job defining variables and using multiple
approaches when
prompted. Very few papers explicitly demonstrated more sophisticated use of a
general problem solving
process (e.g. stating givens and assumptions, giving estimates, checking).
Generalizing other
weaknesses was more difficult. Instructors noted that frequently a student
performed inconsistently on
the four problems, demonstrating good understanding in one setting and having
difficulty with similar
concepts in another setting
Action: Instructors in the Teaching Community will
focus more on problemsolving in fall 2003. The
use of computeraided instruction to teach procedural algebra skills will
provide more class time for
instructors to work on problemsolving and the other DE program learning
outcomes. During the
summer session, eight instructors met for 3hours a week to write and discuss
classroom activities that
focus on conceptual development of algebra content and problemsolving. We will
see if this increased
classroom focus on developing problemsolving skills improves student
performance.
PART II: Skills Outcome
Skills Outcome: Students will recognize and apply
math concepts in a variety of relevant settings and
demonstrate the math skills and knowledge necessary to succeed in subsequent
courses.
Sampling design: From nine Math 25 sections taught
by instructors participating in the Teaching
Community, we used all students who were passing the course prior to taking the
final exam. The
sample contained 136 students from seven sections; 122 took the same final exam.
(One instructor who
taught two sections did not submit data.)
Method: We identified six content objectives
fundamental to Elementary Algebra. Using the common
final exam, we ranked problems (referenced in the table) by skill into three
levels of difficulty. One
instructor gave a different final exam but ranked his problems similarly.
Technique: Instructors tallied the number of
students who made minor errors and the number who got
the problem completely correct.
Level of Difficulty
Skill 
Low 
Medium 
High 
Minor
errors 
Correct 
Minor
errors 
Correct 
Minor
errors 
Correct 
Solve linear equations
Final exam: low=1a, med=1b, high=1f 
16/122
13% 
102/122
84% 
24/122
20% 
86/122
70% 
35/136
26% 
61/136
45% 
Find the equation of a line
Final exam: low=18, med=2, high=7 
8/122
7% 
90/122
74% 
33/136
24% 
76/136
56% 
44/136
32% 
62/136
46% 
Solve quadratic equations
Final exam: low=9, med=1e, high=1g 
31/122
25% 
71/122
58% 
30/122
25% 
67/122
55% 
43/136
32% 
68/136
50% 
Solve a linear inequality
Final exam: med=1h 
XXX 
XXX 
37/122
30% 
59/122
48% 
XXX 
XXX 
Simplify polynomials: add/subtract
Final exam: low=11, med=12 
12/136
9% 
106/136
78% 
40/136
29% 
82/136
60% 
XXX
3/14
21% 
XXX
11/14
79% 
Laws of Exponents
Final exam: low=15a, med=15b 
27/122
22% 
57/122
47% 
35/122
29% 
49/122
40% 
XXX
4/14
29% 
XXX
10/14
71% 
Observations: In all skills at all levels of
difficulty, over 69% of the students could either perform the
skill perfectly or with only minor errors. This sample of students had strengths
in solving linear
equations (without fractions), finding the equation of a line from two points,
and simplifying
polynomials. If we demand complete accuracy, we see weaknesses in solving linear
equations with
fractions, finding equations of lines from verbal descriptions, solving linear
inequalities, and using the
laws of exponents to simplify expressions. In all other skills at all levels of
difficulty at least 50% of
the students performed the skill without errors.
Action: In an attempt to improve students’
procedural algebra skills, the Teaching Community is
experimenting with computeraided instruction in fall 2003. With the PHIM2
software, students will
be able to choose from a variety of instructional methods, receive immediate
feedback, and work
problems until they reach mastery (defined as 85% correct in a given problem
set). We will see if
computeraided instruction improves students’ skills.
