Formative Versus Summative Assessment

Purpose, not instrument

Michael Scriven introduced the pair of terms in 1967, and not about students at all. He was writing about how to evaluate curricula, and he separated two roles evaluation might play: one in the ongoing improvement of a curriculum while it was still being developed, and one that let administrators decide whether the finished curriculum represented a sufficient advance on the available alternatives to justify adopting it. He proposed calling those roles formative and summative.

Two years later Benjamin Bloom moved the distinction onto student learning. He acknowledged the traditional role of tests in judging and classifying students, and set against it a use of formative evaluation that provides feedback and correctives at each stage of the teaching-learning process - brief tests used by teachers and students as aids to learning. Bloom then added the sentence everything downstream depends on: such tests may be graded and used in the judging function, but formative evaluation is much more effective when it is separated from grading and used primarily as an aid to teaching.

Read those origins carefully and the most common student error dissolves. Neither Scriven nor Bloom made formative a property of an instrument. A test is not formative; a use is. Dylan Wiliam states the criterion sharply: assessments are formative if and only if something is contingent on their outcome and the information is actually used to alter what would have happened without it. An exit ticket nobody reads is summative in effect - a record and nothing more. A state accountability test whose results genuinely reshape next summer's teacher workshops has been used formatively, however high its stakes were for the students who sat it.

So the distinction is not about timing, format, formality, or low stakes. Wiliam describes a continuum of cycle lengths instead: short-cycle within a single lesson, five seconds to an hour; medium-cycle between lessons, a day to two weeks; long-cycle between instructional units, four weeks to a year or more. What makes any of them formative is not the length of the loop, where it happens, who runs it, or even who responds. It is that evidence is evoked, interpreted in terms of learning needs, and used to adjust.

For learning, as learning, of learning

A second vocabulary sits alongside the first and is often mistaken for a synonym. Writing for Manitoba Education and the Western and Northern Canadian Protocol, Lorna Earl and Steven Katz set out three purposes of classroom assessment.

Assessment for learning gives teachers information they use to modify and differentiate teaching. It asks not only what students know but how, when and whether they apply it, and it feeds both instructional adjustment and feedback to students.

Assessment as learning puts the student in the driver's seat. Its object is metacognition: students monitoring their own understanding, judging their work against criteria, and adjusting their strategies. Earl and Katz note that systematic assessment as learning has historically been rare, and it is the category most schools underuse. The 2018 revision of the Council of Chief State School Officers' definition makes the same move, extending the goal of the process to supporting students to become self-directed learners.

Assessment of learning is summative: it confirms what students know and can do and produces statements of proficiency that other people - the next teacher, an admissions office, a licensing board - will rely on.

Two cautions. Some writers use only two categories and fold assessment as learning into assessment for learning, so the three-way split is a convention, not a law. More importantly, Earl and Katz argue that it is very difficult and sometimes impossible to serve three assessment purposes at once. An assessment designed to let students expose their confusion without penalty is not the same artifact as one designed to survive scrutiny as a proficiency claim, and trying to do both usually means doing the first one badly.

The neighboring labels

The FAST SCASS collaborative of state assessment directors published a paper for the express purpose of untangling the labels, and each one names something real.

Summative assessment provides information about the level of student, school or program success at an end point in time, after instruction concludes. Its results serve evaluative judgment, inferences about mastery, grades, and accountability requirements.

Interim assessment - also called benchmark, common, or quarterly assessment - covers tests administered periodically through the year to serve some mix of three functions: predicting performance on a later high-stakes test, evaluating programs, and supplying teachers with individual student data. Many such products were marketed as being formative assessments in themselves. FAST SCASS quotes Margaret Heritage's diagnosis that the core problem is the false but widespread assumption that formative assessment is a particular kind of measurement instrument rather than a process. Their own verdict is blunter: interim tests give periodic snapshots, but because they are not proximate to learning as it develops they do not inform ongoing teaching, and at the time of writing there was no meaningful evidence that they enhance student learning.

Diagnostic assessment is a procedure detailed enough to indicate which specific subskills or pieces of enabling knowledge a student does and does not possess. It answers what is not being learned, rather than whether learning is happening. Because it is time-intensive it is normally reserved for students who are not progressing, and it feeds the formative process when results arrive fast enough to act on.

Three further labels round out the set. Curriculum-embedded assessment is built into the materials or activities, so students are never told to stop and take a test. Universal screening is a brief measure given to everyone two or three times a year to flag students who may be at risk. Progress monitoring is a short measure repeated weekly or biweekly to track growth, usually with alternate forms.

Notice that every one of these is defined by an instrument, a population, or a schedule - not by a purpose. That is why each can be used formatively or summatively, and why the purpose test still has to be applied on top of the label. FAST SCASS makes the point about curriculum-embedded work explicitly: it is more likely to be formative when it is not graded, and once it enters the grading process the only thing distinguishing it from any other summative assessment is how smoothly it was blended into the lesson.

Feedback is not automatically good

The engine of formative assessment is supposed to be feedback, so it matters that feedback is not a reliably positive intervention. Avraham Kluger and Angelo DeNisi's 1996 meta-analysis pooled 607 effect sizes from 23,663 observations. The average effect was positive and moderate, d = .41. But over one third of the feedback interventions decreased performance - a substantial minority of cases in which telling people how they were doing made them worse.

Their explanation, Feedback Intervention Theory, is that feedback works by directing attention, and that its effectiveness falls as attention moves up a hierarchy from the task toward the self. Feedback that says this paragraph has no evidence in it points at the work. Feedback that says you are a careless writer, or simply 71 percent, points at the person. The second kind recruits ego defense, and ego defense does not revise paragraphs.

This is the mechanism behind the oldest practical rule in the field. Bloom said in 1969 that formative evaluation is much more effective when separated from grading. Ruth Butler tested it in 1988, giving fifth- and sixth-grade classes numerical grades, individual comments, or both, and framing the contrast as task-involving versus ego-involving evaluation. The reading that has stuck in teaching-center guidance is that adding a score to a comment removes the benefit of the comment. Black and Wiliam reached the same conclusion from the wider literature: pupils given only marks or grades do not benefit from the feedback, which improves learning when it gives specific guidance on strengths and weaknesses, preferably without any overall marks.

The operational consequence is simple. If you want a comment to do work, protect it from the grade - withhold the score until the student has responded, require a revision before releasing it, or do not score the piece at all. Writing both on the same page and hoping the student reads the left column is, on this evidence, wishful.

What good formative practice actually contains

The 2018 FAST SCASS definition calls formative assessment a planned, ongoing process used by all students and teachers during learning and teaching to elicit and use evidence of student learning, improve understanding of intended disciplinary learning outcomes, and support students in becoming self-directed learners. It names five practices that must be integrated: clarifying learning goals and success criteria within a broader progression of learning; eliciting and analyzing evidence of student thinking; engaging in self-assessment and peer feedback; providing actionable feedback; and using that evidence and feedback to move learning forward.

Strip that to the three parts that fail most often.

First, success criteria stated in advance. Students cannot judge their own work against a standard they have not been shown, and a teacher who has not written the criterion down will grade by impression. It is also what separates feedback from opinion.

Second, evidence from every student. A question answered by the three students who raise their hands tells you about three students. Wiliam's illustration is a multiple-choice question about thesis statements where every student holds up a lettered card, so the teacher reads the whole room at once and knows who chose what. He calls the resulting instant a moment of contingency: the point at which instruction can actually change direction. Contrast it with the class discussion that feels productive and yields no readable evidence about anyone.

Third, a decision that follows. This is what the definition is really about, and the part routinely skipped. Evidence that produces no change in what happens next was not formative, however carefully it was gathered.

The word planned in the 2018 definition is doing deliberate work here. FAST SCASS argues that a teacher who appears to be adjusting on the fly is often enacting one of several plans prepared beforehand - anticipating the likely misconceptions, writing the question that will surface them, and deciding in advance what each pattern of answers will trigger. Improvisation of that quality is rehearsed.

How strong is the evidence?

You will meet a number attached to this topic, and you should handle it carefully. In their 1998 Phi Delta Kappan article Inside the Black Box, Paul Black and Dylan Wiliam wrote that typical effect sizes of the formative assessment experiments were between 0.4 and 0.7, larger than most found for educational interventions, and that improved formative assessment helps low achievers most. That sentence became the most-quoted claim in classroom assessment, repeated in textbooks and district slide decks as though it were a measured constant.

It is not one. Neal Kingston and Brooke Nash reviewed more than 300 K-12 studies of formative assessment and found only 13 reporting enough information to compute an effect size. From the 42 independent effect sizes those studies yielded, a random-effects model gave a weighted mean of 0.20, with estimates of 0.32 in English language arts, 0.17 in mathematics and 0.09 in science. Their conclusion was that the commonly claimed figure is not supported by the existing research base, and that the base itself is too thin.

Randy Bennett's critical review attacks the object rather than the estimate. He concluded that formative assessment does not yet name a well-defined set of artifacts or practices, that the breadth of the definitions in use permits implementations with wildly different outcomes, and that the magnitude of the common quantitative claims is suspect because it traces to sources that are untraceable, flawed, dated or unpublished. If the treatment is not well defined, an average effect size for it means little.

The original authors have qualified their own case. Writing in 2006, Wiliam said his own reading of the research suggested medium-cycle formative assessment had shown only modest impact, and that the studies which did show impact tended to be those that changed teachers' day-to-day and minute-to-minute practice. Federal guidance is similarly restrained: the 2009 IES practice guide on using student achievement data assigns a Low level of evidence to all five of its recommendations. Low, the panel is careful to say, means it did not identify a body of research demonstrating effects on achievement - not that the practice is unimportant.

So hold this position. The mechanism is well motivated: eliciting evidence, interpreting it, and acting on it is how teaching becomes responsive rather than scheduled. The size of the effect is unsettled, depends heavily on what was implemented, and should not be quoted as a fixed number.