ten truths about learning assessment

10 truths about learning assessment, courtesy of 13 truths about social media measurement. The parallels are striking.

1. Measurement is not the goal
   

   Neither is third-party validation or accountability. Systematic improvement of teaching and learning is the goal.
   

2. Measuring activity isn’t as important as measuring results
   

   This one I think we've pretty much taken to heart. Outputs not inputs.

3. Metrics are determined by goals

   Determine what you measure based on what you want to accomplish. Some data are more useful for accountability, others for improvement. Consider how actionable and to what ends you can expect it to be before you decide on a measure.

4. Cause and Correlation are different
   

   Correlations are easy to find. Causes aren't.

5. Analysis is at least as hard as the measurement itself
   

   Understanding what the data mean and what to do about it is not only the end goal, it's also the hard part.

6. Standardization has marked limitations
   

   Authentic data useful for improving teaching & learning does not lend itself to accountability or inter-institutional comparisons. Peter Ewell is right: accountability is incongruent with improvement.
   

7. Reporting is not an outcome
   

   Doing something about what you report is.

8. Measurement is in constant evolution
   

   We don't yet have the measurement of learning figured out by any stretch. We are making this up as we go along and probably will for some time yet.
   

9. Measurement is cultural as well as operational
   

   If your culture is such that failure is feared and avoided at all costs, you are going to have problems that flawless execution will not preclude.

10. “Learning isn’t measurable” is an excuse
    

    It really means:
    • I don't know how to operationalize / measure / analyze / interpret
      
    • I don't want to have to / can't afford to spend the time and effort needed
      
    • I’m afraid of what measuring will actually tell me about the effectiveness of my work

Rethinking Education in the Age of Technology

I was excited to read what I hope will be a levelheaded and open-minded account of education in the digital age. I'm more hopeful still seeing a forward (perfunctory, alas, but nevertheless) from John Seely Brown and a blurb from Don Norman. Chapters like "What May be Lost and what May be Gained" seem promising as well.

But I couldn't make it through the preface before getting skeptical again:

"If educators cannot successfully integrate new technologies into what it means to be a school, then the long identification of schooling with education...will dissolve into a world where students with the means and ability will pursue their learning outside of public school."

Students of means and ability will always and forever supplement their learning outside of public school, of course, and I hardly think anyone would see that as problematic, except that a student's means should be much less an issue than it is currently. And indeed, technology is making it easier, more interesting, more social and less expensive to pursue learning informally and independently. I'm right beside the authors in celebrating that fact. What an age we live in when it comes to opportunities to learn, truly.

But if the authors mean to imply that a failure to integrate new technologies somehow threatens to preclude formal schooling from being a meaningful source of education, well, both my FUD and my bullshit detector start going off. I'm barely past the preface, so maybe they don't, but boy, rarely a day goes by that someone doesn't make that facile claim: if higher ed doesn't embrace web 2.0 it will cease to be relevant, to exist in our lifetime. Clotted nonsense. More to come on Rethinking.

The downside of grading rubrics

Students deserve to know and benefit from knowing in explicit detail exactly what is expected of them and what is important. A grading rubric can do exactly that. Unfortunately, a grading rubric can also turn grading into a hollow, unreflective checklist, and assignment preparation into a mechanical compliance effort. Instead of focusing on insight or creativity, one must focus on covering one's bases vis-a-vis the rubric. And -- a key point -- often one needn't worry about anything but covering one's bases. You needn't do anything particularly well, you needn't shed much by way of insight -- just give them what the rubric asked for.

Awarding 100 percent on an essay is preposterous on its face, but I'd wager it's quite common in an orthodox rubric-driven grading regime: 'What do you want me to do, he satisfied all of the items on the rubric?'

I'm moved to note this as i finish a final paper for an instructor who grades by a rigorous rubric. As I wrote, I found myself wishing I could focus on writing a good paper, a paper that was interesting and maybe even insightful, that moved me to take my knowledge of the subject further. Instead, as a rational, satisficing sort would, I instead worked to make sure I would get credit for each item on the rubric checklist. Easier, no question, but dispiritingly hollow too.