ten truths about learning assessment

10 truths about learning assessment, courtesy of 13 truths about social media measurement. The parallels are striking.

1. Measurement is not the goal
   

   Neither is third-party validation or accountability. Systematic improvement of teaching and learning is the goal.
   

2. Measuring activity isn’t as important as measuring results
   

   This one I think we've pretty much taken to heart. Outputs not inputs.

3. Metrics are determined by goals

   Determine what you measure based on what you want to accomplish. Some data are more useful for accountability, others for improvement. Consider how actionable and to what ends you can expect it to be before you decide on a measure.

4. Cause and Correlation are different
   

   Correlations are easy to find. Causes aren't.

5. Analysis is at least as hard as the measurement itself
   

   Understanding what the data mean and what to do about it is not only the end goal, it's also the hard part.

6. Standardization has marked limitations
   

   Authentic data useful for improving teaching & learning does not lend itself to accountability or inter-institutional comparisons. Peter Ewell is right: accountability is incongruent with improvement.
   

7. Reporting is not an outcome
   

   Doing something about what you report is.

8. Measurement is in constant evolution
   

   We don't yet have the measurement of learning figured out by any stretch. We are making this up as we go along and probably will for some time yet.
   

9. Measurement is cultural as well as operational
   

   If your culture is such that failure is feared and avoided at all costs, you are going to have problems that flawless execution will not preclude.

10. “Learning isn’t measurable” is an excuse
    

    It really means:
    • I don't know how to operationalize / measure / analyze / interpret
      
    • I don't want to have to / can't afford to spend the time and effort needed
      
    • I’m afraid of what measuring will actually tell me about the effectiveness of my work

Rethinking Education in the Age of Technology

I was excited to read what I hope will be a levelheaded and open-minded account of education in the digital age. I'm more hopeful still seeing a forward (perfunctory, alas, but nevertheless) from John Seely Brown and a blurb from Don Norman. Chapters like "What May be Lost and what May be Gained" seem promising as well.

But I couldn't make it through the preface before getting skeptical again:

"If educators cannot successfully integrate new technologies into what it means to be a school, then the long identification of schooling with education...will dissolve into a world where students with the means and ability will pursue their learning outside of public school."

Students of means and ability will always and forever supplement their learning outside of public school, of course, and I hardly think anyone would see that as problematic, except that a student's means should be much less an issue than it is currently. And indeed, technology is making it easier, more interesting, more social and less expensive to pursue learning informally and independently. I'm right beside the authors in celebrating that fact. What an age we live in when it comes to opportunities to learn, truly.

But if the authors mean to imply that a failure to integrate new technologies somehow threatens to preclude formal schooling from being a meaningful source of education, well, both my FUD and my bullshit detector start going off. I'm barely past the preface, so maybe they don't, but boy, rarely a day goes by that someone doesn't make that facile claim: if higher ed doesn't embrace web 2.0 it will cease to be relevant, to exist in our lifetime. Clotted nonsense. More to come on Rethinking.

The downside of grading rubrics

Students deserve to know and benefit from knowing in explicit detail exactly what is expected of them and what is important. A grading rubric can do exactly that. Unfortunately, a grading rubric can also turn grading into a hollow, unreflective checklist, and assignment preparation into a mechanical compliance effort. Instead of focusing on insight or creativity, one must focus on covering one's bases vis-a-vis the rubric. And -- a key point -- often one needn't worry about anything but covering one's bases. You needn't do anything particularly well, you needn't shed much by way of insight -- just give them what the rubric asked for.

Awarding 100 percent on an essay is preposterous on its face, but I'd wager it's quite common in an orthodox rubric-driven grading regime: 'What do you want me to do, he satisfied all of the items on the rubric?'

I'm moved to note this as i finish a final paper for an instructor who grades by a rigorous rubric. As I wrote, I found myself wishing I could focus on writing a good paper, a paper that was interesting and maybe even insightful, that moved me to take my knowledge of the subject further. Instead, as a rational, satisficing sort would, I instead worked to make sure I would get credit for each item on the rubric checklist. Easier, no question, but dispiritingly hollow too.

Apple's iPad: More than just poorly named / bright & shiny?

So Apple's 'iPad' tablet device, as anyone with a pulse and a computer is well aware at this point, was officially released yesterday to enormous anticipation and fanfare, even for Apple. There's no question that this iPhone-on-steroids is a Bright Shiny Object (BSO) of the highest order, that the cool factor is high, and that the urge to fondle one when you see those introductory videos is high, but after the oohs and aahs recede, is there really any there there?

The new screen is impressive on first glance - big and bright - and some of the software enhancements look impressive, like the email client for example, but when I ask myself what's transformative and awesome about the iPhone for me personally, I come p with the following: an all-in-one, everything-but-the-kitchen-sink device with fantastic usability and that great multi-touch interface that I can slip in my pocket and thus have with me whenever, wherever.

And what's the iPad? (Other than really, really badly named, that is -- couldn't the Apple gang anticipate the off-color YouTube parodies?) Exactly the same thing only in a form factor that WON'T fit in your pocket and thus isn't likely to be with you whenever, wherever. Cool - yes, but useful? I'm not convinced.

Now, you might agree but counter that it's certain to be an Amazon Kindle killer at a minimum. I mean, it's gorgeous, has that nice color screen and is reasonably priced (at least compared to the preposterously expensive Kindle), right? Good hypothesis, but not so fast. The iPad uses a simple LED-backlit Liquid Crystal Diode display, which is great for general use but isn't likely to cut it as an e-reader. Bright sunlight is still a killer and eyestrain is still a factor. Kindle may not be as sexy but if you need to read War & Peace on either a Kindle or an iPad, trust me, go with the Kindle. An don't take my word for it. The IEEE gives the real lowdown: http://spectrum.ieee.org/tech-talk/computing/hardware/the-ipad-is-not-a-kindle-killer-blame-the-display and trust me, they know of what they speak when it comes to display hardware.

Bottom line: iPad's cool, it's fun and if you get one I'm definitely going to be jealous, but don't expect it to transform the world of computing or e-reading.

Best data visualizations I believe I've ever seen

instructional technology makes strange bedfellows: hacking education or flash frying it

Technology can be used to 'hack' education -- to make it better and more authentic -- or, just as easily, to fast-food-ify it -- to make it cheaper, more convenient, more profitable and much less healthy. Let's be clear about these opposing objectives.

Saw this by way of the always interesting Alex Reid:



There seem to be two camps when it comes to enthusiasm for educational technology. Those who get excited (whether they admit it per se or not, and it seems they rarely do) by technology's Taylorizing potential for productivity, efficiency, scale, precision, and those whose enthusiasm stems from entirely different and maybe contradictory places: the potential for better, more authentic, socially constructed, learner-centric learning.

It's interesting that the two camps are coming from entirely different places and have what I suspect are to a large extent never-the-twain-shall-meet mutually exclusive aims. The ed.-tech-in-the-large camp (those concerned with productivity and the like) does not typically embrace the same causes or travel in the same circles as the ed.-tech-in-the-small gang. They do not typically see eye-to-eye. Yet somehow when it comes to web-facilitated education, the objectives of the two camps are often conflated. Administrators and policy makers concerned with costs, those inclined to decry education as "the last cottage industry" delight in the potential to leverage technology to "transmit" education more efficiently and manage it more "scientifically". Drexel's President Papadakis explains it with characteristic unabashedness:

For Dr. Papadakis, the full-timer issue poses a dilemma, since one of his bedrock ideas is to encourage the use of Internet-based courses that can be taught by inexpensive part-time or non-tenure-track teachers. As he envisions it, experienced Drexel professors will create digital courses containing computerized coursework accessible via the Internet. They’ll be offered on campus but can also be used to teach “distance learning” students who don’t take part in face-to-face instruction in Philadelphia. For the most part, these courses can be taught by junior or part-time faculty.

“Technicians can teach them” at lower cost, says Dr. Papadakis, quickly adding that he is exaggerating when he uses the word “technicians.” (Whole article is available here)

This teachers replaced by technicians nightmare scenario is *surely* not the perspective of the Computer Supported Collaborative Learning folks or the constructivists, or lord knows the EduPunks (although they at least know full well that their interests are not coincident with those of the in-the-large gang) or anyone, I wouldn't think, inspired by the educational potential of Web 2.0.

And the conflation is clear too in the video above. They start out with the in-the-large premise that education is in need of disruption because it's getting more and more expensive at a annual rate of 8-10% or so. Fair enough (and scary enough). Then we launch into Open Education / OCWC boosterism, and while I love the open education movement and its potential to transform informal learning, I'm sorry, but open education is entirely orthogonal to the issue of skyrocketing college costs (making informal education rich, robust, easy and free isn't going to change the cost structure of formal education, even in the medium term; credentialling matters, the social life of information matters). Then we get into the in-the-small pleasing issues of peer-to-peer learning and collaborative note taking and the idea of providing a structure for educational market making, for matching teacher to learner, all of which is exciting and inspiring, none of which is going to make college any cheaper.

Higher ed administrators, faculty and technologists alike need to recognize, acknowledge and talk about these dual objectives and seek to unify them to whatever extent possible or at least approach their mutual exclusivity as a problem of constrained optimization. And while we're at it, let's think harder and better about the future of and relationship between formal and informal education. (I'm looking in your direction, open education people.)

signs of the impending gynarchy: the growing college gender gap

Anticipating a coming gynarchy may be a bit premature (and naive, I know, as there are structural impediments - producing way more female PhDs hasn't led to a female-majority tenured professoriate by any means). I for one welcome our distaff overlords (overladies?); the world would likely be a better place.

Nevertheless the following stats reported in my Current Issues in Education class at Temple are cause for real concern:
Women are going to college at starkly higher rates than men and the disparity continues to grow. The question put to the class was whether we should care, and the consensus, unsurprisingly but disappointingly, was no. Issues of equality are admittedly prickly; we surely should not seek to right this circumstance in justice's name. But the issue is stark in both human capital and "civic capital" terms. In this the information age, as should be particularly clear in such bleak economic times, we owe it to ourselves and our children to maximize our collective stock of knowledge and skills. It's the sole path to modern prosperity. In the small and in the short run, there are alternatives, and I've argued that many folks are pushed in the baccalaureate degree's direction who'd be better served by different pursuits. Those arguments notwithstanding, in total and in the long run, we need as many folks as can stand to benefit to study and to get degrees. And if men as a class appear to be losing interest in college, we are obliged to understand why and to remedy to the extent possible. I know men are a minority, but it's a really substantial minority. Irrespective of your particular naughty bits, we need you to be studying if you are able to benefit from doing so.

And for those who are unmoved by the human capital argument (you know who you are, and I don't like you particularly, by the way), how about the fact that education and civic participation are strongly correlated as well. If prosperity's not your thing, perhaps participative democracy is.

One caveat: With regard to civic participation and to a lesser extent even human capital, not all of the abovementioned correlation is causation: The better educated are more likely to vote not strictly as a result of their education. Undoubtedly to some extent those most likely to seek education are also those ex ante most likely to vote. Still, there's absolutely GOT to be some causation in that correlation, and we need, in unprecedented ways, to wring it out.