Comparative Effectiveness, cont.

The last post on comparative effectiveness was getting long, so I decided to make a separate point in this post.

Something that was implicit in that post is the notion that individuals and systems present different measurement challenges, in both healthcare and education.  Comparative effectiveness research in healthcare aims to improve both quality of care and spending across the board, but it also provides a useful mechanism for evaluating the effectiveness of individual actors in the system.  Both are important, but they shouldn’t be confused.

This is particularly important when you look at the level of process error inherent in any complex procedure.  The thing that immediately comes to mind in education is one of my favorite concepts: “fidelity of implementation.”  Anyone who has been on the procurement side of education reform knows that almost any product/program packaged as an “education solution” comes with two sets of data.  The set that always looks MUCH better is the set from instances in which there was high “fidelity of implementation.”  In the best cases, this actually serves as a check on programmatic quality, but too often it is a concept that masks the level of competency necessary from both the individuals and systems responsible for delivering these programs.

I was once at a round-table discussion where a researcher was arguing that we know as much as we need to know about reading interventions, because an intervention that he had evaluated had great results.  Another participant in that discussion, who worked with large urban school districts, said she had never seen the intervention work, because the training took too long, the procurement systems couldn’t get the intervention into schools, and there weren’t enough teachers with the right level of specialty expertise to deliver the program.  My subsequent observation was: “If this program is so great, why can’t we even implement it in the places where we need it most?”

My point is: the quality of interventions is important, but it’s dangerous to trivialize “fidelity of implementation” in systems that are dominated by human actors and troubled systems.

Advertisements

One Response to Comparative Effectiveness, cont.

  1. Kathleen Smith says:

    Somebody finally understands the black hole that has cost failing districts millions!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: