Success and the Spiritual Life

Just before Christmas, Megan McArdle posted interesting article the Atlantic blog site (here) explaining why  “promising pilot projects often don’t scale “.  She argues that pilot programs often “don’t scale for corporations, and they don’t scale for government agencies.” She continues that   “even when you put super smart people with expert credentials in charge of them” they don’t scale.  And “even when you make sure to provide ample budget resources,” success is not guaranteed.

She then goes on to offer more specific reason as to why “Rolling something out across an existing system is substantially different from even a well run test, and often, it simply doesn’t translate.”

  1. Sometimes the “success” of the earlier project was simply a result of random chance, or what researchers call the Hawthorne Effect.  The effect is named after a factory outside of Chicago which ran tests to see whether workers were more productive at higher or lower levels of light.  When researchers raised the lights, productivity went up.  When researchers lowered the lights, productivity also went up.  Obviously, it wasn’t the light that boosted productivity, but something else–the change from the ordinary, or the mere act of being studied.
  2. Sometimes the success was due to what you might call a “hidden parameter”, something that researchers don’t realize is affecting their test.   Remember the New Coke debacle?  That was not a hasty, ill-thought out decision by managers who didn’t care about their brand.  They did the largest market research study in history, and repeated it several times, before they made the switch.  People invariably told researchers they loved the stuff.  And they did, in the taste test.  But they didn’t love the stuff when it cost them the option of drinking old Coke.
  3. Sometimes the success was due to the high quality, fully committed staff … and [doesn’t work when we are] recruiting from a pool that included folks who were just looking for a job, not a life’s mission to … while adding to the sum of human knowledge.
  4. Sometimes the program becomes unmanageable as it gets larger.
  5. Sometimes the results are survivor bias.  This is an especially big problem with studying health care, and the poor. Health care, because compliance rates are quite low (by one estimate I heard, something like 3/4 of the blood pressure medication prescribed is not being taken 9 months in) and the poor, because their lives are chaotic and they tend to move around a lot, so they may have to drop out, or may not be easy to find and re-enroll if they stop coming.  In the end, you’ve got a study of unusually compliant and stable people (who may be different in all sorts of ways) and oops! that’s not what the general population looks like.

So what does this mean for our life in Christ? Continue reading

VN:F [1.9.22_1171]
Rating: 10.0/10 (2 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Teachers Don’t Like Creative Students — Marginal Revolution

Freedom is one of the central characteristics of holiness; creativity is another. God is called holy because He is not bond by anything external to Himself. Likewise, God’s holiness is seen in His ability to call into being something other than Himself. God creates by freeing us from non-existence.

In the classical Christian understanding of creation, not only does God remain free relative to His creation, He leaves His creation–primarily though not exclusively, humanity–free relative to Himself. Such mutual freedom is essential if we are to turn to God in love.  But I would suggest that not only freedom but also creativity is constitutive of love and therein lines a challenge.

A recent post at Marginal Revolution summarizes some interesting research that argues that classroom teachers generally are not all that supportive of creativity. Interestingly, while teachers say they value creativity in their students, when we actually look at how teachers describe the creative student we discover that they (the teachers) don’t actually value the personality characteristics associated with creativity. Why? Because creative students are disruptive.

I’ve reproduced the whole post after the break. Take a look and tell me what you think. I’d also be interested in what, if any, implications you think the research might have for our life in Christ.

In Christ,

+Fr Gregory Continue reading

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

The Human Limits of Empirical Research

The Human Limits of Empirical Research

Neuroskeptic has an interesting post on the “decline effect.”  In a nutshell, replication is a key to empirical research in both the natural and social sciences.  In the latter however what we are seeing is that often it is not possible for later researches to replicate the findings of earlier studies published in referred journal.  This leads to, for example, what appears to be the declining effectiveness of a psychiatric medication over time.  While early published studies show the drug to be effective, later studies don’t.

One suggested reason for this is that more often than not the studies that get published (at least initially) are the ones that show positive results.  What the research doesn’t submit or what doesn’t get published are studies without any results.  So, how does this happens and why does this matter?

As for the first

The problem is that there are so many ways to statistically analyze any given body of data that it’s easy to test and retest it until you find a “positive result” – and then publish that, without saying (or only saying in the small print) that your original tests all came out negative. Combine this with selective publication of only the best data, and other scientific sins, and you can pull positive results out the hat of mere random noise.

In other words, intentionally or not, the fix is in.  Contrary to the popular understanding, if empirical research is trustworthy at all, it is only so when it is the fruit of virtuous researchers and reviewers who are truthful about ALL their findings.

So why does this matter? Continue reading

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)