Just before Christmas, Megan McArdle posted interesting article the Atlantic blog site (here) explaining why “promising pilot projects often don’t scale “. She argues that pilot programs often “don’t scale for corporations, and they don’t scale for government agencies.” She continues that “even when you put super smart people with expert credentials in charge of them” they don’t scale. And “even when you make sure to provide ample budget resources,” success is not guaranteed.
She then goes on to offer more specific reason as to why “Rolling something out across an existing system is substantially different from even a well run test, and often, it simply doesn’t translate.”
- Sometimes the “success” of the earlier project was simply a result of random chance, or what researchers call the Hawthorne Effect. The effect is named after a factory outside of Chicago which ran tests to see whether workers were more productive at higher or lower levels of light. When researchers raised the lights, productivity went up. When researchers lowered the lights, productivity also went up. Obviously, it wasn’t the light that boosted productivity, but something else–the change from the ordinary, or the mere act of being studied.
- Sometimes the success was due to what you might call a “hidden parameter”, something that researchers don’t realize is affecting their test. Remember the New Coke debacle? That was not a hasty, ill-thought out decision by managers who didn’t care about their brand. They did the largest market research study in history, and repeated it several times, before they made the switch. People invariably told researchers they loved the stuff. And they did, in the taste test. But they didn’t love the stuff when it cost them the option of drinking old Coke.
- Sometimes the success was due to the high quality, fully committed staff … and [doesn’t work when we are] recruiting from a pool that included folks who were just looking for a job, not a life’s mission to … while adding to the sum of human knowledge.
- Sometimes the program becomes unmanageable as it gets larger.
- Sometimes the results are survivor bias. This is an especially big problem with studying health care, and the poor. Health care, because compliance rates are quite low (by one estimate I heard, something like 3/4 of the blood pressure medication prescribed is not being taken 9 months in) and the poor, because their lives are chaotic and they tend to move around a lot, so they may have to drop out, or may not be easy to find and re-enroll if they stop coming. In the end, you’ve got a study of unusually compliant and stable people (who may be different in all sorts of ways) and oops! that’s not what the general population looks like.
So what does this mean for our life in Christ? Simply this, while the social and human sciences can—and do—offer us a great deal of potential insight into the spiritual life, they are only useful if we are aware of the limitations of empirical research. The fact of the matter is that facts don’t exist in a vacuum. The facts are known only within the context of a theory, a vision of what it means to be human for example, and it is that theory is often overlooked.
One reason we jump to the “facts” at the expense of theory is because are looking for something, anything, that just works. This is especially the case when the stakes are high and I am being driven by fear.
But the right use of the social and human science requires that we be guided by a sound anthropology rooted not in the ephemera of empirical research but the catholic vision of the Christian tradition. With that in mind, let’s look quickly at the 5 points above and see what they tell us about the Christian life.
- The Hawthorne Effect suggests that we are never our best selves save when we are with others. We are social or communal beings. If you prefer, to be human means to be-in-communion.
- Even in its most prosaic details, human life and our motivations are mysterious. Human life in its fullness doesn’t simply resist explanation it defies it.
- That said, commitment—that is to say faith and obedience—open up for us the avenues of understanding that are closed to a merely pragmatic or reductionistic view of human life.
- The failure of projects to scale up suggests that the less personal my involvement, the less likely is my fidelity to the project. Success requires a human face, a human connection, and not simply an abstract statement of truth.
- Life triumphs over death but for life to triumph for me requires ascetical self-discipline. This means that both in our in the spiritual life and our everuday life, success is a matter of consistency over the long term.
But if we take points 1-4 into account, such consistency over the long term is impossible if we are not personally engaged by those who embody for us the values and virtues that are central to our Christian life. We need more than a good theory of human life, we need good people in our lives to support and guide us.
I would suggest that whether in the spiritual life or in our personal relationships or our work lives, any success we have will include some combination of these points. The similarity of success in different arenas of life is what helps us live lives that are integrated and well balanced. Additionally, just as we can’t ignore our spiritual lives in favor of our work lives we also can’t ignore the world of work in pursuit of a life of prayer. Human life is a whole and while the exact mix of work and prayer, for example, will be different for each person (and different at different points in her life), the elements out of which a successful life is formed will remain constant.