Clearing out cabinets with old files can be like a mystery adventure. Here's one mystery I came across this week.
In the bustling heyday of 1920s Chicago a revolutionary new theory of crime prevention emerged from the University of Chicago. Tapping into the flowering of biological ecology theory, sociologists Robert Park and Ernest Burgess crafted a social ecology of crime to explain and prevent juvenile delinquency.
Park and Burgess opened a theoretical door that writers like Jane Jacobs walked through a half century later. Whatever came of that early work?
In 1934 Clifford Shaw tested the theory in one of the first ever community crime prevention projects - the Chicago Area Projects (CAP). According to an article in the FBI Bulletin it was Shaw's belief that:
"…the solution to Chicago's gang problem meant reaching out to the gangs and redirecting them into the conventional life of the community. His method, which emphasized a "bottom up," proactive approach, contrasted greatly with traditional, "top down" methods, which stressed punitive or repressive measures to control delinquency."
Pretty impressive stuff. Sounds similar to SafeGrowth. No surprise there. Many modern community development programs owe much to Shaw and the CAP.
What is surprising is the absence of published results. What happened to that project? Volumes of criminology literature exists today due to, in opposition to, or as adjuncts to the ecology of crime theory. Not so for the results of CAP field work.
One of the most famous prevention projects in history has no evaluations? What?
Then I came across another article in my file cabinet.
It turns out CAP WAS successful, especially in Russell Square Park, one of the key sites of gang activity targeted by the program. A RAND report in the 1990s re-evaluated the CAP. Apparently Shaw's early studies from 1932-1937 found delinquency in Russell Square dropped in half, while it did not decline in surrounding neighborhoods. Amazingly, an article in the FBI Bulletin found that, 50 years later, the CAP is still on-going. In fact it has expanded!
If there was ever a landmark story emerging from prevention history, this is it! Why then didn't Shaw publish evaluations?
When I read through the studies in my cabinet I saw that he did. Control groups. Random assignment. The whole bit. He did evaluate it.
Then, after awhile, he didn't.
ECONOMIES OF SCALE
By the 1950s Shaw was telling researchers he didn't think it was even possible to prove statistically CAP worked. Why?
The answer from what I can see is frustration. Shaw got fed up trying to demonstrate with increasing levels of statistical proof that CAP worked, even though the Russell Square research showed just that. After all, his team and the residents themselves saw positive results.
It is an irony of history that the rigors of academic evaluation frustrate on one hand, yet provide the very publications from which future practitioners learn lessons of success and failure. It is why some evaluation is important but too much, not so much.
What Shaw discovered was prevention economies of scale. It is possible to spend more time and money satisfying the rigors of evaluation than actually preventing crime.
I suspect Shaw uncovered what Stanley Leiberson said in the 1990s: "we can be confident that all theories can be shown to be false—simply because it is impossible to specify all possible conditions and, therefore, a literal interpretation of what a theory implies can be taken out of context and lead to a negative result."
In 2005 sociologist Max Travers came to the same conclusion - social science evaluation is not a scientific discipline that produces objective findings.
Evaluation is important. But preventing crime is the point. As long as the former informs the latter in the service of practitioners, we're on track. Otherwise, as Shaw discovered, we've run off the ROTO rails.
What works? Community development, careful direction of municipal services, and involving local residents in safety and livability...that works.