Most psychological and psychiatric surveys and tests are developed to provide the most accurate assessment of current functioning. These are usually validated by looking at their correlations with other standard measurements of similar concepts, determining whether they can be used to predict future behavior or outcomes, or whether they are consistent with some theoretical construct. Many instruments have been used or developed specifically to evaluate the success of behavioral health treatment by measuring clinical outcomes.
At a recent meeting with the vendors of our electronic medical record system, we were trying to specify what sorts of clinical outcome measures we wanted to routinely report as part of a clinical dashboard. We were also looking at how to aggregate and meaningfully compare outcomes across programs and with larger databases. As we looked at several possible scores, it became obvious that many of them have been employed for uses for which they were never intended. To a large degree, these scores had become politicized.
When a particular scale must yield certain scores to justify authorization for ongoing treatment and the instructions for the scale's use have been altered from the original, its ability to assess clinical outcomes is severely compromised.
Instruments have also been subverted by risk management concerns (a specific score must be met to justify admission and discharge) as well as monetary incentives such as bonus payments for specific performance levels.
The school testing scandal in Atlanta is a good example of how evaluation and assessment can be corrupted by incentive systems tied to scores. It's a slippery slope when incentives nudge clinicians and organizations in certain directions, even though their initial motives may have been completely honorable.
In our case a single set of scores are supposed to: (1) show current functioning, (2) demonstrate what functioning would be without treatment, and (3) show progress of treatment, despite the fact that (4) too much progress could make clients ineligible for benefits to pay for ongoing services. I think that might qualify as a quadruple bind.