The measurement of outcomes of behavioral health services is an important function that provides evidence of effective services at client and aggregate levels, and gives planners key data about what is working. Outcomes measurement is coming to all areas and services including to government-funded services, where a variety of forces make outcomes measurement imperative. In this context, the Ohio Department of Mental Health (ODMH) recently announced the creation of the Mental Health Consumer Outcomes Data Mart (ODM). The ODM is a publicly accessible, Web-based tool for analyzing the outcomes data collected on consumers of publicly funded community mental health centers in the state. The ODM is at http://www.mh.state.oh.us/oper/outcomes/outcomes.index.html.
The ODM is part of the Ohio Consumer Outcomes System, which began in 1996 when Ohio's public mental health system launched an initiative to establish a uniform consumer outcomes measurement and reporting system. The collaborative process involved all major stakeholders: provider agencies, county boards of mental health, consumers, consumers' families, and academicians. No major decision about the Outcomes System was made without these constituencies' input.
The ODMH led the Outcomes System initiative at the request of ODMH Director Michael F. Hogan, PhD (Dr. Hogan was the chair of the President's New Freedom Commission on Mental Health). He charged the establishing committee “to identify an initial set of critical consumer outcomes and…recommend to ODMH a standard, statewide, ongoing approach to identifying and measuring consumer outcomes and performance of Ohio's mental health system.”
Today the Outcomes System initiative provides Ohio's public mental health system with an effective mechanism for giving voice to consumers in the treatment process, provides a common set of measures for use in quality improvement, and provides a method for resource accountability. The Outcomes System is comprised of a set of standard instruments and administration protocols for measuring the changes in consumers' lives as a result of services and is an integral part of the ODMH's “Quality Triangle,” along with best practices and quality improvement. Together, these forces aim to build recovery, resiliency, and cultural competency into all services.
The ODM grew out of the need for expanded data analysis capacity, expressed repeatedly in the context of statewide efforts to implement the Outcomes System initiative. In general, technology concerns are the most cited issue, and the inability to produce aggregate reports has been widely noted as a barrier to using outcomes data. Although the ODMH provided various software tools for the management and analysis of outcomes data, most of the tools were aimed at clinical, rather than aggregate, use of data.
The ODM broadly answers the system's needs for aggregate reporting by augmenting boards' and providers' internal data analysis capacity and supplies access to a wide variety of benchmarking data, which providers in other states also might find useful.
The ODM is the latest in a series of data marts the ODMH operates. The other data marts access aggregate claims and inpatient psychiatric hospital use data. The ODMH's dedication to the provision of data marts is based on the belief that knowledge belongs where it can be used most effectively. In Ohio, that means at local mental health boards, at mental health agencies, and throughout the ODMH, although anyone can access these data marts through the Internet.
The obstacles to building the ODM were significant: the data's complexity (six instruments, measures over multiple time points, and many scales and individual items represented), the creation of a simple yet effective user interface, data use policies such as HIPAA, budgetary constraints, and the arduous process of first defining and then building the ODM. The ODM's design was developed by a statewide ODM committee representing agencies; local alcohol, drug addiction, and mental health services boards; the ODMH; and other constituents. The committee met for 20 months, ending in 2004, and issued its recommendations as a detailed conceptual model used for the ODM's development. For 18 months, staff of the Office of Program Evaluation and Research worked closely with their counterparts in the Office of Information Services to build the ODM.
Easy to Use
The ODM's hallmark is ease of use, which was the primary consideration for the intended users—board members and provider staff—who had already explicitly stated a lack of data analysis skills as an obstacle to data use. Other anticipated users are mental health consumers, family members, consumer advocates, ODMH policy makers, and even the press and the public at large.
The ODM can be used without explicit knowledge of the Outcomes System by answering a series of plain-English questions by using pull-down menus and buttons. The general questions are:
Consumer population. Do you want to look at outcomes information for adult or child and adolescent consumers?
Source of data. Consumer outcomes scores come from multiple sources. Whose measurements of consumers would you like to see?
Consumer characteristics. Indicate any special characteristics of the consumers for whom you would like to see outcomes scores.
Time the consumers have been in treatment. Outcomes are measured at various points during consumers' treatment. Indicate the approximate time in treatment (as measured in time since last admission) that best describes the consumers for whom you'd like to see outcomes scores.