Systems for Indicator Success
This newsletter focuses on another component of my dissertation, The Use and Influence of Health Indicators in Municipal Transportation Plans. In my last newsletter, I discussed the key indicator characteristics that made them for useful for transportation decision-making. I'll continue in this newsletter discussing the instrumental use of indicators--the presumption that indicators inform decisions--but focus on the organizational aspects of developing and using indicators.
First of all, none of the people I interviewed (city and transportation planners, advocates, public health practitioners, etc.) felt that data was the problem, despite whatever challenges they may have had with spatial scale, measurability, or the like. One informant was clear:
“I don't think it's because the data isn't usable or available. It's because of the systems. We do not have the systems in place to use the data in the way they're intended to be used” (Informant 35).
In some cases, different departments, such as public health and city planning, collaborated to develop indicators. The development process helped "force different departments to talk to each other" (Informant 12) or otherwise generate some policy alignment through indicator alignment. While helpful, this was not a critical factor in terms of having an influence on administrative decision-making. The three things that mattered most were:
- Elected and/or departmental leadership had adopted a "data-driven" governance paradigm
- Leadership secured sufficient funding for building the data infrastructure to monitor indicators over time and ensured that staff had the technical capacity to manage indicators (or a contract in place for consultant support)
- Leadership had changed organizational procedures with a clear use case for the indicators
The data-driven paradigm, as compared to one based on territoriality or a "cash grab" (Informant 36), was one that valued transparency, accountability, and fairness in decision-making. In two cases, elected leadership drove this approach, which in turn became infused within the culture of the departments. Non-profit organizations helped municipalities be accountable for the goals set in their plans--sometimes they even created their own progress reports.
Having sufficient funding may seem obvious, but some municipalities did not fully appreciate the degree of infrastructure required to monitor indicators over time. Indicator development was no problem--many cities had funding to develop the indicators as part of transportation plan development--but tracking indicators over time required new structures and funding.
Finally, having a clear procedural connection for the indicators was important. In one case, the city had data-driven leadership, sufficient funding, and was tracking indicators over time but they had not created a clear enough link from indicator reporting to decision-making. Specifically, they were reporting on systems monitoring indicators during budgeting decisions. As one informant reported:
It was “a good half-hour of city council just free-associating on their fears and anxieties. It had nothing to do with, like, ‘Are we getting where [we want to be] going?’” (Informant 25)
When reporting on indicators, you need an action proposal. Otherwise, you use indicators as an analytical input to a decision. The most successful cases used evaluation criteria as part of capital improvement programming; that is, a clear nexus between the data and the decision.
In short, to influence administrative decision-making: Ensure the organization has sufficient financial and technical capacity to create data infrastructure. Create accountability mechanisms such as required internally-produced progress reports; if the city doesn't, then advocates can step in. Make sure that indicators are used for a specific routine or analysis in order for them to be influential.
In the next newsletter I'll talk about a different way that indicators can be influential. Not as an analytical input, but as a way to create shared understanding and to set the policy agenda.
