WHAT GETS MEASURED GETS DONE...
True for Vaccines and for AFIX?
By Dale Dauten, King Features Columnist
In God we trust, all others bring data.
W. Edwards Deming
You've no doubt heard the line "What gets measured, gets done." I bet you can't tell me who originated that line. That's because no one is sure. It wasn't Deming or Drucker, although both built on it. The first use might have come from William Kelvin (AKA, Lord Kelvin of Kelvin Scale fame) or maybe it was Georg Joachim de Porris (better known by the mathematicians' version of a rap name, Rheticus), who developed trigonometric tables. Those latter two candidates both said some version of, "If you can measure it, you can manage it."
How true is the "gets done" truism? Well, anyone who has a bathroom scale can probably tell you that measuring alone isn't enough.
But, still, can there can be magic in measurement. Here's my favorite example.
THE MAGIC OF MEASUREMENT
Dr. Joseph Golbus had been put in charge of a chain of hospitals/clinics in the Chicago area. He set out to improve patient satisfaction and chose to measure what's called the Loyalty Score, asking patients,
"Would you refer friends and family to this clinic?"
He found wide variations in loyalty, based on which doctor the patient had encountered. So Golbus started offering training on customer service. Few doctors were interested, taking the old attitude of, "I'm here to cure 'em, not coddle 'em."
To that point, the conclusion was clear: Having a new measurement tool had little effect.
Then Dr. Golbus did the one thing that made all the difference. He started sending out the scores, broken down by each facility and each doctor, to everyone one in the organization. Suddenly the doctors-mostly driven and competitive people-took an interest.
What happened next surprised Golbus:
The same doctors who had ignored his earlier pleas were now coming to him and asking how to improve their scores. He then offered them all the behaviors correlated with higher Loyalty Scores. (It turned out that "looks me in the eye" and "knows my name" were the best predictors of Loyalty Scores.)
Loyalty increased by 20% and Golbus was able to show a corresponding rise in doctors' incomes. Before long the doctors were coming up with customer-oriented new services, like same-day appointments. "From there," Golbus says, "our customers became our sales force."
Beautiful, no? And it suggests lengthening the statement with which we began...
What gets measured... and noticed... and rewarded... gets done."
WILL IT WORK FOR AFIX?
All of which takes us to the program called AFIX (that's Assessment, Feedback, Incentive, eXchange). It's a program to allow states to develop more and better measurement tools for immunization and the CDC is putting significant funding behind it.
Kristi Siahaya, a Public Health Consultant for Scientific Technologies Corporation (STC), describes the need for AFIX:
"This is a way for states to get consistent data for immunization coverage rates.
"Right now states may have different data. For instance, Tdap rates in Mississippi may not define the numerator and denominator the same way as in Washington state. AFIX prescribes how the numerators and denominators are calculated."
Once they have the right numbers, they can offer feedback to clinicians and thus build upon successes.
History of AFIX, as described by the CDC...
In order to raise coverage levels to achieve national immunization goals, the Georgia Department of Public Health implemented a statewide program in 1986 that consisted of annual Assessments of immunization records at its public health clinics. Information was fed back to clinic providers and their staff who, in turn, developed their own solutions to improve coverage. Incentives included rank order listings of clinics by coverage level, awards for sites with the highest levels, and presentations by successful clinics at annual immunization meetings. Other successful strategies included coordinating with WIC, conducting reminder/recall phone activities, and providing performance Feedback to nursing staff. As a result of these efforts, immunization coverage levels in Georgia public clinics increased from 40% to 91% between 1986 and 2001. This intervention has become a national model program to improve immunization rates.
Taking AFIX to the national level means more data, and more consistency. This means...
- States can use the data to assess the effectiveness of varying immunization programs, thus accelerating learning and driving improvements.
- Clinics can see how they stack up against others and this offers the chance for competitiveness to kick in.
- And, of course, the CDC can begin to rank and rate states.
But all the new data are only useful if they are accessible.
Will states be able to see deeply into the data to see what really makes a difference and use that wisdom to drive change?
OR
Will AFIX just offer up a hot, steaming mess of numbers that will end up in unreadable PowerPoint presentations that numb the minds of health officials?
TURNING DATA INTO KNOWLEDGE
If the statistics are boring, you've got the wrong numbers.
Edward Tufte
Having developed the country's most useful health information system, called SMaRT, STC is working with the state of Washington, using a CDC grant, to develop an AFIX type dashboard designed to yield information that is instantly recognizable as important and useful. As Mike Popovich, STC's CEO puts it, "We want the data to be not just useful but irresistible. We want information that people look forward to seeing."
Shown below is a mockup of how the system might appear to the user. Karen Chin, a key member of the team to develop the program for the State of Washington, points out some of the highlights of what users will be able to readily access...
The user can easily access the Immunization Coverage Data Dashboard through the IIS. Here is a list of key features:
- Dynamic data reporting system that will enable the user to quickly view immunization data in close to real-time
- Slice-and-dice the data using the filters on the left to view more specific areas in immunization coverage
- View a breakdown of number of patients who
- Access data that will show the number of patients who have aged out of their vaccine series or who received vaccinations outside of their recommended schedule, which can help immunization programs evaluate vaccination services
- One-stop shop for State Immunization Programs and providers to view their immunization coverage data within the jurisdiction and area they serve
- may not be up-to-date with their vaccine coverage or
- who may have missed opportunities where the provider did not administer all vaccines they were due for during their last vaccination visit so that immunization programs can evaluate gaps in vaccination coverage
The Washington project is about a month from implementation.
Mike Popovich says of that new system:
"We want someone using the system to constantly be saying to colleagues, 'Wow, come see this!' We are creating
you've-got-to-check-this-out! data.
"This isn't just big data, it's muscular data."
