I find it disheartening that those who push for QI and P4P programs based on evidence do not understand the most basic principles of the scientific method. For example, they fail to distinguish between dependent and independent variables. Perhaps they should read this site for children.
Imagine that you are going to study training methods for distance runners so you can evaluate coaching. You enlist 2000 new runners to participate in the a study, excluding athletes, people with flat feet, Morton’s neuroma, asthma, or obesity. The new runners are randomly allocated between two training approaches:
- ‘Intensive’ training with a regimen of five workouts weekly, lasting 45 minutes and consisting of a five minute warm-up and then walk-run periods with steadily increasing run pace and shorted walk time, with a goal of 40 minutes at a 9 minute mile pace.
- ‘Conventional’ training with written materials about training and encouragement to run several times a week.
The study lasts 2 years. Collected data includes the percent of runners able to run an 8 minute mile (primary end point), average running pace during workouts, monthly weight checks, monthly resting pulse and BP, how frequently running shoes are replaced, and how often workout clothes are washed. At the end of the study, the group that trained intensively has a significantly greater number who can run an 8 minute mile and their average running pace is 9:30 per mile, compared to the conventionally trained group that has an average running pace of 11:30 per mile. The intensive training group also has a greater weight loss, lower BP and pulse at rest, has replaced their running shoes more often and has washed their running clothes more often.
It is reasonable to conclude from this study that conventional training is inferior to intensive training (as defined in the study). But it is not reasonable to conclude that low heart rates, weight loss, replacing running shoes regularly, or washing workout clothes more often will improve running speed. The training regimen was the independent variable being tested to see its impact. These other data points are surrogate markers.Changes in the surrogate markers are associated with the primary end-point (faster 1 mile speed) but were the result of the training and did not cause the improvement. Association is not the same as causation.
This basic misunderstanding is frighteningly common, but nowhere is it more evident (or damaging) than in the misuse of A1c measurements in QI and P4P programs.
Multiple studies have been done to assess different treatment approaches to preventing complications in DM. (Note: the goal is not control of sugar, but prevention of complications. It doesn’t really matter whether the average sugar is 250 instead of 130 if the elevated sugar does not cause complications.) I will use the UKPDS-33 and its subset follow-up UKPDS-34 as examples, but the same principle applies to nearly every study of diabetic treatments and outcomes and the misuse of A1c results in QI and P4P programs, including DCCT, DCCT/EDIC, ACCORD, ADVANCE, UGDP, and VADT.
In UKPDS, a large group (3867) of newly diagnosed diabetics was enrolled to test whether an intensive treatment regimen would prevent more complications than conventional treatment with diet alone. Only newly diagnosed diabetics under the age of 65 were included. Patients with heart disease, renal disease, or contraindications to the use of insulin are excluded.
Both groups started with an A1c of 7. The intensive treatment group was treated with combinations of medications (sulfonylureas, metformin and insulin). The control group was treated with diet unless their fasting sugar exceeded 287 or they developed symptoms from the sugars, at which point medications were added. Over 10 years of follow up, it was found that the intensively treated group did better: their average A1c was about 7.1 (range of 6.2 to 8.2) compared to 7.9 (range of 6.9 to 8.8) in the diet group, and microvascular complications (retinopathy, nephropathy and neuropathy) were lower, but there was no change in macrovascular complications (stroke, MI or cardiac death). In UKSPD-34, a 17 year follow-up of the sub-group receiving metformin, 1 in 42 patients in the intensive group had a death prevented at 17 years, but this was associated with metformin use and not directly correlated to A1c.
Does this study mean that lowering the A1c to 7 reduces risk? Absolutely not. The A1c is a dependent variable - it is something that was changed by the intervention of the study.It is a result, not a cause. What this study shows is that using several drugs in various combinations controls sugars and prevents complications better than diet alone, that metformin is associated with improved cardiovascular outcomes, and that intensive therapy lowers A1c more than conventional therapy.
Repeating for emphasis: it does not show that lowering the A1c improves outcomes. (In fact, multiple studies have show than lowering A1c can cause substantial harm.)
And yet, multiple insurers, CMS, and countless health care institutions (including my own) either fail to understand or choose to ignore basic principles of science and data, and focus their QI and P4P goals on surrogate markers like the A1c rather than on actually improving the quality of care.