In an earlier Journal Review comment[1] on Sequist et al.,[2] I stated that the Agency for Healthcare Research and Quality (AHRQ) “tends usually (though not in all cases) to measure disparities in healthcare processes in terms of relative differences in rates of receiving such care, and usually (though not in all cases) to measure disparities in clinical outcomes in terms of relative differences in rates of failing to achieve the desired outcome.” I also discussed the way patterns of changing relative differences in experiencing or avoiding an outcome might be interpreted differently by AHRQ and the National Center for Health Statistics (NCHS), since NCHS measures all disparities in terms of relative differences in adverse outcomes.

The view that AHRQ usually measures disparities in processes in terms of relative differences in rates of receiving certain processes was based on the wording of the core measures used in the National Healthcare Disparities Reports for 2005 and 2006,[3,4] as well as some of the discussion in the reports. Recent preparation for a presentation on measurement issues in the healthcare disparities reports,[5] however, has caused me to recognize that the statements concerning AHRQ’s method of measuring disparities in process outcomes are incorrect. Notwithstanding the wording of the core measures in the disparities reports, in all or almost all cases, AHRQ in fact measures process disparities (as well as clinical outcome disparities) in terms of relative differences between rates of experiencing the adverse outcome (e.g., relative differences between rates of failing to receive prenatal care in the first trimester rather than relative differences between rates of receiving such care).[5,6]

In addition to the quoted incorrect statement, in the comment I stated that, based on declining relative differences in statin use, AHRQ would conclude that racial disparities had declined. In fact, based on increasing relative differences in non-use of statins, AHRQ would conclude that the disparities had increased.

The principal consequence of the correction relating to the theme of the Sequist article is that, whereas my earlier statements would suggest that, at least as to process outcomes, improvements in quality would tend to reduce disparities as measured by AHRQ, in fact improvements in quality would tend to increase disparities in process outcomes as measured by AHRQ.[5,6]

I made similar statements concerning AHRQ’s measurement of healthcare disparities in two other Journal Review comments.[7,8] The first has been corrected,[9] and the other will be corrected shortly.

References:

1. Scanlan JP. Understanding the ways improvements in quality affect different measures of disparities in healthcare outcomes regardless of meaningful changes in the relationships between two groups’ distributions of factors associated with the outcome. Journal Review Aug. 30, 2007: http://www.journalreview.org/view_pubmed_arti...

2. Sequist TD, Adams AS, Zhang F, Ross-Degnan D, Ayanian JZ. The effect of quality improvement on racial disparities in diabetes care. Arch Intern Med 2006;166:675-681:

3. Agency for Healthcare Research and Quality. 2005 National Healthcare Disparities Report: http://www.ahrq.gov/qual/Nhdr05/nhdr05.htm

4. Agency for Healthcare Research and Quality. 2006 National Healthcare Disparities Report: http://www.ahrq.gov/qual/nhdr06/nhdr06.htm

5. Scanlan JP. Measurement Problems in the National Healthcare Disparities Report, presented at American Public Health Association 135th Annual Meeting & Exposition, Washington, DC, Nov. 3-7, 2007: http://www.jpscanlan.com/images/APHA_2007_Pre... http://www.jpscanlan.com/images/ORAL_ANNOTATE...

6. Scanlan JP. Recognizing the way correlations between improvements in healthcare and reductions in healthcare disparities tend to turn on the choice of disparities measure. Journal Review Nov. 9, 2007, responding to Kaytur FA, Clancy CM. Improving quality and reducing disparities. JAMA 2003;289:1033-34: http://www.journalreview.org/view_pubmed_arti...

7. Scanlan JP. Effects of choice measure on determination of whether health care disparities are increasing or decreasing. Journal Review May 1, 2007, responding to Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682 (and several other articles in the same issue):http://www.journalreview.org/view_pubmed_arti...

8. Scanlan JP. Understanding patterns of correlations between plan quality and different measures of healthcare disparities. Journal Review Aug. 30, 2007, responding to Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004: http://www.journalreview.org/view_pubmed_arti...

9. Scanlan JP. Correction to statements concerning the measurement of healthcare disparities in the National Healthcare Disparities Reports in earlier comment on Vaccarino et al. Journal Review Nov. 6, 2007, correcting reference 7: http://www.journalreview.org/view_pubmed_arti...

Sequist et al.[1] attempted to identify the effect of quality improvements in diabetes care on racial disparities in certain outcomes. They found that during a 1997-2001 period when there were substantial increases in (a) annual LDL cholesterol screening (LDL-T), (b) LDL cholesterol control (LDL-C), and (c) statin use (SU), absolute differences between white and black rates decreased for cholesterol screening and cholesterol control while the absolute difference between black and white rates of statin use showed a nonsignificant increase.

The data in Table 2 of the study illustrate the way conclusions about the impact of quality improvements on racial disparities in health care tend to be affected by the choice of measure and the need to understand the way changes in quality tend to affect each measure of difference between the rates of two groups regardless of whether there occurred a meaningful change in the relationship of the groups’ distributions of factors associated with the likelihood of experiencing an outcome. The study measured quality in terms of rates of experiencing certain favorable outcomes and measured racial disparities in terms of absolute differences between black and white rates of experiencing those outcomes. No consideration was given to the way changes in overall prevalence of an outcome may affect differences between rates solely because of characteristics of the differing distributions of factors associated with the outcome or the way other measures of disparities might yield different results.

In prior comments on this site,[2-5], and elsewhere,[6,7] I have explained the ways various measures of health disparities are affected by the prevalence of an outcome. In general, solely as a consequence of two groups’ differing distributions of factors associated with the likelihood of experiencing or avoiding an outcome, various differences between two groups’ rates will tend to change in the following manner as the outcome increases in prevalence.

1. Relative differences in rates of experiencing the outcome tend to decline.

2. Relative differences in rates of failing to experience the outcome tend to increase.

3. Absolute differences may either increase or decrease. Such differences tend to be very small when an outcome is quite rare, grow larger as the outcome becomes more common, then grow small again as the outcome becomes nearly universal. In the case of perfectly normal distributions, when the outcome is in a prevalence range where (a) the relative difference between rates of experiencing an outcome (measured in terms of the ratio of the rate of the group with the higher rate of experiencing the outcome (Group X) to that of the group with the lower rate of experiencing the outcome (Group Y)) is smaller ¬than (b) the relative difference between rates of failing to experience the outcome (measured in terms of the ratio of Group Y’s rate of failing to experience the outcome to Group X’s rate of failing to experience the outcome), further increases in the prevalence of the outcome will tend to reduce the absolute difference between rates of experiencing (or failing to experience) the outcome. To make this point somewhat less abstract, in the case of white and black rates of receiving some beneficial procedure that is generally increasing in prevalence and for which whites have higher average rates than blacks, this would mean that the maximum for the absolute difference would tend to be found where the decreasing ratio of the white to black rates of receiving the procedure (ratio (a)) approximates the increasing ratio of the black to white rates of failing to receive the procedure (ratio (b)).

While Sequist et al. measured racial disparities for each outcome in terms of absolute differences between black and white rates, government agencies would measure the differences otherwise. The Agency for Healthcare Research and Quality (AHRQ) tends usually (though not in all cases) to measure disparities in healthcare processes in terms of relative differences in rates of receiving such care, and usually (though not in all cases) to measure disparities in clinical outcomes in terms of relative differences in rates of failing to achieve the desired outcome.[8] Thus, AHRQ would be inclined to measure disparities in LDL testing and statin use in terms of relative differences between rates of being tested or using statins, while it would be inclined to measure disparities in LDL control in terms of relative differences between rates of failing to control LDL. On the other hand, the National Center for Health Statistics (NCHS) recommends that all disparities be measured in terms of relative differences between rates of failing to experience the favorable outcome.[9,10]

All that said, consider the changes between 1997 and 2001 shown in Table 2 of the Sequist article. For cholesterol screening, from 1997 to 2001 the white rate increased from 43.2% to 65.3%, while the black increased from 29.4%, to 61.6%. Thus, the absolute difference declined from 13.8% to 3.7%. Even without regard to the factors discussed above, that would seem like a meaningful change (reduction) in disparity – that is, one that is unlikely to result solely from the change in prevalence without some change in the relationship of the two groups’ distributions of factors related to experiencing or avoiding the outcome. Given the relationships of the distributions suggested by the absolute differences observed in 1997, the size of the reduction in the absolute difference by 2001intuitively seems too great not to involve a change in the relationships of those distributions. Nevertheless, I note that at the beginning of the period, ratio (a) was larger than ratio (b) and at the end of the period ratio (b) was larger than ratio (a). Thus, the expectation would be that, as screening rates generally increased, the absolute difference would increase for a time and then decline. Hence, while the fact that the decline in the absolute difference between black and white rates seems too great to be solely a function of the change in prevalence, there is no identifiable departure from the expected pattern of changes in absolute differences such as might suggest a meaningful change in susceptibilities. Further, however, the relative difference in rates of screening declined during this period, as did the relative difference between rates of failing to receive screening. While the former would be expected to occur as an outcome increases, the latter would not. Hence, on the basis of the decline in the relative difference in rates of failing to receive screening, one might reasonably infer that there occurred a meaningful decline in black-white differences with respect to factors associated with the likelihood of screening.

For LDL control, from 1997 to 2001, the white rate increased from 17.7% to 44.6%, while the black rate increased from 9.1% to 39.0%. The absolute difference declined from 8.6% to 5.6%. In this case, ratio (a) remained greater than ratio (b) throughout the period. Thus, in circumstances where increases in prevalence would be expected typically to increase absolute differences, the opposite occurred. Such departure from the expected could reasonably be read to suggest a meaningful decline in the black-white disparity with respect to the factors associated with likelihood of LDL cholesterol control. Similarly, while the relative difference in rates of control declined (as ordinarily would occur in the circumstances of an increase in rates of control), the relative difference in rates of failure to control also declined (which is the opposite of what would typically occur in the circumstances of an increase in rates of control). That, too, might reasonably be read as suggesting a meaningful decline in disparity.

For statin use, from 1997 to 2001, the white rate increased from 22.4% to 39.2% while the black rate increased from 15.4% to 29.9%. Thus, the absolute difference between rates increased from 7.0% to 9.3%. Inasmuch as ratio (a) remained greater than ratio (b) throughout the period, such increase is of a kind to be expected solely as a result of changes in prevalence and hence ought not to be regarded as reflecting anything else. The relative difference between rates of using statins declined (as would typically occur in a time of increasing statin use) and the relative difference between rates of non-use increased (as also would typically occur in the circumstances). Thus, the patterns of changing differences between rates offer no basis for determining whether there has been a meaningful change in the relationship of black and white distributions of factors associated with statin uses.

(I ignore the absence of statistical significance with respect to the change in black-white differences in statin use. The discussion applies whether or not the changes are statistically significant. I note, however, that the fact that each pattern of changing disparities is what one would expect in the circumstances provides reason to believe that the observed changes were not random fluctuations.)

With regard to cholesterol screening and LDL control, Sequist et al., AHRQ, and NCHS all would regard disparities to be decreasing, and probably would be correct in these conclusions even if their reasons for such conclusions would not be adequate. With respect to statin use, the authors and NCHS (based, respectively, on increasing absolute differences and increasing relative differences in non-use) would likely conclude that the disparities were increasing, while AHRQ (based on declining relative differences in rates of statin use) would likely conclude that disparities had declined. In this instance, the patterns of directions of change of the various differences between rates do not provide a basis to determine whether any of these conclusions would be correct.

None of this is to say that one can reliably draw inferences about meaningful changes in the manner of the preceding paragraphs given that the distributions are not directly observed and may contain a variety of irregularities. And usually the distributions cannot be directly observed (though that might be possible in the case of the distributions underlying the LDL control rates). (I also note that one might identify different patterns and draw still different conclusions if one examined patterns within the intermediate points in time in Table 2.) But whether or not one can draw reliable inferences while attempting to take the described tendencies into account, one cannot reasonably rely on changes in any of the three measures addressed above as a basis for determining whether disparities have increased or decreased while ignoring the tendencies.

Sequist et al. also studied changes in racial disparities in certain other outcomes. In the case of those outcomes, none of the rates changed very much. And when none of the rates changes very much, there ordinarily will be little change in disparities, meaningful or otherwise.

References:

1. Sequist TD, Adams AS, Zhang F, Ross-Degnan D, Ayanian JZ. The effect of quality improvement on racial disparities in diabetes care. Arch Intern Med. 2006;166:675-681.

2. Scanlan JP. Effects of choice measure on determination of whether health care disparities are increasing or decreasing. Journal Review May 1, 2007: http://www.journalreview.org/view_pubmed_arti...

3. Scanlan JP. Understanding expected patterns of changes in absolute differences between the rates at which racial or gender groups receive adequate care. Journal Review May 1, 2007: http://www.journalreview.org/view_pubmed_arti...

4. Scanlan JP. Understanding when general increases in an outcome tend to result in increasing absolute differences between the rates of two groups. Journal Review June 1, 2007: http://www.journalreview.org/view_pubmed_arti...

5. Scanlan JP. Understanding when general increases in an outcome tend to result in increasing absolute differences between the rates of two groups. Journal Review June 1, 2007: http://www.journalreview.org/view_pubmed_arti...

6. Scanlan JP. Can we actually measure health disparities? Chance 2006:19(2):47-51: http://www.jpscanlan.com/images/Can_We_Actual...

7. Scanlan JP. The misinterpretation of health inequalities in the United Kingdom. Paper presented at: British Society for Population Studies Annual Conference 2006, Southampton, England, Sept. 18-20, 2006: http://www.jpscanlan.com/images/BSPS_2006_Com...

8. National Healthcare Disparities Report, 2006. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/nhdr06/nhdr06.htm

9. Keppel KG, Pearcy JN, Klein RJ. Measuring progress in Healthy People 2010. Healthy People statistical notes. No. 25. Hyattsville, Md.: National Center for Health Statistics: http://www.cdc.gov/nchs/data/statnt/statnt25....

10. Keppel KG, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital and health statistics. Series 2. No. 141. Washington, D.C.: Government Printing Office, 2005. (DHHS publication no. (PHS) 2005-1341.): http://www.cdc.gov/nchs/data/series/sr_02/sr0...