In an earlier Journal Review comment[1] on Trivedi et al.,[2] I stated that the Agency for Healthcare Research and Quality (AHRQ) “tends usually (though not in all cases) to measure disparities in health care processes in terms of relative differences in rates of receiving such care, and usually (though not in all cases) to measure disparities in clinical outcomes in terms of relative differences in failing to achieve the desired outcome.[footnote omitted] Since the criteria for adequate care employed by Trivedi et al. in the instant study are of the latter nature, the AHRQ approach would tend to find disparities to be larger in the plans with the higher rates of adequate care.”

The view that AHRQ usually measures disparities in healthcare processes in terms of relative differences in rates of receiving certain processes was based on the wording of the core measures used in the National Healthcare Disparities Reports for 2005 and 2006,[3,4] as well as some of the discussion in the reports. Recent preparation for a presentation on measurement issues in the healthcare disparities reports,[5] however, has caused me to recognize that the statements concerning AHRQ’s method of measuring disparities in process outcomes are incorrect. Notwithstanding the wording of the core measures in the disparities reports, in all or almost all cases, AHRQ in fact measures process disparities (as well as clinical outcome disparities) in terms of relative differences between rates of experiencing the adverse outcome (e.g., relative differences between rates of failing to receive prenatal care in the first trimester rather than relative differences between rates of receiving such care).[5,6]

Whereas the broad statement about the AHRQ methodology was incorrect, the statement that AHRQ would tend to find disparities in the clinical outcomes examined in the Trivedi study to be larger in plans with higher rates of adequate care was correct.[5,6]

I made similar incorrect statements concerning AHRQ’s measurement of healthcare disparities in two other Journal Review comments.[7,8] These have been corrected.[9,10]

References:

1. Scanlan JP. Understanding patterns of correlations between plan quality and different measures of healthcare disparities. Journal Review Aug. 30, 2007: http://www.journalreview.org/view_pubmed_arti...

2. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004

3. Agency for Healthcare Research and Quality. 2005 National Healthcare Disparities Report: http://www.ahrq.gov/qual/Nhdr05/nhdr05.htm

4. Agency for Healthcare Research and Quality. 2006 National Healthcare Disparities Report: http://www.ahrq.gov/qual/nhdr06/nhdr06.htm

5. Scanlan JP. Measurement Problems in the National Healthcare Disparities Report, presented at American Public Health Association 135th Annual Meeting & Exposition, Washington, DC, Nov. 3-7, 2007: http://www.jpscanlan.com/images/APHA_2007_Pre... http://www.jpscanlan.com/images/ORAL_ANNOTATE...

6. Scanlan JP. Recognizing the way correlations between improvements in healthcare and reductions in healthcare disparities tend to turn on the choice of disparities measure. Journal Review Nov. 9, 2007, responding to Kaytur FA, Clancy CM. Improving quality and reducing disparities. JAMA 2003;289:1033-34: http://www.journalreview.org/view_pubmed_arti...

7. Scanlan JP. Effects of choice measure on determination of whether health care disparities are increasing or decreasing. Journal Review May 1, 2007, responding to Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682 (and several other articles in the same issue):http://www.journalreview.org/view_pubmed_arti...

8. Scanlan JP. Understanding the ways improvements in quality affect different measures of disparities in healthcare outcomes regardless of meaningful changes in the relationships between two groups’ distributions of factors associated with the outcome. Journal Review Aug. 30, 2007, responding to Sequist TD, Adams AS, Zhang F, Ross-Degnan D, Ayanian JZ. The effect of quality improvement on racial disparities in diabetes care. Arch Intern Med 2006;166:675-681: http://www.journalreview.org/view_pubmed_arti...

9. Scanlan JP. Correction to statements concerning the measurement of healthcare disparities in the National Healthcare Disparities Reports in earlier comment on Vaccarino et al. Journal Review Nov. 6, 2007: http://www.journalreview.org/view_pubmed_arti...

10. Scanlan JP. Correction to statements concerning the measurement of healthcare disparities by the Agency for Healthcare Research and Quality in earlier comment on Sequist et al. Journal Review Nov. 15, 2007: http://www.journalreview.org/view_pubmed_arti...

The study by Trivedi et al.[1] examined three issues: (1) the extent to which racial disparities in 4 HEDIS measures in Medicare health plans were the result of racial differences within plans or disproportionate enrollment of blacks in lower performing plans; (2) whether higher quality plans exhibit smaller racial disparities in outcome measures than lower quality plans; and (3) whether any health plans achieved both high overall quality and low racial disparity with respect to 1 or more HEDIS outcome measures. The study found that a much higher proportion of each racial disparity was due to within-plan racial differences than to the disproportionate enrollment of blacks in lower quality plans, and the study’s treatment of that issue seems entirely correct. But the study’s treatments of items (2) and (3) raise certain measurement issues. These issues involve the way the size of differences between group outcome rates are typically affected by the prevalence of the outcome being examined and the manner of categorizing above average plan quality and below average racial disparity. These issues are addressed under the two headings below.

A. The Effect of the Prevalence of an Outcome on the Size of Relative and Absolute Differences Between Rates of Experiencing or Avoiding the Outcome.

Trivedi et al. measured plan quality for each of 4 HEDIS outcomes in terms of the average rate at which black and white enrollees achieved the HEDIS standard (see end note). They measured racial disparities in terms of the absolute difference between black and white rates of achieving those standards. The use of absolute differences as a measure of disparity in this study raises issues similar, if not identical, to those addressed in a prior Journal Review comment [2] discussing an earlier article by the same authors [3] and several other articles on changes in healthcare disparities.[4-6] The earlier article by Trivedi et al. involved a study of whether racial disparities in certain healthcare outcomes were increasing or decreasing. Measuring those disparities in terms of absolute differences between rates of receiving adequate care, the study found that during a 1997-2003 period when the overall rates of adequate care were increasing, absolute differences between black and white rates usually declined.

In the earlier comment, I explained ways various measures of differences between rates of experiencing or failing to experience an outcome tend to be affected by the prevalence of the outcome. With respect to the relative measures that are more often used to appraise a disparity than the absolute differences used by Trivedi et al., I explained that as outcomes like receipt of adequate care (or anything else as to which two groups differ in their susceptibilities) increase in prevalence, relative differences between rates of experiencing the outcome tend to decrease while relative differences between rates of failing to experience the outcome tend to increase. With respect to absolute differences between rates as a measure of disparity, as were used by Trivedi et al. in the earlier study (as in the instant study), I explained that determining the expected direction of a change is more complicated. Such differences tend to be very small when an outcome is quite rare, grow larger as the outcome becomes more common, then grow small again as the outcome becomes nearly universal. In the case of perfectly normal distributions, when the outcome is in a prevalence range where (a) the relative difference between rates of experiencing an outcome (measured in terms of the ratio of the rate of the group with the higher rate of experiencing the outcome (Group X) to that of the group with the lower rate of experiencing the outcome (Group Y)) is smaller ¬than (b) the relative difference between rates of failing to experience the outcome (measured in terms of the ratio of Group Y’s rate of failing to experience the outcome to Group X’s rate of failing to experience the outcome), further increases in the prevalence of the outcome will tend to reduce the absolute difference between rates of experiencing (or failing to experience) the outcome. To make this point less abstract, in the case of white and black rates of receiving some beneficial procedure that is generally increasing in prevalence and where the white rate is greater than the black rate, this means that the maximum for the absolute difference would tend to be found where the decreasing ratio of the white to black rates of receiving the procedure (ratio (a)) approximates the increasing ratio of the black to white rates of failing to receive the procedure (ratio (b)).

In the circumstances of the earlier work by Trivedi et al. and one of the two other works discussed in the earlier comment that relied on absolute differences between rates as a measure of disparity (Sehgal et. al. [6]), ratio (b) was usually larger than ratio (a), and hence increases in the prevalence of an outcome would be expected ordinarily to reduce absolute differences between rates (as was usually observed). On the other hand, in the case of the other study discussed in the commentary that relied on absolute differences (Jha et al. [5]), the rates of the outcomes examined were in ranges where ratio (a) was usually greater than ratio (b), and hence increases in prevalence would be expected ordinarily to increase absolute differences between rates (as also was usually observed).

(The discussion of the earlier Trivedi study in the prior comment, which was fairly summary, noted that absolute differences had declined for 7 of 9 measures of adequate care, but did not distinguish between process outcomes and clinical outcomes. But, while the absolute difference between black and white rates had declined for 6 of 6 process outcomes, such difference had declined for only 1 of 3 clinical outcomes, a matter discussed further some paragraphs below.)

The instant study by Trivedi et al. (which the authors suggest is an extension of the work in the earlier article) raises the same issues, except that the differences in prevalence of adequate treatment involve different Medicare health plans rather than different points in time. That is, the instant study by Trivedi et al. involves various black and white distributions of factors associated with receipt of adequate care. Assuming that the black and white distributions are more or less normal in each plan (or at least exhibit no substantial irregularities, see discussion 11 paragraphs below) and that the relationships between those distributions are similar from plan to plan, the patterns of relationships between black and white rates as quality increases from plan to plan will be similar to those observed with respect to a single pair of differing distributions as the prevalence of an outcome increases over time (as discussed in the earlier comment and as illustrated, say, in the tables and figures of references 7 and 8).[7,8] Trivedi et al. find only weak and nonsignificant correlations between the quality of the plan and the size of the absolute differences between the rates of blacks and whites. But the study did not consider the usual effects of the prevalence of the outcomes in different settings on the absolute differences between the outcome rates of different groups in those settings.

As suggested above, and as discussed in the earlier comment, whether absolute differences between rates would be expected to increase or decrease as an outcome increases or decreases in prevalence is of some complexity even in the situation where the changes in prevalence are in ranges where there is no change with respect to whether ratio (a) or ratio ((b) is larger – that is, where one ratio remains larger throughout the period of the change in prevalence. But the issue becomes even more complex when a change in prevalence is such as to cause a reversal of the relationship between ratios (a) and (b). In such circumstances, the change in prevalence would be expected usually to cause the absolute difference to increase for a time and then to decrease. This means, among other things, that smaller absolute differences can tend to be systematically associated with both the lowest prevalences and the highest prevalences.

Determining an expected direction of change with respect to absolute differences as quality improves across the plans examined by Trivedi et al. – that is, as the prevalence of satisfactory outcomes increases – would appear to raise issues akin to those in the latter situation described in the preceding paragraph. The overall relationship of ratios (a) and (b) for each HEDIS measure can be derived from Figures 1 and 2, and, as augmented by the textual discussion of the effects of socioeconomic status, can be based on (1) the overall rates of each group, (2) the overall white rate with the black rate adjusted so as solely to reflect within-plan disparities, or (3) the overall white rate with the black rate adjusted so as solely to reflect within-plan disparities as further adjusted by socioeconomic status. According to each approach, ratio (b) is somewhat larger than ratio (a). Immediately below are the ratios using approach (2), which seems more pertinent than approach (1) (while approach (3) may or may not be better still, though its use raises some conceptual issues):

Hemoglobin A1 control for among enrollees with diabetes (HA) – ratio (a) = 1.08, ratio (b) = 1.29;

LDL-C control for enrollees with diabetes (LDL-C-D) – ratio (a) = 1.12, ratio (b) = 1.27;

Blood pressure control for enrollees with hypertension (BP) – ratio (a) = 1.08, ratio (b) = 1.12;

LDL-C control among enrollees after an acute coronary event (LDL-C-C) – ratio (a) = 1.18, ratio (b) = 1.39.

In each case ratio (b) is larger than ratio (a), but in no case is it dramatically so. Thus, the overall figures by no means suggest that we would observe a pattern whereby ratio (b) is larger than ratio (a) in all plans. If that were the case, one would expect to observe a tendency throughout the plans whereby increasing quality from plan to plan would be associated with declining absolute differences between black and white rates (as, for example, might be observed with respect to patterns of racial differences in rates of mammography or prenatal care). But the closeness of the ratios suggests that in some number of plans, mainly those with the lowest quality, ratio (a) is larger than ratio (b). Thus, for some part of the universe, the expectation would be that as quality increases from plan to plan, absolute differences between black and white rates would also increase, while at a certain point, further increases in quality would be associated with a decline in absolute differences. Given the relationships between the overall ratios (a) and (b) set out above, it is not surprising that for 3 of the measures – HA, LDL-C-D, and LDL-C-C – the trend lines in Figure 3 show an evident, if not statistically significant or dramatic, association of declining racial differences with increasing quality. That is, it is probable that a majority of cases involve situations where increasing quality is associated with a decline in the absolute difference while somewhat less than half of the cases involve situations where increasing quality is associated with an increase in the absolute difference. And it is worth noting that the instance where the trend line suggests no such pattern – BP – ratios (a) and (b) are almost identical. One possible curious outcome of the use of absolute differences in these circumstances is that, to the extent that plan quality varied substantially by plan type or geography, an analysis of association between plan quality and racial disparity broken down by plan type or geographic area might show that (a) for plan type or geographic area with generally lower quality, increasing quality would be associated with increasing racial differences, while (b) for plan type or geographic area with generally higher quality, increasing quality would be associated with decreasing racial differences.

As discussed above, the typical associations between overall prevalence of an outcome and relative differences in experiencing or failing to experience the outcome are more consistent across the entire range of changes in prevalence than are absolute differences between rates (albeit, the two relative differences tend to change consistently in opposite directions). Thus, had Trivedi et al. examined the association between quality and racial disparities in terms of relative differences, they might well have found that higher quality plans tended to be consistently associated with smaller relative differences in receipt of adequate care but larger relative differences in failing to receive adequate care.

The Agency for Healthcare Research and Quality (AHRQ) tends usually (though not in all cases) to measure disparities in health care processes in terms of relative differences in rates of receiving such care, and usually (though not in all cases) to measure disparities in clinical outcomes in terms of relative differences in failing to achieve the desired outcome.[9] Since the criteria for adequate care employed by Trivedi et al. in the instant study are of the latter nature, the AHRQ approach would tend to find disparities to be larger in the plans with the higher rates of adequate care. The National Center for Health Statistics (NCHS) recommends that all disparities be measured in terms of failing to experience the favorable outcome.[10,11] That approach would also tend to find larger disparities in the higher quality plans.

It also warrants note that, citing their earlier article referenced above [3], Sehgal [6], and Sequist et al.[12], Trivedi et al. point out that improvement in quality may have contributed to the narrowing of disparities in process outcomes, but the same effect has not been evident in clinical outcomes. Sehgal is discussed in the comment on the earlier article by Trivedi et al., which comment attempted to explain why the reduction in absolute differences in certain process outcomes observed in the Sehgal study and the earlier Trivedi study were essentially what one should expect in the circumstances and did not necessarily reflect any improvement in the relative situation of blacks that was not a function of the increases in prevalence of the desirable outcomes (quality). Sequist et al. is the subject of another Journal Review comment,[13] which explores ways in which certain changes in absolute differences observed in that study might or might not be fairly interpreted as reflecting meaningful changes in the relative situation of blacks and whites.

Near the outset of the instant work, Trivedi et al. note that their earlier work had found racial disparities not to have declined for two outcome measures assessing control of glucose and cholesterol. These were two of three clinical outcome measures examined in the earlier study. Table 3 of the earlier study reveals that in the case of glucose control, where a nonsignificant increase in the absolute difference was observed, ratio (a) was smaller than ratio (b) throughout a period of increasing control. Further, contrary to the usual situation, the relative difference between rates of control increased. These factors suggest a meaningful worsening of disparity, according the reasoning in references 7,8, and 13 (significance issues aside). But a significant increase in the absolute difference between cholesterol control for cardiovascular patients occurred during a period of increasing relative rates of control and a period in which for part of the time ratio (a) was greater ratio (b) and for part of the time ratio (b) was greater than ratio (a); and the changes in relative differences in control and failure to control behaved in the usual manner during the period (i.e., relative differences in control rates declined while relative differences in rates of failure to control increased). The same holds with respect to cholesterol control for diabetic patients, where a significant decline in the absolute difference was observed. Thus, in neither of the latter two situations is there an identifiable departure from the standard patterns that might suggest that there occurred a change in the black-white distributions of factors associated with cholesterol control.

Without further belaboring the details of the patterns of differences in clinical outcome rates, it suffices to note that the view of Trivedi et al. that improvements in quality have resulted in declining disparities in processes is based on reductions in absolute differences, without recognition that certain of those changes would be expected solely because of change in prevalence. Thus, there was not a sound basis for concluding that improvements in quality are reducing disparities in processes in a meaningful way, much less for comparing such reductions with the seeming effects of quality on clinical outcomes.

That is not to say that the authors’ discussion of the way disparities in processes may be easier to change than disparities in clinical outcomes is other than entirely reasonable. The factors associated with process outcomes seem much more within the control of physicians and hospital and plan administrators than the factors associated with adherence to regimens that play a large role in clinical outcomes. But the evaluation of whether disparities have changed in a meaningful way – that is, whether the relationship of the two groups’ distributions of factors associated with the various outcomes has changed – is more complicated than the authors recognize.

It might be suggested that the tendencies described with respect to perfectly normal distributions in reference 8 and the near normal distributions in reference 7 may not hold when the distributions depart significantly from the normal and where the standard deviations of the distributions are not the same. And certainly for each racial group there will be a broad range of shapes of distribution of factors affecting varied health and healthcare outcomes. But the general tendencies described regarding the associations between the prevalence of an outcome and relative differences in rates of experiencing the outcome, relative differences in rates of avoiding the outcome, and absolute differences are likely to play some role in almost all cases when the distributions are not substantially irregular. Hence, one cannot draw conclusions about meaningful changes in susceptibilities while ignoring those tendencies. Indeed, one could not confidently draw conclusions while ignoring those tendencies even if the tendencies had a role only occasionally.

Nevertheless, it warrants noting that the distributions underlying the Trivedi study would be expected to depart from normal simply because they are limited to populations that are suffering from certain adverse conditions to which blacks are more prone than whites. This makes the situation akin to one, say, limited to the population falling below point J in Table 1 of reference 8 (the point below which 30% of the advantaged group and 49% of the disadvantaged group falls). Yet, moving from point to point within the population below point J (and limiting the denominator in each fraction to that population), we would observe the same patterns of changes for ratios (a) and (b) and absolute differences that we observe in the overall population (though the patterns of changes in odds ratios would depart substantially from the patterns observed in the overall population).

NHANES data provide a useful illustration in more concrete terms. For example, among subjects in the NHANES 1999-2000 and 2001-2002 samples, in the 55-64 age-group, black men have substantially higher average systolic blood pressure, and are substantially more likely to have systolic blood pressure above 139 (the cutoff for the systolic component of hypertension), than white men in the same age group. Sixty percent (60%) of blacks and 33% of whites have systolic blood pressure above 139. Limiting the universe to that population, the table below presents figures for black and white rates of having blood pressure below each of the points in the data set from 142 to 190. The table also show the ratio of the white to black rate of having blood pressure below that point (ratio (a)), the ratio of the black to white rate of having blood pressure at the point or above (ratio (b)), and the absolute difference between rates of having blood pressure below the point, as well as an indicator of which ratio is larger.

Table: Black and white rates of systolic blood pressure below various points (within population with systolic blood pressure above 139), with (1) ratio of white to black rate of falling below each point (ratio (a)), (2) ratio of black to white rate of falling above each point (ratio (b)), (3) absolute difference between rates, and (4) indicator of which ratio is larger.

Point * B% * W% * RtA * RtB * AbD * LgrRatio

142 ** 0.13 * 0.15 * 1.15 * 1.02 * 0.02 * A

144 ** 0.17 * 0.29 * 1.65 * 1.16 * 0.11 * A

146 ** 0.25 * 0.43 * 1.71 * 1.31 * 0.18 * A

148 ** 0.33 * 0.52 * 1.60 * 1.41 * 0.20 * A

150 ** 0.38 * 0.58 * 1.52 * 1.48 * 0.20 * A

152 ** 0.40 * 0.65 * 1.62 * 1.73 * 0.25 * B

154 ** 0.50 * 0.69 * 1.38 * 1.62 * 0.19 * B

156 ** 0.62 * 0.76 * 1.24 * 1.62 * 0.15 * B

158 ** 0.63 * 0.79 * 1.24 * 1.71 * 0.15 * B

160 ** 0.65 * 0.83 * 1.27 * 2.08 * 0.18 * B

162 ** 0.69 * 0.86 * 1.24 * 2.15 * 0.16 * B

164 ** 0.71 * 0.86 * 1.20 * 2.02 * 0.15 * B

166 ** 0.79 * 0.88 * 1.12 * 1.78 * 0.09 * B

168 ** 0.79 * 0.90 * 1.15 * 2.22 * 0.12 * B

170 ** 0.79 * 0.92 * 1.16 * 2.54 * 0.13 * B

172 ** 0.81 * 0.92 * 1.13 * 2.31 * 0.11 * B

174 ** 0.83 * 0.94 * 1.14 * 2.91 * 0.11 * B

176 ** 0.85 * 0.96 * 1.14 * 4.31 * 0.12 * B

178 ** 0.87 * 0.96 * 1.11 * 3.77 * 0.10 * B

180 ** 0.87 * 0.96 * 1.11 * 3.77 * 0.10 * B

182 ** 0.88 * 0.96 * 1.09 * 3.23 * 0.08 * B

184 ** 0.88 * 0.96 * 1.09 * 3.23 * 0.08 * B

186 ** 0.90 * 0.98 * 1.08 * 4.04 * 0.07 * B

188 ** 0.90 * 0.99 * 1.09 * 8.08 * 0.08 * B

190 ** 0.90 * 0.99 * 1.09 * 8.08 * 0.08 * B

Suppose, then, that within this group of subjects a program is implemented to lower systolic blood pressure. A program that enabled everyone with systolic blood pressure below 146 (the first point at which as many as 10 blacks fall below the point) would create the situation we observe in the third row. Increasingly more successful programs – i.e., those that serially enabled blacks and whites with increasingly higher levels of systolic blood pressure also to lower their systolic blood pressure below 140 – would create the situations observed in the succeeding rows. And we observe that, for the most part, as the programs grow more successful, the relative difference in achieving the favorable outcome would tend to decline and the relative differences in failing to achieve the outcome would tend to increase. The absolute difference, however, increase for a time, but then at approximately the level where ratio (a) ceases to exceed ratio (b), the absolute difference commences to decline, and, for the most part, continues to do so.

This is a rather small sample, including only 84 whites and 52 blacks. But among men in other age groups, and among women with certain age groups, where there are similar racial differences in average systolic blood pressure, one observes similar patterns – though each with the inconsistencies typically found in small data sets. Many of those inconsistencies might well disappear in larger samples. Whether or not that is the case, however, there is ample reason to believe that these are basically the patterns one will usually observe as the effectiveness of interventions to control hypertension or other adverse outcomes as to which blacks and white differ in their susceptibilities increases over time or increases from plan to plan.

The points above should not be read to mean that all observed outcomes are dictated by the described statistical tendencies. But without understanding those tendencies it is not possible to know whether a study is identifying patterns of consequence or is simply yielding the results to be expected because of the choice of method to measure disparities. Even with an understanding of those tendencies, it may be impossible to answer such questions with confidence.[7,8] For, while the tendencies can be expected to be relatively pervasive, their effects may not be predictable enough to effectively take into account.

B. Restricting the Analysis of High Quality Plans with Low Disparities to Plans that Showed a Statistically Significant Departure from the Overall Average.

In examining the extent to which any plans had both high overall quality and low racial disparity in one or more of the outcomes, the authors limited the analysis to plans with 20 or more relevant black subjects and then divided the plans into groupings of “above average,” “average,” or “below average” for both quality and size of disparity. The authors found high quality and low disparity in only 6 plans for HA, 4 plans for LDL-C-D and BP, and 2 plans for LDL-C-C. Only 1 plan had high quality and low disparity in more than one measure.

Because the precise number of plans analyzed for each outcome is not known, it is difficult to determine the extent to which the figures in the preceding paragraph differ from what would occur at random assuming there were no association whatever between quality and absolute differences between black and white rates and the 3 levels of quality or disparity were each comprised of a third of the plans analyzed. In Figure 3, it is easiest to count the number of plans for LDL-C-C, which seems to be about 54. Given 9 combinations of quality and disparity, a random draw where the high quality and low disparity each comprised one third of all plans would on average yield 6 plans with high quality and low disparity – 4 more than were found. Given the apparently higher number of plans analyzed for the other three outcomes, those figures, too, seem to be fewer than would be found at random – again, if the high quality group and the low disparity group each comprised a third of the total plans.

But rather than dividing the plans into thirds, the study treated a plan as above or below average for quality or disparity only if the rate for whites (with respect to quality) or the absolute difference between black and whites rates (with respect to disparity) was different from the overall average at the .05 level of statistical significance. And it seems almost certain that this resulted in very small numbers of plans (considerably less than one-third of those analyzed) being deemed of above average quality and very small numbers of plans being deemed to have below average disparities. In fact, at one point the article refers to the criteria as identifying outliers.

It is far from clear what there was any need to apply a measure of statistical significance to designate a plan as above or below average for purposes of this study. Such a measure might well be appropriate with respect to a statement made about a particular plan (though even then more so with regard to disparity than quality). But for purposes of identifying overall patterns, it would seem more appropriate to divide the plans in thirds – or halves – according to a ranking of the white rates and the racial differences. Even if there were some justification for applying a measure of statistical significance in designating the plans as above or below average, the author ought to make clear that their approach had substantially limited (or expanded) the numbers of cases where one might find both high quality and low disparity.

The point made in this section, however, is made without regard to whether, given the points made in Section A, one could derive meaningful information from such an inquiry in these circumstances. Apart from reason to expect low absolute differences to be associated both with low and high quality plans, there is a question whether a clear association between high quality plans and low disparity (as one might find where ratio (b) was substantially greater than ratio (a) in all cases) would reflect anything other than a statistical phenomenon – just as there would be such a question regarding any association between high quality and small relative differences between rates of adequate care or large relative differences between rates of inadequate care.

End note:

The text of the article, and the language under the x-axes of Figure 3, seem to indicate that, for the main analysis of association between plan quality and racial disparities, quality is measured in terms of the overall rate of meeting the HEDIS standard for blacks and whites combined. Apparently the regression adjusts for the black representation in the plan, which would seem to address any tendency for plans with higher black representation to have lower quality in a way it ought not to. Even so, however, inclusion of the black rate in the determination of plan quality seems to allow the size of the disparity to affect the determination of plan quality. A better approach would seem to be to measure the plan quality solely in terms of the white rate, as was done for the analysis of whether plans of above average quality also had below average disparities.

Also, the precise epidemiologist or statistician might take issue with my use of “prevalence” or “overall prevalence,” since overall prevalence is a function of the black and white rates (and, hence, affected both by the proportion each group makes up of the total population and the size of any disparity). It should be clear, however, that such terms are intended to reflect a general pattern. As a rule, that general pattern can best be identified with reference to the situation of the advantaged group, as, say, is done in the tables and figures of references 8. But I think that, despite the imprecision, describing the general pattern in terms of prevalence or overall prevalence breeds less confusion than doing so in terms the rate of the advantaged group.

References:

1. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004.

2. Scanlan JP. Effects of choice measure on determination of whether health care disparities are increasing or decreasing. Journal Review May 1, 2007: http://www.journalreview.org/view_pubmed_arti...

3. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Trends in the quality of care and racial disparities in Medicare managed care. N Engl J Med 2005;353:692-700.

4. Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682.

5. Jha AK, Fisher ES, Li Z, Orav EJ, Epstein AM. Racial trends in the use of major procedures among the elderly. N Engl J Med 2005;353:683-691.

6. Sehgal AR. Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA 2003;289:996-1000.

7. Scanlan JP. Can we actually measure health disparities? Chance 2006:19(2):47-51: http://www.jpscanlan.com/images/Can_We_Actual...

8. Scanlan JP. The misinterpretation of health inequalities in the United Kingdom. Paper presented at: British Society for Population Studies Annual Conference 2006, Southampton, England, Sept. 18-20, 2006: http://www.jpscanlan.com/images/BSPS_2006_Com...

9. National Healthcare Disparities Report, 2006. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/nhdr06/nhdr06.htm

10. Keppel KG, Pearcy JN, Klein RJ. Measuring progress in Healthy People 2010. Healthy People statistical notes. No. 25. Hyattsville, Md.: National Center for Health Statistics: http://www.cdc.gov/nchs/data/statnt/statnt25....

11. Keppel KG, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital and health statistics. Series 2. No. 141. Washington, D.C.: Government Printing Office, 2005. (DHHS publication no. (PHS) 2005-1341.): http://www.cdc.gov/nchs/data/series/sr_02/sr0...

12. . Sequist TD, Adams AS, Zhang F, Ross-Degnan D, Ayanian JZ. The effect of quality improvement on racial disparities in diabetes care. Arch Intern Med. 2006;166:675-681.

13. Scanlan JP. Understanding the ways improvements in quality affect different measures of disparities in healthcare outcomes regardless of meaningful changes in the relationships between two groups’ distributions of factors associated with the outcome. Journal Review Aug. 20, 2007: http://www.journalreview.org/view_pubmed_arti...