In an earlier comment [1] on Vaccarino et al.[2] and two other articles in the same issue of the New England Journal of Medicine,[3,4] I stated that in the National Healthcare Disparities Reports, the Agency for Healthcare Research and Quality (AHRQ) had typically measured disparities in healthcare in terms of relative differences between rates of receiving a procedure. I also noted that AHRQ had not adopted the recommendation of the National Center for Health Statistics ((NCHS) that all health or healthcare disparities be measured in terms of relative differences in adverse outcomes, which meant that, with respect to the size of healthcare disparities, the two agencies will tend to reach opposite conclusions about the directions of changes. In two later Journal Review comments on related issues,[5,6] and which involved distinctions between clinical outcomes and process outcomes, I stated that AHRQ “tends usually (though not in all cases) to measure disparities in healthcare processes in terms of relative differences in rates of receiving such care, and usually (though not in all cases) to measure disparities in clinical outcomes in terms of relative differences in rates of failing to achieve the desired outcome.”

The statements were based on the wording of the core measures used in the healthcare disparities reports for 2005 and 2006,[7,8] as well as some of the discussion in the reports. Recent preparation for a presentation on measurement issues in the healthcare disparities reports,[9] however, has caused me to recognize that the statements concerning AHRQ’s method of measuring disparities in process outcomes are incorrect. Notwithstanding the wording of the core measures, in all or almost all cases, AHRQ in fact measures disparities in terms of relative differences between rates of experiencing the adverse outcome (e.g., relative differences between rates of failing to receive prenatal care in the first trimester rather than relative differences between rates of receiving such care). This difference in measurement does not materially affect the key points made in these comments, the thrust of which is that, regardless of what measure is used, changes in such measure cannot alone identify a meaningful change in the relative health (or healthcare) of two groups – that is, identify changes that is not solely the result of changes in prevalence. It nevertheless is important to recognize that the statement that two agencies of the government would typically be reaching opposite conclusions on a matter is simply incorrect. It is also important to recognize that as healthcare improves – that is, as favorable process outcomes become more prevalent – AHRQ will not tend to conclude that disparities in process outcomes are declining (as I suggested it would), but will tend to conclude that such disparities are increasing.

In that regard, I note that the first referenced Journal Review comment discussed the findings by Sehgal [10] that, as rates of adequate hemodialysis increased between 1993 and 2000, racial and gender disparities (as measured in terms of absolute differences) had declined. The comment explained that Sehgal’s findings were pretty much what one would expect in the circumstances. Sehgal’s findings have been often cited as showing how improvements in healthcare will reduce disparities, including in a few places by the AHRQ officials principally responsible for the healthcare disparities reports.[11,12] As explained in the initial comment referenced above, however, improvements in quality will not always result even in reductions in absolute differences (though that will usually be the case when the subject is something like adequacy of care). More pertinent here, however, is that the figures underlying Sehgal’s findings of declining absolute differences show increasing relative differences in failure to receive adequate care.[9] Thus, under AHRQ’s approach, disparities in adequacy of hemodialysis – one of the core process measures – would be deemed to be increasing during the period examined by Sehgal. The 2006 report (at 8) in fact shows disparities in adequate hemodialysis rates - measured in terms of relative differences in failing to receive adequate hemodialysis ¬- to have increased (though that finding covers a different period from that examined by Sehgal).

Finally, in only one instance does the 2006 report shows its calculations of the size of changes in particular disparities over time. It does so in a Highlights section (at 6) in these terms:

“From 2000 to 2003, the proportion of adults who received care for illness or injury as soon as wanted decreased for Whites (from 16.2% to 13.4%) but increased for Blacks (from 17.5% to 18.4%). This corresponds to an increase of 9.8% per year in this disparity. However, from 2000 to 2004, the rate of new AIDS cases remained about the same for Whites (from 7.2 to 7.1 per 100,000 population age 13 and over) but decreased for Blacks (from 75.4 to72.1 per 100,000 population), corresponding to a decrease of 7.9% per year in this disparity.”

The former reference describes the process outcome in terms of receipt of a favorable outcome, which is how the core measure is worded, and may have contributed to my misunderstanding of the measurement approach. In fact, however, consistent with the report’s approach of measuring disparities in terms of adverse outcomes, the figures are rates of failing to receive care for illness as soon as wanted, and the disparity is thus adverse to blacks rather than adverse to whites. Moreover, given that the white rate of failing to receive care as soon as wanted declined while the black rate was increased, this would seem to reflect a meaningful worsening of the disparity.

While unrelated to my central point on measuring disparities, I note that both what the report describes as a 9.8% yearly increase in the disparity in receipt of care as soon as wanted and what it describes as a 7.9% yearly decrease in the disparity in new AIDS cases are what would be better described as “percentage point” changes (which is the way NCHS would describe these figures) and that the report’s approach is contrary to the way NCHS would measure the changes.[13]

According to NCHS’s approach and nomenclature, in the former case the disparity change would be deemed a 121% yearly increase and in the latter it would be deemed a 0.9% yearly decrease. That is, in the former case, the ratio of the black rate to the white rate increased from 1.08 to 1.37 over the three-year period. Thus, an 8% greater black rate increased to a 37% greater black rate – a 29 percentage point increase but a 363% increase (29/8). In the latter case, the ratio of the black rate to the white rate declined from 10.47 to 10.14 over the four-year period. Thus, the black rate went from being 947% greater than the white rate to being 914% greater than the white rate – a decline of 33 percentage points but only 3.5% (33/947).

Whereas this might seem merely an error in the calculation or description of these particular figures, note xix at page 5 of the report makes clear that the report is supposed to calculate what it terms the “percent” changes this way – i.e., by taking the percentage point difference without dividing it by the original excess risk. Thus, the various calculations of what are termed “percent” changes for purposes of determining the sizes of changes in disparities over time, which size determinations underlie certain counts in the report, are apparently calculated this way as well. If such is AHRQ’s actual intent, it would be better, for clarity, to refer to the changes as percentage point changes. Inasmuch as I am raising so many questions about the meaning of various changes of measures, I am reluctant to suggest that presenting actual percent changes (also properly termed “percentage changes” but still different from “percentage point changes”) in the relative differences, as NCHS would, is the better approach. But AHRQ should be mindful that its approach seems to differ from that of NCHS in this regard.

References:

1. Scanlan JP. Effects of choice measure on determination of whether health care disparities are increasing or decreasing. Journal Review May 1, 2007: http://www.journalreview.org/view_pubmed_arti...

2. Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682.

3 Jha AK, Fisher ES, Li Z, Orav EJ, Epstein AM. Racial trends in the use of major procedures among the elderly. N Engl J Med 2005;353:683-691.

4. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Trends in the quality of care and racial disparities in Medicare managed care. N Engl J Med 2005;353:692-700.

5. Understanding the ways improvements in quality affect different measures of disparities in healthcare outcomes regardless of meaningful changes in the relationships between two groups’ distributions of factors associated with the outcome. Journal Review Aug. 30, 2007, responding to Sequist TD, Adams AS, Zhang F, Ross-Degnan D, Ayanian JZ. The effect of quality improvement on racial disparities in diabetes care. Arch Intern Med. 2006;166:675-681: http://www.journalreview.org/view_pubmed_arti...

6. Understanding patterns of correlations between plan quality and different measures of healthcare disparities. Journal Review Aug. 30, 2007, responding to Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Relationship between quality of care and racial disparities in Medicare health plans. JAMA 2006;296:1998-2004: http://www.journalreview.org/view_pubmed_arti...

7. Agency for Healthcare Research and Quality. 2005 National Healthcare Disparities Report: http://www.ahrq.gov/qual/Nhdr05/nhdr05.htm

8. Agency for Healthcare Research and Quality. 2006 National Healthcare Disparities Report: http://www.ahrq.gov/qual/nhdr06/nhdr06.htm

9. Scanlan JP. Measurement Problems in the National Healthcare Disparities Report, to be presented at American Public Health Association 135th Annual Meeting & Exposition, Washington, DC, Nov. 3-7, 2007.

10. Sehgal AR. Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA 2003;289:996-1000.

11. Kaytur FA, Clancy CM. Improving quality and reducing disparities. JAMA 2003;289:1033-34.

12. Moy E, Drayton E, Clancy CM. Compliing the evidence: The National Healthcare Disparities Reports. Health Affairs 2005;24(2):376-387.

13. Keppel KG, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital and health statistics. Series 2. No. 141. Washington, D.C.: Government Printing Office, 2005. (DHHS publication no. (PHS) 2005-1341.): http://www.cdc.gov/nchs/data/series/sr_02/sr0...

The special articles on healthcare disparities by Vaccarino et al.[1], Jha et al.[2], and Trivedi et al.[3] in the August 18, 2005 New England Journal of Medicine together illustrate how the choice of a measure of health disparities in a particular setting tends to dictate the nature of the findings regarding whether the disparities are increasing or decreasing. The three articles, like virtually all health disparities research, suffer from the failure to recognize the ways measures of differences between rates at which groups experience some outcome tend to vary depending on the overall prevalence of the outcome.

The most notable of these tendencies is that whereby, when two groups differ in their susceptibility to an outcome, the rarer an outcome, the greater tends to be the relative difference between rates of experiencing it and the smaller tends to be the relative difference between rates of avoiding it.[4-7] The tendency can be illustrated with virtually any data set that allows one to examine the rates at which two groups fall above or below various points on a continuum of factors associated with some outcome. For example, published income data in the United States illustrate the way that reductions in poverty tend to increase relative demographic differences in poverty rates but reduce relative differences in rates of avoiding poverty.[4,6,7]. Hypothetical test score data illustrate the way lowering a cutoff (or improving test performance such as to allow everyone just below the cutoff to pass at the existing cutoff) will generally increase relative differences in failure rates but reduce relative differences in pass rates.[5-7]. The same pattern can be expected to occur wherever two groups differ in their distributions of risks of experiencing or avoiding some outcome, even when, as in the case of mortality and most other health outcomes, the risk distributions cannot be directly observed.

The failure to recognize this tendency is responsible for the near universal perception that racial and socioeconomic disparities in adverse health outcomes have been increasing, a view based on the fact that during periods of declining mortality and other adverse outcomes, relative disparities in experiencing those outcomes have been increasing. But as these outcomes become rarer, it is to be expected that, solely due to factors inherent in the shapes of the risk distributions, relative differences in rates of experiencing the outcomes will increase while relative differences in rates of avoiding the outcomes will decline. Yet increasing relative differences in adverse health outcomes have been regarded as indicating a meaningful worsening of health disparities without consideration of whether the increases are other than what would be expected solely due to declines in the prevalence of the outcomes or whether relative differences in avoiding those outcomes have decreased.

On the other hand, disparities in healthcare have typically been measured in terms of relative differences between rates of receiving a procedure, as in the yearly National Healthcare Disparities Reports of the Agency for Healthcare Research and Quality (AHRQ).[8,9] And, as various health procedures have become more widely available, decreasing disparities in receipt of the procedures have been deemed to indicate meaningful reductions in healthcare disparities without consideration of whether the decreases are other than what would be expected solely because of the increased availability of the procedures or whether the disparities in failure to receive those procedures have increased. Recently, researchers at the National Center for Health Statistics (NCHS), recognizing that one may get different results as to the direction of a change in the size of a disparity depending on whether on examines the favorable or the adverse outcome, have recommended that all disparities, including disparities in procedures like mammography, be measured in terms of the relative differences in the adverse outcome (i.e., rates of failing to receive mammography).[10-12] But the NCHS has failed to recognize that as the overall prevalence of an outcome changes, the relative difference between rates of experiencing the outcome and the relative differences between rates of avoiding the outcome tend to move systematically in opposite directions. So it has merely offered an approach whereby many healthcare disparities that previously were regarded as declining now will instead be regarded as increasing, and without providing any guidance for distinguishing changes in relative differences that are solely the consequence of changes in prevalence from those that reflect something more consequential. As yet AHRQ has not adopted the NCHS recommendation, which means that, with respect to the size of healthcare disparities, the two agencies will tend to reach opposite conclusions about the direction of changes.

Only one of the three special articles, however, relies on relative differences (Vaccarino et al.[1]). The two others rely on absolute differences between rates in determining whether disparities are increasing or decreasing.[2,3] Both AHRQ and NCHS approve of presenting disparities in absolute, as well as relative terms, but give priority to the latter. The use of absolute differences avoids the above-described problem that relative differences between favorable outcomes and between adverse outcomes tend to change systematically in opposite directions as the prevalence of an outcome changes, since the absolute difference is the same regardless of whether one examines the adverse or the favorable outcome. But reliance on absolute difference to measure health disparities is nevertheless problematic because, like relative differences, absolute differences between rates of experiencing (or avoiding) an outcome tend to change solely as a consequence of changes in prevalence of the outcome.

In terms of favorable outcomes like receipt of beneficial healthcare procedures, the absolute difference will be small at the point where few people from either group experience the outcome, will grow larger as the favorable outcome becomes more common, and then will grow small again as the favorable outcome become almost universal. In situations where the distributions are perfectly normal, the maximum value for the absolute difference will coincide with the point where (1) the ratio of the advantaged group’s rate of experiencing the outcome to the disadvantaged group’s rate of experiencing the outcome equals (2) the ratio of the disadvantaged group’s rate of avoiding the outcome to the advantaged group’s rate of avoiding the outcome. The pattern will be similar with near normal distributions, as in the case of black and white income distributions. One will observe such pattern in table 1 and figures 1,2, and 4 of reference 4 (though, because there the black rates are used as the numerator in both ratios, the maximum value for the absolute difference will occur at the point where the relative differences in the favorable and the adverse outcomes are the reciprocals of one another).

Thus, typically, as a favorable outcomes increase, absolute differences will increase in prevalence ranges where ratio (1) is greater than ratio (2), but decline in areas where ratio (2) is greater than ratio (1). The point of intersection of ratios (1) and (2) will vary depending on the difference between the risk distributions of the two groups, but should always occur at a place where more than 50 percent of the advantaged group experiences the favorable outcome. As an example, in circumstances where two groups differ in test scores by one half of a standard deviation, serially lowering a cutoff (and, hence, increasing the prevalence of the favorable outcome) from the point where no one passes will cause the absolute difference between rates of the higher- and lower-scoring groups to reach a high at the point where approximately 60 percent of the higher-scoring group passes the test (and where the decreasing relative difference in pass rates approximates the increasing relative difference in failure rates).[5] In the case of black and whites differences in rates of falling above (or below) various percentages of the poverty line, the absolute difference will reach a high at the point where approximately 63 percent of whites fall above a ratio of the poverty line (250 percent of the poverty line).[4] So, keeping in mind that actual distributions will depart somewhat from models based on perfectly normal and near normal distributions and that, moreover, we cannot directly observe the distributions, we can generally expect that an increase in prevalence that does not bring the advantaged group’s rate above 50 percent will tend to result in an increase in the absolute difference between the rates of the advantaged and disadvantaged groups, while an increase in prevalence where the advantaged group’s rate is above 75 percent will tend to result in a decrease in that difference. Between those levels, identification of the expected direction of the change in absolute difference may be more difficult (particularly since the period being examined may involve situations where a threshold is crossed and an initial increase in the absolute difference may change to a decline in the absolute difference), though comparisons between the size of the relative differences in experiencing the outcome and relative differences in avoiding the outcome may be helpful. In the case of health outcomes like mortality the matter is usually not complicated, because the favorable outcome rates tend to be in ranges where further increases generally lead to declining absolute differences. But the issue can be more complicated in the case of healthcare disparities. Against that background, let us first examine the two studies that relied on absolute differences.

Jha et al. relied on such differences in examining trends in racial disparities in the use of major procedures among the elderly between 1991 and 2001. The study found that nationally for 9 procedures the differences were usually increasing. Focusing primarily on men, the authors found significant increases in absolute differences for 5 procedures, a significant decrease for one procedure, and no significant change for 3 procedures, noting that results were similar for women. Examining rates of total hip replacement, carotid endarterectomy, and coronary-artery bypass grafting in 158 hospital referral regions, the study found that absolute differences widened significantly in 42 regions and narrowed significantly in 22 regions, while not changing significantly in the others.

The predominance of increasing absolute differences in both the national and regional analyses, however, were pretty much what one would expect given the setting. Table 2 of the study showed 10 significant changes in absolute differences for procedures by gender, including 8 increases in the absolute difference and 2 declines. All 8 increases involved procedures that were increasing in frequency. But, while the authors reasonably characterized the procedures as “common,” the rates were in a range where an increase in prevalence tends to be accompanied by an increase in the absolute difference ¬– white final rates ranging from 0.6% (appendectomy for men) to 6.6% (total knee replacement for women) – with the relative difference between rates of receiving the procedures being much higher than the relative difference between rates of failing to receive the procedures. Given that one would expect increases in the absolute differences in these circumstances, there is no reason to regard the observed increases as reflecting some meaningful worsening of the disparity. However, that the relative disparity between rates of receipt increased for 4 of the 8 procedures, being contrary to the norm in such circumstances, might be cautiously regarded as a meaningful worsening of the disparity.

(Of the two significant declines in absolute differences, one (repair of abdominal aortic aneurism for men), involved a situation where the overall prevalence was declining in circumstances (white rate 2.1%) where a decline in prevalence would ordinarily lead to a decline in the absolute difference (though the decline in relative rates of failure to receive the procedure suggests a meaningful reduction in disparity). The other involved a situation where a narrow margin favoring white women changed to a narrow margin favoring black women.)

As to the changes in absolute differences between rates of three procedures in hospital regions, suffice it to say that the procedures being examined (total hip replacement, carotid endarterectomy and coronary artery bypass grafting) all involved rate ranges where the increases in prevalence would typically be accompanied by increasing absolute differences. The study did not provide figures that would allow more in-depth analysis.

Thus, both as to the national and the regional analyses, whether or not close examination of the data might allow one to draw some useful inferences about changes in disparities, in the main the study’s findings were simply what one would expect in circumstances where procedures of relatively limited prevalence were increasing.

The article by Trivedi et al. provides an interesting contrast to that by Jha et al. The Trivedi study also relied on absolute differences in appraising changes between 1997 and 2003 in racial disparities in outcomes that were generally increasing, specifically, receipt of 9 types of adequate care among Medicare beneficiaries. The study found absolute differences between rates to be decreasing in 7 of the 9 measures of adequate care. The frequency rates involved, however, were much higher than those in the Jha study ¬– white final rates ranging from 73% (control of LDL cholesterol as part of cardiovascular care) to 94% (receipt of beta blocker) – and hence in the range where increasing prevalence typically is associated with a decline in the absolute difference. In accordance with the usual pattern where an outcome is increasing in these ranges, the relative difference in receiving adequate care decreased in 8 of 9 cases and the relative difference in rates of failing to receive adequate care increased in 6 of 9 cases. In a letter to the editor, the NCHS statisticians who authored the NCHS policy on measuring disparities in terms of relative differences in adverse outcomes cited the last pattern as a reason why the disparities should be regarded as increasing [13] (to which Trivedi et al. replied by pointing out that the disparities would usually be regarded as declining if one examined the relative differences between rates of receiving adequate care[14]). But while one might draw some inferences about meaningful changes on the on the basis of certain departures from expected patterns, the change in absolute differences relied on by the authors, as well as the other measures the authors might have chosen, were largely what one would expect simply because rates of adequate care in these 9 areas were increasing.

Since increasing attention to quality of care is likely to both increase rates of adequate and result in adequate care rates in ranges where further increases typically lead to declining absolute differences, the findings in the Trivedi study might be expected to be the norm in analyses of absolute differences between rates of adequate care. Seghal had previously relied on absolute differences in examining changes in racial and gender disparities in rates of receiving adequate hemodialysis dose during a period (1993-2000) when rates of adequate dose were rising dramatically (from 46% to 87% for whites and from 36% to 84% for blacks; from 54% to 91% for women and from 31% to 82% for men)).[15] Like Trivedi et al., Seghal found that disparities generally declined over the period. The rates in the latter years, of course, were in the range where overall increases would be expected to reduce absolute disparities. As would be expected as well, the relative difference in rates of receiving adequate doses declined, while the relative difference in rates of inadequate doses increased.

As noted above, only Vaccarino et al.[1] relied on relative differences, examining such differences in rates of receipt of a therapy, the standard approach to measuring disparities in healthcare. The study examined gender and racial difference in the management of acute myocardial infarction from 1994 through 2002, and found that, apart from receipt of aspirin and beta-blockers, disparities persisted over time with no evidence that the difference narrowed. The patterns in the Vaccarino study are not easy to summarize and it would take some space to sort out the way the changes patterns comport with, or vary from, the expected patterns in the circumstances. The main thing to be said about the study, however, is that the prevalence rates of the procedures examined were simply not changing very much. And when prevalence does not change much, measures of disparity tend also not to change much. Notably, in lamenting the lack of progress in the reduction of disparities in recent years, the authors contrasted their findings with the declines in disparities between blacks and whites in the use of coronary angiography and revascularization procedures between 1987 and 1997 that had been observed in a study by Escarce and McGuire.[16] In large part, however, that study found declining relative differences with respect to rates of receiving procedures that were substantially increasing. For that reason alone, it is understandable that Vaccarino et al. would not observe a similar pattern of narrowing relative differences.

With respect to all of the points above, it must be remembered that what I term the expected patterns are merely tendencies. Departures from the patterns do not disprove the tendencies, but merely mean that there may be irregularities in a particular distribution (in which case the tendencies may not be operating at all) or that there may in fact be meaningful changes in the relative situations of two groups (which means that the tendencies may be operating but are outweighed by other factors). At any rate the tendencies are sufficiently widespread and influential that it makes no sense to analyze changes in disparities without consideration of the role of the tendencies. At the same time, however, possible irregularities in the distributions, and several other factors, make interpreting patterns of change while taking the tendencies into account problematic as well.[4-6]. Hence, attributing meaning to departures from expected patterns must be done warily.

Finally, I note that the careful reader may be troubled by the references to prevalence of an outcome, since prevalence is a function of the rates of each group within a population and the proportion each group makes up of the total. Nevertheless, I think it is clear enough what is meant above by a change in prevalence – i.e., a change akin to that observed when a test cutoff is lowered or poverty declines – and reasonable to speak broadly of the implications of changes in overall prevalence.

References:

1. Vaccarino V, Rathore SS, Wenger NK, et al. Sex and racial differences in the management of acute myocardial infarction, 1994 through 2002. N Engl J Med 2005;353:671-682.

2. Jha AK, Fisher ES, Li Z, Orav EJ, Epstein AM. Racial trends in the use of major procedures among the elderly. N Engl J Med 2005;353:683-691.

3. Trivedi AN, Zaslavsky AM, Schneider EC, Ayanian JZ. Trends in the quality of care and racial disparities in Medicare managed care. N Engl J Med 2005;353:692-700.

4. Scanlan JP. Can we actually measure health disparities? Chance 2006:19(2):47-51: http://www.jpscanlan.com/images/Can_We_Actual...

5. Scanlan JP. The misinterpretation of health inequalities in the United Kingdom. Paper presented at: British Society for Population Studies Annual Conference 2006, Southampton, England, Sept. 18-20, 2006: http://www.jpscanlan.com/images/BSPS_2006_Com...

6. Scanlan JP. Race and Mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mort...

7. Scanlan JP. Divining difference. Chance1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Differen...

8. Agency for Healthcare Research and Quality. 2005 National Heathcare Disparities Report. .

9. Agency for Healthcare Research and Quality, 2006 National Healthcare Disparities Report.

10. Keppel KG, Pearcy JN, Klein RJ. Measuring progress in Healthy People 2010. Healthy People statistical notes. No. 25. Hyattsville, Md.: National Center for Health Statistics: http://www.cdc.gov/nchs/data/statnt/statnt25....

11. Keppel KG, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital and health statistics. Series 2. No. 141. Washington, D.C.: Government Printing Office, 2005. (DHHS publication no. (PHS) 2005-1341.): http://www.cdc.gov/nchs/data/series/sr_02/sr0...

12. Keppel, KG, Pearcy JN. Measuring health disparities in terms of adverse outcomes. J Public Health Management Practice. 2005:11(6): 479-83

13. Keppel kg, Pearcy JN, Weissman JS. Unitled letter. N Engl J Med 2005;353:2082-2083: http://content.nejm.org/cgi/content/full/353/...

14. Trivedi AN, Zaslavsky AM, Ayanian JZ . Unitled letter. N Engl J Med 2005;353:2085: http://content.nejm.org/cgi/content/full/353/...

15. Sehgal AR. Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA 2003;289:996-1000.

16. Escarce JJ, McGuire TG. Changes in racial differences in use of medical procedures and diagnostic test among elderly persons: 1986-1997. Am J Public Health 2004:94:1795-1799.