Bland-Altman Analysis

 

The Bland-Altman analysis is not a statistical test measured with a p-value. Instead, it is a process used to assess agreement between two methods of measurement. (Bland and Altman 1986, 2003)  An important requirement of the Bland-Altman method for measuring agreement is that the two methods for measuring the same characteristic use the same scale of measurement.  This implies that when plotted, the points should line up along the line y = x (line of identity).  

It is possible for two measures to have strong linear agreement using a Pearson’s correlation (r) when they are not measuring the same quantity because a correlation analysis does not require that the two measurements be on the same scale or to even be measurements of the same characteristic.

 

The analysis is based on examination of two plots:

·       Plot of identity:  A scatterplot of the two measurements along with the line y = x. If the measurements are in basic agreement then the points in the scatterplot will line up closely to the line y = x.  

·       Bland-Altman plot: A scatterplot of  variable means plotted on the horizontal axis and the differences plotted on the vertical axis which shows the amount of disagreement between the two measures (via the differences) and lets you see how this disagreement relates to the magnitude of the measurements.  This plot includes approximate 95% limits (based on an assumption of normal differences).  If differences observed in this plot are not deemed scientifically (or clinical) important (according to the researcher’s own expertise), this is a confirmation of agreement. (The decision as to what constitutes a clinically important difference should be made in advance of the analysis.)

 

Example

            This example uses data from Bland and Altman (1986): two measurements of peak expiratory flow rate (PEFR) are compared.  One of these measurements uses a “Large” meter and the other a “Mini” meter. The research question of interest is whether these two techniques are measuring the same phenomenon.  The first way to examine the two measure is with A plot of identity. 

 

To get the file used in the analysis

 

http://www.stattutorials.com/SASDATA/Bland_Altman.sas

 

and SAVE the file to the C:\SASDATA directory

 

For this data, the plot is shown here:

 

Note that the line drawn here is the line y = x, not the regression line.   If the scatter of points in this plot lies near the line, it indicates that the two ways of measuring PEFR are similar.  From this plot you can see that the scatterplot falls close to the line which suggests that the two machines are measuring the same characteristic. The SAS code to produce this graph follows.

 

Step 1: Calculate the differences and averages. (Actually the differences are used in the plot of identity above) using this code:

 

DATA BA;

input large mini;

diff=large-mini;

avg=(large+mini)/2;

datalines;
… data follow…

 

Step 2: Specify values for the x and y axes for the plot, and this is done with macro variables in the following code:

 

   %let miny=-100;%let maxy=100;%let ticky=20;

   %let minx=100;%let maxx=700;%let tickx=50;

 

The “tick” variable specifies how often ticks will appear on the axis. For the X axis the minimum and maximum should reflect values that fully encompass the values of the original data (mini and large in this case.)

 

Step 3: Create necessary information to produce the mean line at 0 using the information in the “anno” data set shown below:

 

data anno;

   function='move'; xsys='1'; ysys='1'; x=0; y=0; output;

   function='draw'; xsys='1'; ysys='1'; color='red'; x=100; y=100; size=2;

   output;

   run;

 

Step 4: The plot of identity is produced using this code (using the mean line information from the anno dataset.)

 

   symbol1 i=none v=dot c=black height=1;

   * X AXIS - HORIZONTAL;

         axis1 length=6.5 in width=1 

         value=(font="Arial" h=1) order=&minx to &maxx by &tickx;

   * YAXIS - VERTICAL;

         axis2 length=4.5 in width=1

         value=(font="Arial" h=1) order=&minx to &maxx by &tickx;

   footnote ;

   proc gplot data=PERF;

   plot large*mini / anno=anno haxis=axis1 vaxis=axis2;

   run;

 

            The second plot is a little more difficult to come up with. It is a scatterplot of the average of the two measurements against the difference (large minus mini) between measurements for each subject. This plot is a visual check that the magnitudes of the differences are essentially constant throughout the range of measurement. If the differences are approximately normally distributed, then you would expect about 5% of the points to lie outside the limit lines.  This plot is displayed here:

Notice that the 95% limit lines are at about plus or minus 75 for the differences between readings, and the graph shows a few differences outside or close to the limit lines, i.e. differences in the neighborhood of ±75  (the actual limits are -75.4 and 79.6).   Bland and Altman (1986) argue that such large differences are clinically important (a decision not based on any p-value) and therefore conclude that the two devices do not show sufficient agreement to be used interchangeably.  It is important to note that this lack of agreement is not apparent in the plot of identity. The following steps are used to calculate the values needed for this plot.

 

Step 1: Using the BA data set created above,  calculate the mean and standard deviation of the “diff” value and output that information to a file (named diff in this case.) Merge the mean and std into the original data set (name the combined data set PERF) and use that information to calculate the upper 95% bound (ubound) and lower bound (lbound) using the following SAS code:

 

data prelim;set ba;

proc means noprint data=prelim;var diff;

output out=diff mean=meandiff std=stddiff;

run;

data PERF;set prelim;

if _N_=1 then set diff;

ubound=meandiff+stddiff*2;

lbound=meandiff-stddiff*2;

run;

 

The PERF data set contains the following information shown in the PERF data set below. Notice the meandiff and std columns were from the PROC MEANS and the ubound and lbound were calculated using the code above.

 

 

Step 2: Specify the limits for the axes using macro variables:

 

 * AXIS LIMITS FOR GRAPHS;

   %let miny=-100;%let maxy=100;%let ticky=20;

 

This information is something you must observe from the data – note that the minimum and maximum for the Y axis should be the same magnitude (but one negative) to create a balanced plot.

 

Step 3: The next set of code creates a data set named “b” that contains values that are used to draw the mean line and the two limits (ubound and lbound) on the plot.

 

   data b;set diff;

   Maxaxis=&maxy;

   meanline=50+(meandiff/(maxaxis*2))*100;

   upper = 50+((meandiff+2*stddiff)/(maxaxis*2))*100;

   up=(meandiff+2*stddiff);

   lower = 50+((meandiff-2*stddiff)/(maxaxis*2))*100;

   length color function $8 text $14;

   retain xsys '1' ysys '2' when 'a'; 

   /* The lines are positioned using the percentage of the horizontal axis,*/ 

   * --this is the mean value for the y axis;

   function='move'; xsys='1'; ysys='1';x=0; y=meanline; output;

   function='draw'; xsys='1'; ysys='1';x=100; y=meanline; color='red'; size=2;

   output;

   * Upper confidence limit ------------- ;

   function='move'; xsys='1'; ysys='1'; x=0; y=upper; output;

   function='draw'; xsys='1'; ysys='1'; color='green'; x=100; y=upper; size=1;

   output;

   * Lower confidence limit ------------- ;

   *data Lower;

   function='move'; xsys='1'; ysys='1'; x=0; y=lower; output;

   function='draw'; xsys='1'; ysys='1'; color='green'; x=100; y=lower; size=1;

   output;

   run;

 

Step 4: The Bland –Altman graph is created using the following code. The “symbol” statement specifies the type of dots, color and size of dots. The “axis1” paragraph specifies information about the X-axis and the “axis2” paragraph specifies information about the Y-axis.In particular, the numbers used for each are given by the “order” statements. The plot is produced with the PROC GPLOT statement, which uses the information from the “b” data set to annotate (anno=b) the plot with the three lines calculated above.

 

   symbol1 i=none v=dot c=black height=1;

   * XAXIS - HORIZONTAL - AVERAGE;

   axis1 length=6.5 in width=1 OFFSET=(0,0)

         label=(f="Arial" h=2 'Average')

         value=(font="Arial" h=1)

         order=&minx to &maxx by &tickx;

   * YAXIS - Vertical - Difference;

   axis2 length=4.5 in width=1 OFFSET=(0,0)

         label=(f="Arial" h=2 'Difference')

         value=(font="Arial" h=1)

         order=&miny to &maxy by &ticky;

   footnote h=2 f="Arial" 'Bland-Altman Plot';

   title h=2 f="Arial" 'Example';

   proc gplot data=PERF;

   plot diff*avg/ anno=b haxis=axis1 vaxis=axis2;

   run;

   quit;

 

Reporting the results of a Bland-Altman analysis

The following example illustrates how you might report the results of a Bland-Altman analysis in publication format.

 

Narrative for methods section:

“A Bland-Altman assessment for agreement was used to compare the two peak flow methods.  A range of agreement was defined as mean bias ±2 SD.”

Or

“A Bland-Altman analysis was used to assess the level of agreement between the two methods to compare the new technique to the established one.”

 

Narrative for results section:

 

“The Bland-Altman analysis indicates that the 95% limits of agreement between the two methods ranged from -75.4 to 79.6.  The two methods do not consistently provide similar measures because there is a level of disagreement that includes clinically important discrepancies of up to 80 l/min.”

 

References

•  Barnett, V. and Lewis, T. (1994).  Outliers in Statistical Data, 3^rd edition. New York: John Wiley and Sons.
• Bland JM and Altman DG. (1986).Statistical methods for assessing agreement between two methods of clinical measurement, Lancet, February, pp 307-10.
• Bland JM and Altman DG. (2003). Applying the right statistics: analyses of measurement studies, Untrasound Obstet Gynecol 2003;22: 85-93.
• Elliott, AC, Woodward, WA (2007), Statistical Analysis Quick Reference Guidebook, Sag, pp 107ff.
• Kleinbaum DG, Kupper LL, Muller KE, Nizam A. (1997). Applied Regression Analysis and Other Multivariate Methods (3^rd Edition), North Scituate, MA: Duxbury Press.
• Mantel, N. (1970).  Why stepdown procedures in variable selection,” Technometrics 12, 621-625.
• SAS Institute Inc. (2003). SAS/STAT Software: Reference, Version 9.1, Cary, NC: SAS Institute Inc.

 

For more information... we recommend:

SAS Essentials: Mastering SAS for Research - provides an introduction to SAS statistical software, the premiere statistical data analysis tool for scientific research. Through its straightforward approach, the text presents SAS with step-by-step examples. SAS Essentials introduces a step-by-step approach to mastering SAS software for statistical data analysis. It's also a valuable reference tool for any researcher currently using SAS. Designed for those new to SAS and filled with illustrative examples, the book shows how to read, write and import data; prepare data for analysis; use SAS procedures; evaluate quantitative data; analyze counts and crosstabulation tables; and compare means using the t-test. The book also provides instruction and examples on analysis of variance, correlation and regression, nonparametric analysis, logistic regression, creating graphs, controlling outputs using ODS, as well as advanced topics in SAS programming.ISBN: 0470461292. Order from publisher -Jossey-Bass/Wiley. Barnes & Noble. Amazon.

• WINKS -- a simple to use and affordable statistical software program that will help you analyze, interpret and write-up your results. Download a free trial copy.
   
• Against All Odds VIDEOS - Now in DVD format -- Teaching Videos from Annenberg/PBS --"This highly engaging primer on statistical methods and inference introduces the practical applications of statistics. Produced by the Consortium for Mathematics and Its Applications and Chedd-Angier." Click here for info
   
• BeSmartNotes Reference sheets for SAS, SAS ODS, SAS Functions, SPSS and WINKS  - Click here for info. (www.besmartnotes.com)
   
• Statistical Analysis Quick Reference Guidebook: With SPSS Examples is a practical "cut to the chase" handbook that quickly explains the when, where, and how of statistical data analysis as it is used for real-world decision-making in a wide variety of disciplines. It contains examples using SPSS Statistics software. In this one-stop reference, the authors  provide succinct guidelines for performing an analysis, avoiding pitfalls, interpreting results, and reporting outcomes. Paperback. Sage Publishers ISBN: 1412925606 Order book from Amazon