The Attribute Agreement Analysis is used to assess the accuracy of subjective ratings by people. In general, it is more likely that subjective ratings are accurate and useful if there is substantial agreement in measurements among appraisers.
1. Choose ProcessMA > Quality Tools > Attribute Agreement Analysis.
2. In Rating, select the column containing the measurement data.
3. In Samples, select the column containing the sample indicators.
4. In Appraisers, select the column containing the appraiser indicators.
5. Click OK.
Optional
6. In Known standard, select the column containing the known standard or master value for each sample.
7. Check Attribute data is ordered, if your measurement data have more than two levels and are ordinal.
8. Check Show Kappa and Kendall coef, if you want to display the kappa coefficient tables and Kendall's coefficient tables.
Note To select a column of data into a textbox, double-click on any of the column names shown in the list on the left of the dialog box while in the textbox.
Rating: Text or Numeric.
Samples: Text or Numeric; Must contain equal number of data points as the Rating.
Appraisers: Text or Numeric; Must contain equal number of data points as the Rating.
Known standard: Text or Numeric.
You work in a garment factory and you have just trained 4 new quality controllers. The quality controllers need to determine is the garments are ‘up-to-standard’. You want to assess if these new quality controllers are ready for the job. You asked each quality controller to give their ratings on 10 garments on a five-point scale (-2, -1, 0, 1, 2).
1. Open worksheet ProcessMA > Tools > Data Files > Gage.xls.
2. Choose ProcessMA > Quality Tools > Attribute Agreement Analysis.
3. In Rating, select M – Rating.
4. In Samples, select L – Garment.
5. In Appraisers, select K – Controller.
6. In Known standard, select N – Standard.
7. Check Attribute data is ordered.
8. Uncheck Show Kappa and Kendall coef.
9. Click OK.
Interpretation
The Each Appraiser VS Standard assessment agreement table shows that Frances was only able to matched 5 out of the 10 assessments while Mary was able to match all of them. The confidence interval of %Matched is shown in the table and also plotted as a chart. Based on this study, you conclude than Frances is in most need of more training.