There is no empirical evidence on the sensitivity and specificity of methods to identify the possible overuse and underuse of medical procedures. To estimate the sensitivity and specificity of the RAND/UCLA Appropriateness Method. Parallel three-way replication of the RAND/UCLA Appropriateness Method for each of two procedures, coronary revascularization and hysterectomy. Maximum likelihood estimates of the sensitivity and specificity of the method for each procedure. These values were then used to re-calculate past estimates of overuse and underuse, correcting for the error rate in the appropriateness method. The sensitivity of detecting overuse of coronary revascularization was 68% (95% confidence interval 60-76%) and the specificity was 99% (98-100%). The corresponding values for hysterectomy were 89% (85-94%) and 86% (83-89%). The sensitivity and specificity of detecting the underuse of coronary revascularization were 94% (92-95%) and 97% (96-98%), respectively. Past applications of the appropriateness method have overestimated the prevalence of the overuse of hysterectomy, underestimated the prevalence of the overuse of the coronary revascularization, and provided true estimates of the underuse of revascularization. The sensitivity and specificity of the RAND/UCLA Appropriateness Method vary according to the procedure assessed and appear to estimate the underuse of procedures more accurately than their overuse.