Clarifying various terms for evaluating classifier (or hypothesis testing) performance
                                 by Alex Hartemink
                                 amink@cs.duke.edu



true positives:  TP
true negatives:  TN
false positives: FP (type I error)
false negatives: FN (type II error)

measures whose denominator involves the entire sample:

   prevalence = (TP+FN)/(TP+TN+FP+FN)
   error rate = (FP+FN)/(TP+TN+FP+FN)
   accuracy   = (TP+TN)/(TP+TN+FP+FN)

      prevalence measures the proportion of cases that are positive
         and is thus independent of the classifier; the prevalence 
         of negative cases could also be defined analogously
      accuracy is also called efficiency
      note that error rate + accuracy = 1

measures whose denominator involves only the positive cases:

   true positive rate  = TP/(TP+FN)
   false negative rate = FN/(TP+FN)

      true positive rate is also called recall, recall ratio, sensitivity
      note that true positive rate + false negative rate = 1

measures whose denominator involves only the negative cases:

   true negative rate  = TN/(TN+FP)
   false positive rate = FP/(TN+FP)

      true negative rate is also called specificity
      note that true negative rate + false positive rate = 1

measures whose denominator involves only the positive predictions:

   positive predictive value = TP/(TP+FP)
   false discovery rate = FP/(TP+FP)

      positive predictive value is also called precision, precision ratio
      note that positive predictive value + false discovery rate = 1

measures whose denominator involves only the negative predictions:

   negative predictive value = TN/(TN+FN)

finally, an ROC curve is a plot of true positive rate (or recall or sensitivity or (1-false negative rate)) 
                           versus false positive rate (or (1-true negative rate) or (1-specificity))
                           as the threshold for making positive/negative calls changes