ROC Curve Calculator

How to Use the ROC Curve Calculator

1. 1. Paste the data. Either two columns — the test score, then each subject's true class (1/0, Disease/Healthy, anything with exactly two labels) — or one list of scores for the positives and another for the negatives.
2. 2. Set the direction. Tell the calculator whether a higher score means more likely positive. An AUC below 0.5 is the sign that the direction is the wrong way round.
3. 3. Pick the cut-off rule. Youden's J finds the score that maximises sensitivity + specificity − 1; the other option picks the point nearest the top-left corner. You can also enter a cut-off of your own to see how it performs.
4. 4. Read and download. The AUC comes with DeLong and Hanley–McNeil intervals, and the page lists every point on the curve. Download the plot, or the points as a CSV.

What the ROC Curve and the AUC Show

A continuous test turns into a yes/no result only once you choose a cut-off. The ROC curve shows every choice at once: for each possible cut-off it plots the true positive rate (sensitivity) against the false positive rate (1 − specificity). A useless test follows the diagonal; a perfect one reaches the top-left corner. The area under the curve summarises the whole curve in one number: it is the probability that a randomly chosen case scores higher than a randomly chosen non-case, counting ties as one half (Hanley and McNeil, 1982). That is exactly the Mann–Whitney statistic divided by m × n, so AUC = U / (m × n).

A common rule of thumb from Hosmer and Lemeshow (2000) calls 0.7–0.8 acceptable, 0.8–0.9 excellent and above 0.9 outstanding discrimination. Treat the labels loosely: what is acceptable depends on the condition, on what the test replaces, and on the confidence interval, not on the point estimate.

Confidence Intervals for the AUC

The default interval is DeLong's (DeLong, DeLong and Clarke-Pearson, 1988). Each subject gets a placement value — the share of the opposite class it outranks — and the variance of the AUC is S₁₀/m + S₀₁/n, built from the variances of those values in the two classes. The interval is AUC ± 1.96 × SE, held inside 0 to 1, and it is the estimator R's pROC uses. The older Hanley–McNeil formula assumes an exponential score model and is shown for comparison; the two normally agree closely. The p-value tests AUC = 0.5 with the Mann–Whitney normal approximation, tie-corrected.

Choosing a Cut-Off

Youden's J = sensitivity + specificity − 1 is the vertical distance from the curve up to the diagonal; its maximum picks the cut-off that weights false positives and false negatives equally. The closest-to-top-left rule minimises the straight-line distance to a perfect test. Both ignore the real cost of each kind of error and the prevalence in the population where the test will be used, and a cut-off tuned on the same data that measures it looks better than it will perform on new patients — validate it on fresh data. Where the cut-off sits between scores, the page reports the observed score that starts the positive range.

Worked Example: 24 Cases and 31 Non-Cases

Load example fills in a test score for 24 people with a condition and 31 without; scores are rounded to one decimal, so there are ties.

The AUC of 0.810 is excellent on the rule of thumb, but the interval reaches down to 0.69, so "acceptable" cannot be ruled out. At the best cut-off, 19 of 24 cases and 7 of 31 non-cases test positive (J = 0.566). Moving the cut-off up to 1.0 lifts specificity to 80.6% but drops sensitivity to 54.2%.

• Reporting the AUC without an interval. With a few dozen subjects the interval spans a full band of the rule of thumb.
• Choosing the cut-off and testing it on the same data. The reported sensitivity and specificity are optimistic.
• Quoting predictive values from a case-control sample. PPV and NPV depend on prevalence; a sample with 24 cases in 55 has a prevalence of its own.
• Reading the AUC as accuracy at a cut-off. It summarises ranking across all cut-offs; a useful test can still have an AUC of 0.75.

Frequently Asked Questions

What is a good AUC?

An AUC of 0.5 means no discrimination and 1 means perfect. Hosmer and Lemeshow's rule of thumb calls 0.7 to 0.8 acceptable, 0.8 to 0.9 excellent and above 0.9 outstanding, but the interval matters more than the label, and clinical usefulness depends on what the test is for.

What does the AUC actually measure?

It is the probability that a randomly chosen positive subject has a higher score than a randomly chosen negative one, counting equal scores as half. That is why it equals the Mann–Whitney U statistic divided by the number of case–non-case pairs.

How is the confidence interval for the AUC calculated?

The calculator uses DeLong's method: it computes a placement value for every subject, estimates the variance of the AUC from them, and forms AUC ± 1.96 SE, truncated to the range 0 to 1. The Hanley–McNeil interval, which assumes an exponential model for the scores, is shown beside it.

How do I find the optimal cut-off?

Youden's J picks the score that maximises sensitivity plus specificity minus one, giving false positives and false negatives equal weight. If one error is costlier — missing a cancer versus an unnecessary biopsy — choose a cut-off from the table of points that reflects that, and confirm it on new data.

Why is my AUC below 0.5?

Usually because the direction is reversed: lower scores mark the positive group. Switch the setting to "Less likely positive" and the AUC becomes 1 minus the value shown. An AUC well below 0.5 that persists after the switch points to a labelling mistake.

Is the ROC curve suitable when positives are rare?

It still describes sensitivity and specificity correctly, but because it ignores prevalence it can look flattering when positives are very rare. A precision–recall curve, which shows positive predictive value, is often more informative then, and PPV should be calculated at the real prevalence.