Opening your learning space
Starting the app …
Opening your learning space
Starting the app …
Cohen’s kappa measures agreement between two raters on categorical data and corrects for agreement expected by chance.
Free to browse · flashcards or a multiple-choice quiz · import from Essential
Formula
κ, Po and Pe are dimensionless proportions from 0 to 1
| Symbol | Meaning | Unit |
|---|---|---|
| Chance-corrected agreement statistic | dimensionless | |
| Observed fraction of matching ratings | dimensionless fraction | |
| Expected chance-agreement fraction from the marginal category frequencies | dimensionless fraction |
\kappa=\frac{P_o-P_e}{1-P_e}Kappa compares observed agreement with the agreement expected from the marginal category frequencies. It is classically used for two raters and nominal categories; weighted kappa handles ordered categories separately. κ is sensitive to prevalence and marginal distributions, so reports should include raw agreement, categories and sample. For continuous measurements, the intraclass correlation is a different model.
Two raters independently assign the same units to nominal categories.
Observed agreement is corrected for the chance component expected from marginal frequencies.
Observed agreement
Interpret kappa alongside the margins.
Given Po=0.80 and Pe=0.50, calculate κ.
κ=(0.80−0.50)/(1−0.50)=0.60.
Reading kappa as a percentage of correct ratings.
κ is chance-corrected and can also be negative.
Formula cluster
Po=Pe gives κ=0; perfect observed agreement gives κ=1 when Pe<1.
Two raters agree on Po=0.80 of cases; their marginals imply Pe=0.50 chance agreement. Then κ=(0.80−0.50)/(1−0.50)=0.60. Check: 0.30/0.50=0.60. Kappa is not a percentage and should not be read as a universal quality grade independently of category frequencies.
Assess reliability of categorical coding and diagnostic classification by two raters
Here is how to work through a typical Cohen's Kappa for Categorical Agreement (κ = (Po−Pe)/(1−Pe)) task step by step:
Task
Given Po=0.80 and Pe=0.50, calculate κ.
Solution path
κ=(0.80−0.50)/(1−0.50)=0.60.
agreement between two raters on categorical judgments after chance correction. Its output is κ as a dimensionless metric based on observed and expected agreement.. Po=Pe gives κ=0; perfect observed agreement gives κ=1 when Pe<1. The quantity answers a specific biological or statistical question, not automatically the full cause of an observed pattern. Tie it to the population, sample or model level from which inputs came. A number without that reference can be misleading across studies. Keep description separate from explanation: the metric summarizes data but does not by itself establish the underlying mechanism.
Use Po is observed matching and Pe is chance agreement expected from the margins. The units are both inputs are proportions from zero to one; κ is dimensionless.. Check that numerator and denominator refer to the same time interval, target population and scale. Proportions entered as percentages usually need conversion to values between zero and one. Write units and definitions beside the observations. This helps catch swapped groups, mismatched age intervals or a wrong denominator before they distort the result. Also document measurement definitions and preprocessing, because studies can operationalize the same symbols differently.
The central assumption is: two independent raters classify the same categorical cases. kappa depends on category prevalence and marginal distributions. If for continuous values, ICC is usually more appropriate than nominal kappa., interpretation changes or the formula no longer fits. State sampling, measurement method and biological scale. A model is useful because it simplifies a question, but its limits belong in the report. Do not automatically transfer parameters to different species, populations, laboratories or sampling conditions. Report these assumptions with the result so readers can judge which populations and conditions the conclusion can cover.
In the example, Po=0.80 and Pe=0.50 give κ=0.60. The check is: substitute the proportions and verify 0.30/0.50=0.60. Substitute calculated values back into the original relation and check whether ranges and units are biologically plausible. Rounded data can create a small discrepancy, but should not change direction or order of magnitude. Also test a simple limiting case, such as zero, complete detection or equal groups. This separates arithmetic errors from model limitations. Round intermediate values only at the end, and check the order of magnitude against the biological question.
A nearby quantity is the intraclass correlation. The substantive distinction is that kappa corrects categorical rater agreement; ICC assesses quantitative measurements. Both can appear in the same biological analysis, but they use different inputs and answer different questions. Choose by the requested output and verify the data definition before comparing values. A related name does not imply identical units or interchangeable interpretation. This matters when a descriptive statistic is reported alongside a model parameter. Before comparing values, align definitions, populations, observation periods and analysis rules; otherwise the difference may be methodological.
Quanta exam set
Curated exam set for "Cohen's Kappa for Categorical Agreement":
Question (front)
Which formula describes Cohen's Kappa for Categorical Agreement?
Answer in your set
Question (front)
How do you rearrange Cohen's Kappa for Categorical Agreement for Observed agreement?
Answer in your set
Question (front)
Which common mistake happens with Cohen's Kappa for Categorical Agreement?
Answer in your set
+ 7 more cards: units, variables, derivation, example, exam task
The formula set contains 10 cards. After importing, review them and start practicing; your ratings determine later review intervals.
This list names subject references and further reading. A page or section number appears when it is part of the reference; these are not verbatim quotations.
Create a curated FSRS exam set for κ = (Po−Pe)/(1−Pe): formula recall, variables, derivation, rearrangement, worked example, common mistakes and exam context.
Curated formula set · LaTeX · FSRS spaced repetition · import from Essential