<jats:sec><jats:title>Objectives</jats:title><jats:p>The three objectives of this study were to determine the positive predictive value (PPV) of oestrus diagnosis (heat detection accuracy) by dairy farmers, calculate the diagnostic sensitivity and specificity of the <jats:styled-content style="fixed-case">P4</jats:styled-content> Rapid milk progesterone assay for detecting a corpus luteum and evaluate the economics of using a cow‐side milk progesterone assay designed to aid oestrus diagnosis.</jats:p></jats:sec><jats:sec><jats:title>Methods</jats:title><jats:p>Milk samples were collected from 752 cows diagnosed in oestrus by farm personnel on 14 dairy farms. Samples were tested using the <jats:styled-content style="fixed-case">P4</jats:styled-content> Rapid milk progesterone assay to estimate the PPV of oestrus diagnosis at each farm and a crude pooled mean of <jats:styled-content style="fixed-case">PPV</jats:styled-content> of oestrus diagnosis across all farms. A further 156 milk samples were collected from cows with luteal tissue status determined by transrectal ultrasound and tested by the <jats:styled-content style="fixed-case">P4</jats:styled-content> Rapid assay to enable calculation of the sensitivity and specificity of the <jats:styled-content style="fixed-case">P4</jats:styled-content> Rapid assay.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>For pooled farm samples, the <jats:styled-content style="fixed-case">PPV</jats:styled-content> was 97.0%, with a range between farms of 88.9–100%. Sensitivity of the <jats:styled-content style="fixed-case">P4</jats:styled-content> Rapid milk progesterone assay for detecting a corpus luteum was 90.1% and specificity was 98.7%. Misclassification of oestrus in cows previously identified as pregnant was the most common cause of false‐positive oestrus diagnoses by farm personnel.</jats:p></jats:sec><jats:sec><jats:title>Conclusion</jats:title><jats:p>Routine testing of milk progesterone in all cows diagnosed in oestrus is not economically justified and may even slightly reduce submission rates; conversely, strategic use of cow‐side milk progesterone assays can improve herd reproductive performance by facilitating decisions on whether to rebreed cows previously diagnosed as pregnant.</jats:p></jats:sec>