Talk:Inter-rater reliability

This is the talk page for discussing improvements to the Inter-rater reliability article.
This is not a forum for general discussion of the article's subject.

Put new text under old text. Click here to start a new topic.
New to Wikipedia? Welcome! Learn to edit; get help.

Article policies

Find medical sources: Source guidelines · PubMed · Cochrane · DOAJ · Gale · OpenMD · ScienceDirect · Springer · Trip · Wiley · TWL

Guild of Copy Editors

This article was copy edited by a member of the Guild of Copy Editors.Guild of Copy EditorsWikipedia:WikiProject Guild of Copy EditorsTemplate:WikiProject Guild of Copy EditorsGuild of Copy Editors articles

Statistics Mid‑importance

	This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics articles
Mid	This article has been rated as Mid-importance on the importance scale.

Mathematics Mid‑priority

	Mathematics portal This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.MathematicsWikipedia:WikiProject MathematicsTemplate:WikiProject Mathematicsmathematics articles
Mid	This article has been rated as Mid-priority on the project's priority scale.

Psychology Start‑class Mid‑importance

	Psychology portal This article is within the scope of WikiProject Psychology, a collaborative effort to improve the coverage of Psychology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.PsychologyWikipedia:WikiProject PsychologyTemplate:WikiProject Psychologypsychology articles
Start	This article has been given a rating which conflicts with the project-independent quality rating in the banner shell. Please resolve this conflict if possible.
Mid	This article has been rated as Mid-importance on the project's importance scale.

Medicine Start‑class Mid‑importance

	Medicine portal This article is within the scope of WikiProject Medicine, which recommends that medicine-related articles follow the Manual of Style for medicine-related articles and that biomedical information in any article use high-quality medical sources. Please visit the project page for details or ask questions at Wikipedia talk:WikiProject Medicine.MedicineWikipedia:WikiProject MedicineTemplate:WikiProject Medicinemedicine articles
Start	This article has been given a rating which conflicts with the project-independent quality rating in the banner shell. Please resolve this conflict if possible.
Mid	This article has been rated as Mid-importance on the project's importance scale.

Archives

Archive 1

This page has archives. Sections older than 30 days may be automatically archived by when more than 3 sections are present.

Untitled[edit]

Is Joint Probability the same as Simple agreement? Ben 23:53, 9 March 2007 (UTC)[reply]

What are the consultation methods for asking a group of experts to reach an agreement that can be tested for inter-rater reliability? --131.170.90.3 06:21, 19 September 2007 (UTC)[reply]

Multiple Issues[edit]

I added a multiple issues cleanup tag focusing on (a) the need for neutral citations (there are citations which link to a sales page for the cited book); (b) the need for more reliable references, e.g., the first section (Inter-rater reliability#Sources of inter-rater disagreement) does not have any citations; and (c) the article lacks a logical flow (copy editing needed). If you disagree, let's discuss here first before removing the tag. - Mark D Worthen PsyD (talk) 09:53, 26 December 2018 (UTC)[reply]

And the references are a jumbled mess - they should probably be reworked from the beginning. - Mark D Worthen PsyD (talk) 10:01, 26 December 2018 (UTC)[reply]

Resolving inter-rater disagreements[edit]

The page lacks a section on how to resolve discrepancies among many raters. There are at least two ways to solve them: if data are categorical, the most common decision among raters is chosen as the answer; if the number of raters who chose each option is even, then a random process like a coin toss decides. When the data are quantitative, the ratings are averaged to achieve the final score (arithmetic mean). Gandalf 1892 (talk) 22:03, 19 May 2022 (UTC)[reply]