Talk:Dunning–Kruger effect

"Mount Stupid"

"Mount Stupid" seems to be a popular enough characterisation to merit a redirect to the article. Is it worth mentioning this informal name, albeit with any necessary caveats about this usage? JezGrove (talk) 16:42, 31 December 2022 (UTC)

Thanks for talking about it instead of just doing it. I can see where that might make sense, but I also fear that it would legitimize "mount stupid" as a feature of DK. Wikipedia is such a force in the world, that we have to be wary of creating facts instead of reporting them.

Constant314 (talk) 19:05, 31 December 2022 (UTC)

Agreed, but there have been 454 redirects in the past two years, so it might be worth acknowledging/mentioning? But I don't have a strong opinion either way. JezGrove (talk) 20:21, 31 December 2022 (UTC)

DK Effect is Simply Autocorrelation

The article otherwise largely fails to communicate the degree to which the DK effect is pseudoscience. The statement "the statistical explanation interprets these findings as statistical artifacts" needs to be expanded and made much more prominent to explain why the effect is simply autocorrelation and should not be basis for any cognitive or metacognitive claims despite its appeal. The autocorrelation claim is easy to understand and should be a convincing argument for changing the first paragraph to make clear that while the concept is appealing, it is not based on a valid statistically methodology and should not be taken too seriously.

The Dunning–Kruger effect is a claimed cognitive bias[2] whereby people with low ability, expertise, or experience regarding a certain type of task or area of knowledge tend to overestimate their ability or knowledge. Some researchers also include in their definition the opposite effect for high performers: their tendency to underestimate their skills. Despite its intuitive appeal the effect can be statistically explained as autocorrelation[1] and should not be used to formally explain cognitive biases or metacognitive phenomena regarding self-evaluation of ability, knowledge or experience.

Simply put for any sample of test scores on a 0-10 scale, the likelihood that someone who scores 0 will overestimate the performance is necessarily higher than someone who scores 10. The reverse is also true: anyone who scores 10 will necessarily underestimate their performance more that someone who scores 0.

The ironies are replete, as pointed out in the article: "there is a delightful irony to the circumstances of their [Dunning and Kruger's] blunder. Here are two Ivy League professors arguing that unskilled people have a ‘dual burden’: not only are unskilled people ‘incompetent’ ... they are unaware of their own incompetence. [...] In their seminal paper, Dunning and Kruger are the ones broadcasting their (statistical) incompetence by conflating autocorrelation for a psychological effect."

The popularity of the DK effect may be an interesting study in how bad science can take hold in the popular mind given how many people seem to take it seriously without considering the fatal flaws in the methodology used to identify the alleged phenomenon. DK also serves as an example of how bad science can get through the scientific peer review process, especially if it comes from a highly reputable institution. --Chassin (talk) 16:10, 7 January 2023 (UTC)

This has already been discussed several times. See, for example, Talk:Dunning–Kruger_effect/Archive_5#Neutral_Point_of_View and Talk:Dunning–Kruger_effect/Archive_5#Is_this_true?. For details on the criticisms of the Dunning–Kruger effect, see the section Dunning–Kruger_effect#Criticism_and_alternatives in our article. And your source, https://economicsfromthetopdown.com/, is not reliable. Phlsph7 (talk) 16:35, 7 January 2023 (UTC)

Other sources may indeed be preferred, such as Nuhfer et al (2006). The outcome of previous discussions notwithstanding, the fact remains that the opening paragraphs of the introduction still fail to offer caveats regarding flaws in empirical methods used to support the claims, and instead tend to give credence to it based on its broad application and intuitive appeal. If anything, the only effect Dunning claims to have identified is the tendency for everyone to overestimate their ability, not just people with low ability, and certainly not for those with high ability to underestimate theirs. The first paragraph is misleading in multiple respects and should be revised to address these shortcomings.

The first mention of criticism is in paragraph 4 and characterizes it as "debate" and dismisses it as "not denying the empirical findings", when that is precisely what the statistical criticism does unequivocally. If we compare how criticism of this topic is addressed to the pseudoscience of physiognomy, we can clearly see in the opening of the second paragraph that it is "regarded among academic circles because of its unsupported claims; popular belief in the practice of physiognomy is nonetheless still widespread". The same can be said of the DK effect insofar as the shapes of peoples heads differ and people self-evaluate inaccurately, but the differences in head shapes and self-evaluation errors do not provide the necessary support for their respective theories regarding ability, expertise, or experience. On the contrary, the empirical data when analyzed correctly falsifies them both equally, which in the final analysis is only method we have at our disposal. --Chassin (talk) 19:58, 7 January 2023 (UTC)

the empirical data when analyzed correctly falsifies them both equally Which reliable source says so? Paradoctor (talk) 20:25, 7 January 2023 (UTC)

See https://fortune.com/2023/05/08/what-is-dunning-kruger-effect-smart-intelligence-competence-john-cleese/ 77.204.39.138 (talk) 17:49, 8 June 2023 (UTC)

Adding a note about the common misrepresentation in media

The DK effect is extremely commonly misrepresented and thus misunderstood. The incorrect DK graph constitutes the majority of top search engine results and, although I have no source on this, popular conception of the DKE. I propose adding a small note, if not showing the graph, at least mentioning it and it’s lack of merit. Ferdinand Hoven (talk) 18:08, 16 January 2023 (UTC)

My thinking has come around to agree. Constant314 (talk) 19:41, 16 January 2023 (UTC)

We would need a reliable source, otherwise it's WP:OR. For an earlier discussion of the incorrect graph, see Talk:Dunning–Kruger_effect/Archive_5#Illustration. Phlsph7 (talk) 22:25, 16 January 2023 (UTC)

I think it is a sky is blue type statement. We don't really need a RS to say that the "valley of despair" diagram shows up often. I think we only need a consensus, but I would not force it on a two to one vote. Constant314 (talk) 00:10, 17 January 2023 (UTC)

It depends on what you want to write. I did a google image search for "Dunning-Kruger effect" a few minutes ago. For my search, only one image of the first 8 images (the forth) has the term "valley of despair". Is that sufficient evidence to claim that the "valley of despair" diagram shows up often? You would probably have to generalize over the different images to say that that many of them are "valley of despair" diagrams even though they do not use that term. And even if that is sufficient evidence, the sentence 'The "valley of despair" diagram shows up often (in google image searches for the term "Dunning-Kruger effect")' would be rather pointless to add to the article. It would be more interesting to say that it is incorrect. But this is not a sky is blue type statement. Phlsph7 (talk) 06:55, 17 January 2023 (UTC)

Repeating a misconception risks reinforcing the misconception (for example, when people skim the article). Our role is to record the best available knowledge about the subject. When there is a topic where a large proportion of the discussion in reliable sources is about the misconception, e.g. Parity of zero, then we'd be justified in having a section in the article. Google search results don't qualify as a reliable source, and the accessibility of the results doesn't change that or make a conclusion a "sky is blue type statement". MartinPoulter (talk) 17:00, 17 January 2023 (UTC)

I am not convinced, but it doesn't look like there is any support to add the material, so I am done unless someone else wants to add support. Cheers. Constant314 (talk) 20:02, 17 January 2023 (UTC)

I would argue that the majority of, or at least half of discussion around the dke is about the incorrect graph. I was shown the incorrect graph in an AP stats class when I was in high school, the entire first 50 or so results of “dunning kruger effect” on ecosia (bing) image search are wrong graphs, there’s several youtube videos with millions of views depicting it incorrectly, and I am of the personal view that the majority of people who know about the DKE have the wrong graph in mind. I agree though, that none of these are reliable sources, i just wouldn’t know how to reliably source this, regardless of if it is true. Ferdinand Hoven (talk) 21:26, 18 January 2023 (UTC)

GA Review

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

This review is transcluded from Talk:Dunning–Kruger effect/GA1. The edit link for this section can be used to add comments to the review.

Reviewer: TompaDompa (talk · contribs) 02:08, 14 February 2023 (UTC)

I will review this. TompaDompa (talk) 02:08, 14 February 2023 (UTC)

Hello TompaDompa and thanks for taking the time to review this nomination. I'll try to be responsive and reply to your comments in a timely manner. Phlsph7 (talk) 07:11, 14 February 2023 (UTC)

General comments

• I'm not a copyright expert, but how can the images be CC BY-SA 4.0 when they are from copyrighted works?
• There are a fair number of sentences beginning with "so", which gives a somewhat amateurish impression (being a comparatively informal phrasing). "Thus" is usually a better word to use, and sometimes it can simply be removed (perhaps replacing the preceding period with a semicolon).
• Verb tense is not consistent.
• The "External links" section is empty.
• The article needs copyediting to conform to WP:Make technical articles understandable.
• https://pubmed.ncbi.nlm.nih.gov/19568317 is cited twice in duplicate references.
• I'm missing discussion of the common perception (misconception) among laypeople that the Dunning–Kruger effect means something along the lines of "those who know very little think they know more than (and/or are more confident than) experts". This omission is especially conspicuous considering the caption for the image in the WP:LEAD says Nevertheless, low performers' self-assessment is lower than that of high performers. I notice that the "Mount Stupid" graph (and its interpretation) has been discussed a number of times on the talk page.
• The article cites Dunning (2011) heavily, which might not be the most neutral source.

Lead

• Since its first publication, various criticisms of the effect and its explanation have been made. – going by what the body says, it seems that saying that the effect (rather than the metacognitive explanation) has been subject to various criticisms is not entirely accurate.
• It's rather odd to mention criticism of the proposed explanation of the effect before mentioning what that explanation actually is.
• Some theorists hold that the way low and high performers are distributed makes assessing their skill level more difficult for low performers, thereby explaining their erroneous self-assessments independent of their metacognitive abilities. – this is basically incomprehensible when presented without context like this.

Definition

• I think this section would benefit from a table of the different components of the varying definitions.
• Biases are systematic in the sense that they occur consistently in different situations. They are tendencies since they concern certain inclinations or dispositions that may be observed in groups of people, but are not manifested in every performance. – is this to explain the phrase "systematic tendency"?
• to greatly overestimate their competence or to see themselves as more skilled than they are – are those not just different ways of saying the same thing?
• the lack of skill and the ignorance of this lack – to me, this is an odd phrasing. Specifically, the use of the word "lack" without a complementary "of X" sticks out to me.
• see the relation to metacognition as a possible explanation independent of one's definition – "independent of one's definition"?
• So it is sometimes claimed to include the reverse effect for people with high skill. – "claimed"?
• On this view, – grammar.
• Arguably, – WP:Editorializing.
• This phenomenon has been categorized as a form of the false-consensus effect. – gloss.

Measurement and analysis

• If done afterward, it is important that the participants receive no independent clues during the performance as to how well they did. – this almost comes off as a how-to guide.
• When done in absolute terms, self-assessment and performance are measured according to absolute standards – seems tautological.
• Link quartile at first mention.
• Some researchers focus their analysis on the difference between the two abilities, i.e. on subjective ability minus objective ability, to highlight the negative correlation. – I don't follow. Either this is fairly redundant (how else would you measure an overestimation?) or there's something I'm missing.

Studies

• This section relies a lot on WP:PRIMARY sources. WP:Cite reviews, don't write them.
• I don't see a good reason to list a bunch of studies like this. The findings may be relevant, but a timeline of studies conducted—which is basically what at least the latter part of this section amounts to—is not.
• While many studies are conducted in laboratories, others take place in real-world settings. – the difference is not immediately obvious to me, nor is its significance.
• More recent studies – MOS:RECENT.
• Link percentile at first mention.
• and points out – MOS:SAID.
• It does not yet contain the term "Dunning–Kruger effect", which was introduced later. – the cited WP:PRIMARY source can of course only verify the first part, and it's dubious if a primary source should be used in this way in the first place.
• It does not yet contain the term "Dunning–Kruger effect", which was introduced later. – when?
• the incompetent bank robberies of McArthur Wheeler and Clifton Earl Johnson – is there a strong reason to name these presumably-living people and call them "incompetent" in WP:WikiVoice?
• tries to show – "tries to show"?
• concludes that the Dunning–Kruger effect obtains only in tasks that feel easy – "obtains"?
• As he writes, [...] – this phrasing endorses Dunning's view rather than just reporting it.
• A 2022 study found, consistent with the Dunning–Kruger effect, that people who reject the scientific consensus on issues think they know the most about them but actually know the least. – this is a stronger assertion about the connection to the Dunning–Kruger effect than the cited source supports. It also seems to contradict the earlier Nevertheless, low performers' self-assessment is lower than that of high performers.

Explanations

• This section veers into engaging in disputes rather than merely describing them repeatedly.
• It would almost certainly be preferable to structure this section such that each proposed explanation is discussed separately within its own subsection, along with the arguments for and against it.
• I'm missing the "rational model of self-assessment" discussed in https://www.nature.com/articles/s41562-021-01057-0 and summarized briefly in https://www.nature.com/articles/s41562-021-01101-z.
• Stick to either "metacognitive" or "meta-cognitive".
• Both "account" and "approach" are odd words to choose here. I would go with "explanation", "model", "interpretation", and similar phrasings.
• Some attempts have been made to measure metacognitive abilities directly to confirm this hypothesis. – to investigate it, one would hope.
• There is a large and growing body of criticism of the assumptions on which the metacognitive account is based. – what the cited source says is The classic metacognitive interpretation of the Dunning–Kruger effect has been challenged by alternative explanations.
• This line of argument usually proceeds by providing an alternative approach that promises a better explanation of the observed tendencies. – that seems like it would go without saying.
• One such account is based on the idea that both low and high performers have in general the same metacognitive ability to assess their skill level. – that just seems like the negation of the metacognitive interpretation.
• The explanation for the regression toward the mean interpretation is rather difficult to follow. I'm not even sure it correctly describes the proposed mechanism. The way I understand it from e.g. https://www.nature.com/articles/s41562-021-01101-z, the idea is that the subset of participants with the most extreme actual performances will not have as extreme perceived performances assuming that self-assessments are imperfect.
• But such adjustments do not eliminate the Dunning–Kruger effect, which is why the view that regression toward the mean is sufficient to explain it is usually rejected. – not in the cited source.
• However, it has been suggested – MOS:WEASEL.
• Defenders of the statistical explanation – proponents.
• By choosing the right variables for the randomness due to luck and a positive offset to account for the better-than-average effect, it is possible to simulate experiments – needlessly technical phrasing.
• almost the same correlation between self-assessed ability. – and what? This is an incomplete statement.
• This means that the Dunning–Kruger effect may still have a role to play, if only a minor one. – the meaning of this is not immediately obvious.
• Opponents of this approach – "opponents"?
• But even proponents of this explanation agree that this does not explain the empirical findings in full. – not in the cited source.
• I am not convinced https://www.mcgill.ca/oss/article/critical-thinking/dunning-kruger-effect-probably-not-real is an appropriate source to use in this context. The reference also misspells the author's name.
• Another statistical-artifact-based challenge to the Dunning-Kruger effect is the demonstration that a form of the effect can emerge when the errors of the self-assessment are randomly created. – rather opaque.

Practical significance

• The Dunning–Kruger effect can also have negative implications for the agent in various economic activities – needlessly technical phrasing. Why "agent"?
• Some also concentrate on its positive side, e.g., ignorance can sometimes be bliss. – if there is only one positive side, "e.g." is incorrect. If there are multiple positive sides, "side" is incorrect. "e.g., ignorance can sometimes be bliss" is an odd phrasing to me.

Popular recognition

• This is not really an "In popular culture" section, but with a title like this people might expect it to be, so I would at least change the heading. The Ig Nobel Prize should certainly be mentioned somewhere in the article.
• a satiric Ig Nobel Prize – the Ig Nobel Prize is always satirical. Either swap the indefinite article for a definite one, or provide the explanation that it is satirical elsewhere.
• I'm not convinced mentioning "The Dunning–Kruger Song" is due.

Summary

GA review – see WP:WIAGA for criteria

1. Is it well written?

   A. The prose is clear and concise, and the spelling and grammar are correct:

   B. It complies with the manual of style guidelines for lead sections, layout, words to watch, fiction, and list incorporation:

2. Is it verifiable with no original research?

   A. It contains a list of all references (sources of information), presented in accordance with the layout style guideline:

   B. All in-line citations are from reliable sources, including those for direct quotations, statistics, published opinion, counter-intuitive or controversial statements that are challenged or likely to be challenged, and contentious material relating to living persons—science-based articles should follow the scientific citation guidelines:

   See above.

   C. It contains no original research:

   Spotchecking has revealed several instances of material failing verification.

   D. It contains no copyright violations nor plagiarism:

   Earwig reveals no overt copyvio. I have not spotted any instances of unacceptably WP:Close paraphrasing, but I have not taken a close enough look to be able to rule it out with a reasonable degree of confidence.

3. Is it broad in its coverage?

   A. It addresses the main aspects of the topic:

   B. It stays focused on the topic without going into unnecessary detail (see summary style):

4. Is it neutral?

   It represents viewpoints fairly and without editorial bias, giving due weight to each:

5. Is it stable?

   It does not change significantly from day to day because of an ongoing edit war or content dispute:

6. Is it illustrated, if possible, by images?

   A. Images are tagged with their copyright status, and valid non-free use rationales are provided for non-free content:

   The images are tagged with appropriate licenses, but this seems dubious to me. See above.

   B. Images are relevant to the topic, and have suitable captions:

7. Overall:

   Pass or Fail:

@Phlsph7: I am closing this as unsuccessful. I'm sorry this has taken so long; I started out with the intention to provide detailed feedback, but I have settled for providing a non-exhaustive sample of issues I noted while reading through the article instead. This is an interesting topic and it's a shame close the nomination like this, but there are systemic issues with the article that are not trivially fixable. The writing style is rather WP:TECHNICAL and unnecessarily wordy in places. More seriously, the article misuses sources (violating WP:NOR) and engages in disputes rather than merely describing them (violating WP:NPOV). My suggestion to bring this in line with Wikipedia's WP:Core content policies would be to pick a handful of review articles (or other similar sources that treat the entire overarching topic broadly), and use those to write the article. Sources on specific aspects (especially studies dealing with the effect) can be used to flesh out certain parts of the article by providing additional details, but should not serve as the basis for the article. WP:Cite reviews, don't write them. TompaDompa (talk) 22:20, 23 February 2023 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Discussion after GAN review

@TompaDompa: I'm sorry about the outcome but thanks for your detailed feedback. I've tried to implement many of your suggestions but there is still some work to be done. I've responded to some of your points below. Your feedback here may be helpful to further improve the article.

• Concerning the images: these graphs are not copied from the sources. Instead, their data is reproduced in new graphs. I don't think that the data itself is copyrighted so this shouldn't be a problem.
• Concerning the graph with mount stupid and other popular misconceptions: I would be happy to include a discussion of them if you know of a reliable source on that issue. But there is not much we can do without a reliable source. There are various talk page discussions with the same conclusion, for example, Talk:Dunning–Kruger_effect#Adding_a_note_about_the_common_misrepresentation_in_media.
• Concerning the terms "account" and "approach": The term "account" is used in the reliable sources itself, for example, as the "dual-burdon account". To me, the term "approach" sounds fine but I'm not sure whether it is commonly used in the reliable sources so I changed it.
• Concerning NPOV: This seems to apply mainly to the section "Explanation". In order to rewrite it, it would be helpful to get a better understanding of what exactly you mean here. I'm sure there are a few expressions that might be improved but I see no serious overall bias.
• Concerning the section title "Popular recognition": I didn't get your point here. What title would you suggest?

If some of the changes I made so far are not what you had in mind then please let me know. Phlsph7 (talk) 08:45, 25 February 2023 (UTC)

Like I said, I'm not a copyright expert, but that seems dubious to me. The images are virtually identical to the ones in the sources, and I'll note that Dunning (2011) felt the need to say "Adapted with permission." Let's ask someone who is more knowledgable when it comes to copyright matters than I am. Ping Diannaa: if I remember correctly (and going by your userpage, it would appear so), this is one of your areas of expertise.
The sourcing requirements for saying that people incorrectly believe that the Dunning–Kruger effect means XYZ are fairly low, much lower than the requirements for saying things about the actual Dunning–Kruger effect. This is not a particularly high-quality source, but it is probably reliable enough about what laypeople mistakenly think the Dunning–Kruger effect is.
On wording, I'll just say that Wikipedia is tailored to a much broader audience than our sources, and we should avoid being too technical or "jargon-y" in our phrasings when it is not necessary to use such language for precision (sometimes the distinction between velocity and speed matters, for instance).
The NPOV issues in the "Explanations" section are somewhat better now, but there is still stuff like But even proponents of this explanation agree that this does not explain the empirical findings in full. which is pretty clearly taking a side. Restructuring the section to discuss the different explanations separately in the manner I suggested above would probably go a long way towards achieving neutrality.
As for "Popular recognition", it could perhaps be renamed simply "Recognition" or the Ig Nobel Prize mentioned elsewhere in the article (there is no "History" section at present, but if there were it could be mentioned there). The point is that we don't want people to add "In episode X of television series Y, character Z refers to the Dunning–Kruger effect when [...]". TompaDompa (talk) 10:33, 25 February 2023 (UTC)

The images look okay to keep in my opinion. Simple numerical data can't be copyrighted, and the graphs were created by Wikipedians. We do have a specific noticeboard for media questions, at Wikipedia:Media copyright questions, so if you would like to get another opinion, that would be a good place to go. — Diannaa (talk) 11:34, 25 February 2023 (UTC)

@Diannaa: Thanks for clarifying the issue. Phlsph7 (talk) 11:46, 25 February 2023 (UTC)

(edit conflict) Good to know, thank you very much. I have been looking for somewhere to bring up issues like this more generally, I just never found that one. Is there an equivalent for text—possible WP:Close paraphrasing and whatnot? WP:Copyright problems doesn't really seem like the right place for questions/discussion. TompaDompa (talk) 11:52, 25 February 2023 (UTC)

You could try Wikipedia talk:Copyright problems but occasionally I've seen questions go unanswered there for a long time. General rules are as follows: Under current copyright law, literary works are subject to copyright whether they are tagged as such or not. No registration is required, and no copyright notice is required. So please always assume that all text you find online or in print sources is copyright. Exceptions include works of the US Government and material specifically released under license. Even then, proper attribution is required. — Diannaa (talk) 12:40, 25 February 2023 (UTC)

One more piece of general feedback that I forgot to clarify explicitly in my review: take care not to go beyond what the sources say or engage in any WP:ANALYSIS thereof. It's easy to do accidentally when one is familiar with/knowledgable about the topic, but it constitutes WP:Original research. TompaDompa (talk) 11:59, 25 February 2023 (UTC)

I've added a short discussion of the popular misconception. Thanks for looking up this source. It's not the best one but I hope it is sufficient for the job. You suggested getting a handful of review articles. Do you know of any that provide a detailed discussion of the different explanations? I'll keep the term "dual-burden" account since this is the specific term. But I'll see what I can do about the other mentions of "account". Did any other unnecessarily difficult terms catch your eye? I moved the sentence on the ig nobel price to the section "Practical significance" and renamed it to "Significance". This is not the perfect solution but I don't think the sentence fits well in any of the other sections. I'll keep your advice on WP:OR in mind. Phlsph7 (talk) 12:08, 25 February 2023 (UTC)