Talk:EF English Proficiency Index

From Wikipedia, the free encyclopedia
Jump to: navigation, search

2014 edition[edit]

http://www.ef.co.uk/epi/--86.3.200.81 (talk) 19:54, 13 November 2014 (UTC)

Has the page to be deleted?[edit]

The whole page is about a so called "Index" in a report which is, following the text in the article "the product of EF Education First, a global language training company". The main content of the page, "Ranking", has no source - it comes probably from the private page ef.com. So it seems the page serves to promote this product of a private company.

We read about notability: "If a topic has received significant coverage in reliable sources that are independent of the subject, it is presumed to be suitable for a stand-alone article or list." And we read about "reliable sources": "Articles should rely on secondary sources whenever possible. For example, a review article, monograph, or textbook is better than a primary research paper. When relying on primary sources, extreme caution is advised: Wikipedians should never interpret the content of primary sources for themselves." The question is, are there enough reliable sources for this article, so that it would be worth editing it?

The problem with the whole "Index" is well shown by The Economist:"This was not a statistically controlled study: the subjects took a free test online and of their own accord. They were by definition connected to the internet and interested in testing their English; they will also be younger and more urban than the population at large." Is the Wikipedia the place to promote a study that is not statistically controlled? Maybe it's even possible to find remarks of scientists about the reliability of the "Index"...

To give one example. The article states without giving a specific source: "Europe as a whole speaks the best English, the Middle East the worst." Do we think that the Wikipedia should have such an assertion, if this is based just on some accidentally interested internet users who visit their page? This "index" is dependent on the success of marketing activities. If they promote their page in one country between people who have a high level of English competence, the "ranking" of that country will rise. If they promote the page in another country between people with a low level of English, the "ranking" of that country will go down. Is such an "Index" of a private company worth to be in the Wikipedia?

It seems this is just a marketing article for a language teaching company. "Wikipedia is not a place to promote things". We should note that the article has been created in 2011 by User:K8bell who works for EF Education First and is one of the authors of the EF English Proficiency Index, as the user writes on his page. --Lu Wunsch-Rolshoven (talk) 10:33, 24 November 2016 (UTC)

It seems that in the article it's not clear what the so-called "index" is. At first it's "a report"; can an index be a report? In the criticism it's written: "This pushes the index towards the realm of an online survey rather than a statistically valid evaluation." Probably this is true - the so-called index is nothing more than just an online survey. At least this should be written clearly. --Lu Wunsch-Rolshoven (talk) 11:11, 24 November 2016 (UTC)
The article clearly meets criteria for notability. I have added more citations to demonstrate that the index and findings are widely cited in international press. You can find additional coverage of the report and its findings here: http://www.ef.com/epi/about-epi/press/. The index is an online survey as you point out and as is now stated in the first paragraph: "The index is an online survey first published in 2012". The article is about the EF EPI report, which contains an index, detailed data about each country included in the index, and analysis of trends, correlations with other data sets, etc. I hope that clarifies things. Thanks for your work on making this article better. K8bell (talk) 09:49, 21 January 2017 (UTC)

No sources for EF_English_Proficiency_Index#2016_Rankings[edit]

That's why I deleted this section. If there are no sources next week, probably the section is to be deleted again. --Lu Wunsch-Rolshoven (talk) 19:49, 22 January 2017 (UTC)

Source added K8bell (talk) 10:06, 24 January 2017 (UTC)

EF survey "not based on representative sampling"[edit]

Yakub Marian critically analyzes the EF EPI rankings in his article Why the EF EPI rankings are not what you think. He advises, "don’t take the results seriously; there are huge methodological flaws in the EF EPI study". He shows some strange assertions of the EF test: "According to the results, Poles speak better English, on average, than the Swiss (even though Switzerland has one of the largest numbers of international companies in Europe and around 17% of the Swiss speak English at work)". In fact following the results of EF, a private company based in Sweden, Poland has a score of 61.49 % and Switzerland a score of 60.17 %... Who will believe this?

Marian mentions that following the survey of EF "Hungarians are only slightly worse than Austrians (conflicting the results of another survey, according to which more than 3.5x as many Austrians as Hungarians are even able to speak any English at all)." Amazing, isn't it... (Quite interesting to compare the two maps Marian prepared.)

Marian expresses his opinion that EF is "the only major company that has the nerve to falsely market the results as ranking of countries based on English proficiency." Probably it will be a good idea to compare the EF results for Europe with those of other studies about English knowledge, e. g. in Europe. See for instance the Special Eurobarometer 386. EUROPEANS AND THEIR LANGUAGES, p. 21. Not a test, but a survey based on the own opinion of the respondents. But who believes that the respondents are so far from reality as they should be, if the EF results were real? --Lu Wunsch-Rolshoven (talk) 21:50, 22 January 2017 (UTC)

It is absolutely clear in the report, on the report's webpage, and on this Wikipedia page, that this index is a based on data from a large survey. It is not based on a representative sampling model. There is no disagreement here. Whether or not this or that blogger agrees or disagrees with the report's findings is not really the point of Wikipedia. K8bell (talk) 10:06, 24 January 2017 (UTC)
Thank you for your comment. - You are very right about bloggers. I wonder, if there is any scientific paper which mentions the index and evaluates it - is there?
I would like to emphasize that Marian mainly speaks about the methodology and concludes that this methodology leads to results in contrast to other studies. --Lu Wunsch-Rolshoven (talk) 13:32, 25 January 2017 (UTC)
I'm not aware of any scientific paper which evaluates the EF EPI results, or of any scientific paper that looks at English language proficiency in adults in different countries using any sampling method other than a survey. I'd love to find one, but I don't think it's out there. Unfortunately this data would be very expensive to gather, so I assume that's why nobody's done it. I know from having spoken to Gallup that they charge 1 million USD to add a single question on to one of their existing surveys used in 50 countries! Imagine trying to add an entire English test...
The EU looked at language proficiency in 15-year-olds in a few countries in a PISA-style study called SurveyLang in 2012. That reference is already in the article, but although the study is fascinating, it's not especially helpful as a point of comparison to this survey because it's many fewer countries, much younger test takers, etc. Still, I wish the EU would do it again including more countries this time. K8bell (talk) 08:06, 31 January 2017 (UTC)

It's not clear what the EF "score" means[edit]

How is it calculated for every country? This should be written clearly in the article. --Lu Wunsch-Rolshoven (talk) 21:59, 22 January 2017 (UTC)

More details on methodology can be found here: http://www.ef.com/epi/about-epi/, but I don't think the page should necessarily be made more technical by including all these details. Glad to discuss further if you disagree.K8bell (talk) 10:06, 24 January 2017 (UTC)
Thank you for the link; I included just two sentences in the article; I think this is not too technical. (Please consider putting : on the beginning of your comments; easier to distinguish different user comments.) --Lu Wunsch-Rolshoven (talk)
Thanks for adding these sentences. I really appreciate your contribution to the article and your concern about it being clear and precise. I agree it's not too technical and makes the methodology clearer. If you're curious about the tests, this is the one the majority of the EF EPI data comes from: www.efset.org K8bell (talk) 08:06, 31 January 2017 (UTC)