Wikipedia:Wikipedia Signpost/2011-05-30/Technology report
Wikimedia down for an hour; What is: Wikipedia Offline?
Wikimedia wikis down for an hour
As noted in last week's "Technology Report", Wikimedia wikis underwent a scheduled downtime of one hour on Tuesday 24 May at around 13:00–14:00 UTC. The downtime meant that the Foundation has already missed previous aired targets of limiting downtime to just 5.256 minutes per annum (equivalent to 99.999% uptime) and 52.6 minutes (99.99% uptime) for this calendar year. However, the work does appear to have been successful at reducing the quantity of out-of-date pages served to readers and other similar problems.
During the downtime, designed to allow the operations team sufficient time to "update the router software and tune the configuration", access to Wikimedia sites was intermittent. The episode and associated issues was alluded to by cartoonist Randall Monroe on his comic strip xkcd (see also this week's "In the news" for more details). Wikimedia developers enjoyed dissecting the technical aspects of the cartoon on the wikitech-l mailing list.
What is: Wikipedia Offline?
Wikimedia Labs: soon to be at the cutting edge of MediaWiki development?
23 April 2012
MediaWiki 1.20wmf01 hits first WMF wiki, understanding 20% time, and why this report cannot yet be a draft
16 April 2012
What is: agile development? and new mobile site goes live
12 September 2011
The bugosphere, new mobile site and MediaWiki 1.18 close in on deployment
29 August 2011
Code Review backlog almost zero; What is: Subversion?; brief news
18 July 2011
Wikimedia down for an hour; What is: Wikipedia Offline?
30 May 2011
Bugs, Repairs, and Internal Operational News
25 April 2011
What is: localisation?; the proposed "personal image filter" explained; and more in brief
21 March 2011
Many Wikipedia editors can now access the Internet from multiple locations: at home, at work, even on-the-go with smartphones. In 2010, however, only 30% of the world had any access at all to the so-called "World Wide Web", even when the high rates of availability found in the developed world are allowed to skew the data (source: CIA World Factbook). Since the Wikimedia Foundation's aim is to "encourage the growth, development and distribution of free, multilingual content", it is clear that either the remaining 70% will have to be supplied with the Internet so they can access the online versions of Wikimedia wikis, or the Wikimedia wikis will have to be provided in an offline-friendly format (in contrast, 50% of the world has used a computer, according to Pew Research). The "Wikipedia Offline" project, then, is a WMF initiative aimed at spreading its flagship product freely to the two billion people who use a computer but cannot access the Internet.
There are two parts to the challenge: firstly, in ensuring that there are Wikipedias in as many languages as possible. The number of users for whom a Wikipedia exists in a language they speak was recently estimated as above 98% (foundation-l mailing list); about 82% have a Wikipedia in their native tongue (also foundation-l). The second challenge is the technical one of supplying the information. A current strategy of the Foundation is to continue to make the raw data of Wikipedias available via so-called "dumps", while simultaneously supporting open-source programs that can process these files. In combination, this will allow whole Wikipedias to be either downloaded when an Internet connection is available, or to be shipped on DVDs or other portable media. This runs alongside the Foundation's existing project to select the most useful articles from a given Wikipedia, hence condensing an encyclopedia onto a single CD.
While "dumps" are largely tried and tested (though recent work has focussed on improving their regularity and reliability), there have also been efforts to enable the export of smaller "collections" of articles, for example those relating to major health issues faced by developing countries. This was in part provided by a new export format (ZIM, developed by the openZim project) that can be read by some offline readers. However, ongoing efforts focus mainly on the second half of the strategy: the provision of a good-quality reader capable of displaying off-line versions of wikis. A number of possible readers were tested. The "Kiwix" reader was selected in late 2010, and the Foundation has since devoted time to improving its user interface, including via the translation of its interface. There is also competition from other readers, including "Okawix", the product of the French company Linterweb. User:Ziko blogged last week about the differences he found between the two. Which, if either, will become the standard is unclear, because it is such a fast-moving area.
See also: Wikimedia strategy document, update on Wikimedia's progress (as of March 2011).
In brief
Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for many weeks.
- On 26 May user MZMcBride reported at the administrators' noticeboard that some deletion debates and arbitration pages were being indexed on Google (that is to say, appeared in Google's search listings, despite this being prohibited by the English Wikipedia's "robots.txt"). The problem was traced to a complexity in Google's spidering system, which does not equate ordinary characters (such as ":" and "/") with their encoded forms ("%3A" and "%2F"). As a result, effective blocking requires a number of additional variants to be listed. Within 50 minutes the Foundation's operations engineer Ryan Lane was working on the case, and NOINDEX code was added to relevant templates. NukeBot (run by admin NuclearWarfare) also began to add the directive to each page in turn to enforce non-spidering. Afterwards, bug #29162 was opened to propose automatically handling such cases in future.
- David Gerard launched a vocal attack on the GFDL software licence recommended to developers by the Free Software Foundation. Instead, he advised that "[developers should] use CC by-sa, CC-by or Public Domain ... If it's a software manual, [they should] license or dual-license it under the same licence as the software itself".
- Wikimedia developers honoured a request from the Foundation's legal department (one of what are termed "office actions") to delete certain image files permanently from Wikimedia servers (server admin log).
- The codebase on the anti-vandalism tool Twinkle was updated, prompting a number of bug reports and some non-functionality during the transitionary period. It is now "gadget only", according to its developer User:AzaToth (English Wikipedia's Technical Village Pump). Many other Wikimedia wikis have their own copy of the tool; many of them will have to be updated manually.
- Bug #27465, which prevented the SVG parser from rendering unusual but perfectly valid images, was fixed.
- Magnus Manske, one of the original developers of the MediaWiki software, began a new blog. His first blogpost concerned one of the gadgets he has written for Wikimedia sites, "Commons Commander".
- The Article Feedback extension for rating articles was listed on the Foundation's "Software deployments" page to be expanded to all articles on the English Wikipedia on 31 May. The lack of publicity given to the deployment raised criticism from some quarters, particularly in the light of recent controversies about the Pending Changes feature (example 1, example 2). Erik Möller explained that the page was in error, and instead announced that the tool would be rolled out incrementally over the next few weeks. In related news, a fix preventing the tool from appearing on redirect pages was pushed live to Wikimedia sites (bug #29164).
Discuss this story
"office action"? "legal request"???? Is that all the explanation we will get for the deletion of those images? --Orange Mike | Talk 19:21, 31 May 2011 (UTC)[reply]
Downtime for the maintenance was well below a full hour. Also, planned maintenance does not count against the targeted downtime, as the targeted downtime is for unscheduled downtime (this is normal). That said, there have been a number of other outages that have put us past the 99.999% target, or even the 99.99% target. --Ryan Lane (WMF) (talk) 21:43, 31 May 2011 (UTC)[reply]
"In 2010, however, only 30% of the world had any access at all to the so-called "World Wide Web", even when the high rates of availability found in the developed world are allowed to skew the data" — what is that supposed to mean? It's the nature of an average to take both areas with high(er) and lower availability of Internet/WWW access into account. Areas with higher-than-average penetration rates aren't "skewing the data" any more than areas with a lower-than-average penetration rate are. Perhaps a "pure" average is not a good metric here, though. -- Schneelocke (talk) 12:28, 1 June 2011 (UTC)[reply]
More direct citations would be more useful. To dig up *where* in the Factbook this could be found would take a bit of searching. "In 2010, however, only 30% of the world had any access at all to the so-called "World Wide Web", even when the high rates of availability found in the developed world are allowed to skew the data (source: CIA World Factbook)."Jodi.a.schneider (talk) 17:48, 23 June 2011 (UTC)[reply]