National Security Agency surveillance
XKeyscore or XKEYSCORE (abbreviated as XKS) is a formerly secret computer system first used by the United States National Security Agency for searching and analyzing Internet data it collects worldwide every day. The program has been shared with other spy agencies including Australia's Defence Signals Directorate, New Zealand's Government Communications Security Bureau and the German Bundesnachrichtendienst.
The program's existence was publicly revealed in July 2013 by Edward Snowden in The Sydney Morning Herald and O Globo newspapers, though the codename is mentioned in earlier articles, and like many other codenames can also be seen in job postings, and in the online resumes of employees.
The scope of XKeyscore
XKeyscore is a complicated system and various authors have different interpretations about its actual capabilities. Edward Snowden and Glenn Greenwald explained XKeyscore as being a system which enables almost unlimited surveillance of anyone anywhere in the world, while NSA said that usage of the system is limited and restricted.
According to The Washington Post and national security reporter Marc Ambinder, XKeyscore is an NSA data-retrieval system which consists of a series of user interfaces, backend databases, servers and software that selects certain types of data and metadata that the NSA has already collected using other methods.
According to Snowden and Greenwald
- "You could read anyone’s email in the world, anybody you’ve got an email address for. Any website: You can watch traffic to and from it. Any computer that an individual sits at: You can watch it. Any laptop that you’re tracking: you can follow it as it moves from place to place throughout the world. It’s a one-stop-shop for access to the NSA’s information."
- “…You can tag individuals… Let’s say you work at a major German corporation and I want access to that network, I can track your username on a website on a form somewhere, I can track your real name, I can track associations with your friends and I can build what’s called a fingerprint, which is network activity unique to you, which means anywhere you go in the world, anywhere you try to sort of hide your online presence, your identity.”
According to The Guardian's Glenn Greenwald, low-level NSA analysts can via systems like XKeyscore "listen to whatever emails they want, whatever telephone calls, browsing histories, Microsoft Word documents. And it's all done with no need to go to a court, with no need to even get supervisor approval on the part of the analyst."
He added that the NSA's databank of collected communications allows its analysts to listen "to the calls or read the emails of everything that the NSA has stored, or look at the browsing histories or Google search terms that you've entered, and it also alerts them to any further activity that people connected to that email address or that IP address do in the future".
According to the NSA
In an official statement from July 30, 2013, the NSA said there is no "unchecked analyst access to NSA collection data. Access to XKeyscore, as well as all of NSA's analytic tools, is limited to only those personnel who require access for their assigned tasks." The NSA also states that there are "stringent oversight and compliance mechanisms built in at several levels. One feature is the system's ability to limit what an analyst can do with a tool, based on the source of the collection and each analyst's defined responsibilities."
The agency defended the program, stressing that it was only used to legally obtain information about "legitimate foreign intelligence targets in response to requirements that our leaders need for information necessary to protect our nation and its interests. [...] XKeyscore is used as a part of NSA's lawful foreign signals intelligence collection system. [...] These types of programs allow us to collect the information that enables us to perform our missions successfully -- to defend the nation and to protect U.S. and allied troops abroad."
An NSA presentation about XKeyscore from 2008 says that it's a "DNI Exploitation System/Analytic Framework". DNI stands for Digital Network Intelligence, which means intelligence derived from internet traffic. In an interview with the German Norddeutscher Rundfunk, Edward Snowden said about XKeyscore: "It’s a front end search engine".
XKeyscore consists of over 700 servers at approximately 150 sites where the NSA collects data, like "US and allied military and other facilities as well as US embassies and consulates" in many countries around the world. Among the facilities involved in the program are four bases in Australia and one in New Zealand.
According to an NSA presentation from 2008, these XKeyscore servers are fed with data from the following collection systems:
- F6 (Special Collection Service) – joint operation of the CIA and NSA that carries out clandestine operations including espionage on foreign diplomats and leaders
- FORNSAT – which stands for "foreign satellite collection", and refers to intercepts from satellites
- SSO (Special Source Operations) – a division of the NSA that cooperates with telecommunication providers
In a single, undated slide published by Swedish media in December 2013, the following additional data sources for XKeyscore are mentioned:
- Overhead – intelligence derived from American spy planes, drones and satellites
- Tailored Access Operations – a division of the NSA that deals with hacking and cyberwarfare
- FISA – all types of surveillance approved by the Foreign Intelligence Surveillance Court
- Third party – foreign partners of the NSA such as the (signals) intelligence agencies of Belgium, Denmark, France, Germany, Italy, Japan, the Netherlands, Norway, Sweden, etc.
From these sources, XKeyscore stores "full-take data", which are indexed by plug-ins that extract certain types of metadata (like phone numbers, e-mail addresses, log-ins, and user activity) and index them in metadata tables, which can be queried by analysts. XKeyscore has been integrated with MARINA, which is NSA's database for internet metadata.
However, the system continuously gets so much Internet data that it can be stored only for short periods of time. Content data remain on the system for only three to five days, while metadata is stored for up to 30 days. A detailed commentary on an NSA presentation published in The Guardian in July 2013 cites a document published in 2008 declaring that "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."
For analysts, XKeyscore provides a "series of viewers for common data types", which allows them to query terabytes of raw data gathered at the aforementioned collection sites. This enables them to find targets that cannot be found by searching only the metadata, and also to do this against data sets that otherwise would have been dropped by the front-end data processing systems. According to a slide from an XKeyscore presentation, NSA collection sites select and forward less than 5% of the internet traffic to the PINWALE database for internet content.
Because XKeyscore holds raw and unselected communications traffic, analysts can not only perform queries using "strong selectors" like e-mail addresses, but also using "soft selectors", like keywords, against the body texts of e-mail and chat messages and digital documents and spreadsheets in English, Arabic and Chinese.
This is useful because "a large amount of time spent on the web is performing actions that are anonymous" and therefore those activities can't be found by just looking for e-mail addresses of a target. When content has been found, the analyst might be able to find new intelligence or a strong selector, which can then be used for starting a traditional search.
- Look for the usage of Google Maps and terms entered into a search engine by known targets looking for suspicious things or places.
- Look for "anomalies" without any specific person attached, like detecting the nationality of foreigners by analyzing the language used within intercepted emails. An example would be a German speaker in Pakistan. The Brazilian paper O Globo claims that this has been applied to Latin America and specifically to Colombia, Ecuador, Mexico and Venezuela.
- Detect people who use encryption by do searches like "all PGP usage in Iran". The caveat given is that very broad queries can result in too much data to transmit back to the analyst.
- Showing the usage of Virtual private networks (VPNs) and machines that can potentially be hacked via TAO.
- Track the source and authorship of a document that has passed through many hands.
Most of these things cannot be detected by other NSA tools because they operate with strong selectors (like e-mail and IP addresses and phone numbers) and the raw data volumes are too high to forward them to other NSA databases.
In 2008, it was planned to add a number of new capabilities in the future, like:
The NSA slides published in The Guardian during 2013 claimed that XKeyscore had played a role in capturing 300 terrorists by 2008. This claim could not be substantiated as the redacted documents do not cite instances of terrorist interventions.
A 2011 report from the NSA unit in Griesheim (Germany) says that XKeyscore made it easier and more efficient to target surveillance. Previously, analysis often accessed data they were not interested in. XKeyscore allowed them to focus on the intended topics, while ignoring unrelated data. XKeyscore also proved to be an outstanding tool for tracking active groups associated with the Anonymous movement in Germany, because it allows for searching on patterns, rather than particular individuals. An analyst is able to determine when targets research new topics, or develop new behaviors.
To create additional motivation, the NSA incorporated various features from computer games into the program. For instance, analysts who were especially good at using XKeyscore could acquire "skilz" points and "unlock achievements." The training units in Griesheim were apparently successful and analysts there had achieved the "highest average of skilz points" compared with all other NSA departments participating in the training program.
Usage by foreign partners of the NSA
According to documents Der Spiegel acquired from Snowden, the German intelligence agencies BND (foreign intelligence) and BfV (domestic intelligence) were also allowed to use the XKeyscore system. In those documents the BND agency was described as the NSA's most prolific partner in information gathering. This led to political confrontations, after which the directors of the German intelligence agencies briefed members of the German parliamentary intelligence oversight committee on July 25, 2013. They declared that XKeyscore has been used by the BND since 2007 and that the BfV uses a test version since 2012. The directors also explained that this program is not for collecting data, but only for analyzing them.
As part of the UKUSA Agreement, a secret treaty was signed in 1954 by Sweden with the United States, the United Kingdom, Canada, Australia and New Zealand for the purpose of intelligence collaboration and data sharing. According to documents leaked by Snowden, the Försvarets radioanstalt of Sweden (FRA) has been granted access to XKeyscore.
|Wikimedia Commons has media related to XKeyscore.|
- A full NSA presentation about XKeyscore from 2008
- Building a panopticon: The evolution of the NSA’s XKeyscore
- "Snowden Interview Transcript". NDR. Retrieved 27 January 2014.
- Greenwald, Glenn; Ackerman, Spencer (June 27, 2013). "How the NSA Is Still Harvesting Your Online Data – Files Show Vast Scale of Current NSA Metadata Programs, with One Stream Alone Celebrating 'One Trillion Records Processed'". The Guardian. Retrieved August 5, 2013.
- Layne, Ken (June 18, 2013). "Job Networking Site LinkedIn Filled With Secret NSA Program Names". Retrieved August 6, 2013.
- Nakashima, Ellen (July 31, 2013). "Newly Declassified Documents on Phone Records Program Released". The Washington Post. Retrieved August 6, 2013.
- Fisher, Max (August 1, 2013). "Is XKeyscore Still Active? Defense Contractor Posted a Job Listing for It 2 weeks Ago". WorldViews (blog of The Washington Post). Retrieved August 6, 2013.
- Rea, Kari (July 28, 2013). "Glenn Greenwald: Low-Level NSA Analysts Have 'Powerful and Invasive' Search Tool". ABC News. Retrieved August 4, 2013.
- NSA Press Statement on 30 July 2013
- Wills, Amanda (August 1, 2013). "New Snowden Leak: NSA Program Taps All You Do Online". Mashable (via CNN). Retrieved August 4, 2013.
- Staff (July 31, 2013). "XKeyscore Presentation from 2008 – Read in Full". The Guardian. Retrieved August 6, 2013.
- "Snowden Interview Transcript". Norddeutscher Rundfunk. Retrieved 27 January 2014.
- Staff (undated; circa July 2013). "No alvo dos EUA – O big-brother na América Latina e no mundo" [The U.S. Targets – Big Brother in Latin America and in the World]. O Globo (in Portuguese). Retrieved August 5, 2013.
- Dorling, Philip (July 8, 2013). "Snowden Reveals Australia's Links to US Spy Web". The Sydney Morning Herald. Retrieved August 2, 2013.
- Greenwald, Glenn; Casado, Roberto Kaz e José (July 6, 2013). "EUA expandem o aparato de vigilância continuamente – Software de vigilância usa mais de 700 servidores espalhados pelo mundo". O Globo (in Portuguese). Retrieved August 2, 2013.
- Ambinder, Marc (July 31, 2013). "What's XKEYSCORE?". The Compass (blog of The Week). Retrieved August 4, 2013.
- Gunnar Rensfeldt. "Read the Snowden Documents From the NSA". Sveriges Television. Retrieved 21 December 2013.
- See also: 3 slides about the XKeyscore program
- Greenwald, Glenn (July 31, 2013)."XKeyscore: NSA tool collects 'nearly everything a user does on the internet' – XKeyscore Gives 'Widest-Reaching' Collection of Online Data – NSA Analysts Require No Prior Authorization for Searches – Sweeps Up Emails, Social Media Activity and Browsing History". The Guardian. Retrieved August 1, 2013.
- Gallagher, Sean (August 1, 2013). "NSA's Internet Taps Can Find Systems to Hack, Track VPNs and Word Docs – X-Keyscore Gives NSA the Ability to Find and Exploit Vulnerable Systems". Ars Technica. Retrieved August 4, 2013.
- Greenwald, Glenn; Casado, Roberto Kaz e José (July 13, 2013). "Espionagem dos EUA se espalhou pela América Latina – Depois do Brasil, Colômbia foi o país mais vigiado – Venezuela também entrou na mira de programas americanos" [U.S. Spying Spread Through Latin America – After Brazil, Colombia Was the Country's Most Watched – Venezuela Also Came in the Crosshairs of U.S. Programs]. O Globo (in Portuguese). Retrieved August 5, 2013.
- Laura Poitras, Marcel Rosenbach and Holger Stark, Ally and Target: US Intelligence Watches Germany Closely, August 12, 2013.
- Staff (July 20, 2013). "'Prolific Partner': German Intelligence Used NSA Spy Program". Der Spiegel. Retrieved August 5, 2013.
- Top Level Telecommunications, New slides about NSA collection programs, July 16, 2013
- "Cold War treaty confirms Sweden was not neutral". The Local. Retrieved 12 December 2013.
- Gunnar Rensfeldt. "Read the Snowden Documents From the NSA". Sveriges Television. Retrieved 12 December 2013.