Algorithmic bias

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Jarble (talk | contribs) at 15:28, 12 February 2018 (linking). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

A flow chart showing the decisions made by a recommendation engine, circa 2001.[1]

Algorithmic bias occurs when data used to teach a machine learning system reflects implicit values of humans involved in that data collection, selection, or use.[2] Algorithmic bias has been identified and critiqued for its impact on search engine results,[3] social media platforms,[4] privacy,[5] and racial profiling.[6] In search results, this bias can create results reflecting racist, sexist, or other social biases, despite the presumed neutrality of the data.[7] The study of algorithmic bias is most concerned with algorithms that reflect "systematic and unfair" discrimination.[8]: 332 

As algorithms expand their ability to organize society, politics, institutions, and behavior, sociologists have become concerned with the ways unanticipated output and manipulation can impact the physical world.[9]: 2 [10]: 563 [11]: 294  Because algorithms are often considered to be neutral and unbiased, they can inaccurately project greater authority than human expertise,[12]: 15 [13] and in some cases, reliance on algorithms can displace human responsibility for their outcomes.[12]: 16  Nonetheless, bias can enter into algorithmic systems as a result of pre-existing cultural, social, or institutional expectations; because of technical limitations of their design; or through use in unanticipated contexts or by audiences not considered in their initial design.[8]: 334 

Algorithmic bias has been discovered or theorized in cases ranging from election outcomes[14]: 335  to the spread of online hate speech.[15] Problems in understanding, researching, and discovering algorithmic bias may come from the proprietary nature of algorithms, which are typically treated as trade secrets.[9]: 2  Even with full transparency, understanding algorithms can be difficult because of their complexity,[9]: 2 [16]: 7  and because not every permutation of an algorithm's input or output can be anticipated or reproduced.[17]: 118  In many cases, even within a single use-case (such as a website or app), there is no single "algorithm" to examine, but a vast network of interrelated programs and data inputs,[18]: 367  even between users of the same service.[9]: 5 

Definitions

A 1969 diagram for how a simple computer program makes decisions, illustrating a very simple algorithm.

Algorithms are difficult to define,[19] but may be generally understood as sets of instructions within computer programs that determine how these programs read, collect, process, and analyze data to generate some readable form of analysis or output.[20]: 13  Newer computers can process millions of these algorithmic instructions per second, which has boosted the design and adoption of technologies such as machine learning and artificial intelligence.[21]: 14–15  By analyzing and processing data, algorithms drive search engines,[22] social media websites,[23] recommendation engines,[24] online retail,[25] online advertising,[26] and more.[27]

Contemporary social scientists are concerned with algorithmic processes embedded into hardware and software applications in order to understand their political effects, and to question the underlying assumptions of their neutrality.[9]: 2 [10]: 563 [11]: 294 [28] The term algorithmic bias is used to describe systematic and repeatable errors that create unfair outcomes, i.e., generating one result for certain users and another result for others. For example, a credit score algorithm may deny a loan based on certain factors without being unfair if it is consistently weighing relevant financial criteria. If that algorithm allows loans to some, but denies loans to another set of nearly identical users based on arbitrary criteria, and if this behavior can be repeated across multiple occurrences, an algorithm can be described as biased.[8]: 332  This bias may be intentional or unintentional.[8]: 332 

Methods

Bias can be introduced to an algorithm in several ways. During the assemblage of a database, data must be collected, digitized, adapted, and entered according to human-assisted cataloging criteria.[29]: 3  Next, in the design of the algorithm, programmers assign certain priorities, or hierarchies, in how programs assess and sort that data. This requires human decisions about how data is categorized and which data is discarded.[29]: 4  Some algorithms collect their own data based on human-selected criteria, which can reflect the bias of human users.[29]: 8  Others may practice reinforcing stereotypes and preferences as they process and display "relevant" data for human users, as in selecting information based on previous choices of a user, or group of users.[29]: 6 

Beyond assembling the data, bias can emerge as a result of design.[30] Examples may arise: In sorting processes that determine the allocation of resources or scrutiny (as in determining school placements), or classification and identification processes that may inadvertently discriminate against a category when assigning risk (as in credit scores).[31]: 36  In processing associations, such as recommendation engines or inferred marketing traits, algorithms may be flawed in ways that reveal personal information. Inclusion and exclusion criteria may have unanticipated outcomes for search results, such as in flight recommendation software omitting flights that don't follow the sponsoring airline's preferred flight paths.[30] Algorithms may also display an uncertainty bias, offering more confident assessments when larger data sets are available. This can skew algorithmic processes toward results that more closely correspond with larger sample populations, which may not align with data from underrepresented populations.[32]: 4 

History

Early critiques

The earliest computer programs reflected simple, human-derived operations, and were deemed to be functioning when they completed those operations. Artificial Intelligence pioneer Joseph Weizenbaum wrote that such programs are therefore understood to "embody law".[33]: 40  Weizenbaum describes early, simple computer programs changing perceptions of machines from transferring power to transferring information.[33]: 41  However, he noted that machines might transfer information with unintended consequences if there are errors in details provided to the machine, and if users interpret data in intuitive ways that cannot be formally communicated to, or from, a machine.[33]: 65  Weizenbaum stated that all data fed to a machine must reflect "human decisionmaking processes" which have been translated into rules for the computer to follow.[33]: 70 : 105  To do this, Weizenbaum asserted that programmers "legislate the laws for a world one first has to create in imagination,"[33]: 109  and as a result, computer simulations can be built on models with incomplete or incorrect human data.[33]: 149  Weizenbaum compared the results of such decisions to a tourist in a country who can make "correct" decisions through a coin toss, but has no basis of understanding how or why the decision was made.[33]: 226 

Contemporary critiques

The complexity of analyzing algorithmic bias has grown alongside the complexity of programs and their design. Decisions made by one designer, or team of designers, may be obscured among the many pieces of code created for a single program; over time these decisions and their impact may be forgotten and taken as natural results of the program's output.[17]: 115  These biases can create new patterns of behavior, or "scripts," in relationship to specific technologies as the code interacts with other elements of society.[34] Biases may also impact how society shapes itself around the data points that algorithms require.[35]: 180 

The decisions of algorithmic programs can be weighed more heavily than the decisions of the human beings they are meant to assist,[12]: 15  a process described by author Clay Shirky as "algorithmic authority".[13] Shirky uses the term to describe "the decision to regard as authoritative an unmanaged process of extracting value from diverse, untrustworthy sources," such as search results.[13] This neutrality can also be misrepresented through language frames used when results are presented to the public. For example, a list of news items selected and presented as "trending" or "popular" may be weighed based on significantly wider criteria than their popularity.[29]: 14 

Because of their convenience and authority, algorithms are theorized as a means of delegating responsibility in decision making away from humans.[12]: 16 [36]: 6  This can have the effect of reducing alternative options, compromises, or flexibility.[12]: 16  Sociologist Scott Lash has critiqued algorithms as a new form of "generative power" in that they are a virtual means of generating actual ends.[37]: 71 

Types

Pre-existing

This card was used to load software into an old mainframe computer. Each byte (the letter 'A', for example) is entered by punching holes. Though contemporary computers are more complex, they reflect this human decision-making process in collecting and processing data.[33]: 70 [38]: 16 

Pre-existing bias in an algorithm is a consequence of underlying social and institutional ideologies. Such ideas may reflect personal biases of individual designers or programmers, or can reflect social, institutional, or cultural assumptions. In both cases, such prejudices can be explicit and conscious, or implicit and unconscious.[8]: 334 [11]: 294  Poorly selected input data will influence the outcomes created by machines.[38]: 17  In a critical view, encoding pre-existing bias into software can preserve social and institutional bias, and replicate it into all possible uses of the algorithm into the future.[17]: 116 [36]: 8 

An example of this form of bias is the British Nationality Act Program, designed to automate the evaluation of new UK citizens after the 1981 British Nationality Act.[8]: 341  The program accurately reflected the tenets of the law, which stated that "a man is the father of only his legitimate children, whereas a woman is the mother of all her children, legitimate or not."[8]: 341 [39]: 375  By attempting to appropriately articulate this logic into an algorithmic process, the BNAP inscribed the logic of the British Nationality Act into its algorithm.[8]: 342 

Technical

Technical bias emerges through limitations of a program, computational power, its design, or other constraint on the system.[8]: 332  Such bias can also be a restraint of design, for example, a search engine that shows three results per screen can be understood to privilege the top three results slightly more than the next three, as in an airline price display.[8]: 336  Flaws in random number generation can also introduce bias into results.[8]: 332 

A decontextualized algorithm uses unrelated information to sort results, for example, a flight-pricing algorithm that sorts results by alphabetical order would be biased in favor of American Airlines over United Airlines.[8]: 332  The opposite may also apply, in which results are evaluated in different contexts from which they are collected. For example, data may be collected without crucial external context when facial recognition software is used by surveillance cameras, but evaluated by remote staff in another country or region, or evaluated by non-human algorithms with no awareness of what takes place beyond the camera's field of vision.[10]: 574 

Lastly, technical bias can be created by attempting to formalize decisions into concrete steps on the assumption that human behavior will correlate. For example, software that weighs data points to determine whether a defendant should accept a plea bargain, while ignoring the impact of emotion on a jury, is displaying a form of technical bias.[8]: 332 

Emergent

Emergent bias is the result of the use and reliance on algorithms across new or unanticipated contexts.[8]: 334  New forms of knowledge, such as drug or medical breakthroughs, new laws, business models, or shifting cultural norms, may be discovered without algorithms being adjusted to consider them.[8]: 334, 336  This may exclude groups through technology, without delineating clear outlines of authorship or personal responsibility.[35]: 179 [11]: 294  Similarly, problems may emerge when training data, i.e., the samples "fed" to a machine by which it models certain conclusions, do not align with uses that algorithm encounters in the real world.[40]

In 1990, an example of emergent bias was identified in the software used to place US medical students into residencies, the National Residency Match Program (NRMP).[8]: 338  The algorithm was designed at a time when few married couples would seek residencies together. As more women entered medical schools, more students were likely to request a residency alongside their partners. The process calls for each applicant to provide a list of preferences for placement across the US, which is then sorted and assigned when a hospital and an applicant both agree to a match. In the case of married couples where both sought residencies, the algorithm weighed a "lead member's" location choices first. Once it identified an optimum placement for that person, it removed distant locations from their partner's preferences, reducing their list to the preferred locations within the same city as the partner. The result was a frequent assignment of high-rated schools for the first partner and lower-preference schools to the second partner, rather than sorting for compromises in placement preference.[8]: 338 [41]

Additional emergent biases include:

Correlations

Unpredictable correlations can emerge when large data sets are compared to each other in practice. For example, data collected about web-browsing patterns may align with signals marking sensitive data (such as race or sexual orientation). By "discrimination" against certain behavior or browsing patterns, the end effect would be almost identical to discrimination through the use of direct race or orientation data.[32]: 6  In other cases, correlations can be inferred for reasons beyond the algorithm's ability to understand them, as when a triage program gave lower priority to asthmatics who had pneumonia. Because asthmatics with pneumonia were at the highest risk, hospitals typically give them the best and most immediate care; the algorithm simply compared survival rates.[42]

Unanticipated uses

Emergent bias can occur when an algorithm is used by unanticipated audiences, such as machines that demand users can read, write, or understand numbers. Certain metaphors may not carry across different populations or skill sets.[8]: 334  For example, the British National Act Program was created as a proof-of-concept by computer scientists and immigration lawyers to evaluate suitability for British citizenship. The designers therefore have expertise beyond the user, whose understanding of both software and immigration law would likely be unsophisticated. The agents administering the questions would not be aware of alternative pathways to citizenship outside of the software, and shifting case law and legal interpretations would lead the algorithm to outdated results.[8]: 342 

An area of concern around emergent bias is that it may be compounded as biased technology is more deeply integrated into society. For example, users with vision impairments may not be able to use an ATM, but can easily go to a bank branch. If bank branches begin to close because ATMs replace them, they begin to exclude vision-impaired users from banking, an unintended consequence of a technology.[35]: 179 

Feedback loops

Emergent bias may also create a feedback loop, or recursion, if data collected for an algorithm results in real-world responses which are fed back into the algorithm.[43][44] For example, simulations of the predictive policing software, PredPol, deployed in Oakland, California, suggested an increased police presence in black neighborhoods based on crime data reported by the public.[45] The simulation showed that public reports of crime could rise based on the sight of increased police activity, and could be interpreted by the software in modeling predictions of crime, and to encourage a further increase in police presence within the same neighborhoods.[43][46][47] The Human Rights Data Analysis Group, which conducted the study, warned that in places where racial discrimination is a factor in arrests, such feedback loops could reinforce and perpetuate racial discrimination in policing.[44]

Examples

Facial recognition software used in conjunction with surveillance cameras was found to display bias in recognizing Asian and black faces over white faces.[35]: 191 

College admissions

An early example of algorithmic bias resulted in as many as 60 women and ethnic minorities denied entry to St. George's Hospital Medical School per year from 1982 to 1986, based on implementation of a new computer-guidance assessment system that denied entry to women and men with "foreign-sounding names" based on historical trends in admissions.[48] Other examples include the display of higher-paying jobs to male applicants on job search websites.[49]

Plagiarism detection

The plagiarism-detection software Turnitin compares student-written texts to information found online and returns a probability that the student's work is copied. Because the software compares strings of text, it is more likely to identify non-native speakers of English than native speakers, who might be better able to adapt individual words and break up strings of plagiarized text, or obscure them through synonyms.[12]: 21–22 

Facial recognition

Surveillance camera software may be considered inherently political as it requires algorithms to identify and flag normal from abnormal behaviors, and to determine who belongs in certain locations at certain times.[10]: 572  A 2002 analysis of facial recognition software used to identify individuals in CCTV images found several examples of bias. Software was assessed as identifying men more frequently than women, older people more frequently than younger people, and identified Asians, African-Americans and other races more often than whites.[35]: 191 [50]

Social media

In 2017 a Facebook algorithm designed to remove online hate speech was found to advantage white men over black children when assessing objectionable content, according to internal Facebook documents.[15] The algorithm, which is a blend of computer programs and human content reviewers, was created to protect broad categories rather than specific subsets of categories. For example, posts denouncing "Muslims" would be blocked, while posts denouncing "Radical Muslims" would be allowed. An unanticipated outcome of the algorithm is to allow hate speech against black children, because they denounce the "children" subset of blacks, rather than "all blacks," whereas "all white men" would trigger a block, because whites and males are not considered subsets.[15] Facebook was also found to allow ad purchasers to target "Jew haters" as a category of users, which the company said was an inadvertent outcome of algorithms used in assessing and categorizing data. The company's design also allowed ad buyers to block African-Americans from seeing housing ads.[51]

Impact

Commercial influences

Corporate algorithms could be skewed to invisibly favor financial arrangements or agreements between companies, without the knowledge of a user who may mistake the algorithm as being impartial. For example, American Airlines created a flight-finding algorithm in the 1980s. The software presented a range of flights from various airlines to customers, but weighed factors that boosted its own flights, regardless of price or convenience. In testimony to the United States Congress, the president of the airline stated outright that the system was created with the intention of gaining competitive advantage through preferential treatment.[52]: 2 [8]: 331 

In a 1998 paper describing Google, the founders of the company adopted a policy of transparency in search results regarding paid placement, arguing that “advertising-funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.”[53] This bias would be an "invisible" manipulation of the user.[52]: 3 

Voting behavior

A series of studies of undecided voters in the US and in India found that search engine results were able to shift voting outcomes by about 20%. The researchers concluded that candidates have "no means of competing" if an algorithm, with or without intent, boosted page listings for a rival candidate.[54]

Facebook users who saw messages related to voting were more likely to vote themselves. A randomized trial of Facebook users showing an increased effect of 340,000 votes among users, and friends of users, who saw pro-voting messages in 2010.[55] The legal scholar Jonathan Zittrain has warned that this could create a "digital gerrymandering" effect in elections, "the selective presentation of information by an intermediary to meet its agenda, rather than to serve its users," if intentionally manipulated.[14]: 335 

Gender discrimination

In 2016, the professional networking site LinkedIn was discovered to recommend male variations of women's names in response to search queries for women. The site did not make similar recommendations in searches for male names. For example, "Andrea" would bring up a prompt asking if users meant "Andrew," but queries for "Andrew" did not ask if users meant to find "Andrea". The company said this was the result of an analysis of users' interactions with the site.[56]

In 2012, the department store franchise Target was cited for gathering data points to infer when women customers were pregnant, even if they hadn't announced it, and then sharing that information with marketing partners.[57]: 94 [58] Because the data had been predicted, rather than directly observed or reported, the company had no legal obligation to protect the privacy of those customers.[57]: 98 

Web search algorithms have also been accused of bias. Google's results may prioritize pornographic content in search terms related to sexuality, for example, "lesbian". This bias extends to the search engine surfacing popular but sexualized content in neutral searches, as in "Top 25 Sexiest Women Athletes" articles displayed as first-page results in searches for "women athletes".[59]: 31  In 2017 Google announced plans to curb search results that surfaced hate groups, racist views, child abuse and pornography, and other upsetting and offensive content.[60]

Racial discrimination

Algorithms have been criticized as a method for obscuring racial prejudices in decision-making.[6]: 158  Lisa Nakamura has noted that census machines were among the first to adopt the punch-card processes that lead to contemporary computing, and that their use as categorization and sorting machines for race has been long established and socially tolerated.[6]: 158 

One example is the use of risk assessments in criminal sentencing and parole hearings, an algorithmically generated score intended to reflect the risk that a suspect or prisoner will repeat a crime.[61] From 1920 until 1970, the nationality of a suspect's father was a consideration in such risk assessments.[62]: 4  Today, these scores are shared with judges in Arizona, Colorado, Delaware, Kentucky, Louisiana, Oklahoma, Virginia, Washington, and Wisconsin. An independent investigation by ProPublica found that the scores were inaccurate 80% of the time, and disproportionately skewed to suggest blacks to be at risk of relapse, 77% more often than whites.[61]

In 2015, Google apologized when black users complained that an image-identification algorithm in its Photos application identified them as gorillas.[63] In 2010, Nikon cameras were criticized when image-recognition algorithms consistently asked Asian users if they were blinking.[64] Such examples are the product of bias in biometric data sets.[63] Biometric data is drawn from aspects of the body, including racial features either observed or inferred, which can then be transferred into data points.[6]: 154 

Biometric data about race may also be inferred, rather than observed. For example, a 2012 study showed that names commonly associated with blacks were more likely to yield search results implying arrest records, regardless of any police record of that individual's name.[65]

Sexual discrimination

In 2011, users of the gay hookup app Grindr reported that the app was linked to sex-offender lookup apps in the Android app store's recommendation algorithm. Writer Mike Ananny criticized this association in The Atlantic, arguing that such associations further stigmatized gay men and may discourage closeted men to maintain secrecy.[66] A 2009 incident with online retailer Amazon saw 57,000 books de-listed after a shift in the algorithm expanded its "adult content" blacklist for pornographic works to any books addressing sexuality or gay themes, for example, the critically acclaimed novel Brokeback Mountain.[67][29]: 5 [68]

Challenges

Several challenges impede the study of large-scale algorithmic bias, hindering the application of academically rigorous studies and public understanding.[9]: 5 

Lack of transparency

Commercial algorithms are proprietary, and may be treated as trade secrets.[9]: 2 [16]: 7 [35]: 183  This protects companies, such as a search engine, in cases where a transparent algorithm for ranking results would reveal techniques for manipulating the service.[18]: 366  This makes it difficult for researchers to conduct interviews or analysis to discover how algorithms function.[69]: 20  It can also be used to obscure possible unethical methods used in producing or processing algorithmic output.[18]: 369  The closed nature of the code is not the only concern, however; as a certain degree of obscurity is protected by the complexity of contemporary programs, and the inability to know every permutations of a code's input or output.[35]: 183 

Social scientist Bruno Latour has identified this process as blackboxing, a process in which "scientific and technical work is made invisible by its own success. When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed, the more opaque and obscure they become."[70] Others have critiqued the black box metaphor, suggesting that current algorithms are not one black box, but a network of interconnected ones.[71]: 92 

Complexity

Algorithmic processes are complex, often exceeding the understanding of the people who use them.[9]: 2 [16]: 7  Large-scale operations may not be understood even by those involved in creating them.[72] The social media site Facebook factored in at least 100,000 data points to determine the layout of a user's social media feed in 2013.[73] Furthermore, large teams of programmers may operate in relative isolation from one another, and be unaware of the cumulative effects of small decisions with nested sections of sprawling algorithmic processes.[17]: 118  Not all code is original, and may be borrowed from other libraries, creating a complicated set of relationships between data processing and data input systems.[69]: 22 

Lack of data about sensitive categories

A significant barrier to understanding tackling bias in practice is that categories, such as demographics of individuals protected by anti-discrimination law, are often not explicitly held by those collecting and processing data.[74] In some cases, there is little opportunity to collect this data explicitly, such as in device fingerprinting, ubiquitous computing and the Internet of Things. In other cases, the data controller may not wish to collect such data for reputational reasons, or because it represents a heightened liability and security risk. It may also be the case that, at least in relation to the General Data Protection Regulation, such data falls under the 'special category' provisions (Article 9), and therefore comes with more restrictions on potential collection and processing.

Algorithmic bias does not only relate to protected categories, but can also concern something less easily observable or codifiable, such as political viewpoints. In these cases, there is rarely an easily accessible or non-controversial ground truth, and 'debiasing' such a system becomes considerably more tricky.[75]

Furthermore, false and accidental correlations can emerge from a lack of understanding of protected categories, for example, insurance rates based on historical data of car accidents which may overlap with residential clusters of ethnic minorities.[76]

Rapid pace of change

Personalization of algorithms based on user interactions such as clicks, time on site, and other metrics, can confuse attempts to understand them.[18]: 367 [16]: 7  One unidentified streaming radio service reported it had five unique music-selection algorithms it selected for its users based on behavior. This creates widely disparate experiences of the same streaming product between different users.[9]: 5  Companies also run frequent A/B tests to fine-tune algorithms based on user response. For example, the search engine Bing can run up to ten million subtle variations of its service per day, segmenting the experience of an algorithm between users, or among the same users.[9]: 5 

Rapid pace of dissemination

Computer programs and systems can quickly spread among users, embedding biased algorithms into broader society before their impact can be recognized or remedied.[8]: 331 

Lack of consensus about goals and criteria

A lack of understanding the mechanisms and consequences of an algorithm may be the result of competing motives within an organization, for example, the availability of loans from a bank as opposed to the bank's profit incentive. A simple algorithm designed with a single purpose, such as expanding profits, would reduce loans to higher-risk applicants. In order to minimize discrimination, a banking algorithm would have to balance its interest in short-term profit, and apply the same technical criteria to all applicants, ensure an equal ratio of loans across each group of applicants, and grant loans to an equal ratio of applicants from each category[77].

Regulation

Europe

The General Data Protection Regulation (GDPR), the European Union's revised data protection regime that enters force in 2018, addresses "Automated individual decision-making, including profiling" in Article 22. These rules prohibit "solely" automated decisions which have a "significant" or "legal" effect on an individual, unless they are explicitly authorised by consent, contract, or member state law. Where they are permitted, there must be safeguards in place, such as a right to a human-in-the-loop, and allegedly (although for political reasons, only in a non-binding recital) a right to an explanation of decisions reached. While these are commonly considered to be new, it is the case that nearly identical provisions have existed across Europe since 1995 in Article 15 of the Data Protection Directive, with the original automated decision rules and safeguards originating in French law in the later 1970s.[78] They have rarely been used, given the heavy carve-outs that exist,[79] and are not discussed in any case law of the European Court of Justice.[80]

The GDPR does address algorithmic bias in profiling systems, as well as the statistical approaches possible to clean it, directly in recital 71,[81] noting that

the controller should use appropriate mathematical or statistical procedures for the profiling, implement technical and organisational measures appropriate [...] that prevents, inter alia, discriminatory effects on natural persons on the basis of racial or ethnic origin, political opinion, religion or beliefs, trade union membership, genetic or health status or sexual orientation, or that result in measures having such an effect.

Like the alleged right to an explanation in recital 71, this suffers from the non-binding nature of recitals compared to the binding articles,[82] and while it has been treated as a requirement by the Article 29 Working Party that advise on the implementation of data protection law,[81] its practical dimensions are unclear. It has been argued that the obligatory Data Protection Impact Assessments for high risk data profiling, in tandem with other pre-emptive measures within data protection, may be a better way to tackle issues of algorithmic discrimination than relying on individual transparency rights, as information rights have traditionally fatigued individuals who are too overwhelmed and overburdened to use them effectively.[79]

United States

The United States has no overall legislation regulating controls for algorithmic bias, approaching the topic through various state and federal laws that might vary by industries, sectors, and uses.[83] Many policies are self-enforced or controlled by the Federal Trade Commission.[83] In 2016, the Obama administration released the National Artificial Intelligence Research and Development Strategic Plan,[84] which called for a critical assessment of algorithms and for researchers to "design these systems so that their actions and decision-making are transparent and easily interpretable by humans, and thus can be examined for any bias they may contain, rather than just learning and repeating these biases".[85]: 26 

References

  1. ^ us 7113917 
  2. ^ Nissenbaum, Helen (March 2001). "How computer systems embody values" (PDF). Computer. 34 (3): 120–119. doi:10.1109/2.910905. Retrieved 17 November 2017.
  3. ^ Introna, Lucas; Nissenbaum, Helen (2000). "Defining the Web: the politics of search engines" (PDF). Computer. 33 (1): 54–62. doi:10.1109/2.816269. Retrieved 17 November 2017.
  4. ^ Crawford, Kate (24 June 2015). "Can an Algorithm be Agonistic? Ten Scenes from Life in Calculated Publics". Science, Technology, & Human Values. 41 (1): 77–92. doi:10.1177/0162243915589635.
  5. ^ Tufekci, Zeynep (2015). "Algorithmic Harms beyond Facebook and Google: Emergent Challenges of Computational Agency". Colorado Technology Law Journal Symposium Essays. 13: 203–216. Retrieved 17 November 2017.
  6. ^ a b c d Nakamura, Lisa (2009). Magnet, Shoshana; Gates, Kelly (eds.). The new media of surveillance. London: Routledge. pp. 149–162. ISBN 9780415568128.
  7. ^ Sydell, Laura. "Can Computers Be Racist? The Human-Like Bias Of Algorithms". NPR.org. National Public Radio / All Things Considered. Retrieved 17 November 2017.
  8. ^ a b c d e f g h i j k l m n o p q r s t u Friedman, Batya; Nissenbaum, Helen (July 1996). "Bias in Computer Systems" (PDF). ACM Transactions on Information Systems. 14 (3): 330–347. Retrieved 18 November 2017.
  9. ^ a b c d e f g h i j Seaver, Nick. "Knowing Algorithms" (PDF). Media in Transition 8, Cambridge, MA, April 2013. Retrieved 18 November 2017.
  10. ^ a b c d Graham, Stephen D.N. (July 2016). "Software-sorted geographies". Progress in Human Geography. 29 (5): 562–580. doi:10.1191/0309132505ph568oa.
  11. ^ a b c d Tewell, Eamon (4 April 2016). "Toward the Resistant Reading of Information: Google, Resistant Spectatorship, and Critical Information Literacy". Portal: Libraries and the Academy. 16 (2): 289–310. ISSN 1530-7131. Retrieved 19 November 2017.
  12. ^ a b c d e f Introna, Lucas D. (21 December 2006). "Maintaining the reversibility of foldings: Making the ethics (politics) of information technology visible". Ethics and Information Technology. 9 (1): 11–25. doi:10.1007/s10676-006-9133-z.
  13. ^ a b c Shirky, Clay. "A Speculative Post on the Idea of Algorithmic Authority Clay Shirky". www.shirky.com. Retrieved 20 November 2017.
  14. ^ a b Zittrain, Jonathan (2014). "Engineering an Election" (PDF). Harvard Law Review Forum. 127: 335–341. Retrieved 19 November 2017.
  15. ^ a b c Angwin, Julia; Grassegger, Hannes (28 June 2017). "Facebook's Secret Censorship Rules Protect White Men From Hate Speech But Not Black Children — ProPublica". ProPublica. Retrieved 20 November 2017.
  16. ^ a b c d Sandvig, Christian; Hamilton, Kevin; Karahalios, Karrie; Langbort, Cedric (2014). Gangadharan, Seeta Pena; Eubanks, Virginia; Barocas, Solon (eds.). "An Algorithm Audit" (PDF). Data and Discrimination: Collected Essays. Open Technology Institute.
  17. ^ a b c d Introna, Lucas D. (2 December 2011). "The Enframing of Code". Theory, Culture & Society. 28 (6): 113–141. doi:10.1177/0263276411418131.
  18. ^ a b c d Granka, Laura A. (27 September 2010). "The Politics of Search: A Decade Retrospective" (PDF). The Information Society. 26 (5): 364–374. doi:10.1080/01972243.2010.511560. Retrieved 18 November 2017.
  19. ^ Striphas, Ted. "What is an Algorithm? – Culture Digitally". culturedigitally.org. Retrieved 20 November 2017.
  20. ^ Corman, Thomas H. (2009). Introduction to algorithms (3rd ed.). Cambridge, Mass.: MIT Press. p. 5. ISBN 0262033844. {{cite book}}: Unknown parameter |displayauthors= ignored (|display-authors= suggested) (help)
  21. ^ Kitchin, Rob (25 February 2016). "Thinking critically about and researching algorithms" (PDF). Information, Communication & Society. 20 (1): 14–29. doi:10.1080/1369118X.2016.1154087. Retrieved 19 November 2017.
  22. ^ Google. "How Google Search Works". Google. Retrieved 19 November 2017. {{cite web}}: |author= has generic name (help)
  23. ^ Luckerson, Victor. "Here's How Your Facebook News Feed Actually Works". TIME.com. Retrieved 19 November 2017.
  24. ^ Vanderbilt, Tom. "The Science Behind the Netflix Algorithms That Decide What You'll Watch Next". WIRED. Retrieved 19 November 2017.
  25. ^ Angwin, Julia; Mattu, Surya (20 September 2016). "Amazon Says It Puts Customers First. But Its Pricing Algorithm Doesn't — ProPublica". ProPublica. Retrieved 19 November 2017.
  26. ^ Livingstone, Rob. "The future of online advertising is big data and algorithms". The Conversation. Retrieved 19 November 2017.
  27. ^ Hickman, Leo (1 July 2013). "How algorithms rule the world". The Guardian. Retrieved 19 November 2017.
  28. ^ Crawford, Kate (1 April 2013). "The Hidden Biases in Big Data". Harvard Business Review. {{cite web}}: Cite has empty unknown parameter: |dead-url= (help)
  29. ^ a b c d e f Gillespie, Tarleton; Boczkowski, Pablo; Foot, Kristin. Media Technologies (PDF). Cambridge: MIT Press. pp. 1–30. Retrieved 22 November 2017.
  30. ^ a b Diakopoulas, Nicholas. "Algorithmic Accountability: On the Investigation of Black Boxes |". towcenter.org. Retrieved 19 November 2017.
  31. ^ Lipartito, Kenneth (6 January 2011). "The Narrative and the Algorithm: Genres of Credit Reporting from the Nineteenth Century to Today". SSRN. Social Science Research Network. doi:10.2139/ssrn.1736283. Retrieved 19 November 2017.
  32. ^ a b Goodman, Bryce; Flaxman, Seth (2016). "EU regulations on algorithmic decision-making and a "right to explanation"". arXiv:1606.08813 [stat.ML].
  33. ^ a b c d e f g h Weizenbaum, Joseph (1976). Computer power and human reason : from judgment to calculation. San Francisco: W.H. Freeman. ISBN 0716704641.
  34. ^ Bogost, Ian. "The Cathedral of Computation". The Atlantic. Retrieved 19 November 2017.
  35. ^ a b c d e f g Introna, Lucas; Wood, David (2004). "Picturing algorithmic surveillance: the politics of facial recognition systems". Surveillance & Society. 2: 177–198. Retrieved 19 November 2017.
  36. ^ a b Ziewitz, Malte (1 January 2016). "Governing Algorithms: Myth, Mess, and Methods". Science, Technology, & Human Values. 41 (1): 3–16. doi:10.1177/0162243915608948. ISSN 0162-2439. Retrieved 22 November 2017.
  37. ^ Lash, Scott (30 June 2016). "Power after Hegemony". Theory, Culture & Society. 24 (3): 55–78. doi:10.1177/0263276407075956.
  38. ^ a b Goffrey, Andrew (2008). "Algorithm". In Fuller, Matthew (ed.). Software studies: a lexicon. Cambridge, Mass.: MIT Press. pp. 15–20. ISBN 9781435647879.
  39. ^ Sergot, MJ; Sadri, F; Kowalski, RA; Kriwaczek, F; Hammond, P; Cory, HT (May 1986). "The British Nationality Act as a Logic Program" (PDF). Communications of the ACM. 29 (5): 370–386. Retrieved 18 November 2017.
  40. ^ Gillespie, Tarleton. "Algorithm [draft] [#digitalkeywords] – Culture Digitally". culturedigitally.org. Retrieved 20 November 2017.
  41. ^ Roth, A. E. 1524 –1528. (14 December 1990). "New physicians: A natural experiment in market organization". Science. 250 (4987): 1524–1528. Bibcode:1990Sci...250.1524R. doi:10.1126/science.2274783. Retrieved 18 November 2017.{{cite journal}}: CS1 maint: numeric names: authors list (link)
  42. ^ Kuang, Cliff (21 November 2017). "Can A.I. Be Taught to Explain Itself?". The New York Times. Retrieved 26 November 2017.
  43. ^ a b Jouvenal, Justin (17 November 2016). "Police are using software to predict crime. Is it a 'holy grail' or biased against minorities?". Washington Post. Retrieved 25 November 2017.
  44. ^ a b Chamma, Maurice. "Policing the Future". The Marshall Project. Retrieved 25 November 2017.
  45. ^ Lum, Kristian; Isaac, William (October 2016). "To predict and serve?". Significance. 13 (5): 14–19. doi:10.1111/j.1740-9713.2016.00960.x. Retrieved 25 November 2017.
  46. ^ Smith, Jack. "Predictive policing only amplifies racial bias, study shows". Mic. Retrieved 25 November 2017.
  47. ^ Lum, Kristian; Isaac, William (1 October 2016). "FAQs on Predictive Policing and Bias". HRDAG. Retrieved 25 November 2017.
  48. ^ Lowry, Stella; Macpherson, Gordon (5 March 1988). "A Blot on the Profession". British Medical Journal. 296 (6623): 657. Retrieved 17 November 2017.
  49. ^ Simonite, Tom. "Study Suggests Google's Ad-Targeting System May Discriminate". MIT Technology Review. Massachusetts Institute of Technology. Retrieved 17 November 2017.
  50. ^ Furl, N (December 2002). "Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis". Cognitive Science. 26 (6): 797–815. doi:10.1016/S0364-0213(02)00084-8.
  51. ^ Angwin, Julia; Varner, Madeleine; Tobin, Ariana (14 September 2017). "Facebook Enabled Advertisers to Reach 'Jew Haters' — ProPublica". ProPublica. Retrieved 20 November 2017.
  52. ^ a b Sandvig, Christian; Hamilton, Kevin; Karahalios, Karrie; Langbort, Cedric (22 May 2014). "Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms" (PDF). 64th Annual Meeting of the International Communication Association. Retrieved 18 November 2017.
  53. ^ Brin, Sergey; Page, Lawrence. "The Anatomy of a Search Engine". www7.scu.edu.au. Retrieved 18 November 2017.
  54. ^ Epstein, Robert; Robertson, Ronald E. (18 August 2015). "The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections". Proceedings of the National Academy of Sciences. 112 (33): E4512–E4521. Bibcode:2015PNAS..112E4512E. doi:10.1073/pnas.1419828112. Retrieved 19 November 2017.
  55. ^ Bond, Robert M.; Fariss, Christopher J.; Jones, Jason J.; Kramer, Adam D. I.; Marlow, Cameron; Settle, Jaime E.; Fowler, James H. (13 September 2012). "A 61-million-person experiment in social influence and political mobilization". Nature. 489 (7415): 295. Bibcode:2012Natur.489..295B. doi:10.1038/nature11421. ISSN 0028-0836. PMID 22972300. Retrieved 19 November 2017.
  56. ^ Day, Matt (31 August 2016). "How LinkedIn's search engine may reflect a gender bias". The Seattle Times. Retrieved 25 November 2017.
  57. ^ a b Crawford, Kate; Schultz, Jason (2014). "Big Data and Due Process: Toward a Framework to Redress Predictive Privacy Harms". Boston College Law Review. 55 (1): 93–128. Retrieved 18 November 2017.
  58. ^ Duhigg, Charles (16 February 2012). "How Companies Learn Your Secrets". The New York Times. Retrieved 18 November 2017.
  59. ^ Noble, Safiya (2012). "Missed Connections: What Search Engines Say about Women". Bitch Magazine. 12 (4): 37–41.
  60. ^ Guynn, Jessica (16 March 2017). "Google starts flagging offensive content in search results". USA TODAY. USA Today. Retrieved 19 November 2017.
  61. ^ a b Angwin, Julia; Larson, Jeff; Mattu, Surya; Kirchner, Lauren (23 May 2016). "Machine Bias — ProPublica". ProPublica. Retrieved 18 November 2017.
  62. ^ Harcourt, Bernard (16 September 2010). "Risk as a Proxy for Race". Criminology and Public Policy, Forthcoming. Social Science Research Network. Retrieved 18 November 2017.
  63. ^ a b Guynn, Jessica (1 July 2015). "Google Photos labeled black people 'gorillas'". USA TODAY. USA Today. USA Today. Retrieved 18 November 2017.
  64. ^ Rose, Adam (22 January 2010). "Are Face-Detection Cameras Racist?". Time. Retrieved 18 November 2017.
  65. ^ Sweeney, Latanya (28 January 2013). "Discrimination in Online Ad Delivery". SSRI. Social Science Research Network. doi:10.2139/ssrn.2208240. Retrieved 18 November 2017.
  66. ^ Ananny, Mike. "The Curious Connection Between Apps for Gay Men and Sex Offenders". The Atlantic. Retrieved 18 November 2017.
  67. ^ Kafka, Peter. "Did Amazon Really Fail This Weekend? The Twittersphere Says "Yes," Online Retailer Says "Glitch."". AllThingsD. Retrieved 22 November 2017.
  68. ^ Kafka, Peter. "Amazon Apologizes for "Ham-fisted Cataloging Error"". AllThingsD. AllThingsD. Retrieved 22 November 2017.
  69. ^ a b Kitchin, Rob (25 February 2016). "Thinking critically about and researching algorithms". Information, Communication & Society. 20 (1): 14–29. doi:10.1080/1369118X.2016.1154087.
  70. ^ Bruno Latour (1999). Pandora's hope: essays on the reality of science studies. Cambridge, Massachusetts: Harvard University Press.
  71. ^ Kubitschko, Sebastian; Kaun, Anne (2016). Innovative Methods in Media and Communication Research. Springer. ISBN 9783319407005. Retrieved 19 November 2017.
  72. ^ LaFrance, Adrienne. "The Algorithms That Power the Web Are Only Getting More Mysterious". The Atlantic. Retrieved 19 November 2017.
  73. ^ McGee, Matt (16 August 2013). "EdgeRank Is Dead: Facebook's News Feed Algorithm Now Has Close To 100K Weight Factors". Marketing Land. Retrieved 18 November 2017.
  74. ^ Veale, Michael; Binns, Reuben (2017). "Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data". Big Data & Society. 4 (2): 205395171774353. doi:10.1177/2053951717743530. SSRN 3060763.
  75. ^ Binns, Reuben; Veale, Michael; Kleek, Max Van; Shadbolt, Nigel (2017-09-13). "Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation". Social Informatics. Lecture Notes in Computer Science. 10540. Springer, Cham: 405–415. arXiv:1707.01477. doi:10.1007/978-3-319-67256-4_32. ISBN 9783319672557.
  76. ^ Claburn, Thomas. "EU Data Protection Law May End The Unknowable Algorithm – InformationWeek". InformationWeek. Retrieved 25 November 2017.
  77. ^ Wattenberg, Martin; Viégas, Fernanda; Hardt, Moritz. "Attack discrimination with smarter machine learning". research.google.com. Google Labs. Retrieved 1 February 2018.
  78. ^ Bygrave, Lee A (2001). "AUTOMATED PROFILING". Computer Law & Security Review. 17 (1): 17–24. doi:10.1016/s0267-3649(01)00104-2.
  79. ^ a b Edwards, Lilian; Veale, Michael (2017-05-23). "Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For". Duke Law & Technology Review. 16: 18–84. doi:10.2139/ssrn.2972855.
  80. ^ Laudati, Laraine (2016). SUMMARIES OF EU COURT DECISIONS RELATING TO DATA PROTECTION 2000-2015 (PDF). European Commission.
  81. ^ a b Veale, Michael; Edwards, Lilian (2018). "Clarity, Surprises, and Further Questions in the Article 29 Working Party Draft Guidance on Automated Decision-Making and Profiling". Computer Law & Security Review. doi:10.2139/ssrn.3071679. SSRN 3071679.
  82. ^ Wachter, Sandra; Mittelstadt, Brent; Floridi, Luciano (2017-05-01). "Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation". International Data Privacy Law. 7 (2): 76–99. doi:10.1093/idpl/ipx005. ISSN 2044-3994.
  83. ^ a b Singer, Natasha (2 February 2013). "Consumer Data Protection Laws, an Ocean Apart". The New York Times. Retrieved 26 November 2017.
  84. ^ Obama, Barack (12 October 2016). "The Administration's Report on the Future of Artificial Intelligence". whitehouse.gov. National Archives. Retrieved 26 November 2017.
  85. ^ and Technology Council, National Science (2016). National Artificial Intelligence Research and Development Strategic Plan (PDF). US Government. Retrieved 26 November 2017.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Algorithmic_bias&oldid=825289729"