The implicit-association test (IAT) is a measure within social psychology designed to detect the strength of a person's automatic association between mental representations of objects (concepts) in memory. The IAT was introduced in the scientific literature in 1998 by Anthony Greenwald, Debbie McGhee, and Jordan Schwartz. The IAT is now widely used in social psychology research and, to some extent, in clinical, cognitive, and developmental psychology research. The IAT is the subject of much controversy regarding precisely what it measures, and the lack of reproducibility of many of its results.
- 1 History
- 2 Application and use
- 3 Types
- 4 Theoretical interpretation
- 5 Balanced-identity theory and design
- 6 Criticism and controversy
- 6.1 Validity research
- 6.1.1 Construct validity
- 6.1.2 Internal validity
- 6.1 Validity research
- 7 In popular culture
- 8 See also
- 9 References
- 10 External links
Implicit cognition and measurement
In 1995, social psychology researchers Anthony Greenwald and Mahzarin Banaji asserted that the idea of implicit and explicit memory can apply to social constructs as well. If memories that are not accessible to awareness can influence our actions, associations can also influence our attitudes and behavior. Thus, measures that tap into individual differences in associations of concepts should be developed. This would allow researchers to understand attitudes that cannot be measured through explicit self-report methods due to lack of awareness or social-desirability bias. In essence,the purpose of the IAT was to reliably assess individual differences in a manner producing large effect sizes. The first IAT article was published three years later in 1998.
Since its original publication date, the seminal IAT article has been cited over 4,000 times, making it one of the most influential psychological developments over the past couple of decades. Furthermore, several variations in IAT procedure have been introduced to address test limitations, while numerous applications of the IAT were also developed, including versions investigating bias against obesity, suicide risk, romantic attachment, attitudes regarding sexuality, and political preferences, among others. Finally, as is characteristic of any psychological instrumentation, discussion and debate of the IAT's reliability and validity has continued since its introduction, particularly because these factors vary between different variations of the test.
Application and use
A computer-based measure, the IAT requires that users rapidly categorize two target concepts with an attribute (e.g. the concepts "male" and "female" with the attribute "logical"), such that easier pairings (faster responses) are interpreted as more strongly associated in memory than more difficult pairings (slower responses).
The IAT is thought to measure implicit attitudes: "introspectively unidentified (or inaccurately identified) traces of past experience that mediate favorable or unfavorable feeling, thought, or action toward social objects." In research, the IAT has been used to develop theories to understand implicit cognition (i.e. cognitive processes of which a person has no conscious awareness). These processes may include memory, perception, attitudes, self-esteem, and stereotypes. Because the IAT requires that users make a series of rapid judgments, researchers believe that IAT scores may also reflect attitudes which people are unwilling to reveal publicly. The IAT may allow researchers to get around the difficult problem of social-desirability bias and for that reason it has been used extensively to assess people's attitudes towards commonly stigmatized groups, such as African Americans and Individuals who identify as homosexual.
Task 1 (practice):
Task 2 (practice):
Press E to classify as Pleasant
or I to classify as Unpleasant
Tasks 3 and 4 (data collection):
Press E to classify as Black or Pleasant
or I to classify as White or Unpleasant
Task 5 (practice):
Press E to classify as White
or I to classify as Black
Tasks 6 and 7 (data collection):
Press E to classify as White or Pleasant
or I to classify as Black or Unpleasant
Example of a typical IAT procedure
A typical IAT procedure involves a series of seven tasks. In the first task, an individual is asked to categorize stimuli into two categories. For example, a person might be presented with a computer screen on which the word "Black" appears in the top left-hand corner and the word "White" appears in the top right-hand corner. In the middle of the screen a word, such as a first name, that is typically associated with either the categories of "Black" or "White". For each word that appears in the middle of the screen, the person is asked to sort the word into the appropriate category by pressing the appropriate left-hand or right-hand key. On the second task, the person would complete a similar sorting procedure with an attribute of some kind. For example, the word "Pleasant" might now appear in the top left-hand corner of the screen and the word "Unpleasant" in the top right-hand corner. In the middle of the screen would appear a word that is either pleasant or unpleasant. Once again, the person would be asked to sort each word as being either pleasant or unpleasant by pressing the appropriate key. On the third task, individuals are asked to complete a combined task that includes both the categories and attributes from the first two tasks. In this example, the words "Black/Pleasant" might appear in the top left-hand corner while the words "White/Unpleasant" would appear in the top right-hand corner. Individuals would then see a series of stimuli in the center of the screen consisting of either a name or word. They would be asked to press the left-hand key if the name or word belongs to the "Black/Pleasant" category or the right-hand key if it belongs to the "White/Unpleasant" category. The fourth task is a repeat of the third task but with more repetitions of the names, words, or images.
The fifth task is a repeat of the first task with the exception that the position of the two target words would be reversed. For example, "Black" would now appear in the top right-hand corner of the screen and "White" in the top left-hand corner. The sixth task would be a repeat of the third, except that the objects and subjects of study would be in opposite pairings from previous trials. In this case, "Black/Unpleasant" would now appear in the top right-hand corner and "White/Pleasant" would now appear in the top left-hand corner. The seventh task is a repeat of the sixth task but with more repetitions of the names, words, or images. If the categories under study (e.g. Black or White) are associated with the presented attributes (e.g. Pleasant/Unpleasant) to differing degrees, the pairing reflecting the stronger association (or the "compatible" pairing) should be easier for the participant. In the Black/White-Pleasant/Unpleasant example, a participant will be able to categorize more quickly when White and Pleasant are paired together than when Black and Pleasant are paired if he or she has more positive associations with White people than with Black people (and vice versa if Black and Pleasant are categorized more quickly).
Variations of the IAT include the Go/No-go Association Test (GNAT), the Brief-IAT and the Single-Category IAT. An idiographic approach using the IAT and the SC-IAT for measuring implicit anxiety showed that personalized stimulus selection did not affect the outcome, reliabilities and correlations to outside criteria.
The Go/No-go Association Test (GNAT) is a variation of the IAT that assesses implicit attitudes or beliefs by measuring the relationship between a target concept and two different extremes of an attribute. Specifically, the strength of relationship is assessed by how quickly the items belonging to the target category and specific attribute (yellow and good or yellow and bad) can be picked from surrounding distractor items that are not associated with the target concept or attribute. Respondents are required to press a key when they identify a stimulus that belongs to one of these categories, and not to press a key when they see stimuli that does not belong to those categories. The difference in ability to correctly associate the concept with the specific attributes is described to be the measure of automatic attitude. Unlike the IAT, which measures response latency, the GNAT measures accuracy in identifying the specific relationships between the target concept and specific attributes.
The Single Category IAT, also known as Single-Target IAT (ST-IAT), is unique in that it uses one target category instead of the two required in the original IAT. During the ST-IAT, respondents complete a discrimination block of the evaluative stimuli. The second block consists of sorting target concepts and positive items with one response key and negative items with the other. In the last block, respondents are required to sort target stimuli and negative items together with one key, and positive items with the other key. In comparison to the IAT, which uses contrasts in latency between two concepts and two attributes, the ST-IAT focuses on latency differences in relation to one concept and two attributes.
Valence IATs measure associations between concepts and positive or negative valence. They are generally interpreted as a preference for one category over another. For example, the Race IAT shows that more than 70% of individuals have an implicit preference for Whites over Blacks. On the other hand, only half of Black individuals prefer Blacks over Whites (cf. the earlier "doll experiment" developed by psychologists Kenneth and Mamie Clark during the early civil rights era). Similarly, the Age IAT generally shows that most individuals have an implicit preference for young over old, regardless of the age of the person taking the IAT. The Weight IAT indicates that medical students have lower implicit biases towards obese individuals compared to the general public, but increased explicit biases, although public explicit and implicit biases remained stable. Research with the Sexuality IAT shows that heterosexual individuals have an implicit preference for heterosexuals, associating them with more positive attributes. In contrast, bisexual individuals indicated a preference for heterosexuals over homosexuals, specifically as a result of attributing homosexuals to negative attributes. Neither of these trends of attributing more positive or negative attributes to a specific sexual identity are seen with homosexual respondents. Some other valence IATs include the Weight IAT, the Sexuality IAT, the Arab-Muslim IAT, and the Skin-tone IAT.
Stereotype IATs measure associations between concepts that often reflect the strength to which a person holds a particular societal stereotype. For example, the Gender-Science IAT reveals that most people associate women more strongly with liberal arts and men more strongly with science. Similarly, the Gender-Career IAT indicates that most people associate women more strongly with family and men more strongly with careers. The Asian IAT shows that many people more strongly associate Asian Americans with foreign landmarks and European Americans more strongly with American landmarks. Some other stereotype IATs include the Weapons IAT and the Native IAT.
The self-esteem IAT measures implicit self-esteem by pairing "self" and "other" words with words of positive and negative valence. Those who find it easier to pair "self" with positive words than negative words are purported to have higher implicit self-esteem. Generally, measures of implicit self-esteem, including the IAT, are not strongly related to one another and are not strongly related to explicit measures of self-esteem.
The Brief IAT (BIAT) uses a similar procedure to the standard IAT but requires fewer classifications. It involves approximately four to six tasks rather than seven, only uses combined tasks (corresponding most closely to tasks 3, 4, 6, and 7 on the standard IAT), and has fewer repetitions. Additionally, it requires specification of a focal concept in each task as well as a single attribute, instead of two. For example, although, White, Black, Pleasant, and Unpleasant stimuli all appear, participants would press one key when White and Pleasant words appear and another key when "anything else" appears. Subsequently, participants would press one key when Black and Pleasant words appear and another key when "anything else" appears. Unlike the GNAT, the Brief IAT doesn’t not use accuracy of correctly identifying the specific concept and attribute requested. Instead, the latency is used to acquire results.
The Child IAT (Ch-IAT) allows for children as young as four years of age to take the IAT. Rather than words and pictures, the Ch-IAT uses sound and pictures. For example, positive and negative valence are indicated with smiling and frowning faces. Positive and negative words to be classified are voiced out loud to children.
Studies using the Ch-IAT have revealed that six-year-old White children, ten-year-old White children, and White adults have comparable implicit attitudes on the Race IAT.
According to Greenwald, the IAT provides a "window" into a level of mental operation that operates in unthinking (unconscious, automatic, implicit, impulsive, intuitive, etc.) fashion because associations operating without active thought (automatically) can help performance in one of the IAT's two "combined" tasks, while interfering with the other. Respondents to the IAT experience a higher (conscious, controlled, explicit, reflective, analytic, rational, etc.) level of mental operation, when they try to overcome the effects of the automatic associations. The IAT succeeds as a measure because the higher level fails to completely overcome the lower level.
The interpretation that the IAT provides a "window" to unconscious mental contents has been challenged by Hahn and colleagues, whose results indicated that people are highly accurate in predicting their own IAT scores for a variety of social groups.
De Houwer theorizes that the IAT is a measure of a response compatibility effect, in which participants first learn to associate positive and negative words and concepts with pressing specific keys on the keyboard. Later in the test, when participants are instructed to sort words and concepts that are both negative and positive with the same keyboard key, De Houwer argues that much of the latency and incorrect responses that result from this change are due to the increased cognitive complexity of the task, and not necessarily a reflection of implicit bias.
Brendl, Markman, and Messner have proposed a random walk model process to explain responses in the critical portions of the IAT. They theorize that test respondents base their responses on a process of mental evidence-gathering that continues until the evidence for one option or the other (right or left key) reaches a threshold, at which time a decision is made, and action is taken. This requires consideration of both the concept and the attribute, which can be congruent or incongruent - all factors that affect decision speed. All evidence during the compatible block of the test is congruent, allowing for fast decision-making. However, incongruent concept and attribute in the incompatible task leads to longer processing time. Increased task difficulty also increases evidence threshold criterion, further decreasing decision speed.
An alternative or complementary theory from Mierke and Klauer holds that the cognitive control processes required to switch back and forth between categorizing based on concept versus based on attribute leads to reduced speeds in the critical blocks of the test. In other words, it is much less mentally demanding to sort concepts in the compatible block when only one aspect of the concept must be focused upon. Comparatively, sorting concepts in the incompatible block, which requires focusing on both concept and attribute not only takes longer to process because of the increased complexity, but also because the previous concept may have required a different cognitive effort.
Fiinally, Rothermund and Wentura propose a figure-ground model of explanation for the IAT. In essence, this theory suggests that IAT respondents simplify their task by relying on salience. For example, negative is salient for most people, so if a respondent is to press the right key for negative words, the individual will plan to press the right key for all negative words (figure), and the left key for any other (non-negative) words (ground). This leads to fast decision-making in the compatible task and two of the critical tasks, but not for the third critical task, in which two salient categories require different keys pushed.
There is empirical support for all these explanations of the IAT’s effects, but this is not necessarily evidence against the IAT’s overall validity, as these theories are not mutually exclusive. Furthermore, regardless of the fundamental cognitive processes of the IAT, studies show that multiple implementations of the test validly measure their targeted constructs, and the psychometric value of each implementation varies as a function of its individual characteristics (e.g., construct measured, participant characteristics, testing environment).
Balanced-identity theory and design
Heider's balance theory
In 1958, Fritz Heider proposed the balance theory, which stated that a system of liking and disliking relationships is balanced if the product of the valence of all relationships within the system is positive. In the theory, there are concepts and associations. Concepts are persons, groups, or attributes; and among attribute concepts, there are positive and negative valences. Associations are relations between pairs of concepts, and the strength of association is the potential for one concept to activate another, either by external stimuli or by excitation through their associations with other, already active, concepts. The theory followed the assumption of associative social knowledge: an important portion of social knowledge could be represented as a network of variable-strength associations among person concepts (including self and groups) and attributes (including valence).
When two unlinked or weakly linked nodes are linked to the same third node, the association between these two should strengthen. This is the principle of balance–congruity. The nodes in the principle of balance–congruity are equivalent to the concepts in Heider's balance theory, and the three involved nodes/concepts make up a system. Since every relationship within the system here is positively associated, this, according to a derivation of Heider's theory, also represents a balanced system where the product of the direction of all associations within the system is positive.
Balanced-identity research design
In 2002, Greenwald and his colleagues introduced the balanced-identity design as a method to test correlational predictions of Heider's balance theory. The balanced identity design incorporated Heider's theory, the balance–congruity principle, and the assumption of centrality of self. The assumption of centrality of self is that in an associative knowledge structure, the self's centrality can be represented by its being associated with many other concepts that are themselves highly connected in the structure. The concepts in a typical balanced identity design are the self, a social group/object, and either a valence attribute or nonvalence attribute. There are thus five important associations possible in a typical balanced identity design that connect these three categories of concepts. An attitude is the association of a social group/object with a valence attribute; a stereotype is the association of a social group with one or more nonvalence attribute(s); self-esteem is the association of the self with a valence attribute; a self-concept is the association of the self with one or more nonvalence attribute(s); and the last important association is between the self and a social group/object, which is called an identity. However, in a typical balanced identity design, only three of the five possible associations come into play, and they are usually either identity, self-concept, and stereotype or identity, self-esteem, and attitude. Researchers using a balanced identity design are the ones to determine the set of concepts they want to investigate, and each one of the associations within the system that the researchers created will then be tested and analyzed statistically with both implicit and explicit measures.
Typical results of balanced-identity research design with implicit measures
A typical result of a balanced identity design usually shows that a group's identity is balanced, at least with implicit measures. According to a derivation of Heider's balance theory, since there are three concepts in a typical balanced identity design, the identity is balanced either when all three relations are positive or when one positive and two negative relations are present in the triad system. The triad system of "me—male—being good at math" will be used as an example here, and its typical result acquired from the Implicit Association Test (IAT) will be shown below. For male subjects, the three associations within the triad are usually all positive. For female subjects, the "me—male" association is usually negative, the "male—being good at math" association is usually positive, and the "me—being good at math" association is usually negative. As it's shown, for both the male and female subjects, their group identities are balanced.
Comparison to findings with explicit reports
Self-reporting is also usually used in a balanced identity design. Although self-reports don’t necessarily reflect the predicted consistency patterns from Heider's theory, it is often used to compare with the results from the Implicit Association Test (IAT). Any discrepancies between the self-reports and the IAT results on the same association in a balanced identity design can be an indication of an experience of conflict. The above triad system of "me—male—being good at math" is a good example. For female subjects, whereas the Implicit Association Test (IAT) typically shows a stronger positive association of "male" and "being good at math," the explicit self-reporting usually shows a weaker positive association or even a weaker negative association of "male" and "being good at math." Also, whereas the IAT typically shows a stronger negative association of "me" and "being good at math" for the same female subjects, the self-reporting usually shows a weaker negative or even a weaker positive association of "me" and "being good at math." In this case, the female group is believed to be experiencing a conflict. The common explanation for a group experiencing a conflict is that in an effort to change a stereotypical view that has been around in the society for a really long time, even though people who belong to a certain social group believe that they are able to reject this stereotype (shown in explicit measures), the exact stereotypical thought is still going to remain in the back of their heads (shown in implicit measures), maybe not as much as those who actually believe in that thought. So maybe with time, as a stereotype gradually fades away, that conflict will fade away as well.
The IAT has been widely used as a measure for the balanced identity design because data obtained with this method revealed that predicted consistency patterns from Heider's theory were strongly apparent in the data for implicit measures by IAT but not in those for parallel explicit measures by self-report. The general explanation for why explicit measures by self-report did not reflect the predicted consistency patterns from Heider's theory was that self-report measures can go astray when respondents are either unwilling or unable to report accurately, and these problems could be more than enough to obscure the operation of consistency processes. There are, however, still limitations to the theory. For example, the balanced identity IAT measures only give group results rather than individual results, so it has its limitations when an analysis requires for individual pinpoint data to analyze, for instance, how balanced one's identity is relative to others'. It is hopeful, however, that researchers working with the Implicit Association Test (IAT) are trying hard to overcome challenges such as the one described above.
Criticism and controversy
The IAT has engendered some controversy in both the scientific literature and in the public sphere (e.g. in the Wall Street Journal). For example, it has been interpreted as assessing familiarity, perceptual salience asymmetries, or mere cultural knowledge irrespective of personal endorsement of that knowledge. A more recent critique argued that there is a lack of empirical research justifying the diagnostic statements that are given to the lay public. For instance, feedback may report that someone has a [slight/moderate/strong] automatic preference for [European Americans/African Americans]. Proponents of the IAT have responded to these charges, but the debate continues. According to The New York Times, "there isn't even that much consistency in the same person's scores if the test is taken again". In addition, researchers have recently claimed that results of the IAT might be biased by the participant's lacking cognitive capability to adjust to switching categories, thus biasing results in favor of the first category pairing (e.g. pairing "Asian" with positive stimuli first, instead of pairing "Asian" with negative stimuli first).
According to Jesse Singal, some of these issues have been settled in the research literature, but others continue to inspire debate among researchers and lay people alike.
Since its introduction into the scientific literature in 1998, a great deal of research has been conducted in order to examine the psychometric properties of the IAT as well as to address other criticisms on validity and reliability.
The IAT is purported to measure relative strength of associations. However, some researchers have asserted that the IAT may instead be measuring constructs such as salience of attributes or cultural knowledge.
A 2009 meta-analysis has concluded that the IAT has predictive validity independent of the predictive validity of explicit measures. However, a follow-up meta-analysis questioned some of these results, finding that implicit measures were only weakly predictive of behaviors and no better than explicit measures. Some research has found that the IAT tends to be a better predictor of behavior in socially sensitive contexts (e.g. discrimination and suicidal behaviour) than traditional "explicit" self-report methods, whereas explicit measures tend to be better predictors of behavior in less socially sensitive contexts (e.g. political preferences). Specifically, the IAT has been shown to predict voting behavior (e.g. ultimate candidate choice of undecided voters), mental health (e.g. a self-injury IAT differentiated between adolescents who injured themselves and those who did not), medical outcomes (e.g. medical recommendations by physicians), employment outcomes (e.g. interviewing Muslim-Arab versus Swedish job applicants), education outcomes (e.g. gender-science stereotypes predict gender disparities in nations' science and math test scores), and environmentalism (e.g., membership of a pro-environmental organisation).
The IAT has been questioned for its success with at-risk patients. When patients were tested on their subconscious feelings towards death, suicidal patients are put at risk. Research shows that those experiencing deep suicidal thoughts are not likely to share their true experiences. This is harmful because the patients experiencing these thoughts are the most at risk and resurfacing the suicidal thoughts could be detrimental to their mental health.
In applied settings, the IAT has been used in marketing and industrial psychology. For example, in determining the predictors of risk-taking behaviour of pilots in general aviation, attitudes towards risky flight behaviour as measured through an IAT have shown to be a more accurate forecast of risky flight behaviour than traditional explicit attitude or personality scales. The IAT has also been used in clinical psychology research to test the hypothesis that implicit associations may be a causal factor in the development of anxiety disorders.
Researchers have argued that the IAT may measure salience of concepts rather than associations. Whereas IAT proponents claim that faster response times when pairing concepts indicate stronger associations, critics claim that faster response times indicate that concepts are similar in salience (and slower response times indicate that concepts differ in salience). There is some support for this claim. For example, in an old-young IAT, old faces would be more salient than young faces. As a result, researchers created an old-young IAT that involved pairing young and old faces with neutral words (non-salient attribute) and non-words (salient attribute). Response times were faster when old faces (salient) were paired with non-words (salient) than when old faces (salient) were paired with neutral words (non-salient), supporting the assertion that faster response time can be facilitated by matching salience.
Although proponents of the IAT acknowledge that it may be influenced by salience asymmetry, they argue that this does not preclude interpreting the IAT as a measure of associations.
Culture versus person
Another criticism of the IAT is that it may measure associations that are picked up from cultural knowledge rather than associations actually residing within a person. The counter-argument is that such associations may indeed arise from the culture, but they can nonetheless influence behavior.
To address the possibility that the IAT picks up on cultural knowledge rather than beliefs that are present in a person, some critics of the standard IAT created the personalized IAT. The primary difference between a standard valence IAT and the personalized IAT is that rather than using pleasant and unpleasant words as category labels, it uses "I like" and "I don't like" as category labels. Additionally, the Personalized IAT does not provide error feedback for an incorrect response as in the standard IAT. This form of the IAT is more strongly related to explicit self-report measures of bias.
Proponents of the standard IAT argue that the Personalized IAT increases the likelihood that those taking it will evaluate the concept rather than classify it. This would increase its relationship with explicit measures without necessarily removing the effect of cultural knowledge. In fact, some researchers have examined the relationship between perceptions of general American attitudes and Personalized IAT scores and have concluded that the relationship between the IAT and cultural knowledge is not decreased by personalizing it. However, it is important to note that there was no relationship between cultural knowledge and standard IAT scores either.
Ability to fake result
The IAT has also demonstrated a reasonable amount of resistance to social-desirability bias. Individuals asked to fake their responses on the IAT have demonstrated difficulty in doing so in some studies. For example, participants who were asked to present a positive impression of themselves were able to do so on a self-report measure of anxiety but not an IAT measuring anxiety. Nonetheless, faking is possible, and recent research indicates that the most effective method of faking the IAT is to intentionally slow down responses for pairings that should be relatively easy. Most subjects, however, do not discover this strategy on their own, so faking is relatively rare. An algorithm developed to estimate IAT faking can identify those who are faking with approximately 75% accuracy.
There is a recent study showing that participants can even speed up their responses during the relatively difficult response pairings in an autobiographical implicit association test that aims to test the veracity of autobiographical statement. Specifically, participants who were instructed to speed up their responses to fake the test were able to do so. The effect was larger when participants were trained in speeding up. Most importantly, guilty participants who speed up their responses during the difficult response pairing successfully beat the test to obtain an innocent result. In other words, participants can reverse their test outcome without being detected. Clearly, this poses new challenges to the IATs.
Susceptibility to conscious control
Distinct from faking (the deliberate obscuring of a true association), some studies have shown that heightening awareness about the nature of the test can change the outcome, potentially by activating different fluencies and associations. For example, in one study, a simple reminder from the experimenter ("Please be careful not to stereotype on the next section of the task") was sufficient to significantly reduce the expression of biased associations on a race IAT. Notably, there was not a significant decrease in overall reaction time in this experiment, indicating that this "control" may also be implicit.
A common criticism of the IAT is that it may be difficult to associate positive attributes with less familiar concepts. For example, if a person has had less contact with members of a particular ethnic group, he or she may have a more difficult time associating members of that ethnic group with positive words simply because of this lack of familiarity. There is some evidence against the familiarity based on studies that have ensured equal familiarity with the African American and White names as well as the faces appearing on the Race IAT.
As the IAT relies on a comparison of response times in different tasks pairing concepts and attributes, researchers and others taking the IAT have speculated that the pairing on the first combined task may affect performance on the next combined task. For example, a participant who begins a gender stereotype IAT by pairing female names with family words may subsequently find the task of pairing female names with career words more difficult. Research has indeed shown a small effect of order. As a result, it is recommended to increase the number of classifications required in the fifth IAT task. This gives participants more practice before doing the second pairing, thus reducing the order effect. When studying groups of people, this effect could be countered by giving pairings first to different participants (e.g. half of participants pair female names and family words first, the other half pair female names with career words first).
Cognitive fluency and age
The IAT is influenced by individual differences in average IAT response times such that those with slower overall response times tend to have more extreme IAT scores. Older subjects also tend to have more extreme IAT scores, and this may be related to cognitive fluency, or slower overall response times.
An improved scoring algorithm for the IAT, which reduces the effect of cognitive fluency on the IAT, has been introduced. A summary of the scoring algorithm can be found on Greenwald's webpage.
Repeated administrations of the IAT tend to decrease the magnitude of the effect for a particular person. This issue is somewhat ameliorated with the improved scoring algorithm. An additional safeguard to control for IAT experience is to include a different type of IAT as a comparison. This allows researchers to evaluate the degree of magnitude decrease when administering subsequent IATs.
The act of taking the Race IAT has also been found to exacerbate the negative implicit attitudes that it seeks to assess. Results from four pre-registered experiments demonstrated that completing a Race IAT resulted in increases in White participants’ negative automatic racial evaluations of Black people as measured by two different implicit measures (Single Category IAT and the Affective Misattribution Procedure) but did not generalize to another measure of automatic racial bias(Shooter Bias Task).
The IAT demonstrates inconsistent internal consistency and its test-retest reliability stands at 0.60, a relatively weak level. However, IAT scores do seem to vary between multiple administrations, indicating that it may measure a combination of trait (stable characteristics of people) and state (subject to variation based on situation-specific circumstances) characteristics. One example of the latter case is that scores on the Race IAT are known to be less biased against African Americans when those taking it imagine positive Black exemplars beforehand (e.g. Martin Luther King). Similarly, the Race IAT scores for an individual may indicate bias, but that bias is diminished on another IAT administered after associating with a mixed-race group. In fact, Race IAT scores can be changed even more easily; administering the IAT in different languages yields significantly different scores for bilingual individuals. For example, studies conducted with Moroccan participants fluent in both French and Arabic showed that participants are biased when completing an IAT in their native language; however, that bias is diminished when completing an IAT in another language. Similar results were found in the United States when administering an English and Spanish IAT on bilingual Hispanic Americans. Another state characteristic that may well influence IAT scores is the time of day a person completes the task, with findings that holding a preference for one's own racial group is lowest in the morning, but increases over the course of the day and into the evening; however, this may be more to do with who completes the task at each time of day than a function of circadian rhythms.
In popular culture
After establishing the IAT in the scientific literature, Greenwald, along with Mahzarin Banaji (Professor of Psychology at Harvard University) and Brian Nosek (Associate Professor of Psychology at the University of Virginia), co-founded Project Implicit, a virtual laboratory and educational outreach organization that facilitates research on implicit cognition.
The IAT has been profiled in major media outlets (e.g. in the Washington Post) and in the popular book Blink, where it was suggested that one could score better on the implicit racism test by visualizing respected black leaders such as Nelson Mandela. The IAT was also discussed in a 2006 episode of The Oprah Winfrey Show.
In the episode "Racist Dawg" on King of the Hill, Hank and Peggy take an IAT, colloquially referred to as the "racist test" to see if they prefer the company of white or black people.
- Greenwald, Anthony G.; McGhee, Debbie E.; Schwartz, Jordan L.K. (1998), "Measuring Individual Differences in Implicit Cognition: The Implicit Association Test", Journal of Personality and Social Psychology, 74 (6): 1464–1480, CiteSeerX 10.1.1.489.4611, doi:10.1037/0022-3518.104.22.1684, PMID 9654756
- Greenwald, A. G.; Banaji, M. R. (1995). "Implicit social cognition: Attitudes, self-esteem, and stereotypes". Psychological Review. 102 (1): 4–27. CiteSeerX 10.1.1.411.2919. doi:10.1037/0033-295x.102.1.4. PMID 7878162.
- Nosek, B. A.; Greenwald, A. G.; Banaji, M. R. (2005). "Understanding and using the Implicit Association Test: II. Method variables and construct validity". Personality and Social Psychology Bulletin. 31 (2): 166–180. doi:10.1177/0146167204271418. PMID 15619590.
- "Understanding and Using the Implicit Association Test: IV What We Know (So Far) about the Method - Semantic Scholar". Retrieved 2018-03-13.
- Greenwald, A. G.; McGhee, D. E.; Schwartz, J. K. L. (1998). "Measuring individual differences in implicit cognition: The Implicit Association Test". Journal of Personality and Social Psychology. 74 (6): 1464–1480. CiteSeerX 10.1.1.489.4611. doi:10.1037/0022-3522.214.171.1244. PMID 9654756.
- "Web of Science Journal Citation Reports". Web of Science. 2018. Retrieved March 21, 2018.
- Ortner, Tuulia M.; Van De Vijver, Fons J.R (June 2015). Behavior-based assessment in psychology : going beyond self-report in the personality, affective, motivation, and social domains. Vijver, Fons J. R. van de,, Ortner, Tuulia M. Toronto, Ontario. ISBN 9781613344378. OCLC 907565780.
- Sleek, Scott (2018). "The Bias Beneath: Two Decades of Measuring Implicit Associations". Aps Observer. 31 (2). Retrieved March 21, 2018.
- Greenwald, A. G.; Banaji, M. R. (1995). "Implicit social cognition: Attitudes, self-esteem, and stereotypes". Psychological Review. 102: 8. CiteSeerX 10.1.1.411.2919. doi:10.1037/0033-295x.102.1.4. PMID 7878162.
- Cunningham, William A.; Nezlek, John B.; Banaji, Mahzarin R. (October 2004). "Implicit and explicit ethnocentrism: revisiting the ideologies of prejudice". Personality & Social Psychology Bulletin. 30 (10): 1332–1346. doi:10.1177/0146167204264654. ISSN 0146-1672. PMID 15466605.
- Devine, P.G. (2001), "Implicit Prejudice and Stereotyping: How Automatic Are They?", Journal of Personality and Social Psychology, 81 (5): 757–759, doi:10.1037/0022-35126.96.36.1997
- Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit Association Test at age 7: A methodological and conceptual review (pp. 265–292). In J. A. Bargh (Ed.), Automatic processes in social thinking and behavior. Psychology Press.
- Nosek, Brian A.; Banaji, Mahzarin R. (2001). "THE GO/NO-GO ASSOCIATION TASK" (PDF). Social Cognition. 19 (6): 625–664. doi:10.1521/soco.19.6.625.20886.
- Sriram, N.; Greenwald, A. G. (2009). "The Brief Implicit Association Test". Experimental Psychology. 56 (4): 283–294. doi:10.1027/1618-3188.8.131.523. PMID 19439401.
- Karpinski, Andrew; Steinman, Ross B. (2006). "The single category Implicit Association Test as a measure of implicit social cognition". Journal of Personality and Social Psychology. 91 (1): 16–32. doi:10.1037/0022-35184.108.40.206. PMID 16834477.
- Stieger, Stefan; Göritz, Anja S.; Burger, Christoph (2010). "Personalizing the IAT and the SC-IAT: Impact of idiographic stimulus selection in the measurement of implicit anxiety". Personality and Individual Differences. 48 (8): 940–944. doi:10.1016/j.paid.2010.02.027.
- Nosek, Brian A.; Banaji, Mahzarin R. (2005-06-02). "The Go/No-Go Association Task". Social Cognition. 19 (6): 625–666. doi:10.1521/soco.19.6.625.20886.
- Bluemke, Matthias; Friese, Malte (2008-09-01). "Reliability and validity of the Single-Target IAT (ST-IAT): assessing automatic affect towards multiple attitude objects". European Journal of Social Psychology. 38 (6): 977–997. doi:10.1002/ejsp.487. ISSN 1099-0992.
- Sabin, J. A.; Nosek, B. A.; Greenwald, A. G.; Rivara, F. P. (2009). "Physicians' Implicit and Explicit Attitudes About Race by MD Race, Ethnicity, and Gender". Journal of Health Care for the Poor and Underserved. 20 (3): 896–913. doi:10.1353/hpu.0.0185. PMC 3320738. PMID 19648715.
- Nosek, B. A.; Banaji, M. R.; Greenwald, A. G. (2002). "Harvesting implicit group attitudes and beliefs from a demonstration website". Group Dynamics. 6 (1): 101–115. doi:10.1037/1089-26220.127.116.11.
- Phelan, Sean M.; Puhl, Rebecca M.; Burke, Sara E.; Hardeman, Rachel; Dovidio, John F.; Nelson, David B.; Przedworski, Julia; Burgess, Diana J.; Perry, Sylvia (October 2015). "The mixed impact of medical school on medical students' implicit and explicit weight bias". Medical Education. 49 (10): 983–992. doi:10.1111/medu.12770. PMC 4755318. PMID 26383070.
- Anselmi, Pasquale; Vianello, Michelangelo; Voci, Alberto; Robusto, Egidio (2013-11-18). "Implicit Sexual Attitude of Heterosexual, Gay and Bisexual Individuals: Disentangling the Contribution of Specific Associations to the Overall Measure". PLoS ONE. 8 (11): e78990. doi:10.1371/journal.pone.0078990. PMC 3832517. PMID 24260142.
- Anselmi, Pasquale; Vianello, Michelangelo; Voci, Alberto; Robusto, Egidio (2013-11-18). "Implicit Sexual Attitude of Heterosexual, Gay and Bisexual Individuals: Disentangling the Contribution of Specific Associations to the Overall Measure". PLoS ONE. 8 (11): e78990. doi:10.1371/journal.pone.0078990. PMC 3832517. PMID 24260142.
- Greenwald, A. G.; Farnham, S. D. (2000). "Using the Implicit Association Test to measure self-esteem and self-concept". Journal of Personality and Social Psychology. 79 (6): 1022–1038. CiteSeerX 10.1.1.550.47. doi:10.1037/0022-3518.104.22.1682. PMID 11138752.
- Bosson, J. K.; Swann, W. B. Jr.; Pennebaker, J. W. (2000). "Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited?". Journal of Personality & Social Psychology. 79 (4): 631–643. CiteSeerX 10.1.1.371.9919. doi:10.1037/0022-3522.214.171.1241.
- Baron, A. S.; Banaji, M. R. (2006). "The development of implicit attitudes: Evidence of race evaluations from ages 6, 10 & adulthood". Psychological Science. 17 (1): 53–58. doi:10.1111/j.1467-9280.2005.01664.x. PMID 16371144.
- Televised Lecture "The Psychology of Blink: Understanding How Our Minds Work Unconsciously - Part 1 of 2" (recorded March 5, 2008)
- Hahn, A.; Judd, C.M.; Hirsh, H.K.; Blair, I.V. (2013). "Awareness of implicit attitudes". Journal of Experimental Psychology: General. 143 (3): 1369–1392. doi:10.1037/a0035028. PMC 4038711. PMID 24294868.
- De Houwer, Jan (2001). "A Structural and Process Analysis of the Implicit Association Test". Journal of Experimental Social Psychology. 37 (6): 443–451. doi:10.1006/jesp.2000.1464 – via Elsevier Science Direct.
- Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., Banaji, M. R. (2009). "Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity". Journal of Personality and Social Psychology. 97 (1): 17–41. doi:10.1037/a0015575. PMID 19586237 – via PsycNET.CS1 maint: Multiple names: authors list (link)
- Greenwald, A. G.; Banaji, M. R.; Rudman, L. A.; Farnham, S. D.; Nosek, B. A.; Mellott, D. S. (2002). "A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept". Psychological Review. 109 (1): 3–25. CiteSeerX 10.1.1.366.9580. doi:10.1037/0033-295x.109.1.3. PMID 11863040.
- Azar, B (2008). "IAT: Fad or fabulous?". Monitor on Psychology. 39: 44.
- Singal, Jesse. "Psychology's Favorite Tool for Measuring Racism Isn't Up to the Job". Science of Us. Retrieved 2017-01-13.
- Wax, Amy; Tetlock, Philip (December 1, 2005). "We're all racists at heart". Wall Street Journal. Retrieved 2011-06-09.
- Ottaway, S. A.; Hayden, D. C.; Oakes, M. A. (2001). "Implicit attitudes and racism: Effects of word familiarity and frequency on the implicit association test". Social Cognition. 19 (2): 97–144. doi:10.1521/soco.126.96.36.19906.
- Rothermund, K.; Wentura, D. (2004). "Underlying processes in the Implicit Association Test(IAT): Dissociating salience from associations". Journal of Experimental Psychology: General. 133 (2): 139–165. doi:10.1037/0096-34188.8.131.52. PMID 15149248.
- Arkes, H. R.; Tetlock, P. E. (2004). ""Attributions of Implicit Prejudice, or "Would Jesse Jackson 'Fail' the Implicit Association Test?". Psychological Inquiry. 15 (4): 257–278. CiteSeerX 10.1.1.372.2454. doi:10.1207/s15327965pli1504_01.
- Blanton, Hart; Jaccard, James (January 2006). "Arbitrary Metrics in Psychology" (PDF). American Psychologist. 61 (1): 27–41. CiteSeerX 10.1.1.620.660. doi:10.1037/0003-066X.61.1.27. PMID 16435974.
- Greenwald, Anthony. "Dr. Anthony Greenwald / IAT Materials". Retrieved 2008-10-11.
- Tierney, John (November 17, 2008). "In Bias Test, Shades of Gray". The New York Times. Retrieved 2009-01-09.
- Messner, Claude; Vosgerau, Joachim (2010). "Cognitive Inertia and the Implicit Association Test" (PDF). Journal of Marketing Research. 47 (2): 374–386. doi:10.1509/jmkr.47.2.374.
- Greenwald, A. G.; Poehlman, T. A.; Uhlmann, E.; Banaji, M. R. (2009). "Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity". Journal of Personality and Social Psychology. 97: 17–41. doi:10.1037/a0015575. PMID 19586237.
- Oswald, F.L.; Mitchell, G.; Blanton, H.; Jaccard, J.; Tetlock, P. E. (2013). "Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies". Journal of Personality and Social Psychology. 105 (2): 171–192. doi:10.1037/a0032734. PMID 23773046.
- Matthew K. Nock, Jennifer M. Park, Christine T. Finn, Tara L. Deliberto, Halina J. Dour, & Mahzarin R. Banaji (2010). "Measuring the Suicidal Mind: Implicit Cognition Predicts Suicidal Behavior". Psychological Science. 21 (4): 511–517. doi:10.1177/0956797610364762. PMC 5258199. PMID 20424092.CS1 maint: Multiple names: authors list (link)
- Poehlman, T. Andrew; Uhlmann, Eric Luis; Greenwald, Anthony G.; Banaji, Mahzarin. "Understanding and Using the Implicit Association Test: III. Meta-analysis of Predictive Validity" (PDF). Retrieved 2008-10-11.CS1 maint: Multiple names: authors list (link)
- Arcuri, L.; Castelli, L.; Galdi, S.; Zogmaister, C.; Amadori, A. (2008). "Predicting the vote: Implicit attitudes as predictors of the future behavior of the decided and undecided voters". Political Psychology. 29 (3): 369–387. doi:10.1111/j.1467-9221.2008.00635.x.
- Nock, M. K.; Banaji, M. R. (2007a). "Assessments of self-injurious thoughts using a behavioral test". American Journal of Psychiatry. 164 (5): 820–823. doi:10.1176/appi.ajp.164.5.820.
- Green, A. R.; Carney, D. R.; Pallin, D. J.; Ngo, L. H.; Raymond, K. L.; Iezzoni, L. I.; Banaji, M. R. (2007). "The presence of implicit bias in physicians and its prediction of thrombolysis decisions for Black and White patients". Journal of General Internal Medicine. 22 (9): 1231–1238. doi:10.1007/s11606-007-0258-5. PMC 2219763. PMID 17594129.
- Rooth, D-O (2010). "Automatic associations and discrimination in hiring: Real world evidence". Labour Economics. 17 (3): 523–534. CiteSeerX 10.1.1.653.61. doi:10.1016/j.labeco.2009.04.005.
- Nosek, B. A.; Smyth, F. L.; Sriram, N.; Lindner, N. M.; Devos, T.; Ayala, A.; Bar-Anan, Y.; Bergh, R.; Cai, H.; Gonsalkorale, K.; Kesebir, S.; Maliszewski, N.; Neto, F.; Olli, E.; Park, J.; Schnabel, K.; Shiomura, K.; Tulbure, B.; Wiers, R. W.; Somogyi, M.; Akrami, N.; Ekehammar, B.; Vianello, M.; Banaji, M. R.; Greenwald, A. G. (2009). "National differences in gender-science stereotypes predict national sex differences in science and math achievement". Proceedings of the National Academy of Sciences. 106 (26): 10593–10597. doi:10.1073/pnas.0809921106. PMC 2705538. PMID 19549876.
- Thomas, Gregory (June 2016). "The Development and Validation of an Implicit Measure Based on Biospheric Values" (PDF). Environment & Behavior. 48 (5): 659–685. doi:10.1177/0013916514553836.
- Millner, Alexander, J (October 2018). "The Brief Death Implicit Association Test: Scoring recommendations, reliability, validity, and comparisons with the Death Implicit Association Test". Psychological Assessment. 30 (10): 1356–1366. doi:10.1037/pas0000580. PMC 6172146. PMID 29781668 – via PsycArticles.
- Molesworth, Brett; Chang, Betty (2009). "Predicting pilots' risk-taking behaviour through an Implicit Association Test". Human Factors. 51 (6): 846–857. doi:10.1177/0018720809357756. PMID 20415159.
- Teachman, B. A.; Woody, S. (2004). "Staying tuned to research in implicit cognition: Relevance for clinical practice with anxiety disorders". Cognitive and Behavioral Practice. 11 (2): 149–159. doi:10.1016/s1077-7229(04)80026-9.
- Greenwald, A. G.; Nosek, B. A.; Banaji, M. R.; Klauer, K. C. (2005). "Validity of the salience asymmetry interpretation of the IAT: Comment on Rothermund and Wentura (2004)". Journal of Experimental Psychology: General. 134 (3): 420–425. doi:10.1037/0096-34184.108.40.2060. PMID 16131272.
- Banaji, M. R.; Nosek, B. A.; Greenwald, A. G. (2004). "No place for nostalgia in science: A response to Arkes and Tetlock". Psychological Inquiry. 15 (4): 279–289. doi:10.1207/s15327965pli1504_02.
- Olson, M. A.; Fazio, R. H. (2004). "Reducing the influence of extra-personal associations on the Implicit Association Test: Personalizing the IAT". Journal of Personality and Social Psychology. 86 (5): 653–667. doi:10.1037/0022-35220.127.116.113. PMID 15161392.
- Nosek, B. A.; Hansen, J. J. (2008). "Personalizing the Implicit Association Test increases explicit evaluation of the target concepts". European Journal of Psychological Assessment. 25 (4): 226–236. doi:10.1027/1015-5718.104.22.168.
- Egloff, B.; Schmukle, S.C. (2002), "Predictive Validity of an Implicit Association Test for Assessing Anxiety", Journal of Personality and Social Psychology, 83 (6): 1441–1455, doi:10.1037/0022-3522.214.171.1241
- Kim, D. Y. (2003). "Voluntary controllability of the Implicit Association Test (IAT)". Social Psychology Quarterly. 66 (1): 83–96. doi:10.2307/3090143. JSTOR 3090143.
- Hu, X.; Rosenfeld, J. P. Bodenhausen (2012). "Combating automatic autobiographical associations: The effect of instruction and training in strategically concealing information in the autobiographical implicit association test". Psychological Science. 23 (10): 1079–1085. doi:10.1177/0956797612443834. PMID 22894937.
- Wallaert, Matthew; Ward, Andrew; Mann, Traci. "Explicit control of implicit responses: Simple directives can alter IAT performance". Retrieved 2013-11-10.CS1 maint: Multiple names: authors list (link)
- Dasgupta, N.; McGhee, D. E.; Greenwald, A. G.; Banaji, M. R. (2000). "Automatic preference for White Americans: Eliminating the familiaritexplanation". Journal of Experimental Social Psychology. 36 (3): 316–328. CiteSeerX 10.1.1.77.6362. doi:10.1006/jesp.1999.1418.
- McFarland, S. G.; Crouch, Z. (2002). "A cognitive skill confound on the Implicit Association Test". Social Cognition. 20 (6): 483–510. CiteSeerX 10.1.1.372.1868. doi:10.1521/soco.20.6.483.22977.
- Greenwald, A. G; Nosek, B. A.; Banaji, M. R. (2003). "Understanding and using the Implicit Association Test: I. An improved scoring algorithm". Journal of Personality and Social Psychology. 85 (2): 197–216. CiteSeerX 10.1.1.464.4354. doi:10.1037/0022-35126.96.36.199.
- Hussey, Ian; De Houwer, Jan. "Completing a Race IAT increases implicit racial bias". PsyArXiv. The Society for the Improvement of Psychological Science. Retrieved 20 November 2018.
- Karpinksi, A; Steinman, RB (2006). "The single category implicit association test as a measure of implicit social cognition". Journal of Personality and Social Psychology. 91 (1): 16–32. doi:10.1037/0022-35188.8.131.52. PMID 16834477.
- Payne, K.; Cheng, C.M.; Govorun, O.; Stewart, B.D. (2005). "An inkblot for attitudes: Affect misattribution as implicit measurement". Journal of Personality and Social Psychology. 89 (3): 277–293. CiteSeerX 10.1.1.392.4775. doi:10.1037/0022-35184.108.40.2067. PMID 16248714.
- Correll, J (2014). "The Police Officer's Dilemma: A Decade of Research on Racial Bias in the Decision to Shoo". The Police Officer's Dilemma: A Decade of Research on Racial Bias in the Decision to Shoo. 8 (5): 201–213. doi:10.1111/spc3.12099.
- Dasgupta, N.; Greenwald, A. G. (2001). "On the malleability of automatic attitudes: Combating automatic prejudice with images of admired and disliked individuals". Journal of Personality and Social Psychology. 81 (5): 800–814. doi:10.1037/0022-35220.127.116.110. PMID 11708558.
- Van Bavel, J. J.; Cunningham, W. A. (2009). "Self-categorization with a novel mixed-race group moderates automatic social and racial biases". Personality and Social Psychology Bulletin. 35 (3): 321–335. CiteSeerX 10.1.1.499.4157. doi:10.1177/0146167208327743. PMID 19098257.
- Ogunnaike, O.; Dunham, Y.; Banaji, M. R. (2010). "The language of implicit preferences". Journal of Experimental Social Psychology. 46 (6): 999–1003. doi:10.1016/j.jesp.2010.07.006.
- Zadra, Jonathan; Proffitt, Dennis (2014). "Implicit Associations Have a Circadian Rhythm". PLOS ONE. 9 (11): e110149. doi:10.1371/journal.pone.0110149. PMC 4217730. PMID 25365254.
- Schofield, Timothy (2016). "Time-of-day effects in implicit racial in-group preferences are likely selection effects, not circadian rhythms". PeerJ. 4: e1947. doi:10.7717/peerj.1947. PMC 4841218. PMID 27114886.
- Vedantam, Shankar (2005-01-23). "See No Bias". Washington Post. Retrieved 2008-10-10.
- "Overcoming Prejudice".
This article's use of external links may not follow Wikipedia's policies or guidelines. (January 2017) (Learn how and when to remove this template message)
- Project Implicit – take the test
- IAT Review chapter in Automatic processes in social thinking and behavior (seven years after IAT creation)
- Another IAT Review chapter in Implicit measures of attitudes: Procedures and controversies (seven years after IAT creation)
- IAT Review chapter in Zeitschrift für Experimentelle Psychologie (three years after IAT creation)
- IAT critique in American Psychologist
- IAT critique in Wall Street Journal
- Discussion of IAT critiques in American Psychological Association article
- Further Readings on Unconscious Bias from John Tierney New York Times column on Bias of Bias test
- Dr. Anthony Greenwald's lab page
- Downloadable IAT materials on Dr. Anthony Greenwald's lab page
- IAT validity information on Dr. Anthony Greenwald's lab page
- Dr. Mahzarin Banaji's lab page (co-founder of Project Implicit)
- Dr. Brian Nosek's lab page (co-founder of Project Implicit)
- Televised Lecture "The Psychology of Blink: Understanding How Our Minds Work Unconsciously - Part 1 of 2" (recorded March 5, 2008)
- Word Associations Network