Jump to content

User:Miestersean/DARPA Network Challenge Sandbox

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by Miestersean (talk | contribs) at 00:22, 28 February 2012 (Add sections on strategies of the teams and findings). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

The 2009 DARPA Network Challenge was a prize competition for exploring the roles the Internet and social networking play in the real-time communications, wide-area collaborations, and practical actions required to solve broad-scope, time-critical problems. The competition was sponsored by the Defense Advanced Research Projects Agency (DARPA), a research organization of the United States Department of Defense. The challenge was designed to help the military generate ideas for operating under a range of circumstances, such as natural disasters.[1] Congress authorized DARPA to award cash prizes to further DARPA’s mission to sponsor revolutionary, high-payoff research that bridges the gap between fundamental discoveries and their use for national security.

Specifics of the Competition

Under the rules of the competition, the $40,000 challenge award would be granted to the first team to submit the locations of 10 moored, 8-foot, red, weather balloons at 10 previously undisclosed fixed locations in the continental United States. The balloons were to be placed in readily accessible locations visible from nearby roads. The balloons were deployed at 10:00 AM Eastern Time on December 5, 2009 and scheduled to be taken down at 5:00 PM. DARPA was prepared to deploy them for a second day and wait for up to a week for a team to find all of the balloons.

The contest was announced only about a month before the start date. This limited the amount of time teams had to prepare. The ability of many to do so showed the effectiveness of mass and social media to quickly distribute information and organize people.[2]

DARPA selected the date of the competition to commemorate the 40th anniversary of the Internet.

Results

As mentioned above, DARPA was prepared to deploy the balloons for a second day and accept submissions for up to a week until a team found all 10 balloons. However, the MIT Red Balloon Challenge Team won the competition in under 9 hours.[3] In second place was a team from the Georgia Tech Research Institute (GTRI), who located nine balloons. Additionally, two teams found eight balloons, five found seven, and the iSchools team (which represented Pennsylvania State University, University of Illinois at Urbana-Champaign, University of Pittsburgh, Syracuse University, and University of North Carolina at Chapel Hill), whose strategy is described below, finished tenth with six balloons.[2] In table form, the top ten teams were:[4]

The winners.
The winning team (from L to R): Professor Sandy Pentland, Manuel Cebrian, Anmol Madan, Galen Pickard, Riley Crane, Wei Pan.
Place Name Hometown # Balloons Date/Time
1 MIT Red Balloon Challenge Team Cambridge, MA 10 6:52:41 PM
2 GTRI "I Spy a Red Balloon" Team Atlanta, GA 9 6:59:11 PM
3 Christian Rodriguez and Tara Chang (Red Balloon Race) Cambridge, MA 8 6:52:54 PM
4 Dude It's a Balloon Glen Rock, NJ 8 7:42:41 PM
5 Groundspeak Geocachers Seattle, WA 7 4:02:23 PM
6 Army of Eyes ‐ Mutual Mobile, Inc. Austin, TX 7 4:33:20 PM
7 Team DeciNena Evergreen, CO 7 6:46:37 PM
8 Anonymous Anonymous 7 7:16:51 PM
9 Nerdfighters Missoula, MT 7 8:19:24 PM
10 iSchools DARPA Challenge Team State College, PA 6 6:13:08 PM

Winning MIT strategy

The winning MIT team used a technique similar to multi-level marketing to recruit participants, with the prize money to be distributed up the chain of participants leading to successful balloon spottings, and all prize income remaining after distribution to participants to be given to charity.[5] The team's strategy for public collaboration in finding the balloons was explained on their website:

We're giving $2000 per balloon to the first person to send us the correct coordinates, but that's not all -- we're also giving $1000 to the person who invited them. Then we're giving $500 whoever invited the inviter, and $250 to whoever invited them, and so on... (see how it works). It might play out like this. Alice joins the team, and we give her an invite link like http://balloon.media.mit.edu/alice. Alice then e-mails her link to Bob, who uses it to join the team as well. We make a http://balloon.media.mit.edu/bob link for Bob, who posts it to Facebook. His friend Carol sees it, signs up, then twitters about http://balloon.media.mit.edu/carol. Dave uses Carol's link to join... then spots one of the DARPA balloons! Dave is the first person to report the balloon's location to us, and the MIT Red Balloon Challenge Team is the first to find all 10. Once that happens, we send Dave $2000 for finding the balloon. Carol gets $1000 for inviting Dave, Bob gets $500 for inviting Carol, and Alice gets $250 for inviting Bob. The remaining $250 is donated to charity.

The recursive nature of the reward had two beneficial effects.[2] First, participants had an incentive to involve others, as these new people would not be become competitors for the reward but rather cooperating partners. Second, people not located in the United States were motivated to participate by passing along information even though they had no way of locating a balloon in person. This helped the team garner a large amount (over 5,000) of participants.

To determine whether submissions were legitimate or fake, the team employed at least three strategies. The first strategy was examining whether there were multiple submissions for a location. If this was the case, then the likelihood of a balloon actually being there was thought to be higher. A second strategy was to check whether the IP address of the submitter matched the supposed location of the balloon. A third strategy was to examine photos accompanying the submission. Real photos included a DARPA employee and a DARPA banner, details which were not announced, while faked ones did not.

Second-place Strategy

The second-place GTRI team from Georgia Tech University used a strategy that relied heavily on Internet publicity and social media.[2] They created a Web site three weeks before the launch day and used a variety of media-related efforts, including a Facebook group, in order to increase the visibility of the team and increase the chance that people who spotted the balloons would report the sighting to them.

The team promised to donate all winnings to charity to appeal to the altruism of participants. However, due to the lack of structure that created much incentive as MIT's scheme, their network of participants grew to only about 1,400 people.

With regards to validating submissions, the team assumed that because of the charitable nature of their effort, the number of false submissions would be low. In any case, they primarily relied on personal validation, having phone conversations with submitters.

Tenth-place Strategy

The tenth-place iSchools team, which represented five universities, tried two distinct approaches.[2] The first was directly recruiting team members to look for the balloons on launch day. These members included students, faculty, and alumni on official mailing lists and social media website groups for organizations on the team (e.g., Pennsylvania State University). Only a few of these observers actually participated, however, and only one balloon was found using this strategy.

The second strategy used by the team was using Open Source Intelligence methods to do cyberspace searching for results related to the challenge. This was the main source of their success in locating balloons. This strategy, in turn, consisted of two distinct strategies. The first was to use a group of human analysts who would manually search online on a variety of information sources, including Twitter and the websites of competing teams, compile reported sightings, and then evaluate the validity of sightings based on the reputation of the sources.

The second strategy relating to cyberspace searching was an automated Web crawler which captured data from Twitter and opposing teams' websites and then analyzed it. This technology worked slowly and would have benefited from a longer contest duration, but the Twitter crawler proved to be especially useful because tweets sometimes contained geographic information.

To confirm the validity of possible sightings, recruited team members were used when possible. If none were available, new observers were recruited from organizations located near the sighting. The distributed location of the different organizations in the team allowed this to be a feasible strategy. Photographic analysis was also used to confirm or dispute the validity of claims.

The team also encountered a case of another team accidentally leaking information about a sighting and then trying to cover it up. The iSchools team used a variety of information sources, including social networks, to determine what the real location was. This demonstrated the possibility of using information from a wide variety of public websites to determine the validity of something.

Other Strategies

Prior to the competition numerous people had discussed possible strategies,[6] including satellite photography, aerial photography and crowdsourcing to detect balloons, as well as the possibility of misinformation campaigns to stop other teams from winning.

Findings

The challenge generated a number of insights.[2] First, it showed how mass and social media can act complementarily. While mass media was useful primarily for spreading general information about the challenge, social media was effective for viral dissemination of information about the challenge to potential team recruits. Second, it showed how social media can be useful as a data mining source. For example, the iSchools team did better than many other teams by simply monitoring public websites. Third, the challenge showed the variety of ways in which social networking can be utilized. The MIT and GTRI teams used it primarily to facilitate fast communication between participants, while the iSchools team used it as a source of information.

Fourth, the challenge showed the general effectiveness of using crowdsourcing techniques to solve geographically-distributed, time-sensitive problems. The DARPA program managers were surprised by how quickly the challenge was completed, and the ability of teams to locate and verify sightings was promising. The results have implications for similar problems such as locating missing children, though more time would need to be spent in reducing the noisiness of data collected and more efficiently verifying results to improve the process.

Verified balloon locations

Balloon locations

The officially-verified coordinates of the balloons,[7] listed by their tag numbers, were:

Inspired by the success of the DARPA Network Challenge, DARPA launched the Shredder Challenge in 2011. This competition aimed to explore methods to reconstruct documents shredded by a variety of paper shredding techniques. As with the DARPA Network Challenge, some teams used crowdsourcing to solicit human help in reconstructing the documents.[9] The winning team used a combination of computer-vision algorithm to suggest fragment pairings to human assemblers for verification.[10]

In January 2012, the University of Pennsylvania School of Medicine launched the MyHeartMap Challenge to map Automatic External Defibrillators (AEDs) in the city of Philladelphia.[11] According to the organizer Dr. Raina Merchant, "DARPA succeeded with locating red balloons. AEDs are a natural extension of a brilliant idea."[12]

References

  1. ^ "MIT wins $40,000 prize in nationwide balloon-hunt contest". CNN. 2009. Retrieved 2012-02-21.
  2. ^ a b c d e f John C. Tang, Manuel Cebrian, Nicklaus A. Giacobe, Hyun-Woo Kim, Taemie Kim, Douglas "Beaker" Wickert (2011). "Reflecting on the DARPA Red Balloon Challenge". Communications of the ACM. 54 (4): 78-85.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  3. ^ "MIT Red Balloon Team Wins DARPA Network Challenge" (PDF). DARPA. Retrieved 2009-12-06.
  4. ^ "DARPA Network Challenge Final Standings" (PDF). DARPA. Retrieved 2010-10-07.
  5. ^ "How It Works". MIT Red Balloon Challenge Team.
  6. ^ Adrian Hon (October 31, 2009). "How to Win the DARPA Network Challenge". Mssv.
  7. ^ "DARPA Network Challenge Balloon Coordinates" (PDF). DARPA. Retrieved 2009-12-13.
  8. ^ "Ten red balloons– and one's in Charlottesville!". The Hook. Saturday December 5, 2009. {{cite web}}: Check date values in: |date= (help)
  9. ^ "Crowdsourcing the 'most challenging puzzle ever". CNET. November 17, 2011. Retrieved 2011-12-01.
  10. ^ Drummond, Katie (December 2, 2011). "Programmers Shred Pentagon's Paper Puzzle Challenge". Wired. Retrieved December 5, 2011. {{cite web}}: Italic or bold markup not allowed in: |publisher= (help)
  11. ^ McCullough, Marie (January 31, 2012). "Global contest will lead to help during heart attacks". Philadelphia Enquirer. Retrieved 2012-02-02.
  12. ^ "MyHeartMap Challenge Media Page". University of Pennsylvania. Retrieved 2012-02-03.