Doug Cutting

From Wikipedia, the free encyclopedia
Jump to: navigation, search
Doug Cutting

Douglass Read "Doug" Cutting is an advocate and creator of open-source search technology. He originated Lucene and, with Mike Cafarella, Nutch, both open-source search technology projects which are now managed through the Apache Software Foundation. He is also the creator of Hadoop (Yahoo!, Cloudera).

Education and early career[edit]

He holds a bachelor's degree from Stanford University.[1]

Prior to developing Lucene, Doug held search technology positions at Xerox PARC where he worked on the Scatter/Gather algorithm [2] [3] and on computational stylistics, [4], Excite where he was one of the chief designers of the search engine, and Apple Inc., where he was the primary author of the V-Twin text search framework,[5].

Open source projects[edit]

Lucene, a search indexer, and Nutch, a spider or crawler, are the two key components of an open-source general search platform, which first crawls the Web for content, and then structures it into a searchable index. Cutting's leadership of these two projects extended the concepts and capabilities of general open-source software projects such as Linux and MySQL into the important vertical domain [clarification needed] of search. While it is difficult to track the total number of installations of these platforms, public announcements of the use of Lucene and its direct descendant Solr by various venture-backed startups indicate a significant level of adoption.[6]

Use of MapReduce paradigm[edit]

In December 2004, Google Labs published a paper on the MapReduce algorithm, which allows very large scale computations to be trivially parallelized across large clusters of servers. Cutting, realizing the importance of this paper to extending Lucene into the realm of extremely large search problems, created the open-source Hadoop framework that allows applications based on the MapReduce paradigm to be run on large clusters of commodity hardware. Cutting was an employee of Yahoo!, where he led the Hadoop project full-time; he has since moved on to Cloudera.[7]

Open source foundations and awards[edit]

In July 2009, Doug Cutting was elected to the board of directors of the Apache Software Foundation, and in September 2010, he was elected its chairman.

In 2015, O'Reilly awarded an open source award to Cutting.[8]


  1. ^ "Cloudera management team". Cloudera. Retrieved 2016-08-17. 
  2. ^ Cutting, Douglass R., David R. Karger, Jan O. Pedersen, and John W. Tukey. "Scatter/gather: A cluster-based approach to browsing large document collections." SIGIR '92 Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval. (Reprinted in ACM SIGIR Forum, vol. 51, no. 2, pp. 148-159. ACM, 2017.)
  3. ^ Pedersen, Jan O., David Karger, Douglass R. Cutting, and John W. Tukey. "Scatter-gather: a cluster-based method and apparatus for browsing large document collections." U.S. Patent 5,442,778, issued August 15, 1995.
  4. ^ Karlgren, Jussi; Cutting, Douglass. "Recognizing text genres with simple metrics using discriminant analysis.". Proceedings of the 15th conference on Computational linguistics-Volume 2. Association for Computational Linguistics, 1994. 
  5. ^ "The Lucene search engine: Powerful, flexible, and free". JavaWorld. 2000-09-15. Retrieved 2017-01-25. Cutting is the primary author of the V-Twin search engine (part of Apple's Copland operating system effort)… 
  6. ^ "Wikipedia: Powered by Lucene". Lucene. Retrieved September 5, 2007. 
  7. ^ Handy, Alex (10 August 2009). "Hadoop creator goes to Cloudera". Software Development Times. Archived from the original on 13 March 2012. Retrieved 2011-03-22. 
  8. ^ "O'Reilly Open Source Awards - OSCON 2015". YouTube. O'Reilly. Retrieved 27 July 2015. 

External links[edit]