Talk:Anomaly detection

From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Databases / Computer science  (Rated Start-class, Mid-importance)
WikiProject icon This article is within the scope of WikiProject Databases, a collaborative effort to improve the coverage of database related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Start-Class article Start  This article has been rated as Start-Class on the project's quality scale.
 Mid  This article has been rated as Mid-importance on the project's importance scale.
Taskforce icon
This article is supported by WikiProject Computer science (marked as Mid-importance).
 
WikiProject Computer science (Rated Start-class, Mid-importance)
WikiProject icon This article is within the scope of WikiProject Computer science, a collaborative effort to improve the coverage of Computer science related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Start-Class article Start  This article has been rated as Start-Class on the project's quality scale.
 Mid  This article has been rated as Mid-importance on the project's importance scale.
 
WikiProject Statistics (Rated Start-class, Mid-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

Start-Class article Start  This article has been rated as Start-Class on the quality scale.
 Mid  This article has been rated as Mid-importance on the importance scale.
 

Requested move[edit]

The following discussion is an archived discussion of a requested move. Please do not modify it. Subsequent comments should be made in a new section on the talk page. No further edits should be made to this section.

The result of the move request was: No consensus. — Martin (MSGJ · talk) 11:52, 14 July 2010 (UTC)


Anomaly detectionOutlier detectionRelisted. Vegaswikian (talk) 02:31, 2 July 2010 (UTC) As per WP:COMMONNAME: it seems to me that "outlier" is much more common than "anomaly": [1] are the top articles in data mining. Anomaly detection is only used in the title of #656 and #989 of the top 1000. "outlier" is #87, #108, #119, #123 (this is Local Outlier Factor), #348, #353, #507, #620, #663, #772, #937, #948, #973, #974. I have the impression that "anomaly detection" is more used in the network intrusion context, while outlier detection is in data mining maybe? -- Chire (talk) 13:33, 16 June 2010 (UTC)

  • Anomaly detection is used slightly more often in the scholarly literature, but the articles using outlier detection seem more highly cited. I'd say it's a toss up between the two. Fences&Windows 19:32, 1 July 2010 (UTC)
Do you have some references using "anomaly detection" except the survey in the article? ISBN 1558609016 has a chapter 7.11 titled "Outlier Analysis", where all subpoints include "outlier detection" in their name. In ISBN 0387244352, chapter 7 is titled "outlier detection". Apart from my own experience (in the KDD community, not in network intrusion) it is more common. It also seems to be in industry: PMML seems to have an "outliers" XML attribute; "Oracle Data Mining Concepts" [2] mentions "outliers" but not "anomaly". Java Data Mining seems to use "outlier identification" [3]. The only hit in the WEKA wiki is for "outlier", too. --Chire (talk) 22:15, 6 July 2010 (UTC)
You're cherry-picking sources and assuming that data mining is the only use. Data security articles using "anomaly detection" in their thousands,[4] and so do data mining articles, though less often.[5] Fences&Windows 18:14, 11 July 2010 (UTC)
The above discussion is preserved as an archive of a requested move. Please do not modify it. Subsequent comments should be made in a new section on this talk page. No further edits should be made to this section.

Need citation of independent sources[edit]

Thank you, ‎91.52.6.30. Your edits of the first paragraphs are a nice improvement. I noticed that you also removed the citation needed tags I put on paragraph 2. I still feel that each of the 3 sentences in paragraph 2 make claims that should each be backed up by citations. What do other people think? Karl (talk) 13:38, 26 November 2012 (UTC)

I don't think this needs a reference. Port scans etc. do come in bursts. A lot of people in outlier detection seem to use the KDDCup1999 data set (which actually is flawed: [6] and shouldn't be used). In the variant that I looked at, it had less than 20% "normal" entries, while the largest classes 52% smurf attacks, 18% neptune attacks. So in order to have this data set make sense for outlier detection, you clearly do need to aggregate the data set into something like host features etc. - i.e. detect bursts coming from such attacks. If you really need a reference, how about this one:
Paul Dokas, Levent Ertoz, Vipin Kumar, Aleksandar Lazarevic, Jaideep Srivastava, Pang-Nig Tan
Data Mining for Network Intrusion Detection
There are generally two types of attacks in network intrusion detection: the attacks that involve single connections and the attacks that involve multiple connections (bursts of connections). The standard metrics (Table 1) treat all types of attacks similarly thus failing to provide sufficiently generic and systematic evaluation for the attacks that involve many network connections (bursty attacks). Therefore, two types of analysis may be applied; multi-connection attack analysis for bursty attacks and the single-connection attack analysis for single connection attacks.
I think this is a pretty sound reference (Vipin Kumar certainly is highly regarded) supporting that paragraph. I added it to the article. --Chire (talk) 11:45, 27 November 2012 (UTC)
Great reference. Thank you. Karl (talk) 12:21, 27 November 2012 (UTC)