Talk:Discriminative model

From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Robotics (Rated Stub-class, Mid-importance)
WikiProject icon Discriminative model is within the scope of WikiProject Robotics, which aims to build a comprehensive and detailed guide to Robotics on Wikipedia. If you would like to participate, you can choose to edit this article, or visit the project page (Talk), where you can join the project and see a list of open tasks.
Stub-Class article Stub  This article has been rated as Stub-Class on the project's quality scale.
 Mid  This article has been rated as Mid-importance on the project's importance scale.
 
WikiProject Statistics (Rated Stub-class, Low-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

Stub-Class article Stub  This article has been rated as Stub-Class on the quality scale.
 Low  This article has been rated as Low-importance on the importance scale.
 
WikiProject Computing (Rated Stub-class)
WikiProject icon This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Stub-Class article Stub  This article has been rated as Stub-Class on the project's quality scale.
 ???  This article has not yet received a rating on the project's importance scale.
 




Clarifications required[edit]

"Discriminative models differ from generative models in that they do not allow one to generate samples..."

Samples of what? ---good question, because there are many discriminative models that allow one to generate samples of a target variable by modeling the posterior, but they are just not full probability models for all of the latent variables, or variables one might not care about in certain applications. Therefore, that's a poor statement by the authors if they don't specify. Mcswell 15:13, 12 November 2007 (UTC)


Conflict: In page on Discriminative_model"However, for tasks such as classification and regression that do not require the joint distribution, discriminative models generally yield superior performance. " (compared to generative models)

In page on Generative_model "They(discriminative models) don't necessarily perform better than generative models at classification and regression tasks." vks1712 14:53, 23 April 2012 (EST)

Questions[edit]

Here are some questions that should be clarified in this article:

  • Why is an SVM a discriminative model? It does not model a probability distribution. If modeling probability distributions is not a necessary condition, it should be removed from the definition.
    • Yes it does, just in an odd way. It selects decision boundaries, and these decision boundaries almost always coincide to an optimal set of decision boundaries for some particular probability distributions for the classes. Of course, the classes usually do not obey these distributions, but an SVM will be equivalent to modeling the classes with those distributions, then selecting the optimal decision boundary.
  • Is it true that all supervised learning algorithms are considered to be discriminative because they make predictions in one one direction? (ie X predicts Y, but Y does not necessarily predict X). If so, then why are Decision_trees, Naive_bayes, KNN , etc not on the list next to SVMs?
  • Does this diagram represent the kind of data for which discriminative models are well-suited? Surjection.svg
  • If you train two discriminative models, one to predict X->Y, and one to predict Y->X, do they together form a generative model, since you can use them to predict in both directions? To be considered generative, would the Y->X model need to probabilistically sometimes predict '3' and sometimes predict '4'? Or would it need to return a distribution that specifies the probabilities of '3' and '4'?

Discriminative vs. Discriminant[edit]

Bishop (2007) distinguishes between discriminative approaches that learn p(y|x) use it to construct a decision rule and discriminant approaches that directly learn a discriminant function \hat{y}=f(x) without learning the posterior density. According to this terminology some of the examples listed in this article are not discriminative methods, e.g. SVM. My suggestion would be to introduce the distinction between discriminative and discriminant and create two separate lists. Falk (talk) 21:52, 21 September 2010 (UTC)

Linear discriminant analysis[edit]

According to Linear classifier#Generative models vs. discriminative models, LDA is not a discriminative model despite its name. It models each class by a Gaussian, and the constraints on the covariance matrix determine the shape of the decision boundary (linear, quadratic, ..). Amro (talk) 23:43, 29 September 2010 (UTC)