58_Generative Classifiers

In the previous post(57_Single Layer Classification), we studied methods of classification using discriminant function. In this post, we will explore classification methods using a generative approach, modeling conditional densities and prior probabilities.

 

Starting, as before, with a problem involving two classes. The posterior probability of class ${C_1}$ is as follows. Here, the posterior probability represents calculating the probability of cause A after observing event B, essentially working backwards based on known information to find the answer. In this case, the likelihood function is also used.

 

 

 

Where $\sigma $ is the logistic sigmoid function and represents the log form of the ratio of probabilities for the two classes. When the number of classes is greater than 2, it is as follows.

 

 

This equation is a normalized exponential function and a generalization of the logistic sigmoid. Where ${a_k}$ is as follows.

 

 

Now lets look at the form of the posterior probability when the class-conditional densities are assumed to be Gaussian. The density for class ${C_k}$ is as follows.

 

 

Where ${\Bbb C}$ is the converiance matrix, and it is assumed that all classes share the same converiance matrix.

Based on the previous equations, it can be summarized as follows.

 

 

Since the same converiance matrix is shared, it can be expressed as a linear function of ${\text{x}}$ in the argument of the logistic sigmoid. Therefore, in the general case of $K$ classes, it can be defined as follows.

 

 

Similarly, due to the common converiance, the quadratic terms cancel out, indicating that it is a linear function of ${\text{x}}$. here, if the common coveriance matrix is not used and each class conditional density has its own coveriance matrix, the quadratic terms do not cancel out, resulting in a quadratic discriminant function.