Finite

In a previous post(62_Gradient Information), we discussed the parameter known as the learning rate($\eta $) in the Stochastic Gradient Descent algorithm. $\eta $ is a parameter that needs to be set, and if its value is too small, learning proceeds slowly. On the other hand, if $\eta $ is too large, it can lead to very unstable training of the model. Therefore, setting an appropriate value for $\..

In various neural netwoks, to find a sufficiently good solution, it is necessary to run the gradient-based algorithms described earlier multiple times. Therefore, it is essential to use different randomly selected starting points and compare the performance of each. The gradient of the error function in deep neural networks can be efficiently evaluated using error backpropagation, and this infor..

Similar to regression and classification models, model parameters are defined by optimizing an error function. optimization can be efficiently performed using gradient information, making it essential for the function to be diffferentiable. Here, the derivative is calculated using backpropagation, and the details of backpropagation will be covered later. Finding the optimal values for the weight..

The main idea of neural networks is to select basis functions with learnable parameters and adjust these parameters to minimize an error function defined over all parameters in the model. To achieve this, gradient-based optimization method, like gradient descent, are typically used to optimize the entire model. For example, in a basic neural network model with two layers of learnable parameters,..

Continuing with additional content following the last post. If there is data composed of a dataset and corresponding class labels, a parametric form for the class-conditional densities can be specified. Through this, the values of the parameters can be determined using maximum likelihood. First, let's assume there are two classes. Each class has a Gaussian class-conditional density with a shared..

In the previous post(57_Single Layer Classification), we studied methods of classification using discriminant function. In this post, we will explore classification methods using a generative approach, modeling conditional densities and prior probabilities. Starting, as before, with a problem involving two classes. The posterior probability of class ${C_1}$ is as follows. Here, the posterior pro..

In this post, we will study classificaion problems that can be represented with single-layer neural networks. I would like to emphasize again that I am studing this based on C hris Bishop's "Deep Learning - Foundations and Concepts" Classification aims to assign an input vector $x$ to one of $K$ discrete classes, ${C_k}$. Consequently, the input space is divided into decision regions, and these ..

The maximum likelihood method processes the entire training set at once, which can be computationally expensive with large datasets. Therefore, if the data is large, it is advisable to use a sequential or online algorithm that considers data points one at a time and updates the model paramters subsequently. When applying a sequential algorithm, upon presenting data point $n$, the parameter vecto..

Although covered in a previous post(52_Gaussian Distribution), it seems necessary to provide additional explanation, so in this post, I'll write about the likelihood function. Firstly, the likelihood function is a function that represents how likely the observed data is given the model parameters. Therefore, in deep learning, the likelihood function plays an important role. Probability and likel..

Before delving deeply into deep learning, I have been posting about important foundational knowledge, and today we will study linear regression models through a very simple neural network example. Regression involves predicting one or more target variable values, $t$, when given input variables in the form of an $N$-dimensional vector. In typical supervised learning, a model is trained using a s..