Finite

I plan to study diffusion models, which are widely used method for constructing generative models that involve findinf a distribution $p({\text{z)}}$ for the latent variables ${\text{z}}$ and using deep neural networks to transform ${\text{z}}$ into the data space ${\text{x}}$. Diffusion models work by gradually adding noise to trained images, transforming them into samples of Gaussian distribut..

In real-world applications such as image generation, distributions are highly complex, leading to significant performance improvements with the introduction of deep learning-based generative models. In this post, we will study Generative Adversarial Networks (GANs). In GANs, the generative model learns the distribution from a training dataset using machine learning algorithms and generates new e..

As explained in the previous post, image processing can be performed using Convolutional Neural Networks (CNNs) through graphs. By treating pixels as nodes and edges as pairs of adjacent pixels in the image, an image can be viewed as a specific instance of graph-structured data. Here, $f$ represents a differentiable non-linear activation function, and because the same function is applied across ..

Graphs are commonly composed of a set of objects called nodes, which are connected by edges. Additionally, data can be associated with both nodes and edges. Therefore, graphs can represent many systems across various fields such as social networks, biology, and natural language processing. In this post, we will explore how to apply deep learning to graph-structured data. This involves predicting..

In this post, we will explore an architectural approach known as Convolutional Neural Networks (CNNs), which is a crucial architecture extensively utilized in many deep learning applications. Therefore, we will meticulously examine the contents of the book, referred to as "Chris Bishop's 'Deep Learning - Foundations and concepts' ", to provide a comprehensive understanding of this important topi..

In deep neual networks, increasing the number of layers in the network can improve generalization performance. Additionally, batch normalization can mitigate the problems of vanishing and exploding gradients. However, even with batch normalization, training a network with many layers is challenging. The representational power of a neural network increase exponentially with its depth. However, th..

It has been emphasized that using gradient information is crucial for efficiently training neural networks. While there are many methods to evaluate the gradient of a neural network's error function, one method worth exploring is "automatic differentiation" or "autodiff". Unlike symbolic differentiation, the goal of automatic differentiation is not to find the mathematical expression of the deri..

In the previous post, I described how to fine the derivatives of the weights through backpropagation of the network. In this post, I'll write about the process of calculating the Jacobian matrix, which is given as the derivative of the network's output with respect to the input through backpropagation, and the Hessian matrix, which is a second-order derivative. Each derivative is calculated whil..

In this post, the focus is on efficiently finding the gradient of the error function $E(w)$ of a feedforward neural network. This can be achieved by sending information backward through the network, a process also known as error backpropagation. The term 'backpropagation' is used to clearly describe the procedure for numerically evaluating derivatives, such as the gradient of the error function ..

When training a neural network through a network, if the date to be trained contains extremely large or small values, normalizing the data to a consistent level is very efficient in the learning process. Therefore, normalization is considered a necessary and essential part of neural networks. For example, the presence of extremely large data makes gradient descent training much more difficult. A..