Kernels

kernel Functions

We define a kernel function to be a real-valued function of two arguments \(k(\mathbf{x}, \mathbf{x}^\prime) = <\Phi(\mathbf{x}), \Phi(\mathbf{x}^\prime)>_H \in \mathbb{R}\), for \(\mathbf{x}, \mathbf{x}^\prime \in \mathbb{X}\) that measures similarity between two inputs. Where \(\Phi\) is maps into some inner product space \(H\), sometimes called the feature space.

Positive Definite Kernel

It can be shown that a kernel that corresponds to an inner product in some inner product space coincides with the class of positive definite kernels.

Definition 1: Gram Matrix

Given a kernel \(k\) and inputs \(x_1, ...., x_n \in \mathbb{X}\), the \(n \times n\) matrix:

\[K := (k(x_i, x_j))_{ij}\]

is called the Gram matrix or kernel matrix of \(k\) w.r.t \(x_1, ...., x_n\)

Definition 2: Positive Definite Kernel

The function \(k\) is called a positive definite kernel if:

\[\sum^N_{n=1} \sum^N_{m=1} a_n a_m k(\mathbf{x}_n, \mathbf{x}_m) \geq 0\]

for any real numbers \(a_n, a_m\) and points \(\mathbf{x}_n, \mathbf{x}_m \in \mathbb{X}\) and any \(N \in \mathbb{N}\).