Measure Integral and Real Analysis (4)

Hilbert Spaces

Inner Product Spaces

Definition 8.1: Inner Product; Inner Product Space

An inner product on a vector space \(V\) is a function that takes each ordered pair \(f, g\) of elements of \(V\) to a number \(<f, g> \in \mathbb{F}\) and has the following properties:

• Positivity: \[<f, f> \in [0, \infty), \; \forall f \in V\]
• Definiteness: \[<f, f> = 0 \text{ IFF } f = 0\]
• Linearity in first slot: \[<f + g, h> = <f, h> + <g, h>, \quad <\alpha f, g> = \alpha<f, g>, \quad \forall f,g,h \in V, \alpha \in \mathbb{F}\]
• Conjugate Symmetry: \[<f, g> = \overline{<g, f>}, \; \forall f, g \in V\]

A vector space with inner product is called an inner product space.

If \(\mathbb{F} = \mathbb{R}\), then the complex conjugate can be ignored as the symmetry property.

Theorem 8.3: Basic Properties of an Inner Product

Suppose \(V\) is an inner product space. Then:

1. \(<0, g> = <g, 0> = 0, \; \forall g \in V\)
2. \(<f, g + h> = <f, g> + <f, h>, \; \forall f, g, h \in V\)
3. \(<f, \alpha g> = \bar{\alpha} <f, g>, \; \forall \alpha \in \mathbb{F}, f, g \in V\)

Definition 8.4: Norm Associated with an Inner Product; \(\|\cdot\|\)

Suppose \(V\) is an inner product space. For \(f \in V\), define the norm of \(f\), denoted \(\|f\|\), by:

\[\|f\| = \sqrt{<f, f>}\]

Theorem 8.6: Homogeneity of the Norm

Suppose \(V\) is an inner product space, \(f \in V\), and \(\alpha \in \mathbb{F}\). Then:

\[\|\alpha f\| = |\alpha|\|f\|\]

Definition 8.7: Orthogonal

Two elements of an inner product space are called orthogonal if their inner product equals \(0\).

Definition 8.24: Distance from a Point to a Set

Suppose \(U\) is a nonempty subset of a normed vector space \(V\) and \(f \in V\). The distance from \(f\) to \(U\), denoted \(distance(f, U)\), is defined by:

\[distance(f, U) = \inf\{\|f - g\|: g \in U\}\]

Definition 8.25: Convex

• A subset of a vector space is called convex if the subset contains the line segment connecting each pair of the points in it.
• More precisely, suppose \(V\) is a vector space and \(U \subseteq V\). Then \(U\) is called convex if: \[(1 - t) f + t g \in U, \; \forall t \in [0, 1], f, g \in U\]

Theorem 8.28: Distance to a Closed Convex Set is Attained in a Hilbert Space

• The distance from an element of a Hilbert space to a nonempty closed convex set is attained by a unique element of the nonempty closed convex set.
• More specifically, suppose \(V\) is a Hilbert space, \(f \in V\), and \(U\) is a nonempty closed convex subset of \(V\). Then there exists a unique \(g \in U\) s.t: \[\|f - g\| = distance(f, U)\]

Definition 8.34: Orthogonal Projection; \(P_U\)

Suppose \(U\) is a nonempty closed convex subset of a Hilbert space \(V\). Then orthogonal projection of \(V\) onto \(U\) is the function \(P_U: V \rightarrow V\) defined by setting \(P_U(f)\) equal to the unique element of \(U\) that is closest to \(f\).

This definition only makes sense because of theorem 8.28.

Theorem 8.37: Orthogonal Projection onto Closed Subspace

Suppose \(U\) is a closed subspace of Hilbert space \(V\) and \(f \in V\). Then:

1. \(f - P_U(f)\) is orthogonal to \(g\) for every \(g \in U\).
2. If \(h \in U\) and \(f - h\) is orthogonal to \(g\) for every \(g \in U\), then \(h = P_U(f)\).
3. \(P_U: V \rightarrow V\) is a linear map.
4. \(\|P_U (f)\| \leq \|f\|\), with equality IFF \(f \in U\).

Definition 8.38: Orthogonal Complement; \(U^\perp\)

Suppose \(U\) is a subset of an inner product space \(V\). The orthogonal complement of \(U\) is denoted by \(U^\perp\) and is defined by:

\[U^{\perp} = \{h \in V: <g, h> = 0, \; \forall g \in U\}\]

In other words, the orthogonal complement of a subset \(U\) of an inner product space \(V\) is the set of elements of \(V\) that are orthogonal to every element of \(U\).

Theorem 8.40: Properties of Orthogonal Complement

Suppose \(U\) is a subset of an inner product space \(V\). Then

1. \(U^\perp\) is a closed subspace of \(V\).
2. \(U \cap U^\perp \subseteq \{0\}\).
3. If \(W \subseteq U\), then \(U^\perp \subseteq W^\perp\).
4. \(\bar{U}^{\perp} = U^{\perp}\)
5. \(U \subseteq (U^{\perp})^\perp\)

Theorem 8.41: Orthogonal Complement of the Orthogonal Complement

Suppose \(U\) is a subspace of a Hilbert space \(V\). Then:

\[\overline{U} = (U^\perp)^\perp\]

As a special case, if \(U\) is a closed subspace of a Hilbert space \(V\), then:

\[U = (U^\perp)^\perp\]

Theorem 8.42: Necessary and Sufficient Condition for a Subspace to be Dense

Suppose \(U\) is a subspace of a Hilbert space \(V\). Then:

\[\overline{U} = V\]

IFF

\[U^\perp = \{0\}\]

Theorem 8.43: Orthogonal Decomposition

Suppose \(U\) is a closed subspace of a Hilbert space \(V\). Then every element \(f \in V\) can be uniquely written in the form:

\[f = g + h\]

where \(g \in U\) and \(h \in U^\perp\). Furthermore, \(g = P_U (f)\) and \(h = f - P_U(f)\).

Definition 8.44: Identity Map; \(I\)

Suppose \(V\) is a vector space. The identity map \(I\) is the linear map from \(V\) to \(V\) defined by:

\[I(f) = f\]

for \(f \in V\).

Theorem 8.45: Range and Null Space of Orthogonal Projections

Suppose \(U\) is a closed subspace of a Hilbert space \(V\). Then:

1. \(\text{range}(P_U) = U\) and \(\text{null}(P_U) = U^\perp\).
2. \(\text{range}(P_{U^\perp}) = U^\perp\) and \(\text{null}(P_{U^\perp}) = U\).
3. \(P_{U^\perp} = I - P_U\).

Theorem 8.47: Riesz Representation Theorem

Suppose \(\varphi\) is a bounded linear functional on a Hilbert space \(V\). Then there exists a unique \(h \in V\) s.t:

\[\varphi(f) = <f, h>\]

for all \(f \in V\). Furthermore, \(\|\varphi\| = \|h\|\).

Orthonormal Bases

Definition 8.50: Orthonormal Family

A family \(\{e_k\}_{k \in \Gamma}\) in an inner product space is called an orthonormal family if:

\[ <e_j, e_k> = \begin{cases} 0, \quad \text{if } j \neq k\\ 1, \quad \text{if } j = k \end{cases} \]

for all \(j, k \in \Gamma\)

in other words, a family is an orthonormal family if \(e_j, e_k\) are orthogonal for all distinct \(j, k \in \Gamma\) and \(\|e_k\| = 1\) for all \(k \in \Gamma\).

Theorem 8.52: Finite Orthonormal Families

Suppose \(\Omega\) is a finite set and \(\{e_j\}_{j \in \Omega}\) is an orthonormal family in an inner product space. Then:

\[\|\sum_{j \in \Omega} \alpha_j e_j\|^2 = \sum_{j \in \Omega} |\alpha_j|^2\]

for every family \(\{\alpha_j\}_{j \in \Omega}\) in \(\mathbb{F}\).

Theorem 8.53: Unordered Sum; \(\sum_{k \in \Gamma} f_k\)

Suppose \(\{f_k\}_{k \in \Gamma}\) is a family in a normed vector space \(V\). Then unordered sum \(\sum_{k \in \Gamma} f_k\) is said to be converge if there exists \(g \in V\) s.t for every \(\epsilon > 0\), there exists a finite subset \(\Omega \subseteq \Gamma\) s.t:

\[\|g - \sum_{j \in \Omega^\prime} f_j\| < \epsilon\]

For all finite sets \(\Omega^\prime\) with \(\Omega \subseteq \Omega^\prime \subseteq \Gamma\). If this happens, we set \(\sum_{k \in \Gamma} f_k = g\). If there is no such \(g \in V\), then \(\sum_{k \in \Gamma} f_k\) is left undefined.

• Suppose \(\{a_k\}_{k \in \Gamma}\) is a family in \(\mathbb{R}\) and \(a_k \geq 0\) for each \(k \in \Gamma\). Then the unordered sum \(\sum_{k \in \Gamma} a_k\) converges IFF: \[\sup\{\sum_{j \in \Omega} a_j: \Omega \text{ is a finite subset of $\Gamma$}\} < \infty\]

  If the sum converges then it is the supremum above.

• Suppose \(\{a_k\}_{k \in \Gamma}\) is a family in \(\mathbb{R}\), then the unordered sum converges IFF: \[\sum_{k \in \Gamma} |a_k| < \infty\]

Theorem 8.54: Linear Combinations of an Orthonormal Family

Suppose \(\{e_k\}_{k \in \Gamma}\) is an orthonormal family in a Hilbert space \(V\). Suppose \(\{\alpha_k\}_{k \in \Gamma}\) is a family in \(\mathbb{F}\). Then:

1. The unordered sum \(\sum_{k \in \Gamma} a_k e_k\) converges \(\Longleftrightarrow \sum_{k \in \Gamma} |a_k|^2 < \infty\)

Furthermore, if \(\sum_{k \in \Gamma} a_ke_k\) converges, then:

2. \(\|\sum_{k \in \Gamma} a_k e_k\|^2 = \sum_{k \in \Gamma} |a_k|^2\)

Theorem 8.57: Bessel's Inequality

Suppose \(\{e_k\}_{k \in \Gamma}\) is an orthonormal family in an inner product space \(V\) and \(f \in V\). Then:

\[\sum_{k \in \Gamma} |<f, e_k>|^2 \leq \|f\|^2\]

Theorem 8.58: Closure of the Span of an Orthonormal Family

Suppose \(\{e_k\}_{k \in \Gamma}\) is an orthonormal family in a Hilbert space \(V\). Then:

1. \(\overline{span(\{e_k\}_{k \in \Gamma})} = \{\sum_{k \in \Gamma} \alpha_k e_k: \{\alpha_k\}_{k \in \Gamma} \text{ is a family in $\mathbb{F}$ and } \sum_{k \in \Gamma} |a_k|^2 < \infty\}\).
2. \(f = \sum_{k \in \Gamma} <f, e_k> e_k\), for every \(f \in \overline{span(\{e_k\}_{k \in \Gamma})}\)

Definition 8.61: Orthonormal Basis

An orthonormal family \(\{e_k\}_{k \in \Gamma}\) in a Hilbert space \(V\) is called an orthonormal basis of \(V\) if:

\[\overline{span(\{e_k\}_{k \in \Gamma})} = V\]

An important point to keep in mind is that despite the terminology, an orthonormal basis is not necessarily a basis in sense of theorem 6.54. In fact, if \(\Gamma\) is infinite set (uncountable) and \(\{e_k\}_{k \in \Gamma}\) is an orthonormal basis of \(V\), then it is not a basis of \(V\) (ie. \(span(\{e_k\}_{k \in \Gamma}) \neq V\))

Theorem 8.63: Parseval's Identity

Suppose \(\{e_k\}_{k \in \Gamma}\) is an orthonormal basis of a Hilbert space \(V\) and \(f, g \in V\). Then:

1. \(f = \sum_{k \in \Gamma} <f, e_k> e_k\).
2. \(<f, g> = \sum_{k \in \Gamma} <f, e_k> \overline{<g, e_k>}\).
3. \(\|f\|^2 = \sum_{k \in \Gamma} |<f, e_k>|^2\).

Definition 8.64: Separable

A normed vector space is called separable if it has a countable subset whose closure equals the whole space.

A normed vector space \(V\) is separable if and only if there exists a countable subset \(C\) of \(V\) such that every open ball in \(V\) contains at least one element of \(C\).

Theorem 8.67: Existence of Orthonormal Bases for Separable Hilbert Spaces

Every separable Hilbert space has an orthonormal basis.

Theorem 8.71: Orthogonal Projection in Terms of an Orthonormal Basis

Suppose that \(U\) is a closed subspace of a Hilbert space \(V\) and \(\{e_k\}_{k \in \Gamma}\) is an orthonormal basis of \(U\). Then:

\[P_U(f) = \sum_{k \in \Gamma} <f, e_k> e_k\]

Theorem 8.75: Existence of Orthonormal Bases for All Hilbert Spaces

Every Hilbert space has an orthonormal basis.

Reproducing Kernel Hilbert Space

Definition 8.80: Kernel

Let \(X\) be a non-empty set. A function \(k: X \times X \rightarrow \mathbb{R}\) is called a kernel If there exists an \(\mathbb{R}\)-Hilbert space \(H\) and a map, \(\phi: X \rightarrow H\) s.t \(\forall x, x^{\prime} \in X\): \[k(x, x^{\prime}) := <\phi(x), \phi(x^{\prime})>_H\]

Definition 8.81: Positive Definite Functions

A symmetric function \(k: X \times X \rightarrow \mathbb{R}\) is positive definite if \(\forall n \geq 1\), \(\forall (a_1, ...., a_n) \in \mathbb{R}^n\), \(\forall (x_1, ..., x_n) \in X^{n}\),

\[\sum^n_{i=1}\sum^n_{j=1} a_ia_j k(x_i, x_j) \geq 0\]

The function \(k(\cdot, \cdot)\) is strictly positive definite if for mutually distinct \(x_i\), the equality holds only when all \(a_i\) are zero.

Theorem 8.82: Every Kernel is a Positive Definite Function

Let \(H\) be any Hilbert space, \(X\) is a non-empty set and \(\phi: X \rightarrow H\). Then \(k(x, y) := <\phi(x), \phi(y)>_H\) is a positive definite function. The reverse also holds. A positive definite function is guaranteed to be the inner product in a Hilbert space \(H\) between features \(\phi(x)\).

Thus, we also call a positive definite function kernel function.

Theorem 8.83: Sums of Kernels are Kernels

Given \(\alpha > 0\), and \(k, k_1, k_2\) are all kernels on \(X\), then \(\alpha k\) and \(k_1 + k_2\) are all kernels on \(X\).

Theorem 8.84: Mapping Between Spaces

Let \(X, \tilde{X}\) be sets, and define a map \(A: X \rightarrow \tilde{X}\). Define the kernel \(k\) on \(\tilde{X}\). Then the kernel \(k(A(x), A(x^{\prime}))\) is a kernel on \(X\).

Theorem 8.85: Products of Kernels are Kernels

Given \(k_1\) on \(X_1\) and \(k_2\) on \(X_2\), then \(k_1 \times k_2\) is a kernel on \(X_1 \times X_2\). If \(X_1 = X_2 = X\), then \(k := k_1 \times k_2\) is a kernel on \(X\).

Theorem 8.86:

For any function \(f: X \rightarrow \mathbb{R}\), the expression \(\tilde{k}:= f(x)k(x, y)f(y)\) defines a kernel. In particular:

\[k(x, y) := f(x) f(y)\]

is a kernel.

Definition 8.87: Reproducing Kernel Hilbert Space (First Definition)

Let \(H\) be a Hilbert space of \(\mathbb{R}\)-valued functions defined on a non-empty set \(X\) (that is the elements of \(H\) are \(\mathbb{R}^X\) functions, that is \(\phi: X \rightarrow \mathbb{R}^X\)). A function \(k: X \times X \rightarrow \mathbb{R}\) is called a reproduce kernel of \(H\) and \(H\) is a reproducing kernel Hilbert space, if \(k\) satisfies:

• \(\forall x \in X\), \(k(\cdot, x) \in H\). (The function \(k_x (\cdot) = k(\cdot, x)\) is an element of \(H\), so every element \(x \in X\) is being mapped to \(k_x \in H\)).
• Reproducing Kernel Hilbert Space is a function space which is a set of all possible linear combination of \(k_{x}, \forall x \in X\). \[H:=\overline{span\{k_x: x \in X\}}\]
• \(\forall x \in X\), \(\forall f \in H\), \(<f, k(\cdot, x)>_H = f(x)\) (THE REPRODUCING PROPERTY)

In particular:

\[k(x, y) = <k_x, k_y>\]

where \(\phi(x) = k_x\)

Given \(f = \sum^n_{i=1} \alpha_i k(\cdot, x_i), g = \sum^n_{j=1} \beta_j k(\cdot, x_j)\) The inner product \(<f, g>_H\) is defined as:

\[<f, g>_H = \sum^n_{i=1}\sum^{n}_{j=1} \alpha_i\beta_j k(x_i, y_j)\]

Theorem 8.88: Uniqueness of Reproduce Kernel

Given a kernel, the corresponding RKHS is unique up to isometric isomorphisms. Given an RKHS, the corresponding kernel is unique. In other words, each kernel generates a new RKHS.

Theorem 8.89: Dirac Evaluation Functional

For a Hilbert space \(H\) of real-valued functions on \(X\), and for any point \(x \in X\), the evaluation functional at \(x\) is defined as the map \(\delta_x: H \rightarrow \mathbb{R}\) s.t for all functions \(f \in H\):

\[\delta_x (f) = f(x)\]

Theorem 8.90: Reproducing kernel Hilbert Space (second definition)

Let \(H\) be a Hilbert space of functions \(f: X \rightarrow \mathbb{R}\). Then the evaluation functionals \(\delta_x\) are bounded and continuous functionals IFF \(H\) has a reproducing kernel \(k\).

Fourier Analysis

• \(\sin (x)\cos (y) = \frac{\sin(x - y) + \sin (x + y)}{2}\)
• \(\sin (x) \sin(y) = \frac{\cos(x - y) - \cos(x + y)}{2}\)
• \(\cos(x)\cos(y) = \frac{\cos(x - y) + \cos(x + y)}{2}\)
• \(e^{it} = \cos(t) + i \sin(t)\)
• \(\overline{e^{it}} = e^{-it}\)
• \(z^n = e^{int}\), this is a function on \(\partial \mathbf{D}\).
• \(\overline{z^n} = e^{-int}\)

Fourier Series and Poisson Integral

Definition 11.3: \(\mathbf{D}, \partial \mathbf{D}\)

• \(\mathbf{D}\) denotes the open unit disk in the complex plane: \[\mathbf{D} = \{w \in \mathbb{C}: |w| < 1\}\]
• \(\partial \mathbf{D}\) is the unit circle in the complex plane: \[\partial \mathbf{D} = \{z \in \mathbb{C}: \|z\| = 1\}\]

\(e^{it} = \cos(t) + i \sin(t)\) is a one to one map of \((-\pi, \pi]\) onto \(\partial \mathbf{D}\).

Definition 11.4: Measurable Subsets of \(\partial \mathbf{D}\); \(\sigma\)

• A subset \(E\) of \(\partial \mathbf{D}\) is measurable if \(\{t \in (-\pi, \pi]: e^{it} \in E\}\) is a Borel subset of \(\mathbb{R}\) or \((e^{it})^{-1}(E) \subseteq B(\mathbb{R})\).
• \(\sigma\) is the measureon the measurable subsets of \(\partial \mathbf{D}\) obtained by transferring Lebesgue measure from \((-\pi, \pi]\) to \(\partial \mathbf{D}\), normalized so that \(\sigma(\partial \mathbf{D}) = 1\). In other words, if \(E \subseteq \partial \mathbf{D}\) is measurable, then: \[\sigma(E) = \frac{|\{t \in (-\pi, \pi]: e^{it} \in E\}|}{2\pi}\]

So we can write the Lebesgue integral on \(\partial \mathbf{D}\):

\[\int_{\partial \mathbf{D}} f d\sigma = \int^\pi_{-\pi} f(e^{it}) \frac{dt}{2\pi}\]

for all measurable functions \(f: \partial \mathbf{D} \rightarrow \mathbb{C}\) that above integrals make sense.

Definition 11.5: \(L^p (\partial \mathbf{D})\)

For \(1 \leq p \leq \infty\), define \(L^p(\partial \mathbf{D})\) to mean the complex version of \(L^p(\sigma)\).

Theorem 11.6: Orthonormal Family in \(L^2 (\partial \mathbf{D})\)

\(\{z^n\}_{n \in \mathbb{Z}}\) is an orthonormal family in \(L^2(\partial \mathbf{D})\).

Definition 11.7: Fourier Coefficient; \(\hat{f}(n)\); Fourier Seires

Suppose \(f \in L^1(\partial \mathbf{D})\)

• For \(n \in \mathbb{Z}\), the \(n\)th Fourier coefficient of \(f\) is denoted \(\hat{f}(n)\) and is defined by: \[\hat{f}(n) = \int_{\partial \mathbf{D}} f(z) \bar{z^n} d\sigma(z) = \int^\pi_{-\pi} f(e^{it}) e^{-int} \frac{dt}{2\pi}\]
• The Fourier Series of \(f\) is the formal sum: \[\sum^\infty_{n=-\infty} \hat{f}(n) z^n\]

Theorem 11.9: Algebraic Properties of Fourier Coefficients

Suppose \(f, g \in L^1(\partial \mathbf{D})\) and \(n \in \mathbb{Z}\). Then:

1. \(\widehat{f + g}(n) = \hat{f}(n) + \widehat{g}(n)\)
2. \(\widehat{\alpha f}(n) = \alpha \hat{f}(n)\) for all \(\alpha \in \mathbb{C}\)
3. \(|\hat{f}(n)| \leq \|f\|_1\)

from \(a, b\), we can say that for each \(n \in \mathbb{Z}\), the function that maps from \(L^1(\partial \mathbf{D}) \rightarrow \mathbb{R} := f \mapsto \hat{f}(n)\) is a linear functional.

Theorem 11.10: Riemann-Lebesgue Lemma

Suppose \(f \in L^1(\partial \mathbf{D})\), then:

\[\lim_{n \rightarrow \infty} \hat{f}(n) = 0\]

Definition 11.24(1): Periodic Function

A function \(f\) is said to be periodic with period \(T\) if:

\[f(x) = f(x + T)\]

Theorem 11.24(2): Derivative of a Periodic Function

The derivative of a periodic function is periodic.

Definition 11.24: \(\tilde{f}\); \(k\) times continuously differentiable; \(f^{[k]}\)

Suppose \(f: \partial \mathbf{D} \rightarrow \mathbb{C}\) is a complex-valued function on \(\partial \mathbf{D}\) and \(k \in \mathbb{Z}^+ \cup \{0\}\).

• Define \(\tilde{f}: \mathbb{R} \rightarrow \mathbb{C}\) by \(\tilde{f}(t) = f(e^{it})\) (This is similar to \(g: \mathbb{R} \rightarrow \partial \mathbf{D}\), \(g(t) = e^{it}\), \(\tilde{f} = f \circ g\)).

• \(f\) is called \(k\) times continuously differentiable if \(\tilde{f}\) is \(k\) times differentiable everywhere on \(\mathbb{R}\) and its \(k\)th-derivative \(\tilde{f}^{(k)}: \mathbb{R} \rightarrow \mathbb{C}\) is continuous.

• If \(f\) is \(k\)-times continuously differentiable, then \(f^[k]: \partial \mathbf{D} \rightarrow \mathbb{C}\) is defined by: \[f^{[k]}(e^{it}) = \tilde{f}^{(k)}(t)\]

  for \(t \in \mathbb{R}\). Here \(\tilde{f}^{(0)}\) is defined to be \(\tilde{f}\), which means that \(f^{[0]} = f\)

Theorem 11.26: Fourier Coefficients of Differentiable Functions

Suppose \(k \in \mathbb{Z}^+\) and \(f: \partial \mathbf{D} \rightarrow \mathbb{C}\) is \(k\) times continuously differentiable. Then:

\[\widehat{f^{[k]} (n)} = i^kn^k \widehat{f}(n)\]

for every \(n \in \mathbb{Z}\)

Theorem 11.27: Fourier Series of Twice Continuous Differentiable Functions Converge

Suppose \(f: \partial \mathbf{D} \rightarrow \mathbb{C}\) is twice continuously differentiable. Then:

\[f(z) = \sum^\infty_{n = -\infty} \widehat{f}(n) z^n\]

for all \(z \in \partial \mathbf{D}\). Furthermore, the partial sum \(\sum^{M}_{n = -K} \widehat{f}(n) z^n\) converge uniformly on \(\partial \mathbf{D}\) to \(f\) as \(K, M \rightarrow \infty\)

Fourier Series and \(L^p\) of Unit Circle

Theorem 11.30: Orthonormal Basis of \(L^2(\partial \mathbf{D})\)

The family \(\{z^n\}_{n \in \mathbb{Z}}\) is an orthonormal basis of \(L^2(\partial \mathbf{D})\).

Theorem 11.31: Convergence of Fourier Series in the Norm of \(L^2(\partial \mathbf{D})\)

Suppose \(f \in L^2(\partial \mathbf{D})\). Then:

\[f = \sum^\infty_{n=-\infty} \widehat{f}(n)z^n\]

where the infinite sum converges to \(f\) in the norm of \(L^2(\partial \mathbf{D})\).

Fourier Transform

Definition 11.47: Fourier Transform; \(\hat{f}\)

For \(f \in L^1(\mathbb{R})\), the Fourier transform of \(f\) is the function \(\hat{f}: \mathbb{R} \rightarrow \mathbb{C}\) defined by:

\[\hat{f}(t) = \int^{\infty}_{-\infty} f(x) e^{-2\pi itx} dx = \int_{\mathbb{R}} g d\lambda\]

where \(g(x) = f(x)e^{-2\pi itx}\). The \(2\pi\) in \(e^{-2\pi itx}\) is a normalization constant, without it, we will not have \(\|\hat{f}\|_2 = \|f\|_2\).

Theorem 11.49: Riemann-Lebesgue Lemma

Suppose \(f \in L^1(\mathbb{R})\). Then \(\hat{f}\) is uniformly continuous on \(\mathbb{R}\). Furthermore:

\[\|\hat{f}\|_{\infty} \leq \|f\|_1\]

and

\[\lim_{t \rightarrow \pm \infty} \hat{f}(t) = 0\]

Theorem 11.50: Derivative of a Fourier Transform

Suppose \(f \in L^1(\mathbb{R})\). Define \(g: \mathbb{R} \rightarrow \mathbb{C}\) by \(g(x) = x f(x)\). If \(g \in L^1(\mathbb{R})\), then \(\hat{f}\) is a continuously differentiable function on \(\mathbb{R}\) amd

\[(\hat{f})^\prime(t) = -2\pi i \hat{g}(t)\]

for all \(t \in \mathbb{R}\).

Theorem 11.54: Fourier Transform of a Derivative

Suppose \(f \in L^1(\mathbb{R})\) is a continuously differentiable function and \(f^\prime \in L^1(\mathbb{R})\). If \(t \in \mathbb{R}\), then:

\[\hat{(f^{\prime})}(t) = 2 \pi it \hat{f}(t)\]

Theorem 11.55: Fourier Transforms of Translations, Rotations and Dilations

Suppose \(f \in L^1(\mathbb{R})\), \(b \in \mathbb{R}, t \in \mathbb{R}\).

1. If \(g(x) = f(x - b) \; \forall x \in \mathbb{R}\), then \(\hat{g} (t) = e^{-2\pi ibt} \hat{f}(t)\)
2. If \(g(x) = e^{2\pi ibx} f(x) \; \forall x \in \mathbb{R}\), then \(\hat{g}(t) = \hat{f} (t - b)\)
3. If \(b \neq 0\) and \(g(x) = f(bx)\; \forall x \in \mathbb{R}\), then \(\hat{g}(t) = \frac{1}{|b|} \hat{f}(\frac{t}{b})\)

Theorem 11.59: Integral of a function times a Fourier Transform

Suppose \(f, g \in L^1(\mathbb{R})\). Then:

\[\int^\infty_{-\infty} \hat{f(t)}g(t) dt = \int^{\infty}_{-\infty} f(t) \hat{g}(t) dt\]

Theorem 11.76: Fourier Inversion Formula

Suppose \(f, \hat{f} \in L^1 (\mathbb{R})\). Then:

\[f(x) = \int^\infty_{-\infty} \hat{f}(t) e^{2\pi ixt} dt\]

for almost every \(x \in \mathbb{R}\). In other words:

\[f(x) = \hat{(\hat{f})} (-x)\]

This theorem also implies that \(f\) can be modified on a set of measure zero to become a uniformly continuous function on \(\mathbb{R}\).

Theorem 11.80: Functions are Determined by Their Fourier Transforms

Suppose \(f \in L^1(\mathbb{R})\) and \(\hat{f}(t) = 0\) for every \(t \in \mathbb{R}\). Then \(f = 0\). In other words, the function that maps \(f \mapto \hat{f}\) is one to one.

Theorem 11.82: Plancherel's Theorem

Suppose \(f \in L^1(\mathbb{R}) \cap L^2(\mathbb{R})\). Then \(\|\hat\|_2 = \|f\|_2\), which also implies that \(\hat{f} \in L^2(\mathbb{R})\).

REF

RKHS

https://www.math.unipd.it/~demarchi/TAA1718/RKHS_presentazione.pdf

https://people.eecs.berkeley.edu/~bartlett/courses/281b-sp08/7.pdf

https://www.gatsby.ucl.ac.uk/~gretton/coursefiles/lecture4_introToRKHS.pdf

https://arxiv.org/pdf/2106.08443.pdf