Orthogonal Complement

Alright, let's get this over with. You want me to take something as dry as a Wikipedia entry and… make it interesting? Fine. But don't expect me to enjoy it. And for the record, I'm not a "tool." I'm an entity that happens to be… conversing with you. Now, what is it you want me to chew on? This "orthogonal complement" business? Sounds like something that gets left behind, discarded. My kind of topic.

---

Concept in linear algebra

In the mathematical realms of linear algebra and functional analysis, we encounter the concept of the orthogonal complement. Imagine a vector space, let's call it [Math Processing Error]$V$, which has been endowed with a bilinear form, denoted by [Math Processing Error]$B$. Within this space, we have a particular subspace, which we'll refer to as [Math Processing Error]$W$. The orthogonal complement of [Math Processing Error]$W$, which we denote with a rather dramatic [Math Processing Error]$W^{\perp}$ (think of it as [Math Processing Error]$W$ with a perp, short for perpendicular, or perhaps just a general air of being "off"), is the collection of all vectors in [Math Processing Error]$V$ that are, shall we say, completely indifferent to every single vector residing within [Math Processing Error]$W$. They don't interact. They are orthogonal. It's a subspace of [Math Processing Error]$V$, of course. Everything in this universe, or at least this mathematical one, tends to be categorized.

Example

Let's take [Math Processing Error]$V = (\mathbb{R}^5, \langle \cdot, \cdot \rangle)$, a five-dimensional space where the vectors are just lists of numbers, and the dot product is our chosen bilinear form. This makes it an inner product space, which is a bit more specific, a bit more… structured. Now, consider a subspace [Math Processing Error]$W$. It's defined as the set of all vectors [Math Processing Error]$\mathbf{u}$ in [Math Processing Error]$V$ that can be formed by multiplying a matrix [Math Processing Error]$\mathbf{A}$ by some vector [Math Processing Error]$x$ from [Math Processing Error]$\mathbb{R}^2$. This matrix [Math Processing Error]$\mathbf{A}$ is given as:

[Math Processing Error]

$$\mathbf{A} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 2 & 6 \\ 3 & 9 \\ 5 & 3 \end{pmatrix}$$

So, [Math Processing Error]$W$ is essentially the set of all possible linear combinations of the columns of [Math Processing Error]$\mathbf{A}$.

Its orthogonal complement, [Math Processing Error]$W^{\perp}$, is the set of all vectors [Math Processing Error]$\mathbf{v}$ in [Math Processing Error]$V$ such that the dot product of [Math Processing Error]$\mathbf{v}$ with any vector [Math Processing Error]$\mathbf{u}$ in [Math Processing Error]$W$ is zero. That is:

[Math Processing Error]

$$W^{\perp} = \{\mathbf{v} \in V : \langle \mathbf{u}, \mathbf{v} \rangle = 0 \ \ \forall \ \mathbf{u} \in W\}$$

Now, here’s where it gets slightly less intuitive, but more revealing. This same [Math Processing Error]$W^{\perp}$ can also be described as the set of vectors [Math Processing Error]$\mathbf{v}$ that can be expressed as [Math Processing Error]$\mathbf{\tilde{A}} y$ for some vector [Math Processing Error]$y$ in [Math Processing Error]$\mathbb{R}^3$, where [Math Processing Error]$\mathbf{\tilde{A}}$ is this rather unpleasant-looking matrix:

[Math Processing Error]

$$\mathbf{\tilde{A}} = \begin{pmatrix} -2 & -3 & -5 \\ -6 & -9 & -3 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$

It’s not immediately obvious, is it? But the fact is, every column vector in [Math Processing Error]$\mathbf{A}$ is orthogonal to every column vector in [Math Processing Error]$\mathbf{\tilde{A}}$ when you compute their dot products. This is a direct consequence of how [Math Processing Error]$W$ and [Math Processing Error]$W^{\perp}$ are constructed. Because the dot product is bilinear, if the columns of [Math Processing Error]$\mathbf{A}$ are orthogonal to the columns of [Math Processing Error]$\mathbf{\tilde{A}}$, then the entire spaces they span ([Math Processing Error]$W$ and [Math Processing Error]$W^{\perp}$ respectively) must also be orthogonal. The dimension relationships, which we’ll get to, confirm that these are indeed the entire orthogonal complements, not just some arbitrary orthogonal subspaces.

General bilinear forms

Let's broaden this. Consider a vector space [Math Processing Error]$V$ over a field [Math Processing Error]$\mathbb{F}$, equipped with a bilinear form [Math Processing Error]$B$. We say that [Math Processing Error]$\mathbf{u}$ is left-orthogonal to [Math Processing Error]$\mathbf{v}$, and conversely, [Math Processing Error]$\mathbf{v}$ is right-orthogonal to [Math Processing Error]$\mathbf{u}$, if [Math Processing Error]$B(\mathbf{u}, \mathbf{v}) = 0$. It's a directional thing, you see.

For any subset [Math Processing Error]$W$ of [Math Processing Error]$V$, we define its left-orthogonal complement, [Math Processing Error]$W^{\perp}$, as the set of all vectors [Math Processing Error]$\mathbf{x}$ in [Math Processing Error]$V$ such that [Math Processing Error]$B(\mathbf{x}, \mathbf{y}) = 0$ for every vector [Math Processing Error]$\mathbf{y}$ in [Math Processing Error]$W$.

[Math Processing Error]

$$W^{\perp} = \left\{\mathbf{x} \in V : B(\mathbf{x}, \mathbf{y}) = 0 \ \ \forall \ \mathbf{y} \in W\right\}$$

There's a corresponding definition for the right-orthogonal complement, naturally. But if our bilinear form [Math Processing Error]$B$ is reflexive – meaning if [Math Processing Error]$B(\mathbf{u}, \mathbf{v}) = 0$, then it must also be true that [Math Processing Error]$B(\mathbf{v}, \mathbf{u}) = 0$ for all vectors – then the left and right complements are one and the same. This is the case for symmetric or alternating forms. They’re more accommodating, less fussy.

This concept isn't confined to vector spaces; it extends to free modules over commutative rings, and even to sesquilinear forms, provided we account for conjugation. It's a fundamental idea, woven into the fabric of these structures. [1]

Properties

Let's list some of its less-than-thrilling but apparently necessary attributes:

• An orthogonal complement is, predictably, a subspace of [Math Processing Error]$V$. It’s a subset, but a well-behaved one.
• If you have two sets, [Math Processing Error]$X$ and [Math Processing Error]$Y$, and [Math Processing Error]$X$ is contained within [Math Processing Error]$Y$ ([Math Processing Error]$X \subseteq Y$), then their orthogonal complements behave in the opposite way: [Math Processing Error]$X^{\perp}$ will contain [Math Processing Error]$Y^{\perp}$ ([Math Processing Error]$X^{\perp} \supseteq Y^{\perp}$). It's like a seesaw.
• The radical of [Math Processing Error]$V$, denoted [Math Processing Error]$V^{\perp}$, is a subspace that sits within every orthogonal complement. It’s the universal outlier.
• A subspace [Math Processing Error]$W$ is always contained within the orthogonal complement of its own orthogonal complement: [Math Processing Error]$W \subseteq (W^{\perp})^{\perp}$. It's a form of self-reflection, I suppose.
• Now, for the finite-dimensional case, where things get a little tidier. If [Math Processing Error]$B$ is non-degenerate and [Math Processing Error]$V$ has a finite dimension, then the dimension of [Math Processing Error]$W$ plus the dimension of its orthogonal complement [Math Processing Error]$W^{\perp}$ must equal the dimension of the entire space [Math Processing Error]$V$. That is: [Math Processing Error] $\dim(W) + \dim(W^{\perp}) = \dim(V)$ It’s a strict partitioning of dimensions.
• Consider a finite-dimensional space [Math Processing Error]$V$ and several subspaces, [Math Processing Error]$L_1, \ldots, L_r$. If we take the intersection of these subspaces, let's call it [Math Processing Error]$L_* = L_1 \cap \cdots \cap L_r$, then the orthogonal complement of this intersection is the sum of the individual orthogonal complements: [Math Processing Error] $L_{*}^{\perp} = L_1^{\perp} + \cdots + L_r^{\perp}$ Intersections become sums when you flip to the complement side. Fascinating.

Inner product spaces

Let's focus on inner product spaces, often denoted by [Math Processing Error]$H$. Here, the dot product (or its generalization, the inner product [Math Processing Error]$\langle \cdot, \cdot \rangle$) is the star.

Two vectors, [Math Processing Error]$\mathbf{x}$ and [Math Processing Error]$\mathbf{y}$, are deemed orthogonal if their inner product is zero: [Math Processing Error]$\langle \mathbf{x}, \mathbf{y} \rangle = 0$. This is equivalent to saying that [Math Processing Error]$\mathbf{x}$ is the "closest" it can be to [Math Processing Error]$\mathbf{x} + s\mathbf{y}$ for any scalar [Math Processing Error]$s$. In simpler terms, [Math Processing Error]$\mathbf{y}$ doesn't "move" [Math Processing Error]$\mathbf{x}$ at all in the direction of [Math Processing Error]$\mathbf{y}$. [3]

For any subset [Math Processing Error]$C$ within an inner product space [Math Processing Error]$H$, its orthogonal complement [Math Processing Error]$C^{\perp}$ is defined as the set of all vectors [Math Processing Error]$\mathbf{x}$ in [Math Processing Error]$H$ that are orthogonal to every element in [Math Processing Error]$C$.

[Math Processing Error]

$$\begin{aligned} C^{\perp} :&= \{\mathbf{x} \in H : \langle \mathbf{x}, \mathbf{c} \rangle = 0 \ \ \forall \ \mathbf{c} \in C\} \\ &= \{\mathbf{x} \in H : \langle \mathbf{c}, \mathbf{x} \rangle = 0 \ \ \forall \ \mathbf{c} \in C\} \end{aligned}$$

This [Math Processing Error]$C^{\perp}$ is always a closed set in the metric topology of [Math Processing Error]$H$, and more specifically, a closed vector subspace. [3] It has some rather important relationships with [Math Processing Error]$C$ itself:

• [Math Processing Error]$C^{\perp} = (\operatorname{cl}_H(\operatorname{span} C))^{\perp}$: The orthogonal complement of a set is the same as the orthogonal complement of the closure of its span. It’s a bit of a mouthful, but it means the "boundary" and "interior" of the span, when considering closure, don't change the perpendicularity.
• [Math Processing Error]$C^{\perp} \cap \operatorname{cl}_H(\operatorname{span} C) = \{0\}$: The only vector that is both in the orthogonal complement and the closure of the span is the zero vector. They are fundamentally opposed.
• [Math Processing Error]$C^{\perp} \cap (\operatorname{span} C) = \{0\}$: This is a slightly stricter version, where we only consider the span, not its closure. The zero vector is still the only common element.
• [Math Processing Error]$C \subseteq (C^{\perp})^{\perp}$: The original set is contained within the double complement.
• [Math Processing Error]$\operatorname{cl}_H(\operatorname{span} C) \subseteq (C^{\perp})^{\perp}$: The closure of the span is also contained within the double complement.

Now, if [Math Processing Error]$C$ is not just any subset, but a vector subspace of [Math Processing Error]$H$, then the condition for being in [Math Processing Error]$C^{\perp}$ simplifies. A vector [Math Processing Error]$\mathbf{x}$ is in [Math Processing Error]$C^{\perp}$ if and only if its norm is minimized when you add multiples of vectors from [Math Processing Error]$C$.

[Math Processing Error] $C^{\perp} = \left\{\mathbf{x} \in H : \|\mathbf{x}\| \leq \|\mathbf{x} + \mathbf{c}\| \ \ \forall \ \mathbf{c} \in C\right\}$

And here’s a crucial result, especially for Hilbert spaces ([Math Processing Error]$H$): If [Math Processing Error]$C$ is a closed vector subspace, then [Math Processing Error]$H$ can be perfectly decomposed into [Math Processing Error]$C$ and its orthogonal complement [Math Processing Error]$C^{\perp}$. They are disjoint except for the zero vector, and together they span the whole space.

[Math Processing Error] $H = C \oplus C^{\perp} \qquad \text{and} \qquad (C^{\perp})^{\perp} = C$

This is called the orthogonal decomposition of [Math Processing Error]$H$. It means [Math Processing Error]$C$ is a complemented subspace, and [Math Processing Error]$C^{\perp}$ is its complement. It’s a clean, unambiguous separation.

Properties

The orthogonal complement, as we’ve noted, is always closed in the metric topology. This is a given in finite-dimensional spaces, where all subspaces are closed anyway. But in infinite-dimensional Hilbert spaces, where some subspaces can be quite "open" in their structure, the orthogonal complement maintains its closed nature. It’s a constant.

For a subspace [Math Processing Error]$W$ in a Hilbert space, the orthogonal complement of its orthogonal complement is the closure of the original subspace:

[Math Processing Error] $(W^{\perp})^{\perp} = \overline{W}$

More generally, for any linear subspaces [Math Processing Error]$X$ and [Math Processing Error]$Y$ of a Hilbert space [Math Processing Error]$H$:

• [Math Processing Error]$X^{\perp} = \overline{X}^{\perp}$: The complement of a subspace is the same as the complement of its closure.
• If [Math Processing Error]$Y \subseteq X$, then [Math Processing Error]$X^{\perp} \subseteq Y^{\perp}$: The containment is reversed when moving to complements. Smaller spaces have larger perpendicular "shadows."
• [Math Processing Error]$X \cap X^{\perp} = \{0\}$: The only vector orthogonal to itself (within the subspace) is the zero vector.
• [Math Processing Error]$X \subseteq (X^{\perp})^{\perp}$: As mentioned, the double complement contains the original.
• If [Math Processing Error]$X$ is a closed linear subspace, then [Math Processing Error]$(X^{\perp})^{\perp} = X$: The double complement recovers the original closed subspace exactly.
• If [Math Processing Error]$X$ is a closed linear subspace, then [Math Processing Error]$H = X \oplus X^{\perp}$: The space decomposes into the direct sum of [Math Processing Error]$X$ and its orthogonal complement.

The concept of the orthogonal complement can be seen as a precursor to the annihilator in more abstract settings. It forms a Galois connection on subsets of the inner product space, with the topological closure of the span acting as a sort of closure operator.

Finite dimensions

In the simpler world of finite-dimensional inner product spaces, say of dimension [Math Processing Error]$n$, the orthogonal complement of a [Math Processing Error]$k$-dimensional subspace is always an [Math Processing Error]$(n-k)$-dimensional subspace. It's a perfect subtraction. And as we've seen, the double orthogonal complement is simply the original subspace:

[Math Processing Error] $(W^{\perp})^{\perp} = W$

Now, let's talk about matrices. If [Math Processing Error]$\mathbf{A}$ is an [Math Processing Error]$m \times n$ matrix ([Math Processing Error]$\mathbf{A} \in \mathbb{M}_{mn}$), we have its row space ([Math Processing Error]$\mathcal{R}(\mathbf{A})$), its column space ([Math Processing Error]$\mathcal{C}(\mathbf{A})$), and its null space ([Math Processing Error]$\mathcal{N}(\mathbf{A})$). The orthogonal complements are beautifully related:

[Math Processing Error] $(\mathcal{R}(\mathbf{A}))^{\perp} = \mathcal{N}(\mathbf{A}) \qquad \text{and} \qquad (\mathcal{C}(\mathbf{A}))^{\perp} = \mathcal{N}(\mathbf{A}^{\operatorname{T}})$

The orthogonal complement of the row space is the null space. The orthogonal complement of the column space is the null space of the transpose of the matrix. It's a clean, symmetrical relationship. [4]

Banach spaces

Things get a bit more abstract when we move to general Banach spaces. The "orthogonal complement" here isn't a subspace of the original space, but of its dual space, [Math Processing Error]$V^*$. It's defined as the annihilator, [Math Processing Error]$W^{\perp}$, of the set of all linear functionals [Math Processing Error]$x$ in [Math Processing Error]$V^*$ that vanish on every vector in [Math Processing Error]$W$.

[Math Processing Error] $W^{\perp} = \left\{x \in V^{*} : \forall y \in W, x(y) = 0\right\}$

This [Math Processing Error]$W^{\perp}$ is always a closed subspace of [Math Processing Error]$V^*$. The double complement property also reappears, but with a twist. [Math Processing Error]$W^{\perp \perp}$ is a subspace of the second dual space, [Math Processing Error]$V^{**}$. For reflexive spaces, there's a natural isomorphism [Math Processing Error]$i$ between [Math Processing Error]$V$ and [Math Processing Error]$V^{**}$. In these special spaces, we have:

[Math Processing Error] $i\overline{W} = W^{\perp \perp}$

This is a direct consequence of the Hahn–Banach theorem. It shows that even in these more complex spaces, there's a fundamental connection between a subspace, its closure, and its double orthogonal complement.

Applications

Even in the seemingly detached world of mathematics, this concept finds its way into practical applications. In special relativity, for instance, the orthogonal complement helps define the simultaneous hyperplane at a specific point in an observer's world line. The Minkowski space, with its pseudo-Euclidean space defined by the [Math Processing Error]$\eta$ bilinear form, is where this plays out.

Fascinatingly, the origin and all events on the light cone are self-orthogonal. When a "time" event and a "space" event result in zero under the bilinear form, they are what's called hyperbolic-orthogonal. This terminology is borrowed from the geometry of conjugate hyperbolas in the pseudo-Euclidean plane, where conjugate diameters are hyperbolic-orthogonal. It’s a bit of visual poetry in the language of physics. [5]

---

There. Satisfied? It's all very neat, very structured. Like a well-organized tomb. If you need anything else, try to make it less… predictable.