F

Our next result discloses the connection between reproducing-kernel Hilbert spaces and

positive de¬nite kernels.

Theorem 10.4 Suppose that F is a reproducing-kernel Hilbert function space with re-

producing kernel : — ’ R. Then is positive semi-de¬nite. Moreover, is pos-

itive de¬nite if and only if the point evaluation functionals are linearly independent

in F — .

Proof Since the kernel is real-valued and symmetric, we can restrict ourselves to real

coef¬cients in the quadratic form. For pairwise distinct x1 , . . . , x N and ± ∈ R N \ {0} we

have

2

N N N N N

± j ±k (x j , xk ) = ± j δx j , ±k δ x k = ± j δx j ≥ 0.

j=1 k=1 j=1 k=1 j=1

F— F—

The last expression can and will only be zero if the point evaluation functionals are linearly

dependent.

Hence, the reproducing kernel of a function space F leads to a real-valued positive semi-

de¬nite kernel. Obviously, if the function space F is a complex vector space containing

complex-valued functions then everything said so far remains true with mild modi¬ca-

tions. In particular the reproducing-kernel is now a complex-valued positive semi-de¬nite

function.

From the ¬rst property of De¬nition 10.1 we know that F contains all functions of the

form f = N ± j (·, x j ) if x j ∈ . Furthermore, we know that

j=1

N N N N

= ± j ±k ( (·, x j ), (·, xk ))F = ± j ±k (x j , xk ).

2

f F

j=1 k=1 j=1 k=1

This feature will be used to construct a reproducing-kernel Hilbert space for a given positive

de¬nite kernel in the next section.

136 Native spaces

But before this we want to have a look at how invariance properties of the space in¬‚uence

the kernel.

De¬nition 10.5 Let T be a group of transformations T : ’ . We say F is invariant

under the group T if

(1) f —¦ T ∈ F for all f ∈ F and T ∈ T ,

(2) ( f —¦ T, g —¦ T )F = ( f, g)F for all f, g ∈ F and all T ∈ T .

The invariance of the function space is inherited by the kernel.

Theorem 10.6 Suppose that the reproducing-kernel Hilbert function space F is invariant

under the transformations of T ; then the reproducing kernel satis¬es

(T x, T y) = (x, y)

for all x, y ∈ and all T ∈ T .

Proof From the reproducing-kernel properties and the invariance properties we can read

off

f (y) = f —¦ T ’1 (T y) = ( f —¦ T ’1 , (·, T y))F = ( f, (T ·, T y))F , y∈ .

Of course, we also have f (y) = ( f, (·, y))F , and hence the uniqueness of the reproducing

kernel gives (·, y) = (T ·, T y) for all y ∈ .

The following examples show that radial basis functions arise quite naturally within the

concept of reproducing kernels. For our ¬rst example suppose that = Rd . Let T be the

group of translations on Rd . If we choose the translation T ξ = x ’ ξ for a ¬xed x ∈ Rd

then Theorem 10.6 shows that

(x, y) = (0, x ’ y) =: ’ y),

0 (x

i.e. the kernel is translation invariant.

For our second example suppose that is still Rd but T consists now of the translations

and the orthogonal transformations. Then we can choose an orthogonal transformation

A ∈ Rd—d such that Aξ = ξ 2 e1 , where e1 is the ¬rst unit vector in Rd and ξ = x ’ y.

Hence

(x, y) = (Ax, Ay) = ’ y)) = x ’ y 2 e1 ) =: φ( x ’ y 2 ),

0 (A(x 0(

i.e. is radial.

10.2 Native spaces for positive de¬nite kernels

So far, we have seen that a positive de¬nite function or kernel appears naturally as the

reproducing kernel of a Hilbert function space. But since we normally do not start with

a function space but with a positive de¬nite kernel we are confronted by the problem of

10.2 Native spaces for positive de¬nite kernels 137

¬nding the associated function space that has this kernel as its reproducing kernel. In this

section we want to solve this problem by constructing the corresponding Hilbert function

space. Hence, assume that : — ’ R is a symmetric positive de¬nite kernel.

Motivated by the second remark after the proof of Theorem 10.4 we de¬ne the R-linear

space

F ( ) := span{ (·, y) : y ∈ }

and equip this space with the bilinear form

N M N M

± j (·, x j ), βk (·, yk ) ± j βk (x j , yk ).

:=

j=1 k=1 j=1 k=1

Theorem 10.7 If : — ’ R is a symmetric positive de¬nite kernel then (·, ·) de¬nes

an inner product on F ( ). Furthermore, F ( ) is a pre-Hilbert space with reproducing

kernel .

Proof Obviously (·, ·) is bilinear and symmetric because is symmetric. Moreover, if

we choose an arbitrary function f = N ± j (·, x j ) ≡ 0 from F ( ) we ¬nd that

j=1

N N

( f, f ) = ± j ±k (x j , xk ) > 0,

j=1 k=1

because is positive de¬nite. Finally, we obtain for this f

N

( f, (·, y)) = ± j (x j , y) = f (y),

j=1

which establishes also the reproducing kernel.

The completion F ( ) of this pre-Hilbert space with respect to the · -norm is the

¬rst candidate for a Hilbert function space with reproducing kernel . But the elements of

the completion are abstract elements and we have to interpret them as functions. Since the

point-evaluation functionals are continuous on F ( ), their extensions to the completion

remain continuous. This can be used to de¬ne function values for elements of the completion.

To represent this connection we could use the sloppy notation

f (x) := ( f, (·, x))

for every f ∈ F ( ). But we want to repeat these arguments in the more technical situation

of conditionally positive de¬nite kernels and will therefore be more precise. Thus we de¬ne

the linear mapping

R : F ( ) ’ C( ), R( f )(x) := ( f, (·, x)) .

The resulting functions are indeed continuous because

|R f (x) ’ R f (y)| = ( f, (·, x) ’ (·, y)) ¤ f (·, x) ’ (·, y)

138 Native spaces