thank you very much for this article. You mentioned a third part in which you discuss the implementation of PCR. Is it correct that the third part is missing?

Greetings,
Willem

1. Look at the two dimension case. Consider the image of the unit sphere S={v:‖v‖≤1} under X. Since S is compact and X is continuous there exists a vector v_1 such that ‖Xv_1 ‖ is maximal. That is

‖Xv_1 ‖≥‖Xv‖ ∀ v∈S

Set 〖σu〗_1=Xv_1, where u_1 is a unit vector so that ‖Xv_1 ‖=σ. We will show that for any vector v_2 orthogonal to v_1its image u_2=Xv_2is orthogonal to u_1.

Choose a unit vector u_2 orthogonal to u_1 (ala Gram-Schmidt, for example), and write

Xv_2=αu_1+βu_2

We will show that α=0. Set

v=(σv_1+αv_2)/(σ^2+α^2 )^(1⁄2)

Then

Xv=(σ^2 u_1+α^2 u_1+αβu_2)/(σ^2+α^2 )^(1⁄2) =((σ^2+α^2 ) u_1+αβu_2)/(σ^2+α^2 )^(1⁄2)

Taking the norm of both sides and using the fact that u_1 and u_2 are orthonormal vectors* yields

‖Xv‖=(‖(σ^2+α^2 ) u_1 ‖^2+‖αβu_2 ‖^2 )^(1⁄2)/(σ^2+α^2 )^(1⁄2) =((σ^2+α^2 )^2+(αβ)^2 )^(1⁄2)/(σ^2+α^2 )^(1⁄2) ≥(σ^2+α^2)/(σ^2+α^2 )^(1⁄2)

So that
〖‖Xv‖≥(σ^2+α^2 )〗^(1⁄2)≥σ

Since ‖Xv‖ cannot be strictly greater than σ, equality must hold which means α=0.

*For orthogonal vectors u and v we have:

‖u+v‖^2=(u+v)∙(u+v)=u∙u+u∙v+v∙u+v∙v=‖u‖^2+‖v‖^2

2. The terms “left” and “right” do seem a bit awkward. The come from the positions of the factors U and V in the decomposition X=UΣV^T.

two questions:

1.In the section ‘a geometric fact’ you say ‘…the columns of V are orthonormal by construction’.
It may be so, but it is not obvious, since V columns are defined ‘the pre-images of the principal semiaxes of XS’.

2. In the same section you define ‘u’ as left singular vectors and ‘v’ right singular vectors. Shouldn’t it be the other way around?

Regards,
Eran.

should be
XV = U \Sigma