[{"@context":"http:\/\/schema.org\/","@type":"BlogPosting","@id":"https:\/\/wiki.edu.vn\/en\/wiki43\/projection-linear-algebra-wikipedia\/#BlogPosting","mainEntityOfPage":"https:\/\/wiki.edu.vn\/en\/wiki43\/projection-linear-algebra-wikipedia\/","headline":"Projection (linear algebra) – Wikipedia","name":"Projection (linear algebra) – Wikipedia","description":"Idempotent linear transformation from a vector space to itself “Orthogonal projection” redirects here. For the technical drawing concept, see Orthographic","datePublished":"2022-05-06","dateModified":"2022-05-06","author":{"@type":"Person","@id":"https:\/\/wiki.edu.vn\/en\/wiki43\/author\/lordneo\/#Person","name":"lordneo","url":"https:\/\/wiki.edu.vn\/en\/wiki43\/author\/lordneo\/","image":{"@type":"ImageObject","@id":"https:\/\/secure.gravatar.com\/avatar\/c9645c498c9701c88b89b8537773dd7c?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c9645c498c9701c88b89b8537773dd7c?s=96&d=mm&r=g","height":96,"width":96}},"publisher":{"@type":"Organization","name":"Enzyklop\u00e4die","logo":{"@type":"ImageObject","@id":"https:\/\/wiki.edu.vn\/wiki4\/wp-content\/uploads\/2023\/08\/download.jpg","url":"https:\/\/wiki.edu.vn\/wiki4\/wp-content\/uploads\/2023\/08\/download.jpg","width":600,"height":60}},"image":{"@type":"ImageObject","@id":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/thumb\/e\/eb\/Orthogonal_projection.svg\/252px-Orthogonal_projection.svg.png","url":"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/thumb\/e\/eb\/Orthogonal_projection.svg\/252px-Orthogonal_projection.svg.png","height":"252","width":"252"},"url":"https:\/\/wiki.edu.vn\/en\/wiki43\/projection-linear-algebra-wikipedia\/","wordCount":45967,"articleBody":"Idempotent linear transformation from a vector space to itself“Orthogonal projection” redirects here. For the technical drawing concept, see Orthographic projection. For a concrete discussion of orthogonal projections in finite-dimensional linear spaces, see Vector projection.  The transformation P is the orthogonal projection onto the line m.In linear algebra and functional analysis, a projection is a linear transformation P{displaystyle P} from a vector space to itself (an endomorphism) such that P\u2218P=P{displaystyle Pcirc P=P}. That is, whenever P{displaystyle P} is applied twice to any vector, it gives the same result as if it were applied once (i.e. P{displaystyle P} is idempotent). It leaves its image unchanged.[1] This definition of “projection” formalizes and generalizes the idea of graphical projection.  One can also consider the effect of a projection on a geometrical object by examining the effect of the projection on points in the object.Table of ContentsDefinitions[edit]Projection matrix[edit]Examples[edit]Orthogonal projection[edit]Oblique projection[edit]Properties and classification[edit]Idempotence[edit]Open map[edit]Complementarity of image and kernel[edit]Spectrum[edit]Product of projections[edit]Orthogonal projections[edit]Properties and special cases[edit]Formulas[edit]Oblique projections[edit]A matrix representation formula for a nonzero projection operator[edit]Singular values[edit]Finding projection with an inner product[edit]Canonical forms[edit]Projections on normed vector spaces[edit]Applications and further considerations[edit]Generalizations[edit]See also[edit]References[edit]External links[edit]Definitions[edit]A projection on a vector space V{displaystyle V} is a linear operator P:V\u2192V{displaystyle P:Vto V} such that P2=P{displaystyle P^{2}=P}.When V{displaystyle V} has an inner product and is complete (i.e. when V{displaystyle V} is a Hilbert space) the concept of orthogonality can be used. A projection P{displaystyle P} on a Hilbert space V{displaystyle V} is called an orthogonal projection if it satisfies \u27e8Px,y\u27e9=\u27e8x,Py\u27e9{displaystyle langle Pmathbf {x} ,mathbf {y} rangle =langle mathbf {x} ,Pmathbf {y} rangle } for all x,y\u2208V{displaystyle mathbf {x} ,mathbf {y} in V}. A projection on a Hilbert space that is not orthogonal is called an oblique projection.Projection matrix[edit]The eigenvalues of a projection matrix must be 0 or 1.Examples[edit]Orthogonal projection[edit]For example, the function which maps the point (x,y,z){displaystyle (x,y,z)} in three-dimensional space R3{displaystyle mathbb {R} ^{3}} to the point (x,y,0){displaystyle (x,y,0)} is an orthogonal projection onto the xy-plane. This function is represented by the matrixP=[100010000].{displaystyle P={begin{bmatrix}1&0&0\\0&1&0\\0&0&0end{bmatrix}}.}The action of this matrix on an arbitrary vector isP[xyz]=[xy0].{displaystyle P{begin{bmatrix}x\\y\\zend{bmatrix}}={begin{bmatrix}x\\y\\0end{bmatrix}}.}To see that P{displaystyle P} is indeed a projection, i.e., P=P2{displaystyle P=P^{2}}, we computeP2[xyz]=P[xy0]=[xy0]=P[xyz].{displaystyle P^{2}{begin{bmatrix}x\\y\\zend{bmatrix}}=P{begin{bmatrix}x\\y\\0end{bmatrix}}={begin{bmatrix}x\\y\\0end{bmatrix}}=P{begin{bmatrix}x\\y\\zend{bmatrix}}.}Observing that PT=P{displaystyle P^{mathrm {T} }=P} shows that the projection is an orthogonal projection.Oblique projection[edit]A simple example of a non-orthogonal (oblique) projection isP=[00\u03b11].{displaystyle P={begin{bmatrix}0&0\\alpha &1end{bmatrix}}.}Via matrix multiplication, one sees thatP2=[00\u03b11][00\u03b11]=[00\u03b11]=P.{displaystyle P^{2}={begin{bmatrix}0&0\\alpha &1end{bmatrix}}{begin{bmatrix}0&0\\alpha &1end{bmatrix}}={begin{bmatrix}0&0\\alpha &1end{bmatrix}}=P.}showing that P{displaystyle P} is indeed a projection.The projection P{displaystyle P} is orthogonal if and only if \u03b1=0{displaystyle alpha =0} because only then PT=P.{displaystyle P^{mathrm {T} }=P.}Properties and classification[edit]  The transformation T is the projection along k onto m. The range of T is m and the null space is k.Idempotence[edit]By definition, a projection P{displaystyle P} is idempotent (i.e. P2=P{displaystyle P^{2}=P}).Open map[edit]Every projection is an open map, meaning that it maps each open set in the domain to an open set in the subspace topology of the image.[citation needed]  That is, for any vector x{displaystyle mathbf {x} } and any ball Bx{displaystyle B_{mathbf {x} }} (with positive radius) centered on x{displaystyle mathbf {x} }, there exists a ball BPx{displaystyle B_{Pmathbf {x} }} (with positive radius) centered on Px{displaystyle Pmathbf {x} } that is wholly contained in the image P(Bx){displaystyle P(B_{mathbf {x} })}.Complementarity of image and kernel[edit]Let W{displaystyle W} be a finite-dimensional vector space and P{displaystyle P} be a projection on W{displaystyle W}. Suppose the subspaces U{displaystyle U} and V{displaystyle V} are the image and kernel of P{displaystyle P} respectively. Then P{displaystyle P} has the following properties:P{displaystyle P} is the identity operator I{displaystyle I} on U{displaystyle U}: \u2200x\u2208U:Px=x.{displaystyle forall mathbf {x} in U:Pmathbf {x} =mathbf {x} .}We have a direct sum W=U\u2295V{displaystyle W=Uoplus V}. Every vector x\u2208W{displaystyle mathbf {x} in W} may be decomposed uniquely as x=u+v{displaystyle mathbf {x} =mathbf {u} +mathbf {v} } with u=Px{displaystyle mathbf {u} =Pmathbf {x} } and v=x\u2212Px=(I\u2212P)x{displaystyle mathbf {v} =mathbf {x} -Pmathbf {x} =left(I-Pright)mathbf {x} }, and where u\u2208U,v\u2208V.{displaystyle mathbf {u} in U,mathbf {v} in V.}The image and kernel of a projection are complementary, as are P{displaystyle P} and Q=I\u2212P{displaystyle Q=I-P}. The operator Q{displaystyle Q} is also a projection as the image and kernel of P{displaystyle P} become the kernel and image of Q{displaystyle Q} and vice versa. We say P{displaystyle P} is a projection along V{displaystyle V} onto U{displaystyle U} (kernel\/image) and Q{displaystyle Q} is a projection along U{displaystyle U} onto V{displaystyle V}.Spectrum[edit]In infinite-dimensional vector spaces, the spectrum of a projection is contained in {0,1}{displaystyle {0,1}} as(\u03bbI\u2212P)\u22121=1\u03bbI+1\u03bb(\u03bb\u22121)P.{displaystyle (lambda I-P)^{-1}={frac {1}{lambda }}I+{frac {1}{lambda (lambda -1)}}P.}Only 0 or 1 can be an eigenvalue of a projection. This implies that an orthogonal projection P{displaystyle P} is always a positive semi-definite matrix. In general, the corresponding eigenspaces are (respectively) the kernel and range of the projection. Decomposition of a vector space into direct sums is not unique. Therefore, given a subspace V{displaystyle V}, there may be many projections whose range (or kernel) is V{displaystyle V}.If a projection is nontrivial it has minimal polynomial x2\u2212x=x(x\u22121){displaystyle x^{2}-x=x(x-1)}, which factors into distinct linear factors, and thus P{displaystyle P} is diagonalizable.Product of projections[edit]The product of projections is not in general a projection, even if they are orthogonal. If two projections commute then their product is a projection, but the converse is false: the product of two non-commuting projections may be a projection.If two orthogonal projections commute then their product is an orthogonal projection. If the product of two orthogonal projections is an orthogonal projection, then the two orthogonal projections commute (more generally: two self-adjoint endomorphisms commute if and only if their product is self-adjoint).Orthogonal projections[edit]When the vector space W{displaystyle W} has an inner product and is complete (is a Hilbert space) the concept of orthogonality can be used. An orthogonal projection is a projection for which the range U{displaystyle U} and the null space V{displaystyle V} are orthogonal subspaces. Thus, for every x{displaystyle mathbf {x} } and y{displaystyle mathbf {y} } in W{displaystyle W}, \u27e8Px,(y\u2212Py)\u27e9=\u27e8(x\u2212Px),Py\u27e9=0{displaystyle langle Pmathbf {x} ,(mathbf {y} -Pmathbf {y} )rangle =langle (mathbf {x} -Pmathbf {x} ),Pmathbf {y} rangle =0}. Equivalently:\u27e8x,Py\u27e9=\u27e8Px,Py\u27e9=\u27e8Px,y\u27e9.{displaystyle langle mathbf {x} ,Pmathbf {y} rangle =langle Pmathbf {x} ,Pmathbf {y} rangle =langle Pmathbf {x} ,mathbf {y} rangle .}A projection is orthogonal if and only if it is self-adjoint. Using the self-adjoint and idempotent properties of P{displaystyle P}, for any x{displaystyle mathbf {x} } and y{displaystyle mathbf {y} } in W{displaystyle W} we have Px\u2208U{displaystyle Pmathbf {x} in U}, y\u2212Py\u2208V{displaystyle mathbf {y} -Pmathbf {y} in V}, and\u27e8Px,y\u2212Py\u27e9=\u27e8x,(P\u2212P2)y\u27e9=0{displaystyle langle Pmathbf {x} ,mathbf {y} -Pmathbf {y} rangle =langle mathbf {x} ,left(P-P^{2}right)mathbf {y} rangle =0}where \u27e8\u22c5,\u22c5\u27e9{displaystyle langle cdot ,cdot rangle } is the inner product associated with W{displaystyle W}. Therefore, P{displaystyle P} and I\u2212P{displaystyle I-P} are orthogonal projections.[3] The other direction, namely that if P{displaystyle P} is orthogonal then it is self-adjoint, follows from the implication from \u27e8(x\u2212Px),Py\u27e9=\u27e8Px,(y\u2212Py)\u27e9=0{displaystyle langle (mathbf {x} -Pmathbf {x} ),Pmathbf {y} rangle =langle Pmathbf {x} ,(mathbf {y} -Pmathbf {y} )rangle =0} to\u27e8x,Py\u27e9=\u27e8Px,Py\u27e9=\u27e8Px,y\u27e9=\u27e8x,P\u2217y\u27e9{displaystyle langle mathbf {x} ,Pmathbf {y} rangle =langle Pmathbf {x} ,Pmathbf {y} rangle =langle Pmathbf {x} ,mathbf {y} rangle =langle mathbf {x} ,P^{*}mathbf {y} rangle }for every x{displaystyle x} and y{displaystyle y} in W{displaystyle W}; thus P=P\u2217{displaystyle P=P^{*}}.Proof of existenceLet H{displaystyle H} be a complete metric space with an inner product, and let U{displaystyle U} be a closed linear subspace of H{displaystyle H} (and hence complete as well).For every x{displaystyle mathbf {x} } the following set of non-negative norm-values {\u2016x\u2212u\u2016:u\u2208U}{displaystyle {|mathbf {x} -mathbf {u} |:mathbf {u} in U}} has an infimum, and due to the completeness of U{displaystyle U} it is a minimum. We define Px{displaystyle Pmathbf {x} } as the point in U{displaystyle U} where this minimum is obtained.Obviously Px{displaystyle Pmathbf {x} } is in U{displaystyle U}. It remains to show that Px{displaystyle Pmathbf {x} } satisfies \u27e8x\u2212Px,Px\u27e9=0{displaystyle langle mathbf {x} -Pmathbf {x} ,Pmathbf {x} rangle =0} and that it is linear.Let us define a=x\u2212Px{displaystyle mathbf {a} =mathbf {x} -Pmathbf {x} }. For every non-zero v{displaystyle mathbf {v} } in U{displaystyle U}, the following holds:\u2016a\u2212\u27e8a,v\u27e9\u2016v\u20162v\u20162=\u2016a\u20162\u2212\u27e8a,v\u27e92\u2016v\u20162{displaystyle left|mathbf {a} -{frac {langle mathbf {a} ,mathbf {v} rangle }{|mathbf {v} |^{2}}}mathbf {v} right|^{2}=|mathbf {a} |^{2}-{frac {{langle mathbf {a} ,mathbf {v} rangle }^{2}}{|mathbf {v} |^{2}}}}By defining w=Px+\u27e8a,v\u27e9\u2016v\u20162v{displaystyle mathbf {w} =Pmathbf {x} +{frac {langle mathbf {a} ,mathbf {v} rangle }{|mathbf {v} |^{2}}}mathbf {v} } we see that \u2016x\u2212w\u2016Px\u2016{displaystyle |mathbf {x} -mathbf {w} |{displaystyle langle mathbf {a} ,mathbf {v} rangle } vanishes. Since Px{displaystyle Pmathbf {x} } was chosen as the minimum of the aforementioned set, it follows that \u27e8a,v\u27e9{displaystyle langle mathbf {a} ,mathbf {v} rangle } indeed vanishes. In particular, (for y=Px{displaystyle mathbf {y} =Pmathbf {x} }): \u27e8x\u2212Px,Px\u27e9=0{displaystyle langle mathbf {x} -Pmathbf {x} ,Pmathbf {x} rangle =0}.Linearity follows from the vanishing of \u27e8x\u2212Px,v\u27e9{displaystyle langle mathbf {x} -Pmathbf {x} ,mathbf {v} rangle } for every v\u2208U{displaystyle mathbf {v} in U}:\u27e8(x+y)\u2212P(x+y),v\u27e9=0{displaystyle langle left(mathbf {x} +mathbf {y} right)-Pleft(mathbf {x} +mathbf {y} right),mathbf {v} rangle =0}\u27e8(x\u2212Px)+(y\u2212Py),v\u27e9=0{displaystyle langle left(mathbf {x} -Pmathbf {x} right)+left(mathbf {y} -Pmathbf {y} right),mathbf {v} rangle =0}By taking the difference between the equations we have \u27e8Px+Py\u2212P(x+y),v\u27e9=0{displaystyle langle Pmathbf {x} +Pmathbf {y} -Pleft(mathbf {x} +mathbf {y} right),mathbf {v} rangle =0}But since we may choose v=Px+Py\u2212P(x+y){displaystyle mathbf {v} =Pmathbf {x} +Pmathbf {y} -P(mathbf {x} +mathbf {y} )} (as it is itself in U{displaystyle U}) it follows that Px+Py=P(x+y){displaystyle Pmathbf {x} +Pmathbf {y} =P(mathbf {x} +mathbf {y} )}. Similarly we have \u03bbPx=P(\u03bbx){displaystyle lambda Pmathbf {x} =P(lambda mathbf {x} )} for every scalar \u03bb{displaystyle lambda }.Properties and special cases[edit]An orthogonal projection is a bounded operator. This is because for every v{displaystyle mathbf {v} } in the vector space we have, by the Cauchy\u2013Schwarz inequality:\u2016Pv\u20162=\u27e8Pv,Pv\u27e9=\u27e8Pv,v\u27e9\u2264\u2016Pv\u2016\u22c5\u2016v\u2016{displaystyle left|Pmathbf {v} right|^{2}=langle Pmathbf {v} ,Pmathbf {v} rangle =langle Pmathbf {v} ,mathbf {v} rangle leq left|Pmathbf {v} right|cdot left|mathbf {v} right|}Thus \u2016Pv\u2016\u2264\u2016v\u2016{displaystyle left|Pmathbf {v} right|leq left|mathbf {v} right|}.For finite-dimensional complex or real vector spaces, the standard inner product can be substituted for \u27e8\u22c5,\u22c5\u27e9{displaystyle langle cdot ,cdot rangle }.Formulas[edit]A simple case occurs when the orthogonal projection is onto a line. If u{displaystyle mathbf {u} } is a unit vector on the line, then the projection is given by the outer productPu=uuT.{displaystyle P_{mathbf {u} }=mathbf {u} mathbf {u} ^{mathsf {T}}.}(If u{displaystyle mathbf {u} } is complex-valued, the transpose in the above equation is replaced by a Hermitian transpose). This operator leaves u invariant, and it annihilates all vectors orthogonal to u{displaystyle mathbf {u} }, proving that it is indeed the orthogonal projection onto the line containing u.[4] A simple way to see this is to consider an arbitrary vector x{displaystyle mathbf {x} } as the sum of a component on the line (i.e. the projected vector we seek) and another perpendicular to it, x=x\u2225+x\u22a5{displaystyle mathbf {x} =mathbf {x} _{parallel }+mathbf {x} _{perp }}. Applying projection, we getPux=uuTx\u2225+uuTx\u22a5=u(sgn\u2061(uTx\u2225)\u2016x\u2225\u2016)+u\u22c50=x\u2225{displaystyle P_{mathbf {u} }mathbf {x} =mathbf {u} mathbf {u} ^{mathsf {T}}mathbf {x} _{parallel }+mathbf {u} mathbf {u} ^{mathsf {T}}mathbf {x} _{perp }=mathbf {u} left(operatorname {sgn} left(mathbf {u} ^{mathsf {T}}mathbf {x} _{parallel }right)left|mathbf {x} _{parallel }right|right)+mathbf {u} cdot mathbf {0} =mathbf {x} _{parallel }}by the properties of the dot product of parallel and perpendicular vectors.This formula can be generalized to orthogonal projections on a subspace of arbitrary dimension. Let u1,\u2026,uk{displaystyle mathbf {u} _{1},ldots ,mathbf {u} _{k}} be an orthonormal basis of the subspace U{displaystyle U}, with the assumption that the integer k\u22651{displaystyle kgeq 1}, and let A{displaystyle A} denote the n\u00d7k{displaystyle ntimes k} matrix whose columns are u1,\u2026,uk{displaystyle mathbf {u} _{1},ldots ,mathbf {u} _{k}}, i.e., A=[u1\u22efuk]{displaystyle A={begin{bmatrix}mathbf {u} _{1}&cdots &mathbf {u} _{k}end{bmatrix}}}. Then the projection is given by:[5]PA=AAT{displaystyle P_{A}=AA^{mathsf {T}}}which can be rewritten asPA=\u2211i\u27e8ui,\u22c5\u27e9ui.{displaystyle P_{A}=sum _{i}langle mathbf {u} _{i},cdot rangle mathbf {u} _{i}.}The matrix AT{displaystyle A^{mathsf {T}}} is the partial isometry that vanishes on the orthogonal complement of U{displaystyle U} and A{displaystyle A} is the isometry that embeds U{displaystyle U} into the underlying vector space. The range of PA{displaystyle P_{A}} is therefore the final space of A{displaystyle A}. It is also clear that AAT{displaystyle AA^{mathsf {T}}} is the identity operator on U{displaystyle U}.The orthonormality condition can also be dropped. If u1,\u2026,uk{displaystyle mathbf {u} _{1},ldots ,mathbf {u} _{k}} is a (not necessarily orthonormal) basis with k\u22651{displaystyle kgeq 1}, and A{displaystyle A} is the matrix with these vectors as columns, then the projection is:[6][7]PA=A(ATA)\u22121AT.{displaystyle P_{A}=Aleft(A^{mathsf {T}}Aright)^{-1}A^{mathsf {T}}.}The matrix A{displaystyle A} still embeds U{displaystyle U} into the underlying vector space but is no longer an isometry in general. The matrix (ATA)\u22121{displaystyle left(A^{mathsf {T}}Aright)^{-1}} is a “normalizing factor” that recovers the norm. For example, the rank-1 operator uuT{displaystyle mathbf {u} mathbf {u} ^{mathsf {T}}} is not a projection if \u2016u\u2016\u22601.{displaystyle left|mathbf {u} right|neq 1.} After dividing by uTu=\u2016u\u20162,{displaystyle mathbf {u} ^{mathsf {T}}mathbf {u} =left|mathbf {u} right|^{2},} we obtain the projection u(uTu)\u22121uT{displaystyle mathbf {u} left(mathbf {u} ^{mathsf {T}}mathbf {u} right)^{-1}mathbf {u} ^{mathsf {T}}} onto the subspace spanned by u{displaystyle u}.In the general case, we can have an arbitrary positive definite matrix D{displaystyle D} defining an inner product \u27e8x,y\u27e9D=y\u2020Dx{displaystyle langle x,yrangle _{D}=y^{dagger }Dx}, and the projection PA{displaystyle P_{A}} is given by PAx=argminy\u2208range\u2061(A)\u2061\u2016x\u2212y\u2016D2{textstyle P_{A}x=operatorname {argmin} _{yin operatorname {range} (A)}left|x-yright|_{D}^{2}}. ThenPA=A(ATDA)\u22121ATD.{displaystyle P_{A}=Aleft(A^{mathsf {T}}DAright)^{-1}A^{mathsf {T}}D.}When the range space of the projection is generated by a frame (i.e. the number of generators is greater than its dimension), the formula for the projection takes the form: PA=AA+{displaystyle P_{A}=AA^{+}}. Here A+{displaystyle A^{+}} stands for the Moore\u2013Penrose pseudoinverse. This is just one of many ways to construct the projection operator.If [AB]{displaystyle {begin{bmatrix}A&Bend{bmatrix}}} is a non-singular matrix and ATB=0{displaystyle A^{mathsf {T}}B=0} (i.e., B{displaystyle B} is the null space matrix of A{displaystyle A}),[8] the following holds:I=[AB][AB]\u22121[ATBT]\u22121[ATBT]=[AB]([ATBT][AB])\u22121[ATBT]=[AB][ATAOOBTB]\u22121[ATBT]=A(ATA)\u22121AT+B(BTB)\u22121BT{displaystyle {begin{aligned}I&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathsf {T}}\\B^{mathsf {T}}end{bmatrix}}^{-1}{begin{bmatrix}A^{mathsf {T}}\\B^{mathsf {T}}end{bmatrix}}\\&={begin{bmatrix}A&Bend{bmatrix}}left({begin{bmatrix}A^{mathsf {T}}\\B^{mathsf {T}}end{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}right)^{-1}{begin{bmatrix}A^{mathsf {T}}\\B^{mathsf {T}}end{bmatrix}}\\&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A^{mathsf {T}}A&O\\O&B^{mathsf {T}}Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathsf {T}}\\B^{mathsf {T}}end{bmatrix}}\\[4pt]&=Aleft(A^{mathsf {T}}Aright)^{-1}A^{mathsf {T}}+Bleft(B^{mathsf {T}}Bright)^{-1}B^{mathsf {T}}end{aligned}}}If the orthogonal condition is enhanced to ATWB=ATWTB=0{displaystyle A^{mathsf {T}}WB=A^{mathsf {T}}W^{mathsf {T}}B=0} with W{displaystyle W} non-singular, the following holds:I=[AB][(ATWA)\u22121AT(BTWB)\u22121BT]W.{displaystyle I={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}left(A^{mathsf {T}}WAright)^{-1}A^{mathsf {T}}\\left(B^{mathsf {T}}WBright)^{-1}B^{mathsf {T}}end{bmatrix}}W.}All these formulas also hold for complex inner product spaces, provided that the conjugate transpose is used instead of the transpose. Further details on sums of projectors can be found in Banerjee and Roy (2014).[9] Also see Banerjee (2004)[10] for application of sums of projectors in basic spherical trigonometry.Oblique projections[edit]The term oblique projections is sometimes used to refer to non-orthogonal projections. These projections are also used to represent spatial figures in two-dimensional drawings (see oblique projection), though not as frequently as orthogonal projections. Whereas calculating the fitted value of an ordinary least squares regression requires an orthogonal projection, calculating the fitted value of an instrumental variables regression requires an oblique projection.Projections are defined by their null space and the basis vectors used to characterize their range (which is the complement of the null space). When these basis vectors are orthogonal to the null space, then the projection is an orthogonal projection. When these basis vectors are not orthogonal to the null space, the projection is an oblique projection, or just a general projection.A matrix representation formula for a nonzero projection operator[edit]Let  P{displaystyle P} be a linear operator P:V\u2192V{displaystyle P:Vto V} such that P2=P{displaystyle P^{2}=P} and assume that  P:V\u2192V{displaystyle P:Vto V} is not the zero operator. Let the vectors u1,\u2026,uk{displaystyle mathbf {u} _{1},ldots ,mathbf {u} _{k}} form a basis for the range of the projection, and assemble these vectors in the n\u00d7k{displaystyle ntimes k} matrix A{displaystyle A}. Therefore the integer k\u22651{displaystyle kgeq 1}, otherwise k=0{displaystyle k=0} and P{displaystyle P} is the zero operator. The range and the null space are complementary spaces, so the null space has dimension n\u2212k{displaystyle n-k}. It follows that the orthogonal complement of the null space has dimension k{displaystyle k}. Let v1,\u2026,vk{displaystyle mathbf {v} _{1},ldots ,mathbf {v} _{k}} form a basis for the orthogonal complement of the null space of the projection, and assemble these vectors in the matrix B{displaystyle B}. Then the projection P{displaystyle P} (with the condition k\u22651{displaystyle kgeq 1}) is given byP=A(BTA)\u22121BT.{displaystyle P=Aleft(B^{mathsf {T}}Aright)^{-1}B^{mathsf {T}}.}This expression generalizes the formula for orthogonal projections given above.[11][12] A standard proof of this expression is the following. For any vector x{displaystyle mathbf {x} } in the vector space V{displaystyle V}, we can decompose x=x1+x2{displaystyle mathbf {x} =mathbf {x} _{1}+mathbf {x} _{2}}, where vector x1=P(x){displaystyle mathbf {x} _{1}=P(mathbf {x} )} is in the image of P{displaystyle P}, and vector x2=x\u2212P(x){displaystyle mathbf {x} _{2}=mathbf {x} -P(mathbf {x} )}. So P(x2)=P(x)\u2212P2(x)=0{displaystyle P(mathbf {x} _{2})=P(mathbf {x} )-P^{2}(mathbf {x} )=mathbf {0} }, and then x2{displaystyle mathbf {x} _{2}} is in the null space of P{displaystyle P}. In other words, the vector x1{displaystyle mathbf {x} _{1}} is in the column space of A{displaystyle A}, so x1=Aw{displaystyle mathbf {x} _{1}=Amathbf {w} } for some k{displaystyle k} dimension vector w{displaystyle mathbf {w} } and the vector x2{displaystyle mathbf {x} _{2}} satisfies BTx2=0{displaystyle B^{mathsf {T}}mathbf {x} _{2}=mathbf {0} } by the construction of B{displaystyle B}. Put these conditions together, and we find a vector w{displaystyle mathbf {w} } so that  BT(x\u2212Aw)=0{displaystyle B^{mathsf {T}}(mathbf {x} -Amathbf {w} )=mathbf {0} }. Since matrices A{displaystyle A} and B{displaystyle B} are of full rank k{displaystyle k} by their construction, the k\u00d7k{displaystyle ktimes k}-matrix BTA{displaystyle B^{mathsf {T}}A} is invertible. So the equation  BT(x\u2212Aw)=0{displaystyle B^{mathsf {T}}(mathbf {x} -Amathbf {w} )=mathbf {0} } gives the vector w=(BTA)\u22121BTx.{displaystyle mathbf {w} =(B^{mathsf {T}}A)^{-1}B^{mathsf {T}}mathbf {x} .} In this way, Px=x1=Aw=A(BTA)\u22121BTx{displaystyle Pmathbf {x} =mathbf {x} _{1}=Amathbf {w} =A(B^{mathsf {T}}A)^{-1}B^{mathsf {T}}mathbf {x} } for any vector x\u2208V{displaystyle mathbf {x} in V} and hence P=A(BTA)\u22121BT{displaystyle P=A(B^{mathsf {T}}A)^{-1}B^{mathsf {T}}}.In the case that P{displaystyle P} is an orthogonal projection, we can take A=B{displaystyle A=B}, and it follows that P=A(ATA)\u22121AT{displaystyle P=Aleft(A^{mathsf {T}}Aright)^{-1}A^{mathsf {T}}}.  By using this formula, one can easily check that   P=PT{displaystyle P=P^{mathsf {T}}}. In general, if the vector space is over complex number field, one then uses the Hermitian transpose A\u2217{displaystyle A^{*}} and has the formula  P=A(A\u2217A)\u22121A\u2217{displaystyle P=Aleft(A^{*}Aright)^{-1}A^{*}}. Recall that one can define the Moore\u2013Penrose inverse of the matrix A{displaystyle A} by  A+=(A\u2217A)\u22121A\u2217{displaystyle A^{+}=(A^{*}A)^{-1}A^{*}} since A{displaystyle A} has full column rank, so  P=AA+{displaystyle P=AA^{+}}.Singular values[edit]Note that I\u2212P{displaystyle I-P} is also an oblique projection. The singular values of P{displaystyle P} and I\u2212P{displaystyle I-P} can be computed by an orthonormal basis of A{displaystyle A}. LetQA{displaystyle Q_{A}} be an orthonormal basis of A{displaystyle A} and let QA\u22a5{displaystyle Q_{A}^{perp }} be the orthogonal complement of QA{displaystyle Q_{A}}. Denote the singular values of the matrixQATA(BTA)\u22121BTQA\u22a5{displaystyle Q_{A}^{T}A(B^{T}A)^{-1}B^{T}Q_{A}^{perp }} by the positive values \u03b31\u2265\u03b32\u2265\u2026\u2265\u03b3k{displaystyle gamma _{1}geq gamma _{2}geq ldots geq gamma _{k}}. With this, the singular values for P{displaystyle P} are:[13]\u03c3i={1+\u03b3i21\u2264i\u2264k0otherwise{displaystyle sigma _{i}={begin{cases}{sqrt {1+gamma _{i}^{2}}}&1leq ileq k\\0&{text{otherwise}}end{cases}}}and the singular values for I\u2212P{displaystyle I-P} are\u03c3i={1+\u03b3i21\u2264i\u2264k1k+1\u2264i\u2264n\u2212k0otherwise{displaystyle sigma _{i}={begin{cases}{sqrt {1+gamma _{i}^{2}}}&1leq ileq k\\1&k+1leq ileq n-k\\0&{text{otherwise}}end{cases}}}This implies that the largest singular values of P{displaystyle P} and I\u2212P{displaystyle I-P} are equal, and thus that the matrix norm of the oblique projections are the same.However, the condition number satisfies the relation \u03ba(I\u2212P)=\u03c311\u2265\u03c31\u03c3k=\u03ba(P){displaystyle kappa (I-P)={frac {sigma _{1}}{1}}geq {frac {sigma _{1}}{sigma _{k}}}=kappa (P)}, and is therefore not necessarily equal.Finding projection with an inner product[edit]Let V{displaystyle V} be a vector space (in this case a plane) spanned by orthogonal vectors u1,u2,\u2026,up{displaystyle mathbf {u} _{1},mathbf {u} _{2},dots ,mathbf {u} _{p}}. Let y{displaystyle y} be a vector. One can define a projection of y{displaystyle mathbf {y} } onto V{displaystyle V} asprojV\u2061y=y\u22c5uiui\u22c5uiui{displaystyle operatorname {proj} _{V}mathbf {y} ={frac {mathbf {y} cdot mathbf {u} ^{i}}{mathbf {u} ^{i}cdot mathbf {u} ^{i}}}mathbf {u} ^{i}}where repeated indices are summed over (Einstein sum notation). The vector y{displaystyle mathbf {y} } can be written as an orthogonal sum such that y=projV\u2061y+z{displaystyle mathbf {y} =operatorname {proj} _{V}mathbf {y} +mathbf {z} }. projV\u2061y{displaystyle operatorname {proj} _{V}mathbf {y} } is sometimes denoted as y^{displaystyle {hat {mathbf {y} }}}. There is a theorem in linear algebra that states that this z{displaystyle mathbf {z} } is the smallest distance (the orthogonal distance) from y{displaystyle mathbf {y} } to V{displaystyle V} and is commonly used in areas such as machine learning.  y is being projected onto the vector space V.Canonical forms[edit]Any projection P=P2{displaystyle P=P^{2}} on a vector space of dimension d{displaystyle d} over a field is a diagonalizable matrix, since its minimal polynomial divides x2\u2212x{displaystyle x^{2}-x}, which splits into distinct linear factors. Thus there exists a basis in which P{displaystyle P} has the formP=Ir\u22950d\u2212r{displaystyle P=I_{r}oplus 0_{d-r}}where r{displaystyle r} is the rank of P{displaystyle P}.  Here Ir{displaystyle I_{r}} is the identity matrix of size r{displaystyle r}, 0d\u2212r{displaystyle 0_{d-r}} is the zero matrix of size d\u2212r{displaystyle d-r}, and \u2295{displaystyle oplus } is the direct sum operator.  If the vector space is complex and equipped with an inner product, then there is an orthonormal basis in which the matrix of P is[14]P=[1\u03c3100]\u2295\u22ef\u2295[1\u03c3k00]\u2295Im\u22950s.{displaystyle P={begin{bmatrix}1&sigma _{1}\\0&0end{bmatrix}}oplus cdots oplus {begin{bmatrix}1&sigma _{k}\\0&0end{bmatrix}}oplus I_{m}oplus 0_{s}.}where "},{"@context":"http:\/\/schema.org\/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"item":{"@id":"https:\/\/wiki.edu.vn\/en\/wiki43\/#breadcrumbitem","name":"Enzyklop\u00e4die"}},{"@type":"ListItem","position":2,"item":{"@id":"https:\/\/wiki.edu.vn\/en\/wiki43\/projection-linear-algebra-wikipedia\/#breadcrumbitem","name":"Projection (linear algebra) – Wikipedia"}}]}]