Definite quadratic form

In mathematics, a definite quadratic form is a quadratic form over some real vector space $V$ that has the same sign (always positive or always negative) for every non-zero vector of $V$ . According to that sign, the quadratic form is called positive-definite or negative-definite.

A semidefinite (or semi-definite) quadratic form is defined in much the same way, except that "always positive" and "always negative" are replaced by "never negative" and "never positive", respectively. In other words, it may take on zero values for some non-zero vectors of $V$ .

An indefinite quadratic form takes on both positive and negative values and is called an isotropic quadratic form.

More generally, these definitions apply to any vector space over an ordered field.^[1]

YouTube Encyclopedic

1/3
Views:
54 932
1 504
14 310

Transcription

MARTINA BALAGOVIC: Hi. Welcome to recitation. Today's problem is about positive definite matrices. And it's asking us, for which values of the parameter c, which is sitting here in the matrix, is the matrix B, 2, minus 1, minus 1, minus 1, 2, minus 1, minus 1, minus 1, 2 plus c, positive definite, and for which values of c is it positive semidefinite? I'm going to leave you alone with the problem. You should pause the video and then come back and compare your solution. And we're back. As you remember from the lectures, there are several tests that you can do on matrices to find out if they're positive definite and if they're positive semidefinite. And I'm going to demonstrate three to you. First, I'm going to do the one that you should do in case you have very little time and you're asked to do a problem like this on the test, which is of course the determinant test. The determinant test asks us to compute determinants of the matrices in the upper left corner of all sizes. And it says that it's going to be positive definite if they're all greater than 0 and positive semidefinite if some 0's sneak into into that sequence. So let's calculate the determinants. The first determinant is the determinant of this tiny matrix here. So it's just 2. The next one is the determinant of this two by two submatrix, 2, minus 1, minus 1, 2, which is equal to 4 minus 1, which is 3. And finally, we have the determinant of B, which I'm going to calculate for you. I'm going to calculate it, I'm going to decompose it along the first line, first row. So it's 2 times the determinant of this submatrix, 2, minus 1, minus 1, 2 plus c, minus minus 1, this one here, times this determinant, which is minus 1, minus 1, minus 1, 2 plus c. And then plus minus 1, this one here, times this determinant, which is minus 1, 2, minus 1, minus 1. And in total, that's 2 times 4 plus 2c minus 1 plus minus 2 minus c minus 1 and minus 1 plus 2. And in total, this should give us 6 plus 4c minus 3 minus c minus 3, which is 3c. So let's look at the determinants. 2 is positive. 3 is positive. 3c is positive. So the answer is the matrix is positive definite if c is bigger than 0, and it's positive semidefinite if it's either strictly bigger than 0 or equal to 0. And that's all. If you're on the test, this is everything that you should do. Now let me show you two more tests to demonstrate that first, they take longer, and second, to see these numbers and their quotients show up in other tests and to try to convince you that these tests really are equivalent. Let me do the pivots test for you. So we take our matrix B, 2, minus 1, minus 1, minus 1, 2, minus 1, minus 1, minus 1, 2 plus c. And let's pretend we're solving a system that has this as a matrix. So we multiply the first row by 1/2, and we add it to the second and to the third row. We get 2, minus 1, minus 1, 0, 3/2, minus 3/2, 0, minus 3/2, 3/2 plus c. So the first column is good. And then we just replace the third column with the third column plus the second column. And we get 2, minus 1, minus 1, 0, 3/2, minus 3/2, 0, 0, and c. And so the pivots are 2, 3/2, and c. And again, the answer is as before. It's positive definite if c is strictly bigger than 0, and it's positive semidefinite if c is greater or equal to 0. But I want you to notice something else here. So before, we had 2, 3, and 3c. And now for determinants, as these pivots, we have 2, 3/2, which is the second determinant over the first determinant, and c, which can be thought of as 3c/3, so the third determinant over the second determinant. And something like this is always going to happen. And finally, let me do the energy test, or completing the square. So one of the definitions of positive definite, one could say the definition because of which we are really interested in such matrices, is the following. It's positive definite if when we consider this quadratic form, so a form that maps x, y, and z to this expression here that's going to be quadratic in x, y, and z, it's positive semidefinite if this is always greater or equal than 0. And it's positive definite if, when we have an expression like this and try to solve this equals 0, the only solution is that x, y, and z all have to be 0. So let's try calculating this form, completing the squares on it, and seeing these numbers show up again. So when I multiply this like this, put in a B, do the multiplication, we get something like this. 2x squared plus 2y squared plus 2 plus c z squared minus 2xy minus 2xz minus 2yz. And now let's try completing the squares using the formulas that I prepared for you up here in this pop-up window in the corner. So let's try completing, we have a formula for the square of a plus b plus c and the square of a plus b. So first, we try to get something squared so that this something has all the x's that appear here, so that we get something squared plus some expression that only takes y's and z's. I'm not going to do the calculation in front of you, further embarrass myself with it. But let me tell you that what you get is 2 times x minus 1/2 y minus 1/2 z squared. And this ate all the x's that showed up here. The remainder only has y's and z's. When you use the second of those formulas in a pop-up window to complete the square of y's and z's, you get 3/2 times y minus z squared. So this took up all the y's, and we're left just with a z that comes as c times z squared. And now, let's look at the following. This is a square of some real number. So that's positive. This is a square of some real number, so that's positive. And this is a square of some real number, so that's positive. They're all multiplied by positive numbers, 2, 3/2, and c, which we've already seen here. And so if c is bigger or equal than 0, this is certainly always bigger or equal than 0. Now to the question of if this can be equal to 0 without x, y, and z all being 0, well, let's look at two cases. If c is strictly bigger than 0, then let me write this matrix here as 2, 3/2 and c times 1, minus 1/2, minus 1/2, 0, 1, minus 1, and 0, 0, 1. Let's imagine c is strictly bigger than 0. And let's see when can this expression be equal to 0. Well, as we said, it's a sum of certain squares. They're all greater or equal to 0, so they all have to be equal to 0 for this expression to be 0. In other words, z needs to be equal to 0. y minus z needs to be equal to 0. And x minus 1/2 y minus 1/2 z also needs to be equal to 0. And since this matrix has all the pivots, this can only happen if x, y, and z are all equal to 0. On the other hand, if c is equal to 0, then let me write this matrix here again. Take 2 out here. Take 3/2 out here. And write again 1, minus 1/2, minus 1/2, 0, 1, minus 1, and 0, 0, 0. And so imagining that c is 0, when can this whole expression be equal to 0? Well, the last bit is already 0. We need to have y minus z equal to 0. And we need to have 1 times x minus 1/2 times y minus 1/2 times z also equal to 0. So we need to have x, y, z in the kernel of this matrix. But this matrix only has two pivots. It has one free variable. So we can find a solution to this times x, y, z equals 0 that's not 0 itself. Namely, if you remember how to solve systems like this, we set z equal to 1. From this, we calculate that y has to be 1. And then from this, we calculate that x has to be 1. And you can check that for in case c is 0, this thing here, when you plug it in here, you really get 0. In fact, this thing here is in the kernel of the matrix B. In fact, in case c is 0, the columns of matrix B sum up to 0 because of this here. And that's all I wanted to say for today.

Associated symmetric bilinear form

Quadratic forms correspond one-to-one to symmetric bilinear forms over the same space.^[2] A symmetric bilinear form is also described as definite, semidefinite, etc. according to its associated quadratic form. A quadratic form $Q$ and its associated symmetric bilinear form $B$ are related by the following equations:

{\begin{aligned}Q(x)&=B(x,x)\\B(x,y)&=B(y,x)={\tfrac {1}{2}}[Q(x+y)-Q(x)-Q(y)]~.\end{aligned}}

The latter formula arises from expanding $\;Q(x+y)=B(x+y,x+y)~.$

Examples

As an example, let $V=\mathbb {R} ^{2}$ , and consider the quadratic form

Q(x)=c_{1}{x_{1}}^{2}+c_{2}{x_{2}}^{2}

where $~x=[x_{1},x_{2}]\in V$ and $c$ ₁ and $c$ ₂ are constants. If $c$ ₁ > 0 and $c$ ₂ > 0 , the quadratic form $Q$ is positive-definite, so Q evaluates to a positive number whenever $\;[x_{1},x_{2}]\neq [0,0]~.$ If one of the constants is positive and the other is 0, then $Q$ is positive semidefinite and always evaluates to either 0 or a positive number. If $c$ ₁ > 0 and $c$ ₂ < 0 , or vice versa, then $Q$ is indefinite and sometimes evaluates to a positive number and sometimes to a negative number. If $c$ ₁ < 0 and $c$ ₂ < 0 , the quadratic form is negative-definite and always evaluates to a negative number whenever $\;[x_{1},x_{2}]\neq [0,0]~.$ And if one of the constants is negative and the other is 0, then $Q$ is negative semidefinite and always evaluates to either 0 or a negative number.

In general a quadratic form in two variables will also involve a cross-product term in $x$ ₁· $x$ ₂:

Q(x)=c_{1}{x_{1}}^{2}+c_{2}{x_{2}}^{2}+2c_{3}x_{1}x_{2}~.

This quadratic form is positive-definite if $\;c_{1}>0\;$ and $\,c_{1}c_{2}-{c_{3}}^{2}>0\;,$ negative-definite if $\;c_{1}<0\;$ and $\,c_{1}c_{2}-{c_{3}}^{2}>0\;,$ and indefinite if $\;c_{1}c_{2}-{c_{3}}^{2}<0~.$ It is positive or negative semidefinite if $\;c_{1}c_{2}-{c_{3}}^{2}=0\;,$ with the sign of the semidefiniteness coinciding with the sign of $\;c_{1}~.$

This bivariate quadratic form appears in the context of conic sections centered on the origin. If the general quadratic form above is equated to 0, the resulting equation is that of an ellipse if the quadratic form is positive or negative-definite, a hyperbola if it is indefinite, and a parabola if $\;c_{1}c_{2}-{c_{3}}^{2}=0~.$

The square of the Euclidean norm in $n$ -dimensional space, the most commonly used measure of distance, is

{x_{1}}^{2}+\cdots +{x_{n}}^{2}~.

In two dimensions this means that the distance between two points is the square root of the sum of the squared distances along the $x_{1}$ axis and the $x_{2}$ axis.

Matrix form

A quadratic form can be written in terms of matrices as

x^{\mathsf {T}}A\,x

where $x$ is any $n$ ×1 Cartesian vector $\;[x_{1},\cdots ,x_{n}]^{\mathsf {T}}\;$ in which at least one element is not 0; $A$ is an $n \times n$ symmetric matrix; and superscript ^T denotes a matrix transpose. If $A$ is diagonal this is equivalent to a non-matrix form containing solely terms involving squared variables; but if $A$ has any non-zero off-diagonal elements, the non-matrix form will also contain some terms involving products of two different variables.

Positive or negative-definiteness or semi-definiteness, or indefiniteness, of this quadratic form is equivalent to the same property of $A$ , which can be checked by considering all eigenvalues of $A$ or by checking the signs of all of its principal minors.

Optimization

Definite quadratic forms lend themselves readily to optimization problems. Suppose the matrix quadratic form is augmented with linear terms, as

x^{\mathsf {T}}A\,x+b^{\mathsf {T}}x\;,

where $b$ is an $n$ ×1 vector of constants. The first-order conditions for a maximum or minimum are found by setting the matrix derivative to the zero vector:

2A\,x+b={\vec {0}}\;,

giving

x=-{\tfrac {1}{2}}\,A^{-1}b\;,

assuming $A$ is nonsingular. If the quadratic form, and hence $A$ , is positive-definite, the second-order conditions for a minimum are met at this point. If the quadratic form is negative-definite, the second-order conditions for a maximum are met.

An important example of such an optimization arises in multiple regression, in which a vector of estimated parameters is sought which minimizes the sum of squared deviations from a perfect fit within the dataset.

Notes

^ Milnor & Husemoller 1973, p. 61.
^ This is true only over a field of characteristic other than 2, but here we consider only ordered fields, which necessarily have characteristic 0.

References

Kitaoka, Yoshiyuki (1993). Arithmetic of quadratic forms. Cambridge Tracts in Mathematics. Vol. 106. Cambridge University Press. ISBN 0-521-40475-4. Zbl 0785.11021.
Lang, Serge (2004), Algebra, Graduate Texts in Mathematics, vol. 211 (Corrected fourth printing, revised third ed.), New York: Springer-Verlag, p. 578, ISBN 978-0-387-95385-4.
Milnor, J.; Husemoller, D. (1973). Symmetric Bilinear Forms. Ergebnisse der Mathematik und ihrer Grenzgebiete. Vol. 73. Springer. ISBN 3-540-06009-X. Zbl 0292.10016.

This page was last edited on 10 June 2022, at 18:41

From Wikipedia, the free encyclopedia