To install click the Add extension button. That's it.

The source code for the WIKI 2 extension is being checked by specialists of the Mozilla Foundation, Google, and Apple. You could also do it yourself at any point in time. 4,5
Kelly Slayton
Congratulations on this excellent venture… what a great idea!
Alexander Grigorievskiy
I use WIKI 2 every day and almost forgot how the original Wikipedia looks like.
Live Statistics
English Articles
Improved in 24 Hours
Added in 24 Hours
Languages
Recent
Show all languages
What we do. Every page goes through several hundred of perfecting techniques; in live mode. Quite the same Wikipedia. Just better.
.
Leo
Newton
Brights
Milds Sample mean and covariance

The sample mean or empirical mean and the sample covariance are statistics computed from a collection (the sample) of data on one or more random variables. The sample mean and sample covariance are estimators of the population mean and population covariance, where the term population refers to the set from which the sample was taken.

The sample mean is a vector each of whose elements is the sample mean of one of the random variables – that is, each of whose elements is the arithmetic average of the observed values of one of the variables. The sample covariance matrix is a square matrix whose i, j element is the sample covariance (an estimate of the population covariance) between the sets of observed values of two of the variables and whose i, i element is the sample variance of the observed values of one of the variables. If only one variable has had values observed, then the sample mean is a single number (the arithmetic average of the observed values of that variable) and the sample covariance matrix is also simply a single value (a 1x1 matrix containing a single number, the sample variance of the observed values of that variable).

Due to their ease of calculation and other desirable characteristics, the sample mean and sample covariance are widely used in statistics and applications to numerically represent the location and dispersion, respectively, of a distribution.

• 1/5
Views:
71 908
1 206 949
31 595
789 086
5 873
• ✪ Covariance Example
• ✪ Central limit theorem | Inferential statistics | Probability and Statistics | Khan Academy
• ✪ Lesson 10 - Statistics Population Mean & Sample Mean
• ✪ Sampling distribution of the sample mean | Probability and Statistics | Khan Academy
• ✪ Calculating and Understanding Covariance (COVARIANCE.S) in Excel

Sample mean

Let $x_{ij}$ be the ith independently drawn observation (i=1,...,N) on the jth random variable (j=1,...,K). These observations can be arranged into N column vectors, each with K entries, with the K ×1 column vector giving the ith observations of all variables being denoted $\mathbf {x} _{i}$ (i=1,...,N).

The sample mean vector $\mathbf {\bar {x}}$ is a column vector whose jth element ${\bar {x}}_{j}$ is the average value of the N observations of the jth variable:

${\bar {x}}_{j}={\frac {1}{N}}\sum _{i=1}^{N}x_{ij},\quad j=1,\ldots ,K.$ Thus, the sample mean vector contains the average of the observations for each variable, and is written

$\mathbf {\bar {x}} ={\frac {1}{N}}\sum _{i=1}^{N}\mathbf {x} _{i}={\begin{bmatrix}{\bar {x}}_{1}\\\vdots \\{\bar {x}}_{j}\\\vdots \\{\bar {x}}_{K}\end{bmatrix}}$ Sample covariance

The sample covariance matrix is a K-by-K matrix $\textstyle \mathbf {Q} =\left[q_{jk}\right]$ with entries

$q_{jk}={\frac {1}{N-1}}\sum _{i=1}^{N}\left(x_{ij}-{\bar {x}}_{j}\right)\left(x_{ik}-{\bar {x}}_{k}\right),$ where $q_{jk}$ is an estimate of the covariance between the jth variable and the kth variable of the population underlying the data. In terms of the observation vectors, the sample covariance is

$\mathbf {Q} ={1 \over {N-1}}\sum _{i=1}^{N}(\mathbf {x} _{i}.-\mathbf {\bar {x}} )(\mathbf {x} _{i}.-\mathbf {\bar {x}} )^{\mathrm {T} },$ Alternatively, arranging the observation vectors as the columns of a matrix, so that

$\mathbf {F} ={\begin{bmatrix}\mathbf {x} _{1}&\mathbf {x} _{2}&\dots &\mathbf {x} _{N}\end{bmatrix}}$ ,

which is a matrix of K rows and N columns. Here, the sample covariance matrix can be computed as

$\mathbf {Q} ={\frac {1}{N-1}}(\mathbf {F} -\mathbf {\bar {x}} \,\mathbf {1} _{N}^{\mathrm {T} })(\mathbf {F} -\mathbf {\bar {x}} \,\mathbf {1} _{N}^{\mathrm {T} })^{\mathrm {T} }$ ,

where $\mathbf {1} _{N}$ is an N by 1 vector of ones. If the observations are arranged as rows instead of columns, so $\mathbf {\bar {x}}$ is now a 1×K row vector and $\mathbf {M} =\mathbf {F} ^{\mathrm {T} }$ is an N×K matrix whose column j is the vector of N observations on variable j, then applying transposes in the appropriate places yields

$\mathbf {Q} ={\frac {1}{N-1}}(\mathbf {M} -\mathbf {1} _{N}\mathbf {{\bar {x}}^{\mathrm {T} }} )^{\mathrm {T} }(\mathbf {M} -\mathbf {1} _{N}\mathbf {{\bar {x}}^{\mathrm {T} }} ).$ Like covariance matrices for random vector, sample covariance matrices are positive semi-definite. To prove it, note that for any matrix $\mathbf {A}$ the matrix $\mathbf {A} ^{T}\mathbf {A}$ is positive semi-definite. Furthermore, a covariance matrix is positive definite if and only if the rank of the $\mathbf {x} _{i}.-\mathbf {\bar {x}}$ vectors is K.

Unbiasedness

The sample mean and the sample covariance matrix are unbiased estimates of the mean and the covariance matrix of the random vector $\textstyle \mathbf {X}$ , a row vector whose jth element (j = 1, ..., K) is one of the random variables. The sample covariance matrix has $\textstyle N-1$ in the denominator rather than $\textstyle N$ due to a variant of Bessel's correction: In short, the sample covariance relies on the difference between each observation and the sample mean, but the sample mean is slightly correlated with each observation since it is defined in terms of all observations. If the population mean $\operatorname {E} (\mathbf {X} )$ is known, the analogous unbiased estimate

$q_{jk}={\frac {1}{N}}\sum _{i=1}^{N}\left(x_{ij}-\operatorname {E} (X_{j})\right)\left(x_{ik}-\operatorname {E} (X_{k})\right),$ using the population mean, has $\textstyle N$ in the denominator. This is an example of why in probability and statistics it is essential to distinguish between random variables (upper case letters) and realizations of the random variables (lower case letters).

$q_{jk}={\frac {1}{N}}\sum _{i=1}^{N}\left(x_{ij}-{\bar {x}}_{j}\right)\left(x_{ik}-{\bar {x}}_{k}\right)$ for the Gaussian distribution case has N in the denominator as well. The ratio of 1/N to 1/(N − 1) approaches 1 for large N, so the maximum likelihood estimate approximately equals the unbiased estimate when the sample is large.

Variance of the sampling distribution of the sample mean

For each random variable, the sample mean is a good estimator of the population mean, where a "good" estimator is defined as being efficient and unbiased. Of course the estimator will likely not be the true value of the population mean since different samples drawn from the same distribution will give different sample means and hence different estimates of the true mean. Thus the sample mean is a random variable, not a constant, and consequently has its own distribution. For a random sample of N observations on the jth random variable, the sample mean's distribution itself has mean equal to the population mean $E(X_{j})$ and variance equal to ${\frac {\sigma _{j}^{2}}{N}},$ where $\sigma _{j}^{2}$ is the population variance.

Weighted samples

In a weighted sample, each vector $\textstyle {\textbf {x}}_{i}$ (each set of single observations on each of the K random variables) is assigned a weight $\textstyle w_{i}\geq 0$ . Without loss of generality, assume that the weights are normalized:

$\sum _{i=1}^{N}w_{i}=1.$ (If they are not, divide the weights by their sum). Then the weighted mean vector $\textstyle \mathbf {\bar {x}}$ is given by

$\mathbf {\bar {x}} =\sum _{i=1}^{N}w_{i}\mathbf {x} _{i}.$ and the elements $q_{jk}$ of the weighted covariance matrix $\textstyle \mathbf {Q}$ are 

$q_{jk}={\frac {1}{1-\sum _{i=1}^{N}w_{i}^{2}}}\sum _{i=1}^{N}w_{i}\left(x_{ij}-{\bar {x}}_{j}\right)\left(x_{ik}-{\bar {x}}_{k}\right).$ If all weights are the same, $\textstyle w_{i}=1/N$ , the weighted mean and covariance reduce to the sample mean and covariance mentioned above.

Criticism

The sample mean and sample covariance are not robust statistics, meaning that they are sensitive to outliers. As robustness is often a desired trait, particularly in real-world applications, robust alternatives may prove desirable, notably quantile-based statistics such as the sample median for location, and interquartile range (IQR) for dispersion. Other alternatives include trimming and Winsorising, as in the trimmed mean and the Winsorized mean.