Covariance, Variance Of Sums: 7 Important Facts

COVARIANCE, VARIANCE OF SUMS, AND CORRELATIONS OF RANDOM VARIABLES

  The statistical parameters of the random variables of different nature using the definition of expectation of random variable is easy to obtain and understand, in the following we will find some parameters with the help of mathematical expectation of random variable.

Moments of the number of events that occur

    So far we know that expectation of different powers of random variable is the moments of random variables and how to find the expectation of random variable from the events if number of event occurred already, now we are interested in the expectation if pair of number of events already occurred, now if X represents the number of event occurred then for the events A1, A2, ….,An define the indicator variable Ii as

gif

the expectation of X in discrete sense will be

gif

because the random variable X is

gif

now to find expectation if number of pair of event occurred already we have to use combination as

gif

this gives expectation as

gif
gif

from this we get the expectation of x square and the value of variance also by

gif

By using this discussion we focus different kinds of random variable to find such moments.

Moments of binomial random variables

   If p is the probability of success from n independent trials then lets denote Ai for the trial i as success so

gif
gif
gif
gif

and hence the variance of binomial random variable will be

gif

because

gif

if we generalize for k events

gif
gif

this expectation we can obtain successively for the value of k greater than 3  let us find for 3

gif
gif

gif
gif

using this iteration we can get

gif

Moments of hypergeometric random variables

  The moments of this random variable we will understand with the help of an example suppose n pens are randomly selected from a box containing N pens of which m are blue, Let Ai denote the events that i-th pen is blue, Now X is the number of blue pen selected is equal to the number of events A1,A2,…..,An that occur because the ith pen selected is equally likely to any of the N pens of which m are blue

gif

and so

A %7Bi%7D%29%20%3D%5Cfrac%7Bm%7D%7BN%7D%20%5Cfrac%7Bm 1%7D%7BN 1%7D
gif
gif

this gives

gif

so the variance of hypergeometric random variable will be

gif
gifgif

in similar way for the higher moments

gif
gif

hence

gif

Moments of the negative hypergeometric random variables

  consider the example of a package containing n+m vaccines of which n are special and m are ordinary, these vaccines removed one at a time, with each new removal equally likely to be any of the vaccine that remain in the package. Now let random variable Y denote the number of vaccines that need to be withdrawn until a total of r special vaccines have been removed, which is negative hypergeometric distribution, this is somehow similar with negative binomial to binomial as to hypergeometric distribution. to find the probability mass function if the kth draw gives the special vaccine after k-1 draw gives r-1 special and k-r ordinary vaccine

gif

now the random variable Y

Y=r+X

for the events Ai

gif
gif

as

gif

hence to find the variance of Y we must know the variance of X so

gif
gif
gif
gif

hence

gif

COVARIANCE             

The relationship between two random variable can be represented by the statistical parameter covariance, before the definition of covariance of two random variable X and Y recall that the expectation of two functions g and h of random variables X and Y respectively gives

gif
gif
gif
gif
gif

using this relation of expectation we can define covariance as

   “ The covariance between random variable X and random variable Y denoted by cov(X,Y)  is defined as

gif

using definition of expectation and expanding we get

gif
gifgif

it is clear that if the random variables X and Y are independent then

gif
gif

but the converse is not true for example if

gif

and defining the random variable Y as

gif

so

gif

here clearly X and Y are not independent but covariance is zero.

Properties of covariance

  Covariance between random variables X and Y has some properties as follows

gif
gif
gif
gif

using the definition off the covariance the first three properties are immediate and the fourth property follows by considering

em%3E%7Bj%3D1%7D%5E%7Bm%7D%20Y %7Bj%7D%20%5Cright%20%5D%20%3D%5Csum %7Bj%3D1%7D%5E%7Bm%7D%20v %7Bj%7D

now by definition

covariance

Variance of the sums

The important result from these properties is

gif

as

gif
gif
gif
gif

If Xi ‘s are pairwise independent then

Example: Variance of a binomial random variable

  If X is the random variable

gif

where Xi are the independent Bernoulli random variables such that

gif

 then find the variance of a binomial random variable X with parameters n and p.

Solution:

since

gif
gif

so for single variable we have

gif
gif
gif

so the variance is

gif

Example

  For the independent random variables Xi with the respective means and variance and a new random variable with deviation as

gif

then compute

gif

solution:

By using the above property and definition we have

gif
gif
gif

now for the random variable S

COVARIANCE

take the expectation

gif

Example:

Find the covariance of indicator functions for the events A and B.

Solution:

for the events A and B the indicator functions are

gif
gif

so the expectation of these are

gif
gif
gif
gif

thus the covariance is

gif
B%29%20 %20P%28A%29%5D

Example:

     Show that

gif

where Xi are independent random variables with variance.

Solution:

The covariance using the properties and definition will be

gif
gif
gif
gif

Example:

  Calculate the mean and variance of random variable S which is the sum of n sampled values if set of N people each of whom has an opinion about a certain subject that is measured by a real number v that represents the person’s “strength of feeling” about the subject. Let  represent the strength of feeling of person  which is unknown, to collect information a sample of n from N is taken randomly, these n people are questioned and their feeling is obtained to calculate vi

Solution

let us define the indicator function as

gif

thus we can express S as

gif

and its expectation as

gif

this gives the variance as

gif
gif

since

gif
gif

we have

gif
gif
gif
gif
gif

we know the identity

gif

so

gif
gif
gif
gif

so the mean and variance for the said random variable will be

gif
gif

Conclusion:

The correlation between two random variables is defined as covariance and using the covariance the sum of the variance is obtained for different random variables, the covariance and different moments with the help of definition of expectation is obtained  , if you require further reading go through

https://en.wikipedia.org/wiki/Expectation

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH.

For more post on mathematics, please follow our Mathematics page