Gamma Distribution Exponential Family: 21 Important Facts

Content

  1. Special form of Gamma distributions and relationships of Gamma distribution
  2. Gamma distribution exponential family
  3. Relationship between gamma and normal distribution
  4. Poisson gamma distribution | poisson gamma distribution negative binomial
  5. Weibull gamma distribution
  6. Application of gamma distribution in real life | gamma distribution uses | application of gamma distribution in statistics 
  7. Beta gamma distribution | relationship between gamma and beta distribution
  8. Bivariate gamma distribution
  9. Double gamma distribution
  10. Relation between gamma and exponential distribution | exponential and gamma distribution | gamma exponential distribution
  11. Fit gamma distribution
  12. Shifted gamma distribution
  13. Truncated gamma distribution
  14. Survival function of gamma distribution
  15. MLE of gamma distribution | maximum likelihood gamma distribution | likelihood function of gamma distribution
  16. Gamma distribution parameter estimation method of moments | method of moments estimator gamma distribution
  17. Confidence interval for gamma distribution
  18. Gamma distribution conjugate prior for exponential distribution | gamma prior distribution | posterior distribution poisson gamma
  19. Gamma distribution quantile function
  20. Generalized gamma distribution
  21. Beta generalized gamma distribution

Special form of Gamma distributions and relationships of Gamma distribution

  In this article we will discuss the special forms of gamma distributions and the relationships of gamma distribution with different continuous and discrete random variables also some estimation methods  in sampling of population using gamma distribution is briefly discuss.

Gamma distribution exponential family

  The gamma distribution exponential family and it is two parameter exponential family which is largely and applicable family of distribution as most of real life problems can be modelled in the gamma distribution exponential family and the quick and useful calculation  within the exponential family can be done easily, in the two parameter if we take probability density function as

x%7Dx%5E%7B%5Calpha%20

if we restrict the known value of α (alpha) this two parameter family will reduce to one parameter exponential family

x%7D a%20%5C%20%5C%20log%5Clambda%20%5Cfrac%7Bx%5E%7B%5Calpha%20

and for λ (lambda)

gif

Relationship between gamma and normal distribution

  In the probability density function of gamma distribution if we take alpha nearer to 50 we will get the nature of density function as

Gamma distribution exponential family
Gamma distribution exponential family

even the shape parameter in gamma distribution we are increasing which is resulting in similarity of normal distribution normal curve, if we tend shape parameter alpha tends to infinity the gamma distribution will be more symmetric and normal but as alpha tends to infinity value of x in gamma distribution will tends to minus infinity which result in semi infinite support of gamma distribution infinite hence even gamma distribution becomes symmetric but not same with normal distribution.

poisson gamma distribution | poisson gamma distribution negative binomial

   The poisson gamma distribution and binomial distribution are the discrete random variable whose random variable deals with the discrete values specifically success and failure in the form of Bernoulli trials which gives random success or failure as a result only, now the mixture of Poisson and gamma distribution also known as negative binomial distribution is the outcome of the repeated trial of Bernoulli’s trial, this can be parameterize in different way as if r-th success occurs in number of trials then it can be parameterize as

gif

and if the number of failures before the r-th success then it can be parameterize as

gif

and considering the values of r and p

gif
gif

the general form of the parameterization for the negative binomial or poisson gamma distribution is

gif.latex?P%28X%3Dx%29%3D%5Cbinom%7Bx+r 1%7D%7Bx%7Dp%5E%7Br%7D%281

and alternative one is

gif.latex?P%28X%3Dx%29%3D%5Cbinom%7Bx+r

this binomial distribution is known as negative because of the coefficient

gif.latex?%5Cbinom%7Bx+r 1%7D%7Bx%7D%20%3D%5Cfrac%7B%28x+r 1%29%28x+r 2%29...r%7D%7Bx%21%7D%20%5C%20%3D%20%28 1%29%5E%7Bx%7D%5Cfrac%7B%28 r %28x 1%29%29%28 r %28x 2%29%29...%28 r%29%7D%7Bx%21%7D%20%5C%20%3D%20%28 1%29%5E%7Bx%7D%5Cfrac%7B%28 r%29%28 r 1%29..

and this negative binomial or poisson gamma distribution is well define as the total probability we will get as one for this distribution

gif

The mean and variance for this negative binomial or poisson gamma distribution is

gif
gif

the poisson and gamma relation we can get by the following calculation

%5Cbeta%20%7D%20d%5Clambda
%5Cbeta%20%29%7Dd%5Clambda
gif
gif

Thus negative binomial is the mixture of poisson and gamma distribution and this distribution is used in day to day problems modelling where discrete and continuous mixture we require.

Gamma distribution exponential family
Gamma distribution exponential family

Weibull gamma distribution

   There are generalization of exponential distribution which involve Weibull as well as gamma distribution as the Weibull distribution has the probability density function as

gif

and cumulative distribution function as

gif

where as pdf and cdf of gamma distribution is already we discussed above the main connection between Weibull and gamma distribution is both are generalization of exponential distribution the difference between them is when power of variable is greater than one then Weibull distribution gives quick result while for less than 1 gamma gives quick result.

     We will not discuss here generalized Weibull gamma distribution that require separate discussion.

application of gamma distribution in real life | gamma distribution uses | application of gamma distribution in statistics 

  There are number of  application where gamma distribution is used to model the situation such as insurance claim to aggregate, rainfall amount accumulation, for any product its manufacturing and distribution, the crowd on specific web,  and in telecom exchange etc. actually the gamma distribution give the wait time prediction till next event for nth event. There are number of application of gamma distribution in real life.

beta gamma distribution | relationship between gamma and beta distribution

    The beta distribution is the random variable with the probability density function

gif

where

gif

which has the relationship with gamma function as

gif

and beta distribution related to gamma distribution as if X be gamma distribution with parameter alpha and beta as one and Y be the gamma distribution with parameter alpha as one and beta then the random variable X/(X+Y) is beta distribution.

or If X is Gamma(α,1) and Y is Gamma (1, β) then the random variable X/(X+Y) is Beta (α, β) 

and also

gif

bivariate gamma distribution

     A two dimensional or bivariate random variable is continuous if there exists a function f(x,y) such that the joint distribution function

gif

where

gif
gif

and the joint probability density function obtained by

gif

there are number of bivariate gamma distribution one of them is the bivariate gamma distribution with probability density function as

gif

double gamma distribution

  Double gamma distribution is one of the bivariate distribution with gamma random variables having parameter alpha and one with joint probability density function as

em%3E%7B2%7D%29%7Dy %7B1%7D%5E%7B%5Calpha %7B1%7D%20 1%7Dy %7B2%7D%5E%7B%5Calpha %7B2%7D%20 1%7D%20exp%28 y %7B1%7D%20 y %7B2%7D%29%2C%20y %7B1%7D%26gt%3B%200%2C%20y %7B2%7D%26gt%3B%200

this density forms the double gamma distribution with respective random variables and the moment generating function for double gamma distribution is

em%3E%7B2%7D%7D%20%7D

relation between gamma and exponential distribution | exponential and gamma distribution | gamma exponential distribution

   since the exponential distribution is the distribution with the probability density function

and the gamma distribution has the probability density function

clearly the value of alpha if we put as one we will get the exponential distribution, that is the gamma distribution is nothing but the generalization of the exponential distribution, which predict the wait time till the occurrence of next nth event while exponential distribution predict the wait time till the occurrence of the next event.

fit gamma distribution

   As far as fitting the given data in the form of gamma distribution imply finding the two parameter probability density function which involve shape, location and scale parameters so finding these parameters with different application and calculating the mean, variance, standard deviation and moment generating function is the fitting of gamma distribution, since different real life problems will be modelled in gamma distribution so the information as per situation must be fit in gamma distribution for this purpose various technique in various environment is already there e.g in R, Matlab, excel etc.

shifted gamma distribution

     There are as per application and need whenever the requirement of shifting the distribution required from two parameter gamma distribution the new generalized three parameter or any another generalized gamma distribution shift the shape location and scale , such gamma distribution is known as shifted gamma distribution

truncated gamma distribution

     If we restrict the range or domain of the gamma distribution for the shape scale and location parameters the restricted gamma distribution is known as truncated gamma distribution based on the conditions.

survival function of gamma distribution

                The survival function for the gamma distribution is defined the function s(x) as follows

gif

mle of gamma distribution | maximum likelihood gamma distribution | likelihood function of gamma distribution

we know that the maximum likelihood take the sample from the population as a representative and this sample consider as an estimator for the probability density function to maximize for the parameters of density function, before going to gamma distribution recall some basics as for the random variable X the probability density function with theta as parameter has likelihood function as

this we can express as

and method of maximizing this likelihood function can be

if such theta satisfy this equation, and as log is monotone function we can write in terms of log

and such a supremum exists if

em%3E%7Bk%7D%29

now we apply the maximum likelihood for the gamma distribution function as

gif

the log likelihood of the function will be

gif

so is

gif

and hence

gif

This can be achieved also as

gif.latex?%5Ctextbf%7BL%7D%28%5Calpha%20%2C%5Cbeta%20%7C%20x%29%3D%5Cleft%20%28%20%5Cfrac%7B%5Cbeta%20%5E%7B%5Calpha%20%7D%7D%7B%5CGamma%20%28%5Calpha%20%29%7D%20x %7B1%7D%5E%7B%5Calpha%20 1%7D%20e%5E%7B %5Cbeta%20x %7B1%7D%7D%20%5Cright%20%29...%5Cleft%20%28%20%5Cfrac%7B%5Cbeta%20%5E%7B%5Calpha%20%7D%7D%7B%5CGamma%20%28%5Calpha%20%29%7D%20x %7Bn%7D%5E%7B%5Calpha%20 1%7D%20e%5E%7B %5Cbeta%20x %7Bn%7D%7D%20%5Cright%20%29%20%3D%5Cleft%20%28%20%5Cfrac%7B%5Cbeta%20%5E%7B%5Calpha%20%7D%7D%7B%5CGamma%20%28%5Calpha%20%29%7D%20%5Cright%29%5E%7Bn%7D%20%28x %7B1%7D%20%28x %7B2%7D...%28x %7Bn%7D%29%5E%7B%5Calpha%20 1%7D%20e%5E%7B

by

gif

and the parameter can be obtained by differentiating

gif
gif
gif

gamma distribution parameter estimation method of moments | method of moments estimator gamma distribution

   We can calculate the moments of the population and sample with the help of expectation of nth order respectively, the method of moment equates these moments of distribution and sample to estimate the parameters, suppose we have sample of gamma random variable with the probability density function as

gif

we know the first tow moments for this probability density function is

em%3E%7B2%7D%3D%5Cfrac%7B%5Calpha%20%28%5Calpha%20+1%29%20%7D%7B%5Clambda%20%5E%7B2%7D%7D

so

gif

we will get from the second moment if we substitute lambda

em%3E%7B1%7D%5E%7B2%7D%7D%3D%5Cfrac%7B%5Calpha%20+1%7D%7B%5Calpha%20%7D

and from this value of alpha is

em%3E%7B2%7D %5Cmu%20 %7B1%7D%5E%7B2%7D%7D

and now lambda will be

em%3E%7B2%7D %5Cmu%20 %7B1%7D%5E%7B2%7D%7D

and moment estimator using sample will be

gif

confidence interval for gamma distribution

   confidence interval for gamma distribution is the way to estimate the information and its uncertainty which tells the interval is expected to have the true value of the parameter at what percent, this confidence interval is obtained from the observations of random variables, since it is obtained from random it itself is random to get the confidence interval for the gamma distribution there are different techniques in different application that we have to follow.

gamma distribution conjugate prior for exponential distribution | gamma prior distribution | posterior distribution poisson gamma

     The posterior and prior distribution  are the terminologies of Bayesian probability theory and they are conjugate to each other, any two distributions are conjugate if the posterior of one distribution is another distribution, in terms of theta let us show that gamma distribution is conjugate prior to the exponential distribution

if the probability density function of gamma distribution in terms of theta is as

gif

assume the distribution function for theta is exponential from given data

gif

so the joint distribution will be

gif

and using the relation

gif

we have

gif
gif
gif

which is

gif

so gamma distribution is conjugate prior to exponential distribution as posterior is gamma distribution.

gamma distribution quantile function

   Qauntile function of gamma distribution will be the function that gives the points in gamma distribution which relate the rank order of the values in gamma distribution, this require cumulative distribution function and for different language different algorithm and functions for the quantile of gamma distribution.

generalized gamma distribution

    As gamma distribution itself is the generalization of exponential family of distribution adding more parameters to this distribution gives us generalized gamma distribution which is the further generalization of this distribution family, the physical requirements gives different generalization one of the frequent one is using the probability density function as

gif

the cumulative distribution function for such generalized gamma distribution can be obtained by

gif

where the numerator represents the incomplete gamma function as

em%3E%7B0%7D%5E%7B%5Cinfty%7Dt%5E%7Ba 1%7De%5E%7B t%7Ddt

using this incomplete gamma function the survival function for the generalized gamma distribution can be obtained as

gif

another version of this three parameter generalized gamma distribution having probability density function is

gif

where k, β, θ are the parameters greater than zero, these generalization has convergence issues to overcome the Weibull parameters replaces

using this parameterization the convergence of the density function obtained so the more generalization for the gamma distribution with convergence is the distribution with probability density function as

gif.latex?F%28x%29%20%3D%20%5Cbegin%7Bcases%7D%20%5Cfrac%7B%7C%5Clambda%20%7C%7D%7B%5Csigma%20.t%7D.%5Cfrac%7B1%7D%7B%5CGamma%20%5Cleft%20%28%20%5Cfrac%7B1%7D%7B%5Clambda%20%5E%7B2%7D%7D%20%5Cright%20%29%7D.e%5Cleft%20%5B%20%5Cfrac%7B%5Clambda%20.%5Cfrac%7BIn%28t%29 %5Cmu%20%7D%7B%5Csigma%20%7D+In%5Cleft%20%28%20%5Cfrac%7B1%7D%7B%5Clambda%20%5E%7B2%7D%7D%20%5Cright%20%29 e%5E%7B%5Clambda.%5Cfrac%7BIn.%28t%29

Beta generalized gamma distribution

   The gamma distribution involving the parameter beta in the density function because of which sometimes gamma distribution is known as the beta generalized gamma distribution with the density function

gif
gif

with cumulative distribution function as

gif

which is already discussed in detail in the discussion of gamma distribution, the further beta generalized gamma distribution is defined with the cdf as

gif

where B(a,b) is the beta function , and the probability density function for this can be obtained by differentiation and the density function will be

gif

here the G(x) is the above defined cumulative distribution function of gamma distribution, if we put this value then the cumulative distribution function of beta generalized gamma distribution is

%5CGamma%20%28%5Cbeta%20%29%7D%7D%5Comega%20%5E%7Ba 1%7D%20%281 %5Comega%20%29%5E%7Bb 1%7D%20d%5Comega

and the probability density function

gif

the remaining properties can be extended for this beta generalized gamma distribution with usual definitions.

Conclusion:

There are different forms and generalization of gamma distribution and Gamma distribution exponential family as per the real life situations so possible such forms and generalizations were covered in addition with the estimation  methods of gamma distribution in population sampling of information, if you require further reading on Gamma distribution exponential family, please go through below link and books. For more topics on Mathematics please visit our page.

https://en.wikipedia.org/wiki/Gamma_distribution

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH