To install click the Add extension button. That's it.

The source code for the WIKI 2 extension is being checked by specialists of the Mozilla Foundation, Google, and Apple. You could also do it yourself at any point in time. 4,5
Kelly Slayton
Congratulations on this excellent venture… what a great idea!
Alexander Grigorievskiy
I use WIKI 2 every day and almost forgot how the original Wikipedia looks like.
Live Statistics
English Articles
Improved in 24 Hours
Languages
Recent
Show all languages
What we do. Every page goes through several hundred of perfecting techniques; in live mode. Quite the same Wikipedia. Just better.
.
Leo
Newton
Brights
Milds Poisson regression

In statistics, Poisson regression is a generalized linear model form of regression analysis used to model count data and contingency tables. Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters. A Poisson regression model is sometimes known as a log-linear model, especially when used to model contingency tables.

Negative binomial regression is a popular generalization of Poisson regression because it loosens the highly restrictive assumption that the variance is equal to the mean made by the Poisson model. The traditional negative binomial regression model, commonly known as NB2, is based on the Poisson-gamma mixture distribution. This model is popular because it models the Poisson heterogeneity with a gamma distribution.

Poisson regression models are generalized linear models with the logarithm as the (canonical) link function, and the Poisson distribution function as the assumed probability distribution of the response.

• 1/5
Views:
14 466
18 441
9 437
9 161
22 684
• ✪ Poisson regression models for count data; Gabriele Durrant (part 1 of 3)
• ✪ An Introduction to the Poisson Regression Model
• ✪ 03 03 Poisson regression, part 1 of 2
• ✪ Poisson Regression | Modelling Count Data | Statistical Models
• ✪ GLM in R: Poisson regression | crime data | fuller version

Transcription

So today I'm going to talk about Poisson regression models for count data. | I will first of all give a brief review of regression analysis. I will then introduce poisson regression and looking at a simple model without a covariate first of all the so called a equiprobable model. I will be then assessing this particular model with the pearson chi-squared test and the log likelihood ratio test statistics and also I will be looking at some residual analysis as well and then I will be introducing the Poisson regression model with a co-variate so basically a Poisson time trend model. | You may have come across different types of regression models already for example a linear regression model or a continuous dependent variable. You may have used logistic regression models already for a binary outcome variable. There are obviously other types of regression as well that are also part of the generalized linear models. So basically for example the multinomial logit model for a multi-category , un-ordered variable and also the sort of so called commutative logit for multi-category ordered variable so ordinal regression. | Here we are going to basically go step further. We are going to look at the outcome variable that is a count available using Poisson regression. Sometimes in the literature you may also find the expression of a log linear model. | Data for this particular session are assumed to be first of all a count variable Y so for example the number of accidents or the number of suicides in a particular geographical area or time period then we've got a categorical variable X for example which lets say a capital C possible categories such as days of the week or months. So basically Y here in this particular case has capital C possible outcomes so y 1 y 2 and so on until I yc . Obviously generally in Poisson regression modelling you may think of a number of categorical variables that you have or a number of even a continuous variables as explanatory variables in your models. Here we're going to start with something relatively simple. | Just to sort of introduce the basic principles of Poisson regression so basically it's a form of regression analysis here to model count data and in particular case if all the explanatory variables are categorical then we basically model a contingency table so basically cell counts. And the model basically models expected frequencies. The model specifies also how the count variable obviously relates to any of these explanatory variables or for example the level of the categorical variables. | Poisson models is a form of generalized linear modeling. It uses the logarithm the log as the canonical link function in this particular case. We basically assume that the outcome variable Y the the dependent variable the variable that we are particularly interested in has a Poisson distribution and the logarithm basically is its expected value that can be modeled by a linear combination of any of these unknown parameters so basically of this unknown beta coefficients the regression coefficients in your model. Sometimes it's referred to as a Log linear model in particular when used to model contingency tables. | Let's have a look at a brief example for example the number of suicides by weekday in France So we've got a number of week days in the first column and the second column just simply the frequencies the occurrences the events and then let's say the percentages how it is distributed according to two days of the week. | So that's the type of a model or the type of data that we would like to model. Let's first look at a very simple case the equiprobably model. The equiprobable model means that basically all outcomes are equally probable so they are equally likely that is for our particular example we assume a uniform distribution for the outcome across days of week so Y does not vary with the days of week X basically. | So the equiprobable model is basically given by this formula here: So the probability of a particular event across these categories basically of the days of the week is equally distributed so it's 1/C so 1 over the capital C so we basically expect an equal distribution across days of week. And given this particular data we can test then the assumption of our interest , basically the assumption of the equiprobable model so H0 that this assumption holds. So looking at our example again let's say suicides by week days in France. Basically H0 the assumption that we would like to test means that each day is equally likely for the suicides to happen that means the expected proportion of suicides is about 100 / 7 so 7 days of the week. So basically just over 14% per day and if you look at the third column of the table we see the actual observed distribution and obviously that depends a little bit on each day of the week possibly and diverges a little bit from 14% per day. But may be the divergence is not very much and we are satisfied with actually our assumption and to do that properly we obviously would need to do a formal test and I'll come to that in in the next session and i will explain the extra formal test in in further detail. | Looking at another example example 2 looking at traffic accidents per week again the amount to make the H0 assumption of the equiprobable model that means that each day is equally likely for an accident that means the expected proportion is again at the number of accidents is 100/7 so basically just over 14% per day we would expect. And there may be in this particular example we see a greater distribution in particular for Sunday that seems to be a greater percentage than in just fourteen percent. So you may want to continue testing if the observed distribution that we have is may be different from the expected distribution or if it's still ok to assume that they are actually equal. | Looking at hypothesis testing we may say in this particular case H0 that each day is equally likely for an accident to happen but we can also think of other alternative null hypothesis for example that each working day is equally likely for an accident or that maybe saturday sunday the weekends are equally likely for an accident. You could also think of course of other extra or additional variables for example the distance driven each day of the week and you may want to take into account those types of an explanatory variables as well. Just thinking about this a bit further, basically we cannot express the equiprobable model more formally as actual Poisson regression model without a covariate and that models the expected frequencies. So basically we assume a poisson distribution with parameter move for a random component that means the response variable Y follows a poisson distribution that means basically that that Y follows this notation here or this formula here using the exponential function and move the parameter of interest and also the Y the outcome variable of interest where Yis just simply the count variable 1 2 and 3 and so on. So basically Y is a random variable that takes on only positive integer values and also this Poisson distribution has only one single parameter µ which actually is the mean and the variance of this distribution. And we assume that our outcome follows i.e. this Poisson distribution follows the integer count distribution. Looking at basically that the simple model to start with, we aim to model the expected value of y and it can be shown that this is the parameter µ , hence we aim to model the parameter µ effectively in our Poisson model. So defining the equiprobable model that I had on an earlier slide and sort of the intuitive notation, I'm now formalising this writing it down as the expected value of y, the parameter µ and that is 1/ C because we are making the assumption of the equal probability across weekdays. Or using the link function the log of µ would then be a coefficient alpha so that is basically the coefficient that i would like to estimate as part of my model and alpha is then basically the log of 1/C in this particular case.

Regression models

If $\mathbf {x} \in \mathbb {R} ^{n}$ is a vector of independent variables, then the model takes the form

$\log(\operatorname {E} (Y\mid \mathbf {x} ))=\alpha +\mathbf {\beta } '\mathbf {x} ,$ where $\alpha \in \mathbb {R}$ and $\mathbf {\beta } \in \mathbb {R} ^{n}$ . Sometimes this is written more compactly as

$\log(\operatorname {E} (Y\mid \mathbf {x} ))={\boldsymbol {\theta }}'\mathbf {x} ,\,$ where x is now an (n + 1)-dimensional vector consisting of n independent variables concatenated to a vector of ones. Here θ is simply α concatenated to β.

Thus, when given a Poisson regression model θ and an input vector x, the predicted mean of the associated Poisson distribution is given by

$\operatorname {E} (Y\mid \mathbf {x} )=e^{{\boldsymbol {\theta }}'\mathbf {x} }.\,$ If Yi are independent observations with corresponding values xi of the predictor variables, then θ can be estimated by maximum likelihood. The maximum-likelihood estimates lack a closed-form expression and must be found by numerical methods. The probability surface for maximum-likelihood Poisson regression is always concave, making Newton–Raphson or other gradient-based methods appropriate estimation techniques.

Maximum likelihood-based parameter estimation

Given a set of parameters θ and an input vector x, the mean of the predicted Poisson distribution, as stated above, is given by

$\lambda :=\operatorname {E} (Y\mid x)=e^{\theta 'x},\,$ and thus, the Poisson distribution's probability mass function is given by

$p(y\mid x;\theta )={\frac {\lambda ^{y}}{y!}}e^{-\lambda }={\frac {e^{y\theta 'x}e^{-e^{\theta 'x}}}{y!}}$ Now suppose we are given a data set consisting of m vectors $x_{i}\in \mathbb {R} ^{n+1},\,i=1,\ldots ,m$ , along with a set of m values $y_{1},\ldots ,y_{m}\in \mathbb {N}$ . Then, for a given set of parameters θ, the probability of attaining this particular set of data is given by

$p(y_{1},\ldots ,y_{m}\mid x_{1},\ldots ,x_{m};\theta )=\prod _{i=1}^{m}{\frac {e^{y_{i}\theta 'x_{i}}e^{-e^{\theta 'x_{i}}}}{y_{i}!}}.$ By the method of maximum likelihood, we wish to find the set of parameters θ that makes this probability as large as possible. To do this, the equation is first rewritten as a likelihood function in terms of θ:

$L(\theta \mid X,Y)=\prod _{i=1}^{m}{\frac {e^{y_{i}\theta 'x_{i}}e^{-e^{\theta 'x_{i}}}}{y_{i}!}}.$ Note that the expression on the right hand side has not actually changed. A formula in this form is typically difficult to work with; instead, one uses the log-likelihood:

$\ell (\theta \mid X,Y)=\log L(\theta \mid X,Y)=\sum _{i=1}^{m}\left(y_{i}\theta 'x_{i}-e^{\theta 'x_{i}}-\log(y_{i}!)\right).$ Notice that the parameters θ only appear in the first two terms of each term in the summation. Therefore, given that we are only interested in finding the best value for θ we may drop the yi! and simply write

$\ell (\theta \mid X,Y)=\sum _{i=1}^{m}\left(y_{i}\theta 'x_{i}-e^{\theta 'x_{i}}\right).$ To find a maximum, we need to solve an equation ${\frac {\partial \ell (\theta \mid X,Y)}{\partial \theta }}=0$ which has no closed-form solution. However, the negative log-likelihood, $-\ell (\theta \mid X,Y)$ , is a convex function, and so standard convex optimization techniques such as gradient descent can be applied to find the optimal value of θ.

Poisson regression in practice

Poisson regression may be appropriate when the dependent variable is a count, for instance of events such as the arrival of a telephone call at a call centre. The events must be independent in the sense that the arrival of one call will not make another more or less likely, but the probability per unit time of events is understood to be related to covariates such as time of day.

"Exposure" and offset

Poisson regression may also be appropriate for rate data, where the rate is a count of events divided by some measure of that unit's exposure (a particular unit of observation). For example, biologists may count the number of tree species in a forest: events would be tree observations, exposure would be unit area, and rate would be the number of species per unit area. Demographers may model death rates in geographic areas as the count of deaths divided by person−years. More generally, event rates can be calculated as events per unit time, which allows the observation window to vary for each unit. In these examples, exposure is respectively unit area, person−years and unit time. In Poisson regression this is handled as an offset, where the exposure variable enters on the right-hand side of the equation, but with a parameter estimate (for log(exposure)) constrained to 1.

$\log(\operatorname {E} (Y\mid x))=\log({\text{exposure}})+\theta 'x$ which implies

$\log(\operatorname {E} (Y\mid x))-\log({\text{exposure}})=\log \left({\frac {\operatorname {E} (Y\mid x)}{\text{exposure}}}\right)=\theta 'x$ Offset in the case of a GLM in R can be achieved using the offset() function:

glm(y ~ offset(log(exposure)) + x, family=poisson(link=log) )


Overdispersion and zero inflation

A characteristic of the Poisson distribution is that its mean is equal to its variance. In certain circumstances, it will be found that the observed variance is greater than the mean; this is known as overdispersion and indicates that the model is not appropriate. A common reason is the omission of relevant explanatory variables, or dependent observations. Under some circumstances, the problem of overdispersion can be solved by using quasi-likelihood estimation or a negative binomial distribution instead.

Ver Hoef and Boveng described the difference between quasi-Poisson (also called overdispersion with quasi-likelihood) and negative binomial (equivalent to gamma-Poisson) as follows: If E(Y) = μ, the quasi-Poisson model assumes var(Y) = θμ while the gamma-Poisson assumes var(Y) = μ(1 + κμ), where θ is the quasi-Poisson overdispersion parameter, and κ is the shape parameter of the negative binomial distribution. For both models, parameters are estimated using Iteratively reweighted least squares. For quasi-Poisson, the weights are μ/θ. For negative binomial, the weights are μ/(1 + κμ). With large μ and substantial extra-Poisson variation, the negative binomial weights are capped at 1/κ. Ver Hoef and Boveng discussed an example where they selected between the two by plotting mean squared residuals vs. the mean.

Another common problem with Poisson regression is excess zeros: if there are two processes at work, one determining whether there are zero events or any events, and a Poisson process determining how many events there are, there will be more zeros than a Poisson regression would predict. An example would be the distribution of cigarettes smoked in an hour by members of a group where some individuals are non-smokers.

Other generalized linear models such as the negative binomial model or zero-inflated model may function better in these cases.

Use in survival analysis

Poisson regression creates proportional hazards models, one class of survival analysis: see proportional hazards models for descriptions of Cox models.

Extensions

Regularized Poisson regression

When estimating the parameters for Poisson regression, one typically tries to find values for θ that maximize the likelihood of an expression of the form

$\sum _{i=1}^{m}\log(p(y_{i};e^{\theta 'x_{i}})),$ where m is the number of examples in the data set, and $p(y_{i};e^{\theta 'x_{i}})$ is the probability mass function of the Poisson distribution with the mean set to $e^{\theta 'x_{i}}$ . Regularization can be added to this optimization problem by instead maximizing

$\sum _{i=1}^{m}\log(p(y_{i};e^{\theta 'x_{i}}))-\lambda \left\|\theta \right\|_{2}^{2},$ for some positive constant $\lambda$ . This technique, similar to ridge regression, can reduce overfitting.