Niklas Inde

Welcome to Regression

To begin we need to understand that the different distributions ask different questions.

Simple linear regression and multivariable regression we ask to have the error (or the residuals) to normally distributed with $N(0,\sigma^2)$.
So in simple linear regression we just ask the line to be in the center of the data. And in multivariable linear regression we just span this thought to higher dimensions.

Simple Linear regression
Multivariable Regression

In logistic regression we ask to approximate the $\lambda$ in where instead of the $Y_i \sim N(\alpha+\beta x_i,\sigma^2)$, it has the distribution $Y_i \sim Pois(\lambda_i)$, where $lambda_i$ is something you can find in the link below.

Logistic Regression (also referred to as a binomial regression with log it link function)

Poisson Regression
Tree regression Link to another website

Common distributions with typical uses and canonical link functions

Distribution	Support of distribution	Typical uses	Link name	Link function	Mean function
Normal	real: ${\displaystyle (-\infty ,+\infty )}$	Linear-response data	Identity	${\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\mu \,\!}$	${\displaystyle \mu =\mathbf {X} {\boldsymbol {\beta }}\,\!}$
Exponential	real: ${\displaystyle (0,+\infty )}$	Exponential-response data, scale parameters	Inverse	${\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\mu ^{-1}\,\!}$	${\displaystyle \mu =(\mathbf {X} {\boldsymbol {\beta }})^{-1}\,\!}$
Gamma
Inverse Gaussian	real: ${\displaystyle (0,+\infty )}$		Inverse squared	${\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\mu ^{-2}\,\!}$	${\displaystyle \mu =(\mathbf {X} {\boldsymbol {\beta }})^{-1/2}\,\!}$
Poisson	integer: ${\displaystyle 0,1,2,\ldots }$	count of occurrences in fixed amount of time/space	Log	${\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\ln {(\mu )}\,\!}$	${\displaystyle \mu =\exp {(\mathbf {X} {\boldsymbol {\beta }})}\,\!}$
Bernoulli	integer: ${\displaystyle \{0,1\}}$	outcome of single yes/no occurrence	Logit	${\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\ln {\left({\frac {\mu }{1-\mu }}\right)}\,\!}$	${\displaystyle \mu ={\frac {\exp {(\mathbf {X} {\boldsymbol {\beta }})}}{1+\exp {(\mathbf {X} {\boldsymbol {\beta }})}}}={\frac {1}{1+\exp {(-\mathbf {X} {\boldsymbol {\beta }})}}}\,\!}$
Binomial	integer: ${\displaystyle 0,1,\ldots ,N}$	count of # of "yes" occurrences out of N yes/no occurrences
Categorical	integer: ${\displaystyle [0,K)}$	outcome of single K-way occurrence
	K-vector of integer: ${\displaystyle [0,1]}$, where exactly one element in the vector has the value 1
Multinomial	K-vector of integer: ${\displaystyle [0,N]}$	count of occurrences of different types (1 .. K) out of N total K-way occurrences