4.1 Basic linear model

Mathematical representation

 ${y}_{i}={\mathbf{x}}_{i}^{\prime }\beta +{𝜀}_{i},\phantom{\rule{2em}{0ex}}\phantom{\rule{1em}{0ex}}{𝜀}_{i}\sim \mathrm{N}\left(0,\frac{1}{\tau }\right)$ (4.1)
• the model is estimated using $N$ observations
• ${y}_{i}$ is the value of the dependent variable for observation $i$
• ${\mathbf{x}}_{i}$ is a $K\phantom{\rule{0.3em}{0ex}}×\phantom{\rule{0.3em}{0ex}}1$ vector that stores the values of the $K$ independent variables for observation $i$
• $\beta$ is a $K\phantom{\rule{0.3em}{0ex}}×\phantom{\rule{0.3em}{0ex}}1$ vector of parameters
• $\tau$ is the precision of the error term: ${\sigma }_{𝜀}^{2}=\frac{1}{\tau }$

Priors

 Parameter Probability density function Default hyperparameters $\beta$ $p\left(\beta \right)=\frac{|\mathbf{P}{|}^{1∕2}}{{\left(2\pi \right)}^{K∕2}}exp\left\{-\frac{1}{2}{\left(\beta -\mathbf{m}\right)}^{\prime }\mathbf{P}\left(\beta -\mathbf{m}\right)\right\}$ $\mathbf{m}={\mathbf{0}}_{K}$, $\mathbf{P}=0.001\cdot {\mathbf{I}}_{K}$ $\tau$ $p\left(\tau \right)=\frac{{b}_{\tau }^{{a}_{\tau }}}{\Gamma \left({a}_{\tau }\right)}{\tau }^{{a}_{\tau }-1}{e}^{-\tau {b}_{\tau }}$ ${a}_{\tau }=0.001$, ${b}_{\tau }=0.001$

Syntax

$\left[$<model name> = $\right]$ lm( y ~ x1 x2  xK $\left[$, <options> $\right]$ );

where:

• y is the dependent variable name, as it appears in the dataset used for estimation
• x1 x2 $\dots$xK is a list of the $K$ independent variable names, as they appear in the dataset used for estimation; when a constant term is to be included in the model, this must be requested explicitly

The optional arguments for the simple linear model are:1

 Gibbs parameters "chains" number of chains to run in parallel (positive integer); the default value is 1 "burnin" number of burn-in draws per chain (positive integer); the default value is 10000 "draws" number of retained draws per chain (positive integer); the default value is 20000 "thin" value of the thinning parameter (positive integer); the default value is 1 "seed" value of the seed for the random-number generator (positive integer); the default value is 42 Hyperparameters "m" mean vector of the prior for $\beta$ ($K\phantom{\rule{0.3em}{0ex}}×\phantom{\rule{0.3em}{0ex}}1$ vector); the default value is ${\mathbf{0}}_{K}$ "P" precision matrix of the prior for $\beta$ ($K\phantom{\rule{0.3em}{0ex}}×\phantom{\rule{0.3em}{0ex}}K$ symmetric and positive-deﬁnite matrix); the default value is $0.001\phantom{\rule{0.3em}{0ex}}\cdot \phantom{\rule{0.3em}{0ex}}{\mathbf{I}}_{K}$ "a_tau" shape parameter of the prior for $\tau$ (positive number); the default value is $0.001$ "b_tau" rate parameter of the prior for $\tau$ (positive number); the default value is $0.001$ Dataset and log-marginal likelihood "dataset" the id value of the dataset that will be used for estimation; the default value is the ﬁrst dataset in memory (in alphabetical order) "logML_CJ" boolean indicating whether the Chib (1995)/Chib & Jeliazkov (2001) approximation to the log-marginal likelihood should be calculated (true$|$false); the default value is false

Reported Parameters

 $\beta$ variable_name vector of parameters associated with the independent variables $\tau$ tau precision parameter of the error term, ${𝜀}_{i}$ ${\sigma }_{𝜀}$ sigma_e standard deviation of the error term: ${\sigma }_{𝜀}=1∕{\tau }^{1∕2}$

Stored values and post-estimation analysis
If a left-hand-side id value is provided when a simple linear model is created, then the following results are saved in the model item and are accessible via the ‘.’ operator:

 Samples a matrix containing the draws from the posterior of $\beta$ and $\tau$ x1,$\dots$,xK vectors containing the draws from the posterior of the parameters associated with variables x1,$\dots$,xK (the names of these vectors are the names of the variables that were included in the right-hand side of the model) tau vector containing the draws from the posterior of $\tau$ logML the Lewis & Raftery (1997) approximation of the log-marginal likelihood logML_CJ the Chib (1995)/Chib & Jeliazkov (2001) approximation to the log-marginal likelihood; this is available only if the model was estimated with the "logML_CJ"=true option nchains the number of chains that were used to estimate the model nburnin the number of burn-in draws per chain that were used when estimating the model ndraws the total number of retained draws from the posterior ($=$chains $\cdot$ draws) nthin value of the thinning parameter that was used when estimating the model nseed value of the seed for the random-number generator that was used when estimating the model

Additionally, the following functions are available for post-estimation analysis (see section B.14):

• diagnostics()
• test()
• pmp()

Examples

Example 1

myData = import("$BayESHOME/Datasets/dataset1.csv"); myData.constant = ones(rows(myData), 1); lm( y ~ constant x1 x2 x3); Example 2 myData = import("$BayESHOME/Datasets/dataset1.csv");
myData.constant = ones(rows(myData), 1);

myModel = lm(y ~ constant x1 x2 x3,
"m"=ones(4,1), "P" = 0.1*eye(4,4),
"a_tau"=0.01, "b_tau"=0.01,
"burnin"=10000, "draws"=40000, "thin"=4, "chains"=2,
"logML_CJ" = true, "dataset"=myData);

diagnostics("model"=myModel);

plot(myModel.tau,
"title"="draws from the posterior of tau",
"grid"="on");

1Optional arguments are always given in option-value pairs (eg. "chains"=3).