The best line minimizes the residual sum of squares

\[ RSS=\sum\limits_{i=1}^n(Y_i - \hat{Y}_i)^2\]

We could do a Monte Carlo approach: try a bunch of slopes passing through \((\bar{X},\bar{Y})\) and calculate the RSS, then pick the smallest, but math offers a simpler solution.

Recall the sum of squares: \[\ SS_Y = \sum\limits_{i=1}^n(Y_i - \bar{Y})^2\]

Sample variance: \[\ s^2_Y=\frac{\sum\limits_{i=1}^n(Y_i - \bar{Y})^2}{n-1}\]

SS equivalent to: \[\ SS_Y = \sum\limits_{i=1}^n(Y_i - \bar{Y})(Y_i - \bar{Y})\]

With 2 variables, we can define sum of cross products \[\ SS_{XY} = \sum\limits_{i=1}^n(X_i - \bar{X})(Y_i - \bar{Y})\]

By analogy to the sample variance, we define sample covariance \[\ s_{XY} = \frac{\sum\limits_{i=1}^n(X_i - \bar{X})(Y_i - \bar{Y})}{(n-1)}\]

By hand on whiteboard

x <- c(2 , 4, 3)
y <- c(1, 5, 6)

In ordinary least squares regression (OLS), the slope of the best fit line is defined as: \[ \frac{covariance\ of\ XY}{variance\ of\ X}\]

Or: \[\hat{\beta}_1 = \frac{s_{XY}}{s^2_X} = \frac{SS_{XY}}{SS_X}\] note: this simplifies because both are divided by same denomenator

The intercept is easy to find, because the best fit line passes through \((\bar{X},\bar{Y})\). Thus: \[ \bar{Y} = \hat{\beta}_0 + \hat{\beta}_1 \bar{X} \]

Or:

\[ \hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}\]

Recall: \[Y_i = \beta_0 + \beta_1X_i + \epsilon_i\]

Regression assumes that that \(\epsilon\) is a random normal variable with mean 0 and variance of \(\sigma^2\), which is related to the scatter around the line.

We estimate \(\sigma^2\) like this: \[\frac{RSS}{n-2}\]

The square root of this is standard error of regression: \[\hat{\sigma}^2=\sqrt{\frac{RSS}{n-2}}\]

In a linear relationship, the total variance we want to explain is \(SS_Y\).

Some variance is attributable to our error term (measured by \(RSS\)).

The remaining variance is explained by our regression, thus: \[SS_{reg}=SS_Y-RSS\]

Therefore: \[SS_Y=SS_{reg}+RSS\]

This is referred to as partitioning a sum of squares.

This leads to calculation of \(r^2\), AKA the coefficient of determination

\[r^2 = \frac{SS_{reg}}{SS_Y}=\frac{SS_{reg}}{SS_{reg} + RSS}\]

The square root of this value is known as \(r\) or the product-moment correlation coefficient.

Note: the sign of \(r\) comes from the sign of the slope of the line.

You will always get parameter estimates for the intercept and slope. The next question is: are they signficant?

The slope measures the strength of the effect of \(X\) on \(Y\). The slope is a measure of effect size

• \(\bar{Y}\) = mean of Y
• \(\hat{Y}_i\) = predicted value from regression
• \(Y_i\) = value of Y

We can also construct confidence intervals around our parameter estimates (formulae not shown).

Notice that our intervals are narrowest near the mean, and fatter the further you go away from mean.

plot(lm(response~predictor))

Any systematic correlation between the residuals and the fitted is bad.

Systematic departures from linearity can show up in the residual plots

response <- predictor^2 + rnorm(1000, sd=50)
plot(lm(response~predictor))

Model II regression requires a separate package lmodel2

In bioanthro literature, this regression is known as RMA (reduced major axis), but known in this package as MA (RMA in this package is something else)

The function is different, but the call is still simple.

We are going to look at the scaling relationship between overall size and BBH (basion-bregma-height) in humans and fossil hominins

• heads - Howell Craniometric Data
• fossils - Fossil Data

Calculate the geomean for each individual in both data sets using the variables GOL, BBH, XCB, BNL, BPL, ASB

Add this data to a new column called geomean

geomean <- function(x, na.rm = TRUE) {
  product <- prod(x, na.rm = TRUE)
  n <- sum(!is.na(x))
  geo <- product ^ (1/n)
  return(geo)
  }

Do an Ordinary Least Squares regression of log(BBH) as a function of log(geomean) for the modern data and a separate one for the fossils

Make a plot of log(BBH) as a function of log(geomean)

• plot the modern humans
  • and add the modern human regression line to the plot
• add a new layer with the fossil data, being sure to differentiate them visually
  • add the fossil regression line to the plot, using a different kind of line (e.g. dashed, or a different color)

How do the fossils compare to the modern humans?

How does LB1 (H. floresiensis) compare to other species?