---
title: "MATH-650 Assignment 8"
author: 'Saket Choudhary (USCID: 2170058637) (skchoudh@usc.edu)'
date: "09/28/2015"
output:
  pdf_document: default
  html_document:
    keep_md: yes
  word_document: default
---

## Chapter 11: 10

```{r}
library(ggplot2)
data <- read.csv('data_ch9_16.csv', header=T)
data$PollenRemovedlogit = log(data$PollenRemoved/(1-data$PollenRemoved))
data$DurationOfVisitlog = log(data$DurationOfVisit)
ggplot(data, aes(x=DurationOfVisitlog, y=PollenRemovedlogit, color=BeeType)) +
    geom_point(shape=1) +
    scale_colour_hue(l=50) + 
    geom_smooth(method=lm,  
                se=FALSE)
```

\begin{align*}
\mu\{PollenRemovedLogit| DuratioOfVisitlog, BeeType \} &= \beta_0 + \beta_1 DurationOfVisitlog\\ 
&+ \beta_2 BeeType + \beta_3 BeeType*DurationOfVisitlog
\end{align*}


```{r}
lmfit <- lm(PollenRemovedlogit ~ BeeType + DurationOfVisitlog 
            + BeeType*DurationOfVisitlog, data=data)
summary(lmfit)
r <- residuals(lmfit)
yh <- predict(lmfit)

p1<-ggplot(lmfit, aes(.fitted, .resid))+geom_point()
p1 <- p1 +geom_hline(yintercept=0)+geom_smooth() + 
geom_text(aes(label=ifelse((.resid>4*IQR(.resid)|.fitted>4*IQR(.fitted)),paste('', "\n", .fitted, ",", .resid),"")), hjust=1.1)
p1
```

From the residual plot, there seem to be no outliers(see outlier detection part in the last code chunk wheere a outlier is defined if it is greater than 4*IQR(x)).

Also the p-value of cross interaction term $BeeTypeWorker:DurationofVisitlog$ is 0.342 and hence at a significance level of 0.05 can be safely neglected.



## Chapter 11: 21

$$ SS(\beta_0,\beta_1\dots \beta_n) = \sum_{i=1}^Nw_i(Y_i-\beta_0-\beta_1X_{1i}-\beta_2X_{2i}-\cdots-\beta_pX_{pi})^2 $$

\begin{align*}
\frac{\partial SS}{\partial \beta_0} &= 2 \sum_{i=1}^Nw_i(Y_i-\beta_0-\beta_1X_{1i}-\beta_2X_{2i}-\cdots-\beta_pX_{pi})\times -1 = 0\\
\end{align*}
\begin{align*}
n\beta_0\sum w_i+\beta_1\sum w_iX_{1i}+\beta_2\sum w_i X_{2i}+\cdots+\beta_p\sum w_i X_{pi} &= \sum_{i=1}^N w_iY_i\\
\end{align*}

\begin{align*}
\frac{\partial SS}{\partial \beta_1} &= 2 \sum_{i=1}^Nw_i(Y_i-\beta_0-\beta_1X_{1i}-\beta_2X_{2i}-\cdots-\beta_pX_{pi})\times -X_{1i} = 0\\
\end{align*}
\begin{align*}
\beta_0\sum w_iX_{1i}+\beta_1 \sum w_iX_{1i}^2+\beta_2\sum w_i X_{2i}X_{1i}+\cdots+\beta_p\sum w_i X_{pi}X_{1i} &= \sum_{i=1}^N w_iX_{1i}Y_i\\
\end{align*}


Similarly,

\begin{align*}
\frac{\partial SS}{\partial \beta_p} &= 2 \sum_{i=1}^Nw_i(Y_i-\beta_0-\beta_1X_{1i}-\beta_2X_{2i}-\cdots-\beta_pX_{pi})\times -X_{1i} = 0\\
\end{align*}
\begin{align*}
\beta_0\sum w_i X_{pi}+\beta_1 \sum w_i X_{1i}X_{pi}+\beta_2\sum w_i X_{2i}X_{pi}+\cdots+\beta_p\sum w_i X_{pi}^2 &= \sum_{i=1}^N w_iX_{pi}Y_i\\
\end{align*}

To prove that this is indeed the minimum, we need to show that $$\frac{\partial^2 SS}{\partial \beta_i^2}$$ is convex:

\begin{align*}
\frac{\partial^2 SS}{\partial \beta_0^2} &= 2w_i \geq 0
\end{align*}


\begin{align*}
\frac{\partial^2 SS}{\partial \beta_1^2} &= 2\sum_i w_i X_{1i}^2 \geq 0
\end{align*}

Similarly for any $1 \leq j \leq p$:
\begin{align*}
\frac{\partial^2 SS}{\partial \beta_j^2} &= 2\sum_i w_i X_{ji}^2 \geq 0
\end{align*}

And for $$ k \neq j $$:

\begin{align*}
\frac{\partial^2 SS}{\partial \beta_j \beta_k} &= 2\sum_i w_i X_{ji}X_{ki} \geq 0
\end{align*}

\begin{align*}
\begin{pmatrix}
\sum_i w_iX_{1i}^2 & \sum_i w_iX_{1i}X_{2i} & \dots & \sum_i w_iX_{1i}X_{ni}\\
\sum_i w_iX_{2i}X_{1i} & \sum_i w_iX_{2i}^2 & \dots & \sum_i w_iX_{2i}X_{ni}\\
 \vdots
\sum_i w_iX_{ni}X_{1i} & \sum_i w_iX_{2i}^2 & \dots & \sum_i w_iX_{ni}^2
\end{pmatrix}
\end{align*}
(We can take the $w_i$ out by factoring that as a separate vector) and then 
each element in the  remaining matrix in this case can be written as $H_{ij}=\mathbf{x_i}^T\mathbf{x_j}$ and hence this is a Gram matrix and positive definite, hence minima at the above point is guaranteed.
