Explanation

Let's analyze each statement:

Statement I: Akaike's information criteria are consistent

• FALSE. Akaike's Information Criterion (AIC) is not consistent. Consistency in model selection means that as sample size increases, the probability of selecting the true model approaches 1. AIC tends to overfit (select models that are too complex) because it has a weaker penalty for additional parameters compared to Schwarz's Bayesian Information Criterion (BIC).

Statement II: Akaike's information criterion always gives model orders that are at least as large as those obtained under the Schwarz's information criterion

• TRUE. AIC has a penalty term of 2k (where k is the number of parameters), while BIC has a penalty term of k·ln(n) (where n is sample size). Since ln(n) > 2 for n > 7, BIC penalizes additional parameters more heavily. Therefore, AIC will typically select models with at least as many parameters as BIC, and often more.

Statement III: If the residual sum of squares falls after the addition of an extra term, the value of the information criterion must fall

• FALSE. Information criteria balance model fit (measured by RSS or log-likelihood) with model complexity. The formula for AIC is: AIC = n·ln(RSS/n) + 2k. While adding a parameter may reduce RSS, it also increases k. If the reduction in RSS is small relative to the penalty increase, the information criterion can actually rise.

Statement IV: The adjusted R-squared is an example of an information criterion

• TRUE. Adjusted R² is indeed an information criterion, though not commonly referred to as such. It balances model fit (R²) with complexity by penalizing additional parameters: Adjusted R² = 1 - [(1-R²)(n-1)/(n-k-1)]. It has a weaker penalty term compared to AIC or BIC.

Correct statements: II and IV

Key Points:

• AIC is not consistent; BIC is consistent
• AIC typically selects more complex models than BIC
• Information criteria can increase even when RSS decreases if the penalty outweighs the improvement in fit
• Adjusted R² is a form of information criterion with a relatively weak penalty for complexity