Frequentist Model-Averaged Confidence Intervals

Abstract

Model averaging is a technique used to account for model uncertainty in the process of multimodel inference. A frequentist model-averaged estimator is defined as a weighted average of the single-model estimates resulting from each candidate model. We may also desire a model-averaged confidence interval, which similarly accounts for the uncertainty in model selection. Preexisting constructions for such an interval have assumed a normal sampling distribution for model-averaged estimators, and thereby construct Wald intervals centered around the model-averaged estimate. This approach is problematic, since the assumption of normality is generally incorrect. Furthermore, it relies upon accurate estimation of the standard error term, the form of which is not well understood.

We propose and study a new approach to the construction of frequentist model-averaged confidence intervals, which is analogous to that of a credible interval in Bayesian model averaging. This new construction is called a model-averaged tail area (MATA) interval, since it involves a weighted average of single-model nominal error rates. Through a variety of simulation studies, we compare MATA intervals against the preexisting approach of centering a Wald interval around a model-averaged estimate, and also against Bayesian model averaging. Intervals are assessed in terms of their achieved coverage rate and relative width. We consider several information criteria for the construction of frequentist model weights, and a variety of Bayesian prior distributions for the candidate models and parameters.

The frequentist MATA interval was observed to have the best coverage properties in the normal linear setting. In addition, constructing model weights using Akaike's information criterion (AIC) appeared to benefit the performance of frequentist model-averaged intervals. A different result was observed in non-normal settings, where the Bayesian approach more fully accounted for model uncertainty. Bayesian model averaging produced wider intervals with superior coverage rates, relative to any frequentist approach. Furthermore, the use of a data-dependent prior probability mass function for the set of candidate models resulted in Bayesian intervals with coverage rates nearest to the nominal value.