Re: model performance with and without interactions...

[Spencer Graves <spencer.graves@PDF.COM>]

	  George Box (1999) has said that the proper role of statistics is to 
catalyze the process of scientific learning.
	  What one does with the interactions depends at least partly on 
whether one is doing exploratory or confirmatory analysis.  Even in a 
confirmatory investigation, I think that failing to also consider the 
data from an exploratory perspective is negligent and a huge waste of 
the cost.  In this regard, I like to think of the multiple levels of 
each main effect as arrayed on a continuum, with the continuum to be 
elucidated.  In this regard, the (g-1)x(h-1), etc., degrees of freedom 
can be decomposed into that many individual terms that correspond to a 
Taylor series expansion in terms of the latent variables behind each 
main effect.  This leads us to consider Tukey's one degree of freedom 
for non-additivity and generalizations by Mandel and others, discussed 
in Milliken and Johnson (1989).
hope this helps.  spencer graves

	  Box, George E. P. (1999) “Statistics as a Catalyst to Learning by 
Scientific Method”, Journal of Quality Technology, 31: 16-72;  reprinted 
in Tiao et al. (eds) Box in Quality and Discovery (NY:  Wiley 2000, pp. 
170-188).
	  Milliken, G. A., and Johnson, D. E. (1989) Analysis of Messy Data, 
vol. 2:  Nonreplicated Experiments (NY:  Chapman and Hall, pp. 7-40).
Naomi Altman wrote:
> I would not drop interactions based on the relative size of F-tests.  I 
> look at the interaction plots, as suggested here.
> Some statisticians would NEVER drop interactions, because there are 
> inferential problems having to do with this type of "pre-test" modeling.
> --Naomi Altman
>
> At 11:53 AM 6/20/2003 -0300, Fowler, Mark wrote:
>
> > Rob,
> >
> >         Yes, we did that. That was exactly the justification for removing
> > interaction terms (the interactions did not suggest troubling 
> > differences in
> > trends, just differences like greater and lesser slopes of trends in the
> > same direction). So having reviewed the interactions, I'm not worried 
> > about
> > opting for the main effects model in this instance. My question is more
> > general. The rationale for dropping consideration of the interactions was
> > not explicitly because the interactions did not appear to contradict the
> > main effects interpretation. The interactions were also perceived as 
> > minor
> > concerns because the magnitude of the F values associated with the
> > interactions were much smaller than those associated with the main 
> > effects.
> > It is this rationale that concerns me, given the likelihood of it being
> > applied in other situations if accepted in this case. In other words, the
> > next time interactions with smaller F's than main effects are 
> > encountered,
> > this reasoning could preclude bothering to look at the interactions. I'm
> > questioning the principle in general, not the consequences for this 
> > example.
> > The example just illustrates the context in which the decision was made.
> >
> > -----Original Message-----
> > From: Rob Balshaw [mailto:Rob.Balshaw@syreon.com]
> > Sent: Friday, June 20, 2003 9:51 AM
> > To: Fowler, Mark
> > Subject: RE: [S] model performance with and without interactions...
> >
> >
> > Have you looked at the predictions themselves, perhaps through a 
> > series of
> > plots?   If over the range you are likely to see in your data the
> > interactions do not result in substantive changes in the 'accuracty' 
> > of your
> > model, you might be justified in leaving them out.
> >
> > Just a thought.
> >
> > Rob
> >
> > -- Robert Balshaw, Ph.D.
> > -- Senior Biostatistician, Syreon Corp.
> > -- Phone: 604.676.5900x220; Fax: 604.676.5911
> >
> > CONFIDENTIAL
> > This e-mail and all attached documents are for the intended
> > recipient(s) only.  They are confidential and may contain
> > privileged information; any unauthorized dissemination,
> > distribution, copying or printing is strictly prohibited.
> > If you have received this transmission in error, please
> > notify the sender immediately and delete this e-mail and
> > its content from your system.
> >
> >
> > > -----Original Message-----
> > > From: s-news-owner@lists.biostat.wustl.edu
> > > [mailto:s-news-owner@lists.biostat.wustl.edu]On Behalf Of Fowler, Mark
> > > Sent: Friday, June 20, 2003 3:51 AM
> > > To: S-news (E-mail)
> > > Subject: [S]
> > >
> > >
> > > I had presented the following ANOVA to depict the potential 
> > relevance of
> > > interactions, where we are seeking to determine if main effects
> > > alone could
> > > be used to represent the essentials of the model with respect to
> > > predictions
> > > of values for yland (annual time series):
> > >
> > >               Df      Deviance        Resid. Df       Resid. Dev      F
> > > Value         Pr(F)
> > > NULL          NA      NA              4519            
> > 5135.242        NA
> > > NA
> > > cfv           34      517.41791       4485            4617.824
> > > 18.68637      0.000000e+000
> > > yland         15      348.6338        4470            4269.191
> > > 28.539124     0.000000e+000
> > > mland         9       164.50611       4461            4104.684
> > > 22.444086     0.000000e+000
> > > AREA          5       105.90937       4456            3998.775
> > > 26.009187     0.000000e+000
> > > yland:mland   117     279.9389        4339            3718.836
> > > 2.937919      0.000000e+000
> > > yland:AREA    67      182.53822       4272            3536.298
> > > 3.345349      0.000000e+000
> > > mland:AREA    41      90.57283        4231            3445.725
> > > 2.712542      2.896125e-008
> > >
> > >
> > >  I thought that the magnitude of the deviance associated with
> > > year:month and
> > > year:area, coupled with significance of the F test, indicated a need to
> > > interpret the interactions (i.e. they were potentially relevant in a
> > > pragmatic sense). This assertion was countered with the argument 
> > that the
> > > smaller F's associated with the interaction terms indicated that main
> > > effects would be adequate to do the job. Examination of the
> > > interactions was
> > > felt to support this argument, as predictions from the interaction 
> > model
> > > were mostly variations in degree, not contradictory. However I'm 
> > concerned
> > > that the compatability between main effects and interaction model
> > > predictions may be circumstantial (just happened to be true for this
> > > particular example), and that the interpretation of F magnitudes
> > > to reflect
> > > relevance of a term might be inappropriate. Opinions?
> > >
> > > --------------------------------------------------------------------
> > > This message was distributed by s-news@lists.biostat.wustl.edu.  To
> > > unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
> > > the BODY of the message:  unsubscribe s-news
> > --------------------------------------------------------------------
> > This message was distributed by s-news@lists.biostat.wustl.edu.  To
> > unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
> > the BODY of the message:  unsubscribe s-news
>
> Naomi S. Altman                                814-865-3791 (voice)
> Associate Professor
> Dept. of Statistics                              814-863-7114 (fax)
> Penn State University                         814-865-1348 (Statistics)
> University Park, PA 16802-2111
>
>
> --------------------------------------------------------------------
> This message was distributed by s-news@lists.biostat.wustl.edu.  To
> unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
> the BODY of the message:  unsubscribe s-news