Skip to main content

Can I look at reported standard errors (SE) and decide if means differ?


No guarantees, but roughly if means differ by 3*SE then they are statistically significant.  This is based on the Least Significant Difference, which is 2*sqrt(2)*SE.  Often people use non-overlapping confidence intervals as a decision rule, but this is equivalent to 4*SE, which is a bit conservative.

Things that make 3*SE fail:
1)  Actually statistical differences depend on the standard error of difference, SED, not SE.  Anything in the model that makes these differ will make the rule fail, such as covariates and blocking factors.
2)  In general, mixed models with random effects will make the rule fail, because random variance is included in SE, but not in SED.  But this will make 3*SE rule conservative, 3*SED will be even smaller.  If 3*SE suggests a statistical difference, difference most likely exists.

Also take a look at Error Bars paper.

Comments

Popular posts from this blog

DANDA - A macro collection for easier SAS statistical analysis

Objective :  You are running ANOVAs or regressions in SAS, and wish there was a way to avoid writing the dozens of commands needed to conduct the analysis and generate recommended diagnostics and summary of results, not to mention the hundreds of possible options that might be needed to access recommended methods.  A possible solution is to download a copy of danda.sas below, and use this macro collection to run the dozens of commands with one statement.  We will also have future posts covering various uses of danda.sas, giving examples as always. danda.sas is under continued development, check this page for updates. Date                       Version               Link 2021/03/15             2.12.030          danda.sas 2021/03/15             2.12                UserManual.pdf     2012/08/30                 2.11                danda211.sas Example :  You have an RBD split-plot design, so typical SAS code for mixed model ANOVA is proc mixed data=one;   class block treat week;   m

UTF character data, encoding of text

Objective and Background :  You have text data that is UTF encoded and need SAS/R to read and write datasets with that encoding.  If you have ever printed or viewed text information, and seen something like Giuffr?Ÿ’e?ƒe?Ÿƒ?ÿ?›ƒ?ªƒ?›?Ÿ’e›ƒ?ª­?Ÿƒeee, then you are running into this encoding issue.  Computers store text using numbers, with each number assigned to a particular character.  See  https://en.wikipedia.org/wiki/ASCII  to find that the character & is stored as 38 when using the ASCII encoding.  Unicode is popular internationally because it encodes special characters such as accented letters, and UTF-8 is a widely used version ( https://en.wikipedia.org/wiki/UTF-8 ).  In UTF-8 the & character is stored as 26, and you can imagine how the jumbled example above arises from the confusion of what letters are being stored. Solution 1 :  Use options to request that individual datasets be read and written in a particular encoding.  In SAS, specify encoding options on the vario

Factorial ANOVA with control treatment not integrated into the factorial

Factorial treatment designs are popular, due to advantages of research on multiple treatment factors and how they interact.  But if the design includes a control treatment that is not part of the factorial, problems occur in estimation of least squares means.  A typical example is shown here, with 2 fertilizer and 3 irrigation treatments, giving 6 factorial treatment combinations, plus a control that is defined by a 3rd level of fertilizer, and a 4th level of irrigation: Fert1:Irrig2               Fert2:Irrig1             Fert1:Irrig1 Fert2:Irrig3               Fert1:Irrig3             Fert2:Irrig2           Control Other situations might have the control sharing a level of one of the factors, for example the control might be defined as Fert2:Irrig4.  But this still causes problems with estimation of least squares means due to the levels of one factor not occurring with all levels of the other factor. Let's jump into a SAS example, but using random numbers to allow easy creati