2 Replies Latest reply on Jun 23, 2017 9:01 PM by csanderson99

    Does Target account for multiple family errors (Stats) ?


      Basically what the title says- a statistics question.


      For everyone else interested. Multiple family errors (alpha inflation) happens when testing more than one variant to 95% confidence (p = .05). While each test individually has a 5% chance of Type I error, as a family grouping the error rate increased with more variants. (Formula is: 1-(1-alpha)^n where alpha is .05 and n is the number of tests). So if we are testing 8 variations (+ 1 default), the formula would be: 1-(1-.05)^8 = .34 or a 34% chance of seeing a false positive among our grouping if we are testing 8 variants.


      Does Target account for this automatically, or do we have to apply a correction post-hoc (or as a priori)?