Forget NHST: conference bans all conclusions

Once again, CMU is hosting the illustrious notorious SIGBOVIK conference.

Not to be outdone by the journal editors who banned confidence intervals, the SIGBOVIK 2015 proceedings (p.83) feature a proposal to ban future papers from reporting any conclusions whatsoever:

In other words, from this point forward, BASP papers will only be allowed to include results that “kind of look significant”, but haven’t been vetted by any statistical processes…

This is a bold stance, and I think we, as ACH members, would be remiss if we were to take a stance any less bold. Which is why I propose that SIGBOVIK – from this day forward – should ban conclusions…

Of course, even this provision may not be sufficient, since readers may draw their own conclusions from any suggestions, statements, or data presented by authors. Thus, I suggest a phased plan to remove any potential of readers being mislead…

I applaud the author’s courageous leadership. Readers of my own SIGBOVIK 2014 paper on BS inference (with Alex Reinhart) will immediately see the natural synergy between conclusion-free analyses and our own BS.

4 thoughts on “Forget NHST: conference bans all conclusions”

  1. In other words, from this point forward, BASP papers will only be allowed to include results that “kind of look significant”, but haven’t been vetted by any statistical processes.‡ I imagine this will also include studies where the cohorts “are not quite the same if you sort of look at this graph, I guess.”

    I don’t get it. What does the author think that a “statistically significant” difference can tell us? What exactly is being vetted?

    1. Look: I’m not a big defender of p-values and hypothesis tests. I thought I’d already made it clear, but apparently not: Yotam as well as the commenters there keep arguing against p-values. When I say “I agree, let’s forget p-values, but CIs can be useful,” then the response is “No, you’re wrong, you don’t need p-values…” Um.

      In all seriousness, I believe confidence intervals (and some other statistical inferential tools) are useful as (imperfect) summaries of how precisely our experiment measured the target quantity. Throw out p-values if you like, but banning all statistical inference is overkill. The editors of BASP did just that, which is why I enjoy this SIGBOVIK comment poking fun at them.

  2. I read your post and was reminded of Deborah Mayo. I am more of a frequentist than not, and have been accused (on a Less Wrong clone site) of working the dark magic of Pearson-Neyman and even Fisher hypothesis testing.

    I decided to visit the SIGBOVIK website. It is a parody of the ACM special interest groups, correct? This is the footer:

    The Association of Computational Heresy, your leading source for cutting-edge research on State Dominance, Diversity at the Edge, and Cloud Rendering.

    I lol’d! It is cute!

Comments are closed.