Thread by @BenCKinney

By popular demand, #NeuroThursday will tackle last weekend's big psychological finding in the author world: this study that supposedly claims people read faster with 2 spaces after a period. washingtonpost.com/news/speaking-…

The upshot is: some researchers got a group of people who naturally write with 1 space or 2 space, and made them read both kinds of text. The two-spacer people read a tiny bit faster when they saw 2-space text. But.

Now, there's an easy answer why this article is wrong. Let me start there and get it out of the way so I can then tell you something more subtle & interesting.

(But beware, this thread is not going where I originally thought it was. All this analysis led me to revise my originally-planned conclusion. Twists ahead!)

So, easy answer: as noted down low in the WaPo article, the researchers used Courier New. That's a fixed width font. But the whole point of single-space formatting is to take advantage of non-fixed-width spacing!

Single space between sentences lets the computer/font determine a good space width.

So: this research tells us that double spaces are good for typewriter fonts. Yawn. But follow me, dear NeuroFriends, and we can learn something more interesting from it…

I dug into the article itself. It only has one figure, and this is it. Higher = better (faster reading speed). The two clusters are person type (1-space or 2-space habit), the color is text spacing.

The second and fourth bars (in each cluster) are funny - the researchers decided to test two spaces after commas, too. It's a fun control, but it's not really what we're interested in.

In that figure, their conclusion seems clear. In the 2-space people (right cluster), reading 1 space (first bar) was slower than reading 2 spaces (third bar).

But "clear" is a funny thing in science. We need some techniques to tell anecdote from data, to tell discovery from chance. (Or try to.) Time for statistics, oh boy!

Nah just kidding, don't worry, I won't try to teach you any math here.

All you need to know is that standard statistics try to detect "significant" differences. That means we think the difference comes from some real change in the world, not just a chance byproduct of normal variation.

In the text, the authors examined 3 effects: the text's period spacing, the text's comma spacing, and the group's usage preference. And the only one of these with an effect of its own was comma spacing.

No surprise there! People hate reading with two spaces after commas. I will refrain from demonstrating, to avoid fisticuffs.

It's not a huge shock that the other two factors show no effect. Neither group is intrinsically faster at reading; neither spacing is intrinsically better. It only matters when you combine them both (i.e., spacing only matters to one group).

This is what we in the bizness call an "interaction effect." A & B don't have to do something alone, but they do together. Here's a nice toy example of an interaction effect between a drug (placebo vs 50mg dose) and sex (M/F) on pain.

Now, let's pretend to ignore one factor: sex. If you average male and female, just smoosh 'em together, those two left-side points average out to a single point around 20. Same goes for the right side.

Hey, 20 on both sides! That means if you ignore those sex differences, the amount of pain under one treatment (Placebo) is the same as the amount under the other treatment (50mg of ExampleDrug).

There's no direct effect of drug, but when you look at both together, there's obviously something going on. Interaction effect!

(Side note: the National Institutes of Health requires that scientists look at sex as a biological variable. This is good, since it's better than ignoring sex, e.g. testing only males. But it's also incomplete...

...Even if you try to focus only on "biological sex," reality is more complex than that. TBF that's kinda true for everything we study: we know we miss a lot of variation...

...We hope it all averages out across a big group. But also this is why we need to be careful about looking for statistical significance! When so much remains uncontrolled, as it always does in bio/medicine, natural variability is quite high.)

Now, where were we? Something about typing? Oh right! As I was saying, in our little study, the researchers found no direct effect of group or period-spacing.

(Note: "no effect of group" means that while the two clusters may look different – right cluster taller than left – this difference isn't strong enough to convince us it's real. Little differences like 10-20 WPM happen a lot just by chance.)

So what about interaction effects? Well, the authors found a significant three-way interaction. One that requires all three variables! The two-space writers did better when reading exactly what they write (1 space comma, 2 space period).

And this is where I apologize, because I had a different conclusion in mind for this thread. When I started, I thought: "Who cares about comma spacing? All we want to know is the two-way interaction between writing style & period spacing."

But that’s wrong. Looking at two variables at a time just means lumping the third one all together - combining "1 space after comma" and "2 spaces after comma" into one group ("however many spaces after comma").

However, I'm pretty sure nobody writes with a [random, or variable, or 1.5-on-average] number of spaces after a comma.

So the three-way interaction IS the right question. Because it tells us exactly which combination of the 3 variables gave us that particularly high bar, second from far-right

When I wrote this last weekend, I was wrong. The authors' statistics are just fine. But their conclusions are still bogus because they're testing typewriter fonts.

External Tweet loading...
If nothing shows, it may have been deleted
by @BenCKinney view original on Twitter

Our #NeuroThursday takeaway this week is that data analysis is hard. If you ask the question in a slightly wrong way, look at the data from the wrong angle, results can seem to appear or disappear into thin air!

As we say in the old country, science is hard.

References: here's the full science article. Pricey unless you have some kind of library access to the journal Attention, Perception, & Psychophysics. link.springer.com/article/10.375…

Thanks for listening everyone! Go type however many spaces you want, share this around if you like, and maybe enjoy some of my other writing too. benjaminckinney.com/publications/

And lo, here's the whole thread on double-spacing and scientific analysis condensed into Threadreader form! threadreaderapp.com/thread/9947600…