“If you keep waving that damn thing around, I’m gonna start thinking you’re trying to keep me from reading it.”
Nico set the tablet down on the break room table and slid it along the coffee-stained surface to Simran, who picked it up and started to skim through the material.
“Look, Simran, I’ve thought this over again and I still think it doesn’t make sense. Yes, by brightness they should be over 600 light-years away, but those spectra are too strange of a coincidence.”
“We’ve had this talk before! I keep telling you, we’ve seen spectra like that before. Yeah, it’s higher trace element concentrations than ‘usual’, but still…”
“But so many stars right next to each other? It’s too coincidental.”
“Look at what I did; don’t just scroll through it. Yeah, yeah, I know it’s lunchtime, but turn your brain back on and work.”
“Fine. Cookie!” She pointed at the cardboard box on the counter, covered in crumbs and with a distinct oil stain on the side.
“Those are from last week’s group meeting, Simran.”
Grunting, Nico got up and walked to the counter as the door swung open, revealing a short woman with carefully cut, frizzled yellow hair, bright red glasses, and the suggestion of an impish grin.
“Those are from last week, Nico. I’d be careful.”
“Hello to you too, Dr. Kaxiros.”
“If you drop dead from E. coli —”
“It would be mold.”
“I’m not a biologist, leave me alone — then Simran would win this little argument you’re clearly back at.”
“It’s for Simran,” Nico shot back with a faux-irritated glare.
“So you’re trying to win the argument by poisoning her?”
“Dr. Kaxiros, I…”
At this, Simran stood up, took the cookie from Nico’s extended hand, and pressed the tablet into it as if in exchange, and turned to him.
“OK, Nico. Walk me through it.”
“I ran some basic calculations, and if there is a cloud, we could get the same final spectra even if it’s nearly within the 400 light-year limit.”
“But a cloud with that composition would be so odd…”
“Odd yes, but so is your ‘it’s-all-just-a-coincidence-of-trace-heavy-metals’ —”
“You’re making risky assumptions!”
“So are you!”
“Kids, kids —” Dr. Kaxiros intervened. “Have you ever heard why one of the Pleiades is fainter than the others?”
Dr. Kaxiros cleared her throat and began:
“Once a young warrior was wandering through the woods on a spiritual journey. One night, he heard a beautiful singing in the distance, and followed it for hours through the wood until finally reaching the edge of a great lake.
“Across the lake he saw seven maidens so beautiful that they glowed in the moonlight, engaged in a dance of incredibly complexity and grace, until finally, in perfect concert and without a word between them, the young women came together and levitated into the sky. They rose and rose until all that could be seen were seven orbs of brilliant blue, all of an equal fire.
“For seven nights he returned to the lake and watched their dance, growing enamored of the most beautiful of them all, till finally the young warrior revealed himself and asked her to wife. She was fond of the warrior, but told him that he must come with her to live in the Sky Land if they married. He readily agreed, and together the seven maidens and the husband of the fairest rose into the sky.
“We know this is true, because even today the seven sisters are in the sky. Six are clearly seen, but the seventh sits back in the shadow with her husband.”
Silence held for a moment in the air, the precious silence that holds the last remains of the otherworldly before a return to reality. Simran was the first to return.
“So you think A5234 has marital problems?”
“The Wyandot did. And it fits the data, doesn’t it?”
“Well, yes, but…”
“Anything can fit one set of data or make sense — plausibility is the barest start to an answer. Test it! Test it elsewhere. Find clouds in front of a star within the parallax limit, and start watching temporal patterns in observed spectra. Do they match what your model would predict, Nico?”
“Well, I haven’t…”
“Oh, don’t look so sheepish,” Dr. Kaxiros grinned. “And give me a cookie.”
Background: The width of the Earth’s orbit around the sun allows astronomers to use triangulation and the parallax effect to determine the distance of stars up to approximately 400 light-years away. Beyond that limit, triangulation becomes impossible and rough estimates are made by comparing observed brightness with the predicted brightness objects “ought” to have. (See here for more details.)
Disclaimer: In case you didn’t read the intro: I am not an astrophysicist, and most of this astrophysics is made up. These stories are allegories for general classes of failings and issues of rigor in science.
The Moral: “It makes sense” or “it fits the data” are both scientific fallacies.
Something “making sense” is only valuable in the development of hypotheses: it provides no evidence in and of itself. (A set of fascinating writings on the fallacies of understanding and “makes sense” can be found here and here.)
The value of a model lies in its ability to explain, fit, or predict data outside of the those to which it was fitted. It’s very easy to create a model — like that of the devoted married star — that explains a single group of data. In statistics and machine learning, this phenomenon is known as “overfitting”.
But the only useful model is one which can be extended to new situations (within limits, of course): Dr. Kaxiros is proposing that Nico test whether his model, built from the data on the ambiguously distant star cluster, can give accurate predictions about stars whose true distance is already known (because they are within the parallax boundary).
Image Source: NASA via Wikipedia.
Story Source: A Native American Wyandot story, “The Origin of the Pleiades,” with text from Myths and Legends of British North America, via the course UnTextbook.