Monday, March 05, 2012

Scut work science

I've been thinking about scientific progress.

There's a lot of scut work in science. That's always been true - just look at how many freaking beetles and finches Darwin collected.

Ideally, the scut work serves to clear the path to a final "product" - whatever that may be. If you do the scut work wrong, some scientist's project fails because of the bad data you submitted, and they angrily punch you right in your reputation.

However, recently we've been seeing the scut work get politicized and profitable. At some point, people began to think "well, the important thing is what is true, and that's what science can tell us".

Ehhhhh-ehhhh. That's the scut work. "True or false" is scut work. "How does this fucking thing actually work" is what science is about.

So we've got all these scientists getting paid to find out whether something is true or false. Most famously, drug trials: every new drug is apparently effective.

Of course, that's bullshit. The truth isn't that every new drug works best, and the truth isn't that every new drug works worse. The problem is that whether a drug works better or worse is politicized. Everyone who finds the drug works gets published, everyone who finds it doesn't, doesn't.

And who's gonna call you on it? It's not the kind of scut work where someone else will try to develop from that data and find that it is clearly wrong. There's no further development built on that data. It's just marketing data masquerading as science. "Cover our ass" science.

To me, that's a big problem. Obviously, due diligence is required when creating products, especially medicines. But paying scientists to do scut work while at the same time removing the layer that checks their work? That's a recipe for disaster.

That's like telling a kid you'll pay him a dime for every question he gets right on his tests at school, but instead of looking at the tests, you just ask "how many questions did you get right?"

He'll probably do it earnestly for a while, but when he realizes you never check how many he actually got right, he's going to start getting a surprising number right. Maybe it'll eventually bite him in the ass when grades come out, but even then you'll just yell at him a little.

... That's my opinion of "cover our ass" science. I'd really like to see us pay a lot more attention to the scut work, and whether it is actually worthwhile, useful, build-uponnable... or whether it was the one trial out of fifty that just happened to get good results and was therefore the only one published.

As ever, the goal with science is to figure out how things work. Trials and tests serve that end. Any trial or test for another reason isn't science, it's just due diligence and/or marketing.

No comments: