Reason.com/blog: Are Most Scientific Results Bunk?

[twa]

At the risk of being called a science denier I offer what I think is a needed look at making science better

http://reason.com/blog/2013/10/28/are-most-scientific-results-bunk

 

 

I recently enjoyed listening to Stanford University statistician John Ioannidis and University of Virginia psychologist Brian Nosek talk about how to make scientific evidence more reliable. Ioannides gained some well-deserved fame with his 2005 article in PLoS One, "Why Most Published Research Findings Are False," and Brian Nosek has just established the Center for Open Science which is offering its Open Science Framework that aims to improve the validity of scientific research.

In a recent article, "Evaluation of Very Large Treatment Effects of Medical Interventions," in the Journal of the American Medical Association, Ioannid1s and his colleagues combed through 85,000 medical interventions collected in the Cochrane Database of Systematic Reviews seeking to uncover highly effective treatments. What they found is that treatments that supposedly produce very large benefits (odds ratio greater than 5) were almost always found initially in small studies and that when they were replicated in larger studies the benefits became relatively modest. In the end only one treatment was found to provide a major benefit, e.g., supplying extracorporeal oxgyen to premature babieswith severe respiratory failure. Last year, Nature reported the shocking finding that nine out of 10 preclinical peer-reviewed cancer research studies cannot be replicated.

Another big problem is the bias toward publishing positive results, while sticking negative results in the file drawer. In an interesting 2010 studypublished in PLoS One, University of Edinburgh researcher Daniele Fanelli found that as the science under consideration got "softer' the more positive results were reported. From the abstract:...more @link

 

[s0crates]

This seems more like a critique of research methodology and scientific practice than science itself. In fact I would say the types of things being criticized are actually violations of the principles of empirical science.

[Springfield]

Between alexy, PeterMP and twa, I think we've had enough science vs faith for the year. I'm tapping out.

[PeterMP]

This is an issue for newer science at any given particular point in time.

 

If you look back in time, there was "science" that smoking was not just not bad for you, but actually good for you.

 

Over time, that stuff tends to get weeded out.

 

Maybe not as fast as it should/can, but on a larger scale of what science it, I don't think it is much of an issue.

 

On more local scales (e.g. the last 10 years or less in most cases), if you are trying to use science to understand how to live or something like that, then this sort of thing is an issue (where you are willing to assume that the underlying assumptions behind science are likelyt be true at all).

 

And as with alexey's new thread, this seems like to me it should be merged into the same thread.

 

Its like this is becoming the Stadium where everybody thinks all their thoughts on a topic are worthy of a new thread.

[twa]

This seems more like a critique of research methodology and scientific practice than science itself. In fact I would say the types of things being criticized are actually violations of the principles of empirical science.

 

 

Precisely

[Elessar78]

 

In the end only one treatment was found to provide a major benefit, e.g., supplying extracorporeal oxgyen to premature babieswith severe respiratory failure. 

 

 

 

What??? Science needed to be told that a baby with malfunctioning lungs would benefit (major benefit) from helping it breathe?

[Bang]

the science of science is to always try to disprove itself until there is no doubt left that the conslusion is sound.

I would think that until something moves from theory to 'law' such as gravity, etc. it is always up for scrutiny, and potential de-bunking. (and even after.)

So in that regard, I think the word "results" is sort of misleading, Results of any given study or experiment very rarely equates to concrete findings, mostly lead to more questions and other areas to study to continue to build the whole.

"Results" could take decades of study, centuries, even. Accepted "facts" of a decade ago may be challenged and revised based on new discoveries.

When i was a kid we thought dinosaurs were slow plodding monsters and we presented the 'reality' of them in unreal depictions. We've since imagined/ learned more based on other sciences that have shed light on things, and we now have a completely different representation of the entire subject. (Will we ever truly "know"? Not without a time machine, the one true impossibility in my opinion. but we can try to use as much information that we have learned to try and form the most logical picture.)

to that end, I have always placed more 'faith' in it, because it doesn't typically just profess an idea as fact without absolutely and thoroughly trying to prove it's not.

Of course, money taints everything, and whenever i see a study and it's results, my first question is always 'who funded this'. i think that sheds a lot of light on the results they may or may not have come up with.

~Bang

[twa]

Of course, money taints everything, and whenever i see a study and it's results, my first question is always 'who funded this'. i think that sheds a lot of light on the results they may or may not have come up with.

~Bang

 

that is certainly a problem,but the inverse is true as well.....results geared toward winning funding is another

 

http://www.biocentury.com/scibx/translationalnotes/2011-04-14/translational-notes-hedging-against-academic-risk-s4

 

reproducing results and protocols are fundamentals that expose faulty research

 

laziness,greed and desire for recognition are failings not replaced with a PHD, though certainly weeded out

[Bang]

Yep, that is certainly true. Money is indeed the root of all evil.

~Bang

[twa]

Yep, that is certainly true. Money is indeed the root of all evil.

~Bang

 

the love or burning desire for it, money itself is blameless....like science itself 

 

cold cash or cold fact are beneficial