The Cochrane Collaboration is one of the best sources of high quality evidence on health and has about 5,000 systematic reviews reviews in its library. It's possible to search the reviews using the Cochrane summaries site, and the abstracts provided there are useful, but a major drawback of the site is that there is no way to access a compilation of the results quickly, which would be very useful as a way of accessing Cochrane's results in a time-efficient way.
Since many people don't have time to make a lengthy study of many abstracts/reviews, but could potentially benefit from the findings contained in some of the reviews, I decided to pull together some results of Cochrane reviews in a more user-friendly way.
The most useful reviews for improving your life
Criteria used in the search
Since many people don't have time to make a lengthy study of many abstracts/reviews, but could potentially benefit from the findings contained in some of the reviews, I decided to pull together some results of Cochrane reviews in a more user-friendly way.
The most useful reviews for improving your life
I considered where to start. Many Cochrane reviews are on the evidence for particular medical interventions (drugs, surgical techniques, and so on). People would need to see a doctor in order to use many of these treatments. I came to the conclusion that reviews on interventions that people would try at home would be the most useful place to start compiling results.
There is a whole range of things we can think of as aiming at "life improvement": for example, people commonly think that drinking beverages with antioxidants and cutting down on sodium is a healthy choice. The question is: which "life-improving" behaviors are supported by high-quality evidence?
With this question in mind, I selected reviews from the Cochrane library by reading through the review titles in Cochrane's full list.
There is a whole range of things we can think of as aiming at "life improvement": for example, people commonly think that drinking beverages with antioxidants and cutting down on sodium is a healthy choice. The question is: which "life-improving" behaviors are supported by high-quality evidence?
With this question in mind, I selected reviews from the Cochrane library by reading through the review titles in Cochrane's full list.
- Reviews pertaining to a population without an existing serious disorder or dysfunction. So for example, I didn't select reviews for treatment of asthma, heart disease, thyroid disorders, cancer and so on.
- Reviews on conditions that people commonly have without going to a doctor - for instance, treatments for aches and pains like headaches and backache, weight loss treatments, and so on.
- Reviews that were applicable to a general adult population. I did not select reviews specifically for elderly people, children/infants, teenagers, smokers, or pregnant women.
The findings
67 studies met my criteria. [Note: though there may be 10-20 I missed after scanning the titles ~5,000 titles for relevant reviews and scanning again to double-check, I doubt there are many more that meet the criteria I used.] Here are my definitions of categories used in the table below:
67 studies met my criteria. [Note: though there may be 10-20 I missed after scanning the titles ~5,000 titles for relevant reviews and scanning again to double-check, I doubt there are many more that meet the criteria I used.] Here are my definitions of categories used in the table below:
Quality of evidence
- High: the authors note a low risk of bias at most; they do not mention the need for further studies in order to add to the reliability of the findings
- Moderate: the authors note a medium/high risk of bias or other problems for some of the included studies and often mention a need for further high-quality randomized controlled trials.
- Low: the authors note a high risk of bias in some or all of the studies and/or there are very few/small-sized studies; the authors mention a clear need for further high quality studies in order to come to reliable conclusions.
Effect
In the cases where "helpful/no effect/harmful" ratings are given, there was enough evidence to support this conclusion to an extent according to the authors of the review; note that in the attribution is in some cases tentative because of weak evidence.
In the cases where "helpful/no effect/harmful" ratings are given, there was enough evidence to support this conclusion to an extent according to the authors of the review; note that in the attribution is in some cases tentative because of weak evidence.
- Helpful: studies showed that there was evidence in favor of the intervention's being helpful.
- No effect: the authors conclude there is some evidence of no effect.
- Harmful: studies showed that there was evidence that the intervention is harmful.
- Not enough evidence: according to the authors of the review, there is not enough evidence to draw a conclusion at all, either because of a lack of high-quality studies, or because the available high-quality studies are too small.
See the full results in a shared Google spreadsheet (see "full version" tab).
Recommended use: download the file to be able to view the cells fully, and use filters to search for reviews you're most interested in. Below is my abridged version; note that the full version contains effect sizes with supporting quotations.
Visualizing the results
The labels refer to the above categories; the numbers within the bubbles refer to the number of studies in each pair of categories (for example: "high quality of evidence for a positive effect" is in the upper right-most bubble). The size of the bubbles corresponds to the number of studies in each category.
Recommended use: download the file to be able to view the cells fully, and use filters to search for reviews you're most interested in. Below is my abridged version; note that the full version contains effect sizes with supporting quotations.
Visualizing the results
The labels refer to the above categories; the numbers within the bubbles refer to the number of studies in each pair of categories (for example: "high quality of evidence for a positive effect" is in the upper right-most bubble). The size of the bubbles corresponds to the number of studies in each category.

This is awesome!
ReplyDeleteYes! This is great.
ReplyDeleteThe review calling exercise helpful for weight loss made me suspicious, so I clicked and saw that the effect size was 1 kilogram and that there was nothing reporting whether the weight was kept off. There REALLY needs to be an "effect size" metric here...
ReplyDeleteOkay, on going back up I see that there is a version that has effect size. But I wouldn't have left it out of the abridged version. Small effect sizes are one of the plagues of modern statistics and a chief warning sign of irreproducible results.
...and on clicking the link to the full version, all I see is the abridged version again! ??
I am not seeing any studies in the full version that are different from the one in the abridged version.
DeleteI see only two that refer to obesity: exercise (with an effect size of 1kg) and cognitive behavior therapy (8 lbs). There is no metastudy listed here that shows exercise has a significant effect size. Maybe you mistakenly thought the CBT study was referring to exercise?
Thanks, Joe and Julia! EY - I agree that effect size is really crucial. This information (with supporting quotes) is contained in the "full version," which is a tab in the linked google doc above. (I re-set so that the "full version" tab appears immediately rather than having to click over to it.) I think it may be good to add this column to the abridged version as well, and thanks for your suggestion!
ReplyDelete"Acupuncture for neck pain" is listed here as helpful (with moderate evidence). Seriously? My skeptical alarm bells are ringing here.
ReplyDeleteReally interesting, thanks for putting it together!/Linda & Simon
ReplyDeleteVery cool---surprising I haven't seen this elsewhere!
ReplyDelete