While I'm not a professional researcher, the topic of studies has interested me since I
was taught to discuss and dissect them for Psychology in college.
Obviously there's a difference between behavioural studies and those
created to test skincare, haircare or make up products, but the basic
questions we need to ask in order to determine just how reliable and
trustworthy a study is remain the same. I'll be the first to admit
that I'm drawn in by amazing trial results just as much as good
reviews or recommendations for a product, but are these results
always creditworthy? And should we be looking deeper than just
percentages on a page?
The
first thing to think about when presented with a study for a cosmetic
item is- why was the study commissioned? In the commercial
world where the results are readily posted onto publicity campaigns
the answer will almost always be to sell more products, and this is
why we need to think about whether the study is scientifically
reliable or just part of the advertising.
Another
thing to think about is who conducted the study? Was it the
brand themselves using an in-house team (least reliable), a third
party paid by the brand (more reliable) or an unrelated group
conducting a study for example to compare similar products (most
reliable). It is almost a guarantee that for the first two the
results will be completely positive, and that moves us on to thinking
about how it was conducted.
One
of the most important things when looking at results is how many
participants were included? The main place we see high positive
percentages are in mascara adverts, like the one above. The fact that
94% saw visible lift is an amazing statistic, but follow the asterisk
and we find out that these results are part of a 'consumer panel
survey'- which could have only included 5 people. Even when
trials have what seems like a large amount of participants- say 300,
is this a good representation of the ~54 million people that live in
England alone? Would the participants include a good mix of ages, skin
types, hair types etc and does it indeed actually include the people
to whom the product is targeted (eg mature skin for an antiwrinkle
product)?
Next
we move on to the study itself- how long was the study, how was it conducted and results collected? This can be tricky because unless you can find the full published study there will only be a limited amount of details available, but there are important points to take from the information we can find.
Obviously a trial of a longer duration is going to give more reliable results- I'm sure there have been many examples of when you tried a product and loved it on the first go, but as time went on you realised you didn't get on with it as well as you thought.
Obviously a trial of a longer duration is going to give more reliable results- I'm sure there have been many examples of when you tried a product and loved it on the first go, but as time went on you realised you didn't get on with it as well as you thought.
How
the study was conducted can also make a big difference to the
results- it could've been a survey that people received on a postcard
to fill in and send back to be entered into a prize draw, maybe a scientific trial carried out by researchers, or even experiments on cell samples under a microscope. 'In Vivo' clinical trials performed on a whole living organism such as a person will generally give more helpful results than those studies performed on samples in an artificial environment for example 'Ex Vivo' or 'In Vitro' performed in a test tube or petri dish. For extra In Vivo study credibility points, look for phrases like 'placebo-controlled' (meaning the product was tested against another product that was meant to provide no real effects), 'blind/single blind' (meaning the participants didn't know what the product was or what it was claiming to do) and the gold standard 'double blind' (meaning neither the participants nor the researchers knew who was using a placebo and who was using the studied product).
In terms of how the results were collected, physical measurements will always be more accurate and credible than visual observations or self assessment: someone says your hair looks shiny today, but does that mean its actually healthier? There is a wide range of scientific equipment that can measure everything from how hydrated your skin is to how much breakage your hair has undergone and can determine whether important results are just cosmetic.
In terms of how the results were collected, physical measurements will always be more accurate and credible than visual observations or self assessment: someone says your hair looks shiny today, but does that mean its actually healthier? There is a wide range of scientific equipment that can measure everything from how hydrated your skin is to how much breakage your hair has undergone and can determine whether important results are just cosmetic.
Finally,
we need to consider if there are any factors that might have added to
the results, such as if the item was used in a particular way or
other products were used at the same time. For this we need to follow
the asterisks, small arrows or little numbers next to the big results
to find the small print. A popular French brand claims one of
it's products provides results equal to prescription skincare, on
studying the fine print we find out that this claim is only true when
using the product plus a 0.025% topical retinoid, something not
mentioned any where else. The small print is where you can find extra
information, or disclaimers, for trial results appearing in
advertising and they can often help you in deciding whether the study
is reliable or just a ruse.
We need not though be too cynical when it comes to interpreting results, a lot of studies do highlight just how great a product
really is and I think actually what's more worrying than a sneaky
study is a company who doesn't give any source at all for their
claims or statistics.
Do you go on studies or recommendations for
your purchases?