Why alcohol experiments are problematic

I recently published a post (Has the WHO lost its way regarding alcohol?) pointing out that what is recognized to be the best form of scientific experiment is what is called a “double-blind treatment–control” experiment (or sometimes a “randomized controlled trial”, or RCT). This procedure cannot usually be done ethically on people, and therefore no-one has ever admitted doing it in medical science. So, we will never have the best possible scientific evidence about the effects of alcohol on human health.

This does not mean that we do not have experiments about wine and health. What the scientists do is the best that they ethically can; and some of the pros and cons of this process is what I will discuss in this post. I cover several different but important topics.

What the researchers do is to follow groups of drinkers and non-drinkers through time, and see how these people get on — this is called a “descriptive” study rather than a “manipulative” one (as described above), of which there are several types as listed in the above picture. Here, we are concerned with the first one in the list, “observational”. In this type of study, health and behavior experts measure all of the consistent differences they can find between the studied people, to see what matches their patterns of their drinking and non-drinking. The results of the famous 1926 study by Raymond Pearl are shown in the next graph, as but one early example.

Also, we should ideally do all of this in such a manner that the people involved do not know which of the experimental groups they are in, and nor do the people evaluating their behavior (this is what “double blind” means). Is this actually feasible? Of course not. We can’t force people into the experimental groups, we can only ask them to volunteer to participate. That is, we rely on self–reporting of their alcohol consumption (often via detailed questionnaires filled in by the participants). Let’s look at the consequences of this now.

Here, is one useful recent discussion of how we might justifiably proceed (Causal inference about the effects of interventions from observational studies in medical journals):

Building on the extensive literature on causal inference across diverse disciplines, we suggest a framework for observational studies that aim to provide evidence about the causal effects of interventions based on 6 core questions: what is the causal question; what quantity would, if known, answer the causal question; what is the study design; what causal assumptions are being made; how can the observed data be used to answer the causal question in principle and in practice; and is a causal interpretation of the analyses tenable?

This is all well and good, but the biggest recognized issue is how to choose the studied people. Basically, the choice should be literally random, but this is impossible. As a discussion of the problems with one example of this, we have:

• The Mendelian randomization by Millwood et al.: observations and expressions of concern

In particular, people volunteer to take part in experiments, and this can never be described as “random”. Notably, people’s admissions regarding their own drinking may not be accurate (see also: Wine ratings involve both the accuracy and bias of the raters). This is discussed here:

• Risky drinkers underestimate their own alcohol consumption
• Sources, under-reporting of alcohol consumption

As one specific example discussion (Reweighting UK Biobank corrects for pervasive selection bias due to volunteering):

Volunteers tend to be healthier and of higher socio-economic status than the population from which they were sampled ... Volunteer bias in all associations, as naively estimated in UKB, was substantial — in some cases so severe that unweighted estimates had the opposite sign of the association in the target population. For example, older individuals in UKB reported being in better health, in contrast to evidence from the UK Census.

So, we often end up with this rather jaundiced (but realistic) view:

• All moderate alcohol consumption studies are “all over the charts”

Moving on, another potential approach, to getting around the sampling problems discussed here, is to use animals as a substitute for humans (eg. mice or dogs). This generates a lot of emotional response from parts of the public, which I will not delve into here. Instead, I will focus on the actual experiments.

One useful discussion is (The flaws and human harms of animal experimentation):

Nonhuman animal (“animal”) experimentation is typically defended by arguments that it is reliable, that animals provide sufficiently good models of human biology and diseases to yield relevant information, and that, consequently, its use provides major human health benefits. I demonstrate that a growing body of scientific literature critically assessing the validity of animal experimentation generally (and animal modeling specifically) raises important concerns about its reliability and predictive value for human outcomes and for understanding human physiology ... The resulting evidence suggests that the collective harms and costs to humans from animal experimentation outweigh potential benefits and that resources would be better invested in developing human-based testing methods.

One important point is whether animal experiments actually lead to any benefit for human medical treatments. Sadly, it seems mostly not (Analysis of animal-to-human translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory approval for human applications):

There is an ongoing debate about the value of animal experiments to inform medical practice, yet there are limited data on how well therapies developed in animal studies translate to humans. We aimed to assess 2 measures of translation across various biomedical fields: (1) The proportion of therapies which transition from animal studies to human application, including involved timeframes; and (2) the consistency between animal and human study results ... The overall proportion of therapies progressing from animal studies was 50% to human studies, 40% to RCTs, and 5% to regulatory approval.

I think that you can all see the bottom line here: things are not likely to get any better any time soon, regarding experiments of alcohol intake by humans. The medical scientists are doing the best that they ethically and practically can, in the real world. This, however, does not match what they would be doing in the theoretical world of scientific experiments, which is what would be the best for devising effective medical ideas.

Nevertheless, I am not the only one who has noted that there is: ‘No good evidence’ of risk from low-level alcohol consumption. Basically, the risks of one or two drinks per day are so low that they are very difficult to estimate; and drinking with meals also seems to be unproblematic (Drinking wine with meals linked to better health outcomes). Alternatively, there definitely are risks associated with heavy alcohol consumption, and people with known health problems related to alcohol may not have any safe level of consumption, as well as pregnant women. People with a family history of alcohol abuse also need to be careful.