This Keto Life: All Up In Your Science (Confirmation Bias)

Scientists are not immune to confirmation bias – no one is. That is, to a large extent, why we have peer review: because otherwise we could publish any old shit that we thought was convincing and not ever have anyone point out “Yes, but you would find that convincing. Have you thought instead that your data could mean something quite different?” or even “Yes, but you failed to account for this confounding factor,” or “Your analysis includes several underlying assumptions that are not met, and thus it is invalid.”

I am reading a few more books about the ketogenic diet (and yes, I am seeking out criticisms online to see if they are valid for me. I already did this before I decided to go low-carb, but I continue to do it. I think it’s important to stay on top of these arguments).

It’s become clear to me that nutrition is a minefield of confirmation bias, much of which is dangerously close to woo, and that if I’d read the wrong books first, I would not necessarily be convinced.

The first books that I read and found convincing were, firstly, The Art and Science of Low Carbohydrate Living and The Art and Science of Low Carbohydrate Performance by Drs Phinney and Volek. Here are the reasons I found their work convincing:

  • they are qualified researchers with expertise in the field about which they are writing (you wouldn’t think that would be hard to find, but I’ll say more about this below)
  • they go into exhaustive detail about the studies they have done, how they set them up, and how they have drawn the conclusions they did
  • they go into similarly exhaustive detail when they critique other studies (which they do in a calm and consistent fashion)
  • they let you follow the dots

Now, the first of these books does get a bit technical – only slightly, but it might be enough to scramble one’s head a bit. This is because the initial intended audience included healthcare providers. There are a couple of chapters, particularly regarding saturated, polyunsaturated and monounsaturated fats and the mobilisation of these in muscle tissues and muscle membranes that I am going to have to re-read, because that information did not stay in my head.

Before I read these books, I read a few posts from the blog of Dr Peter Attia, who likewise goes into clear and precise detail about a number of key concepts. He writes as I like scientists to write: he provides a few basic definitions, diagrams where appropriate, and proceeds from there.

In the last couple of days, I downloaded Keto Clarity and Cholesterol Clarity by Jimmy Moore. I’d read a few posts on his blog, and they seemed to make reasonable sense, but were a bit more simplified. The books were reasonably cheap and I like the word “clarity” in the titles. I thought they might be a good shortcut to get my head around some important metabolic concepts.

And I hate to say it – no disrespect to the man intended – but I have not found these books (I have not finished the latter, to be fair) nearly as convincing.

Moore does a good job of making it clear that his experience is only his experience, and you should not over-extrapolate from it; I like that. He describes his personal experiments as “n=1”, and I like that too. He also says things like, “At this point, you may be wondering why the hell you should listen to anything I have to say, as I have no qualifications. That’s excellent! You should question people! My job is to convince you!”

I really like that. I think it’s a good position to have, and since Moore has no qualifications in science or nutrition other than researching heavily (like I have no qualifications in nutrition apart from that sort of researh), it’s a very responsible position. What he has done is tried to get a large body of research and a large body of opinions from other experts and distill it down into more digestible (ha, see what I did there?) material for the layperson.

I think he has done a reasonably good job, but for two problems.

Firstly, I do not like his choice of experts.

Don’t get me wrong, a lot of them do tick the box for me. His co-author, Dr Eric Westman, mostly explains things well (although he has a couple of metaphors which are just plain silly and trying a bit too hard to communicate to the common folk, what is the word on the street, yo). There are a large number of medical doctors, qualified nutritionists and biomedical researchers in his arsenal of consulted experts. There are cardiologists. There are lipidologists. There are diabetes specialists. These are the people you want to back you up when you make the case for a ketogenic lifestyle or the role of cholesterol (or lack thereof) in heart disease.

However, there are also food bloggers. There are personal trainers (I trust a qualified personal trainer to understand anatomy and sports science and how I should move in order to maximise efficient muscle development, flexibility, coordination and aerobic efficiency barring medical hurdles. That is about as far as it goes).

There are- and I honestly find this quite damning- people who are highly qualified academics in computer science or astrophysics. These people are no more qualified than I am to offer advice (which is one reason I mostly restrict these posts to my own experience and any advice I give has been heavily, heavily validated elsewhere, i.e., salt), except that they may have spent longer doing their own amateur research. I was entirely dismayed to see in the list the name of a person who has been associated with the worst kind of sneakily published, pay-to-publish-in-an-inappropriate-journal junk science listed as an expert (by pure coincedence, I’d read a write-up of this junk science the very day before I bought the book. I stared at the photo in the bio in genuine horror).

As scientists, are they well equipped to deal with the primary literature? Yes, of course, better than most people. That’s the value of scientific training (to a lesser extent, research training in general teaches you to evaluate claims and evidence). Are they probably, in many cases, correct in what they are saying? Yes, mostly, in the context I’d give them the benefit of doubt. Are they people whose blogs I would read and find interesting? Perhaps. Blogging expertise is a low bar (I posit myself as Exhibit A); anyone can have an opinion and make an argument, but to do that well is a skill I admire.

But are they people who should be being quoted by a health blogger in his book as experts in the field? No, they are not; and I think that by including them as experts in his book Moore makes a terrible error in judgment, reducing the credibility of his case and his own credibility by association.

Perhaps he is playing a numbers game. He has clearly met all these people and respects them highly, and is thinking, “Wow, all these intelligent and respectable people are in my corner here, I should list them all and quote them all!” but in my opinion he would have done better to thin the herd, cut a good number of his “Moment of Clarity” quotes and not been so determined to quote all his mates (to be fair, it was his first book).

The second problem I have with his book is a big one: in spite of his apparent determination to do so, he does not always back up his claims.

This doesn’t make me want to drop keto, because I’ve already read Phinney and Volek and Attia, and they make many of the same claims and they do back them up with references and well-designed studies; but if Moore was the first book I had read, I would doubt some of the claims he makes. He backs some of them up well, but others he just leaves sitting there or – and this ties in with my first nitpick – he relies on his stable of experts to back them up for him.

Which would be fine if all his experts were experts, but some, as I said, are not necessarily deserving of the title.

I have been convinced that the ketogenic diet is one that leads to a healthier metabolic situation for people who are either insulin resistant or heading in that direction. The evidence is very good that a monitored ketogenic diet is extremely helpful for Type 2 Diabetes, and that it maybe be helpful for Type 1 Diabetics; however, in the latter case it must be medically monitored. Type 1 Diabetics are the only effective risk group for ketoacidosis – i.e., where blood ketones get too high, in the order of 10mM or so. Nutritional ketosis shouldn’t get much past 5mM for most people because we will produce insulin to reduce it when it gets too high, but deliberately provoking ketosis in a system that does not produce insulin can be a wee bit problematic.

I have been effectively convinced that the ketogenic diet, by swapping one’s fuel specialisation from sugar-burning to fat-burning, increases one’s energy levels after adaptation, because we can store a twenty times more fat than we can glycogen. I’m finally feeling fully adapted, and it is amazing how awake and functional I am all day (as long as I avoid sugars). It’s unprecedented for me.

I have also been convinced that the ketogenic diet will reduce inflammation (as expressed by C-reactive protein) and improve lipid profiles. I haven’t been convinced by personal experience because I can’t test those myself; rather I have been convinced by a number of extensive studies, literature reviews and meta-analyses that have been published in reputable journals. Inflammation is important to me because my body is prone to sit around having an inflammatory response for no good reason, and then I end up having an HS flare (hidradenitis suppurativa, which I’ve mentioned before. Don’t Google it, please, it’s pretty gross). Inflammation is also implicated in a couple of other minor conditions to which my body is prone – and, I think not coincidentally, those conditions are correlated with Type 2 Diabetes and insulin resistance, suggesting that if perhaps you react badly to certain glucose situations, you end up with a lot of inflammation happening, and there might be shared causes there. I am therefore hoping that a ketogenic diet really is anti-inflammatory, and while I don’t expect my HS to disappear entirely, I am hoping that the incidence of attacks will decline. I did notice that attacks declined significantly after I quit eating wheat regularly in 2009 (my own “n=1” observation).

My plan of attack here is to get a thorough lipid profile (and inflammation blood work) done when I have been doing keto for six months.

Why am I listing all these things?

Because I have not been convinced that the ketogenic diet is anti-carcinogenic. The only people making this claim are people who are not actually qualified experts in health – those people usually know better than to make wild anti-carcinogenic claims. The people making these claims are also the people making claims about the “paleo” diet (more on that another time, I promise, but a quick summary on my view is that people can choose whatever labels they want, and if that makes them happy, fine, but it doesn’t make it scientifically accurate), and the evils of processed foods (which can be evil, sure, but are not necessarily carcinogenic. That is a big damn claim and you need to back that up).

People who have terminal cancer are in many cases desperate and upset, which makes perfect sense. It’s hard for most people to hold onto rationality and skepticism under these circumstances. Many do so, but it’s not necessarily easy. I feel that it is grossly unethical to promote a particular way of eating that focuses on that desperation without any evidence to back it up. Promote it because of the documented positive effects on inflammation, heart health and metabolic processes – that’s fine. A cancer sufferer might find that makes them feel better for other reasons, and the improved health (if they are a person who will benefit from keto; not everybody will) might turn the tide; but honestly, keto-adaptation in a cancer patient is also going to make a very sick person feel even worse (at least temporarily), and whether it’s worth it or not will vary from person to person.

That claim really rubbed me the wrong way.

In summary, confirmation bias is a sneaky thing. It’s not that I don’t want to believe these claims, but the minute any lifestyle change is purported to solve everything, cure everything, change everything, I get skeptical. I can’t help it. It sounds like snake oil. There is no magic cure for everything because life is complicated, and biology is complicated. Frankly, if I hadn’t seen these studies for myself, I’d be thinking even the well-supported claims made concerning the keto diet were a bit dodgy, just because there are so many of them.

Wanting to believe something is very, very dangerous, and this is why I will keep reading criticisms of keto as well as defenses and explanations.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s