Craig Pickering

Sports Science Monthly – April 2019

/by

Every month we take a deep dive into the latest research in sports science. In the April Sports Science Monthly we start off by looking at a new framework for evaluating research. Then we focus on new findings about specific topics like gluten free diets for athletes, the role of testosterone in female performance, sports nutrition, the speed gene, and more.

As always, the full Sports Science Monthly is available exclusively to HMMR Plus Members. You can browse the past topics on our archive page. The first topic below is free to everyone, but sign up now to read about all the latest research. To get an idea of what Sports Science Monthly is all about, the April 2016 edition is available in its entirety for free.

This Month’s Topics

How to evaluate research for your own practice

» Quick summary: As evidence-based or evidence-led practitioners, you’re likely to continually be scouring research to improve your practice. The framework suggested in this paper allows you to analyze the results of a study, and determine whether they’re applicable to you and your athletes.

If you regularly read these columns, it’s a fair bet that you’d consider yourself an evidence-based or evidence-lead practitioner. This is, of course, a good thing, but it can also be challenging; how do you determine which studies should guide you, and which to reject? Fortunately, in a recent Sports Medicine paper, a group of leading sports nutritionists put together a framework, termed the “Paper-2-Podium Matrix”, which can be used to guide you in this area. Whilst it’s aimed at sports nutritionists, it has important implications for anyone wishing to transfer from research to practice, and so it’s worthwhile taking a deeper look.

The matrix itself is comprised of nine points, each of which are scored from -2 to +2. I can’t reproduce the table here for copyright reasons, but by clicking here and navigating to table 1, you can see it for yourselves. If you’re left with an overall negative score after scoring each component, you should be cautious in applying the research findings in your practice; a score of 0 to a “low positive” (which isn’t quantified) indicates the research findings may be appropriate, but caution should be exercised, and a “moderate to high” positive score indicates that it is likely an appropriate study to guide practice.

The first component part is that of research context; basically, what type of research is it? Studies in non-human cells score poorly here, as you’d expect, whilst studies in isolated human cells receive a score of zero. Essentially, you want to use studies that recruit humans, and subject them to some sort of exercise intervention, either acutely or chronically, with bonus points added if the mechanisms underpinning what is actually happening are elucidated.

The second component is that of research participants. A poor study wouldn’t inform you as to the activity level and ability of the participants, so be wary of this. In my first ever HMMRMedia article, which was on evaluating science, I wrote that the findings of a study are only ever truly applicable to the sample that made up that study; to that end, you should apply more weight to studies that recruit subjects similar to the athletes you work with. If you work with elite athletes, then ideally you want elite athletes to be the study population; if this is not possible (and elite athletes typically don’t take part in interventional research), then a fair—but not completely perfect—compromise is well-trained subjects. Conversely, if you’re working with developing youngsters, then research on adults is likely inappropriate. In essence—give more weight to studies that use subjects whom best represent your athletes.

The next component is that of research design. The gold standard for research is a randomized cross-over trial; this means that the subjects are randomized into two or more groups, and receive both treatments (most commonly the intervention and a placebo). As an example, if I wanted to test whether caffeine worked, I would recruit around 20 subjects, and randomize them into a caffeine group and a placebo group for experiment one, and then switch these for experiment two. To guard against any placebo or expectancy effects, the subjects should be blinded to the treatment; again, returning to the caffeine example, we know from published research that if think caffeine is performance enhancing, and know you’ve consumed it, you see a greater performance benefit than if you were blinded. Conversely, if you think that caffeine is performance enhancing, and you know that you haven’t consumed it, then your performance is typically worse than when you were blinded to this. Included in the research design component is sample size. As a very brief, simple explanation, for the results of a study to be valid, the subjects have to accurately mirror the target population, and the larger the sample size, the more likely this is. As a double check, you can read papers in a similar field to the one you’re analyzing, and see if the sample sizes are similar.

Component four discusses whether the study utilized dietary and/or exercise controls. This is more important in sports nutrition research, but is a worthwhile point to consider nonetheless. This component basically determines whether or not the researchers have taken into account the baseline status of the subjects. Again, using caffeine as an example, habitual caffeine use might alter the performance benefits a subject receives from caffeine, and a caffeine naïve individual might see greater negative side-effects from caffeine use compared to a regular user. As a result, in a caffeine study I would want to quantify the baseline caffeine intake of all my subjects, and take this into account when analyzing the data. This baseline data can be self-reported (e.g. “I tend to consume 3 cups of coffee per day”), but, because any self-report data is subject to recall bias, the gold standard is to try to monitor this through validated measures, such as a real-time food diary, or collection of urine for caffeine metabolite analysis. Both of these a tricky and costly, so you can see why doing good science is hard, but being able to critically analyze this in your reading of a paper can be crucial.

Next is the validity and reliability of measurement methods in the study. Validity essentially means “does this test measure what it’s supposed to test?” Many methods have validation studies to support their use, and the authors of a given study should cite these papers, so watch out for those within the paper you’re reading. Reliability means “if I were to repeat this test again, would I get the same results?” Again, this is obviously important because you want the result to be representative of where the athlete is, as opposed to random noise. Again, better studies will either cite reliability studies for their given method, or carry out their own reliability study. As a final note, subjects should be familiar with the test used for measurement, as this increases the reliability of the test; in most studies, the subjects will undertake at least one, but often multiple, familiarization trials prior to the study starting in order to cover this—check that this is covered in the paper you’re reading.

» Learn more: Craig Pickering explains some tips to becoming a better consumer of sports science.

Component six covers the data analysis portion of the paper. Papers that don’t analyze data, or only use descriptive statistics (i.e. don’t use significance tests or similar methods) score poorly here. At a minimum, the researchers should attempt to report whether the effect they’re reporting is likely to be “real” or not (most commonly done through the use of significance testing; i.e. is the p-value equal to or less than 0.05?), with bonus points if they quantify the real-world importance of this finding, for example through the use of effect sizes, or potentially the much maligned magnitude based inferences method.

The next three components cover the application of the findings to your own individual practice. First up, there is the feasibility of application; basically, how easy is it for you to make changes based on the study you’re analyzing? This comes down to a) cost of the intervention; b) simplicity of the intervention; and c) compliance of the athletes to the intervention. As an example, let’s say you want to implement a sleep monitoring tool. You’d have a greater chance of success if you utilized a free smartphone app that asked two questions than you would with a detailed, thirty question questionnaire that required daily printing on paper. The highest points here are awarded for interventions that are cheap, simple, and easy to comply with. Next, there is the risk/reward aspect; how safe is the intervention, relative to how much it might improve performance. In an ideal world, we’re looking for low risk changes, with few side effects. Again, using caffeine as an example, any intervention can be tested in training (representing low risk), with minimal risk of harm provided the caffeine dose is sensible, and no risk of an anti-doping violation as caffeine is not banned. The upshot is that the potential reward can be huge; caffeine is a well-established and well-replicated performance enhancer, and so we would expect to see a performance improvement. Finally, there is the timing of the intervention; is it appropriate given the athletes age and time of the season, and can you test the intervention in training prior to a competition?

The above all represent a decent checklist to work through as you read a study, allowing you to better determine whether the intervention tested is likely to be useful and/or applicable to you and your athletes, and, hopefully, improve their performance.


Join Now to Keep Reading

This is just the beginning. To continue reading this article you must be a HMMR Plus member. Join now to get access to this and more content. Learn more and sign up here. If you are already a member, log in here.