Finding, Evaluating and Applying Baseball Research (Part 1): Guest Blog By Dr. Ed Fehringer
A few days ago, I had a spirited twitter discussion about conducting and evaluating research in the baseball training and coaching industries. That exchange spurred me to reach out to my friend, Dr. Ed Fehringer, a highly regarded Omaha, Nebraska orthopedic surgeon who specializes in shoulder and elbow surgery. Dr. Fehringer has been involved in several published clinical studies and he his one of the editors of Rockwood and Masten’s The Shoulder — widely regarded as the penultimate shoulder resource for students and clinicians in orthopedic community.
I asked Dr. Fehringer to write a guide for baseball coaches and instructors on finding, evaluating and applying relevant research. The FBR brainstormed and produces a list of 11 questions we thought baseball coaches, players and parents might like to know about research. Dr. Fehringer responded almost immediately and his answers were exactly what we were looking for. Here are the 11 questions:
1. Why do we need good research in baseball?
2. Why is it important for a coach/instructor to be able to review, understand and critique research.
3. How do I find relevant research (online search hacks)?
4. What are the parts/design of a typical study?
5. How do I evaluate the quality of a study?
6. What are the dangers of only reading the abstract.
7. What are the typical statistical tools used and what do they mean?
8. How do I evaluate the authors’ methods, discussion and conclusions?
9. What are the most common pitfalls in reading research?
10. How do I begin to apply the results of research to help my players/team?
11. What are the possible consequences of not learning to search out and evaluate quality research?
Over the next several days, we will publish his answers to these questions (3 or 4 at a time), but first I’ll hand the mic over to Dr. Fehringer to give a little background about his experience and credentials.
Before the Q & A – A Background on Dr. Fehringer
Randy, et al. Thanks for giving me an opportunity. I am not a research guru; I am far from it. I am fortunate in that I have been able to take part in a fair amount of clinical research and basic science research; the experiences have been invaluable. But I had no formal research training. I didn’t have any summer jobs or internships during college or medical school. I mowed lawns, cut down trees in power lines, and dug graves in a graveyard. I had to just jump right in and figure it out and learn from many great friends along the way for whom I am grateful. Early on I disliked research and had no desire to do any. It was just a necessary evil to get into an orthopedic surgery residency. Even in residency, I had to do a research project…but I didn’t like…yet. But I did two. As a shoulder and elbow fellow, I had the opportunity to learn from a master shoulder surgeon and researcher…and I will never be him. One thing that I learned is that honesty, willingness to work, and a good baloney radar are all key. We must be skeptical….but we cannot be so skeptical as to be obstinate and arrogant such that we cast a view that suggests all research is inadequate.
After about 5 years in my academic career I began doing more basic science research with the help of some terrific collaborators in various departments. I still partake in a bit today. I become keenly aware after doing some clinical research that, because of an infinite number of variables, the studies were far from perfect. There are so many variables in human beings that, as a perfectionist, I found myself less and less confident in making conclusions based upon studies that involved many uncontrolled variables. I felt that basic science studies would be more controlled. In some respects, this is true. In others, I was just naïve.
As a young boy, I once told my mother that I liked to take attributes that I found admirable from everyone I met and combine them into something much greater. Moreover, I said I’d not mimic the traits I found less desirable in others. While that hasn’t always worked exactly to my liking, I believe there is a kernel of truth or a nugget or two in nearly any research project. Sometimes one learns from a project exactly how not to conduct research! My advice: be honest, skeptical but realistic, and jump right in. I can tell you that the more I learn the more I understand that I know very little concretely. So I try to approach each project with an open, discerning mind. I’ll now respond to the list of questions offered. To anyone reading this, if you have further questions want to add to the discussion, please respond in the comments section of this blog.
Question #1: Why do we need good research in baseball?
Answer: In my opinion baseball needs good research just like any other facet of our lives. One could say a $2 billion dollar/year industry at the MLB level deserves research of some sort. But the game also affects the lives of 11.5 million high school and younger players in the U.S., and throwing arm injuries have increased dramatically as described by many. Certainly, we need to get better. Simply changing is just that: changing. And if we are changing without measuring, we don’t truly know whether changes have had any impact on outcomes, especially if more than one change is made at a time.
I believe getting better can occur if we study and begin to better understand multiple aspects of the game. Baseball, as much as many other sports, has often relied on anecdotes, history, and wisdom for training and for playing. And, frankly, research is difficult to do well, it is often boring, it is laborious, and it is expensive. Finally, there are so many layers in the training and playing process that to even cut through those and have the language and communication be consistent throughout is a challenge in and of itself. It is no wonder the game has relied on anecdotes, history, and wisdom. Researching through all of that is a monumental task.
Question #2: Why is it important for a coach/instructor to be able to review, understand and critique research?
Answer: Continuing education is important in whatever field we choose to study or work. All areas are evolutionary processes. If we better understand research performed in the area in which we work (& play), we may be better informed and make wiser decisions regarding coaching and training. It can be intimidating to talk about reviewing “research.” Some get nauseous with mention of the word. I had no interest in research during most of my medical training. As part of our orthopedic surgical residency training, I was required to do a project that was worthy of being published. I did two but disliked every second of the work. As part of my shoulder and elbow fellowship at the University of Washington, I was required to do two projects. It was there that I caught the research bug and became involved in 8 projects that were ultimately published. I began to see the benefits and how we were making a difference. I gained a greater appreciation and acumen re: shoulder and elbow disorders. As importantly, I came to understand how difficult good research can be but how it also helped me better understand pathology…and lack of pathology. Finally, I became more efficient and effective in evaluating research performed by others.
When I returned to the University of Nebraska Medical Center to train orthopedic surgery residents how to diagnose and treat shoulder and elbow disorders, I helped run the orthopedic surgery resident research committee. While more than 90% of residents would not go on to any sort of academic or research careers, we felt comprehensive education of orthopedic surgery residents required they be involved in research so that they would be able to critically evaluate research the remainder of their careers. Having had to perform research themselves, they would be better critics that would be able to see through a poor paper and celebrate a great paper.
The more educated one is in their respective field, often the more that can be translated to one’s team. In baseball, we all want to protect players and not change for the sake of change. I have not met a parent or coach whose number one concern is not the health of their players. Success is important to all, but everyone knows their chances are better if everyone is healthy. Changing protocols based upon the “latest research” or what one recently read on-line or heard from another coach can work. But to become more educated in the process such that one can “weed through” the quoted research or more that is available can be valuable. It often requires simply jumping in and reading….and learning….and continually evaluating….and tinkering….without necessarily making wholesale changes….while developing relationships with folks that we trust and whose feedback we value. Common sense matters when evaluating research. As wise mentor once said to me, “If it doesn’t make sense, it’s because it probably doesn’t.” I did not find that profound in 2000. It becomes more profound each day. Another said, “If it doesn’t make sense to you, how’s it gonna make sense to a bunch of statistics?” This was humorous but also profound. Never lose one’s common sense. Coaches have great common sense and “feel.” They use what works. They are suspicious of change. They should be skeptical but not obstinate. And, as advocates for themselves and their players, continuing education cannot hurt.
As part of the research process, coaches have the benefit of seeing and training players on a daily basis, some for years. The doctor does not. I look at coaches’ roles in the research process as being analogous to my role in the operating room. When doing basic science research, it can be relatively easy for the Ph.D. in a lab to go down a path in their project that does not reflect what happens in the operating room. So, it is my job to help communicate what happens in the operating room so that it can be better replicated in a lab. For coaches to take part in research, their role can often be to communicate what actually occurs on a daily basis in training or what they are seeing. As a coach you have the benefit of being able to analyze the study design in any research being presented to ascertain whether it replicates what is happening and/or if it is practical.
Question#3: How do I find relevant research ?
Answer: One can do a simple Google search. The difficult task becomes discriminating which studies are “good ones” and if they are in “good journals.” There are more and more open access journals that require authors to pay to have their articles published. Clearly these have an inherent bias and have generally (but not always) weaker studies. Journals that do not require author payment will generally be higher quality. For the consumer, it’s nearly impossible to identify whether a journal is an open access journal and/or whether the article is in a quality journal that has had a credible peer review. Yet, there are often good articles that are rejected over and over by reputable journals in their peer review process for a variety of reasons that end up being published in lesser journals. As well, some poorer articles are published in higher quality journals due to the same variety of reasons. Publishing research is a multi-layered process with multiple variables that are difficult to succinctly describe. But, as a general rule, more reputable journals have more reputable articles.
Pubmed is the search tool that I use to search for medical studies (similarly to a Google search). One will be able to track the articles by way of author names or subject names. In the orthopedic shoulder and elbow world, typically the Journal of Shoulder and Elbow Surgery, the Journal of Bone and Joint Surgery, and the American Journal of Sports Medicine are reputable sources. However, there are many more, too many to be named here.
Great start Doc. We can’t wait to hear your response to the rest of our questions. We hope everyone has enjoyed Part 1 of this series. I know we are thrilled to share it with you. Parts 2 and 3 will follow in the next several days.
See you at The Ranch,
Randy Sullivan, MPT, CSCS