Aa
The Inherent Problem with "Anecdotal" or "Patient-based" Evidence
We have see several treatments of MS where there appeared to be a great benefit to many patient's MS. An article will list a "large" number of people who have taken this route to treating their MS and who claim a huge reduction in relapses or even improvement in symptoms. These reports are called "anecdotal" in medicine. An anecdote is a story. So we are looking at the testimony of these people as they report the effects of this or that particular treatment.
There have been several, but off the top of my head I can point to The Swank MS Diet, Low-Dose Naltrexone, and the Marshall Protocol. All of these have thier advocates and their strongest point of persuasion is that "so many" people have benefited from their particular treatment.
My intent here is to explain why testimonial evidence must be viewed with great suspicion. I am not feeling terribly clear-headed here, so I will try to explain the problem with testimonial, anecdotal, or patient-based reports of improvement. Please feel free to ask me to be clearer if need be. Also realize that I am not saying that this kind of patient report should be disregarded. If anything it should lead to rigorous testing.
When we hear that 100 people have been treated with a method, say LDN, and they all say they have benefitted greatly, it sounds wonderful. A hundred people got better. This is great! But, if we think about it, we have to think about what we don't know.
Is it possible that actually 1,000 (or 10,000) people tried this method? Of course it is. All of these things are in the internet with glowing reports of how well people have done. Many of the couple million MS patients in the world are aware of these treatments and many will be motivated to give them a try. Is it possible that 900 (9,900) didn't find any improvement in it and stopped using it? After all, it was not a formal study. It was pretty much grass roots. Of course it is, but we don't know. Maybe a huge number tried it on their own, weren't helped, dropped it and never thought about it again. That is, they didn't stay in touch with the interested group. They didn't talk about and moved on to other attempts to treat their disease. So their "participation' is not even known about. There may be 900 (9,900) people out there that failed to be helped for every 100 who had glorious success.
It makes sense that people who try LDN and don't get better from it or don't see a stabilization or who continue to get worse, will likely drop it. We will never know about them, because they are not on the internet clamoring that they weren't helped. They have moved on. We know from experience here on the forum that 2/3 of the people who have tried it have either had no improvement, or they actually got worse. Of course, our numbers are tiny but they prove that some, perhaps many, did not benefit from LDN.
So, just looking at those made-up (but reasonable) numbers, we see that perhaps 10% appeared to be helped by the LDN. Man, thats 1 in 10! Still seems like it might have promise, no?
But, what else do we know about MS? Well, one pertinent thing is that we know that about 12% - 15% of all people with MS will have a very benign course throughout their entire lives. No matter what treatment they, try or don't try, they will do well. If some (or all) of those LDN users were among the 15% that were destined to have a benign or mild course tried LDN, they might very well believe that their disease was mild because of the LDN, and not because they were never going to have bad disease in the first place.
The same is true of the Swank Diet. Dr. Swank claimed to have done a "study". By real scentific standards which allow verification of a theory, Dr. Swank did NOT do a real study. What he did was follow a large group of 140+ patients for 30+ years and documented that they did well on his diet. What he did not do, is determine how many people tried the diet, found it made no difference and who stopped following it. He had no control group, nor did he follow those that did not stay on his diet. We have no idea of this number. Again, people whose MS turned out to follow a benign course would find that they had very few symptoms on the diet. Therefore, they were most likely to stay with it and to believe that their efforts with the diet were responsible for having so few problems with MS. We have no way of knowing what their course would have been without the diet.
Most people, when they here that they have MS, picture a wheelchair early in their lives. Most people's view of MS is much more severe than the disease usually is. So, when they don't have a really bad time, they may quite possibly give credit to anything they were doing that they thought might help, whether this was a diet, a supplement, a medication, an exercise or whatever.
The discussion above dose not even deal with the issue of placebo. We all know that some people in a study who believe they are getting the medication report improvement, even though they were getting a sugar pill. We are all susceptible to "hope" and the effects it can have in making us feel better.
So, when we look at a promised treatment or which is based on the "reports" of individual people reporting that they got better, we need to remember a couple things.
1) We are not hearing from any people that failed to improve on the treatment. That group may be tiny, but it also may be huge. Either way that group tends to be silent and invisible. A true scientific study keeps track of all the groups.
2) We need to know that the internet is anonymous. We don't know that all the people reported in the numbers are real. We don't even know that all the testimonies are real. In other treatments where someone is looking to make money, fraud investigators have found faked testimonials of people who didn't exist. I am not claiming that this has occurred in MS treatments, but we have no way of knowing. We would like to believe that people only print the truth online or that they never exaggerate even for a cause that they feel passionately about, but, who knows?
3) Sometimes we see a celebrity adding their testimonial that something is effective. Unless that celebrity is a) not benefitting financially and b) is in a position to know scientifically that a treatment is effective, their experience and story about it should carry no more weight than your neighbor's .
4) A strong showing of testimonial evidence is certainly a reason to look at a treatment scientifically. I recommend that any of us looking at such a treatment, be VERY clear on what we don't see. Let's not fool ourselves into thinking that "patient-based" evidence is anything other than some people saying it worked.
I am actually not against the use of LDN. We know that it is safe and that a couple scientific studies have shown that it can significantly improve the quality of life. Whether or not it has an effect an the accrual of disability or the number of relapses is completely unknown and cannot be determined by putting together all of the reports of people themselves.
Quix
There have been several, but off the top of my head I can point to The Swank MS Diet, Low-Dose Naltrexone, and the Marshall Protocol. All of these have thier advocates and their strongest point of persuasion is that "so many" people have benefited from their particular treatment.
My intent here is to explain why testimonial evidence must be viewed with great suspicion. I am not feeling terribly clear-headed here, so I will try to explain the problem with testimonial, anecdotal, or patient-based reports of improvement. Please feel free to ask me to be clearer if need be. Also realize that I am not saying that this kind of patient report should be disregarded. If anything it should lead to rigorous testing.
When we hear that 100 people have been treated with a method, say LDN, and they all say they have benefitted greatly, it sounds wonderful. A hundred people got better. This is great! But, if we think about it, we have to think about what we don't know.
Is it possible that actually 1,000 (or 10,000) people tried this method? Of course it is. All of these things are in the internet with glowing reports of how well people have done. Many of the couple million MS patients in the world are aware of these treatments and many will be motivated to give them a try. Is it possible that 900 (9,900) didn't find any improvement in it and stopped using it? After all, it was not a formal study. It was pretty much grass roots. Of course it is, but we don't know. Maybe a huge number tried it on their own, weren't helped, dropped it and never thought about it again. That is, they didn't stay in touch with the interested group. They didn't talk about and moved on to other attempts to treat their disease. So their "participation' is not even known about. There may be 900 (9,900) people out there that failed to be helped for every 100 who had glorious success.
It makes sense that people who try LDN and don't get better from it or don't see a stabilization or who continue to get worse, will likely drop it. We will never know about them, because they are not on the internet clamoring that they weren't helped. They have moved on. We know from experience here on the forum that 2/3 of the people who have tried it have either had no improvement, or they actually got worse. Of course, our numbers are tiny but they prove that some, perhaps many, did not benefit from LDN.
So, just looking at those made-up (but reasonable) numbers, we see that perhaps 10% appeared to be helped by the LDN. Man, thats 1 in 10! Still seems like it might have promise, no?
But, what else do we know about MS? Well, one pertinent thing is that we know that about 12% - 15% of all people with MS will have a very benign course throughout their entire lives. No matter what treatment they, try or don't try, they will do well. If some (or all) of those LDN users were among the 15% that were destined to have a benign or mild course tried LDN, they might very well believe that their disease was mild because of the LDN, and not because they were never going to have bad disease in the first place.
The same is true of the Swank Diet. Dr. Swank claimed to have done a "study". By real scentific standards which allow verification of a theory, Dr. Swank did NOT do a real study. What he did was follow a large group of 140+ patients for 30+ years and documented that they did well on his diet. What he did not do, is determine how many people tried the diet, found it made no difference and who stopped following it. He had no control group, nor did he follow those that did not stay on his diet. We have no idea of this number. Again, people whose MS turned out to follow a benign course would find that they had very few symptoms on the diet. Therefore, they were most likely to stay with it and to believe that their efforts with the diet were responsible for having so few problems with MS. We have no way of knowing what their course would have been without the diet.
Most people, when they here that they have MS, picture a wheelchair early in their lives. Most people's view of MS is much more severe than the disease usually is. So, when they don't have a really bad time, they may quite possibly give credit to anything they were doing that they thought might help, whether this was a diet, a supplement, a medication, an exercise or whatever.
The discussion above dose not even deal with the issue of placebo. We all know that some people in a study who believe they are getting the medication report improvement, even though they were getting a sugar pill. We are all susceptible to "hope" and the effects it can have in making us feel better.
So, when we look at a promised treatment or which is based on the "reports" of individual people reporting that they got better, we need to remember a couple things.
1) We are not hearing from any people that failed to improve on the treatment. That group may be tiny, but it also may be huge. Either way that group tends to be silent and invisible. A true scientific study keeps track of all the groups.
2) We need to know that the internet is anonymous. We don't know that all the people reported in the numbers are real. We don't even know that all the testimonies are real. In other treatments where someone is looking to make money, fraud investigators have found faked testimonials of people who didn't exist. I am not claiming that this has occurred in MS treatments, but we have no way of knowing. We would like to believe that people only print the truth online or that they never exaggerate even for a cause that they feel passionately about, but, who knows?
3) Sometimes we see a celebrity adding their testimonial that something is effective. Unless that celebrity is a) not benefitting financially and b) is in a position to know scientifically that a treatment is effective, their experience and story about it should carry no more weight than your neighbor's .
4) A strong showing of testimonial evidence is certainly a reason to look at a treatment scientifically. I recommend that any of us looking at such a treatment, be VERY clear on what we don't see. Let's not fool ourselves into thinking that "patient-based" evidence is anything other than some people saying it worked.
I am actually not against the use of LDN. We know that it is safe and that a couple scientific studies have shown that it can significantly improve the quality of life. Whether or not it has an effect an the accrual of disability or the number of relapses is completely unknown and cannot be determined by putting together all of the reports of people themselves.
Quix
COMMUNITY LEADER
Hmmmm, i was in reseach for over 11 years, mum about 20, one thing that i can say for sure is that "push polling" is a factor at play in a lot of research, basically the questions are worded to produce a positive. I love stats, though its getting harder to understand, i absolutely love reading %'s, its part of the puzzle and sometimes gives a lot of insight.
100% sounds good, then you work out the total data base was 2 people, so its meaningless, stats alone dont always tell the full story. For example 20% develope life threatening complications it doesnt sound that bad, change it to 1 in 5 makes you think twice. Now change the data base to 10,000 people, that would make 2,000 people developed life threatening complications, now its scarry.
Stats are just one piece of the puzzle, common sense when reading anything that sounds really really promising should always be veiwed with a pinch of skeptisism.
Cheers......JJ
100% sounds good, then you work out the total data base was 2 people, so its meaningless, stats alone dont always tell the full story. For example 20% develope life threatening complications it doesnt sound that bad, change it to 1 in 5 makes you think twice. Now change the data base to 10,000 people, that would make 2,000 people developed life threatening complications, now its scarry.
Stats are just one piece of the puzzle, common sense when reading anything that sounds really really promising should always be veiwed with a pinch of skeptisism.
Cheers......JJ
You have heard this expression before, "take what you read, with a grain of salt", also the expression, "don't always believe, everything you read".
Those of us, with MS, have to be very cautious about supposed cures or medicines that seem to help symptoms of MS.
After all, money is money and there is always going to be people out there trying to get your money, somehow. They know you suffer, and people that suffer, want relief.
If not from medicine prescribed by a medical professional, then by alternatives created by others.
The old term is, "snake oil".
Comparison, population wise, there are not many of us with MS, in the world, but we are a significant number (that is unfortunately increasing, daily), more than 400,000 in the United States, last I read.
Even the approved medicines, on the market, for treating MS, do not have significant long term studies on the benefits or long term side effects of their own medications.
Example, Rebif (the medication I currently take), the company's study for the benefits and long term side effects, is for only 2 years.
Though, there are many people who have taken Rebif for more than 2 years, some of which are members of this site.
So, does this mean, that Rebif will no longer be effective after 2 years, no, it just means that's how long the company studied it.
A college professor of mine, once showed me a correlation example:
In my city, it had the most churches, per city block, than any city it's size.
It also had the most liquor stores, per city block, than any city it's size.
Correlation was that most church goers also drink lots of alcohol.
You can twist any study, to show what you wish it to show, to prove your point!
So, I agree, if the study was of 1,000 people with MS, and the benefits of the medicine in question, helped 100 people (i.e. those 100 people claimed their symptoms were improved, etc.), then 900 people with MS, saw no benefit at all.
The company slanted their findings to get the desired result. Since the medicine "helped" 100 people with MS, then it is a known medical break through?
The answer is, NO, but because we want medicines that "might" help us, we fall
for their "slanted truth".
We need to make the FDA force these companies to give us the "full picture".
How many people were in the test (or trial) overall, and how many actually benefited.
Those of us, with MS, have to be very cautious about supposed cures or medicines that seem to help symptoms of MS.
After all, money is money and there is always going to be people out there trying to get your money, somehow. They know you suffer, and people that suffer, want relief.
If not from medicine prescribed by a medical professional, then by alternatives created by others.
The old term is, "snake oil".
Comparison, population wise, there are not many of us with MS, in the world, but we are a significant number (that is unfortunately increasing, daily), more than 400,000 in the United States, last I read.
Even the approved medicines, on the market, for treating MS, do not have significant long term studies on the benefits or long term side effects of their own medications.
Example, Rebif (the medication I currently take), the company's study for the benefits and long term side effects, is for only 2 years.
Though, there are many people who have taken Rebif for more than 2 years, some of which are members of this site.
So, does this mean, that Rebif will no longer be effective after 2 years, no, it just means that's how long the company studied it.
A college professor of mine, once showed me a correlation example:
In my city, it had the most churches, per city block, than any city it's size.
It also had the most liquor stores, per city block, than any city it's size.
Correlation was that most church goers also drink lots of alcohol.
You can twist any study, to show what you wish it to show, to prove your point!
So, I agree, if the study was of 1,000 people with MS, and the benefits of the medicine in question, helped 100 people (i.e. those 100 people claimed their symptoms were improved, etc.), then 900 people with MS, saw no benefit at all.
The company slanted their findings to get the desired result. Since the medicine "helped" 100 people with MS, then it is a known medical break through?
The answer is, NO, but because we want medicines that "might" help us, we fall
for their "slanted truth".
We need to make the FDA force these companies to give us the "full picture".
How many people were in the test (or trial) overall, and how many actually benefited.
I am one , as much as I try, don't understand the language and terminolgy
to understand all this information. This forum means so much to me amd is so helpful
Thank goodness my daughter also is a big help in guiding me.
(She is a molecular/cell bio with masters and is in research.)
Also, she can access scienctific publications and has been to NIH numerous
times
thank you all sincerely
Linda
to understand all this information. This forum means so much to me amd is so helpful
Thank goodness my daughter also is a big help in guiding me.
(She is a molecular/cell bio with masters and is in research.)
Also, she can access scienctific publications and has been to NIH numerous
times
thank you all sincerely
Linda
Lot of misguided folks out there on countless fronts because they don't understand the language/terminology, or the process in which valid studies are conducted.
thank you much for this,
-shell
thank you much for this,
-shell
I agree with what both of you have said, I have seen some wild things on the internet that were suppose to cure MS. I always consider myself smart enough to know that 99.99% are just crazy. And it saddens me to see people so desparate for help that they fall prey to these charlatans.
As far as these special diets go, I'm sure that it can help some, but not cure, as we and our bodies are all different and no one can be sure these diets are for everyone. I tried the protein diet and it actually caused my fatigue to worsen.
As for the LDN, which I take. I don't know if the scientific evidence shows any significant positive effect or change for MS. You can read anything you want to hear, but as with other things I just take it all with a grain of salt...haha
The LDN has helped me a little. I don't have the benefit of health insurance, so it's a low cost way for me to feel just a little better.
I took it for about 6 months and saw improvement in my fatique and leg weakness. Not a major difference, but enough that I was more able to finish my daily tasks without wanting to pass out by 1 pm....haha
I decided to stop taking it, just to see if it was really making a difference or if I was just imagining it. After the 2nd week of not taking my LDN, my legs became so weak I couldn't walk across the room, and the fatique was completely overwhelming. I stayed off the LDN for about 5 wks, until I couldn't deal with the fatigue any longer.
I'm back on it now, and the fatigue is a little better and so are the legs. I have no scientific way of knowing for sure if the LDN is really helping, but it does make me feel better, and since I have no alternative, I take it, and will continue to do so. But would NEVER tell anyone else they should take LDN instead of a DMD!!
As with the DMD's, some work for some people, and other's don't. I read all the time about patients changing from one DMD to another as the one they're on doesn't seem to be helping.
I think everyone should be wary and hyper aware of all treatments, but the DMD's, as none of them have had the necessary scientific trials. But everyone should also remember that all courses of MS are as different as the people who have it and each person responses differently to treatment.
Thanks Quix for the informative post, I totally agree that there are too many "cures" for MS out here that aren't worth the paper they are written on. Giving false hope to those who are suffering, is wrong, and I hope everyone will be careful and not fall for these scams.
doni
As far as these special diets go, I'm sure that it can help some, but not cure, as we and our bodies are all different and no one can be sure these diets are for everyone. I tried the protein diet and it actually caused my fatigue to worsen.
As for the LDN, which I take. I don't know if the scientific evidence shows any significant positive effect or change for MS. You can read anything you want to hear, but as with other things I just take it all with a grain of salt...haha
The LDN has helped me a little. I don't have the benefit of health insurance, so it's a low cost way for me to feel just a little better.
I took it for about 6 months and saw improvement in my fatique and leg weakness. Not a major difference, but enough that I was more able to finish my daily tasks without wanting to pass out by 1 pm....haha
I decided to stop taking it, just to see if it was really making a difference or if I was just imagining it. After the 2nd week of not taking my LDN, my legs became so weak I couldn't walk across the room, and the fatique was completely overwhelming. I stayed off the LDN for about 5 wks, until I couldn't deal with the fatigue any longer.
I'm back on it now, and the fatigue is a little better and so are the legs. I have no scientific way of knowing for sure if the LDN is really helping, but it does make me feel better, and since I have no alternative, I take it, and will continue to do so. But would NEVER tell anyone else they should take LDN instead of a DMD!!
As with the DMD's, some work for some people, and other's don't. I read all the time about patients changing from one DMD to another as the one they're on doesn't seem to be helping.
I think everyone should be wary and hyper aware of all treatments, but the DMD's, as none of them have had the necessary scientific trials. But everyone should also remember that all courses of MS are as different as the people who have it and each person responses differently to treatment.
Thanks Quix for the informative post, I totally agree that there are too many "cures" for MS out here that aren't worth the paper they are written on. Giving false hope to those who are suffering, is wrong, and I hope everyone will be careful and not fall for these scams.
doni
I meant to add that "Patient-Based Evidence" - in my mind - is not "evidence" at all. It is "data". It is a series of data-points that prove nothing, but may suggest that something is true and needs to be studied for true evidence. Calling it evidence doesn't make it evidence.
Quix
Quix
Even with drug studies I look at who funded them. Recently a medical journal had an article with the results of a study of how safe and effective Copaxone is. I looked at who did the stud, Teva the drug maker. I am not saying Copaxone is not a good drug I just look at the study a little more objectively.
Also when the "New" possible cures for MS hit the news I do not get that excited. The news tends to exagerate or over simplify the findings. I do further research and find it is usually one study with may be 100 participants. This is promising but not a home run.
LDN is a wide spectrum cure. The internet sites have it helping many things. I am personally leery of anything that helps a whole host of diseases. The endorphin stimulation could make you feel better so in that case it may be alright but I can't see it helping the course of MS.
I definitely stay away from anything that claims to cure MS. Diets, supplements, etc.
I do my homework.Even when a Doctor suggests a treatment I read up about it before I decide if it is right for me.
I believe in a well balanced diet and exercise. If nothing else keeping my over all health under control and not having extra pounds to drag around.
My MS Specialist lent me a book Curing MS: How Science is Solving the Mysteries of Multiple Sclerosis by Dr. Howard L. Weiner. This guy has been trying to find a cure for MS and even understand MS his whole career. He goes step by step through the history of research. I gained a great appreciation for the complication of understanding this disease and for even coming up with treatments. Each step takes a long time.
I have not given up hope. I am just realistic. No one understands this disease yet.
Alex
Also when the "New" possible cures for MS hit the news I do not get that excited. The news tends to exagerate or over simplify the findings. I do further research and find it is usually one study with may be 100 participants. This is promising but not a home run.
LDN is a wide spectrum cure. The internet sites have it helping many things. I am personally leery of anything that helps a whole host of diseases. The endorphin stimulation could make you feel better so in that case it may be alright but I can't see it helping the course of MS.
I definitely stay away from anything that claims to cure MS. Diets, supplements, etc.
I do my homework.Even when a Doctor suggests a treatment I read up about it before I decide if it is right for me.
I believe in a well balanced diet and exercise. If nothing else keeping my over all health under control and not having extra pounds to drag around.
My MS Specialist lent me a book Curing MS: How Science is Solving the Mysteries of Multiple Sclerosis by Dr. Howard L. Weiner. This guy has been trying to find a cure for MS and even understand MS his whole career. He goes step by step through the history of research. I gained a great appreciation for the complication of understanding this disease and for even coming up with treatments. Each step takes a long time.
I have not given up hope. I am just realistic. No one understands this disease yet.
Alex
Have an Answer?
You are reading content posted in the Multiple Sclerosis Community
Top Neurology Answerers
Australia
ON
Queensland, Australia
Queen Creek, AZ
Find out how beta-blocker eye drops show promising results for acute migraine relief.
In this special Missouri Medicine report, doctors examine advances in diagnosis and treatment of this devastating and costly neurodegenerative disease.
Here are 12 simple – and fun! – ways to boost your brainpower.
Discover some of the causes of dizziness and how to treat it.
Discover the common causes of headaches and how to treat headache pain.
Two of the largest studies on Alzheimer’s have yielded new clues about the disease
