A Conversation On Anecdotal Evidence

This is an email conversation between Randall Johnson and Con Slobodchikoff on the value of anecdotal evidence in biology.

Randall Johnson:
I've submitted a new Dog Behavior Blog comment (http://www/dogbehaviorblog.com) in response to your latest entry, "A Dog's Concern for a Cat". Focus: anecdotal evidence as a useful source of information.

Comment on Dog Behavior Blog: This incident joins a long list of other reported observations that clearly show that dogs are capable of feelings we associate with empathy and sympathy and that they extend these feelings toward other species.  If we had only a handful of stories like this one, then we could say this is anecdotal evidence, but when there are hundreds of stories reporting similar behavior, we no longer have anecdotal evidence, but rather a solid body of observational evidence. Since the human environment is the dog’s natural habitat, then these reports can also be considered as field observations, which is a respected source of data-gathering. I don’t know if anyone has decided how many anecdotes it takes to cross over to the realm of observational field data, but if this hasn’t been done yet, it’s high time some kind of marker was devised. And, in defense of anecdotal evidence, Hungarian ethologist Vilmos Csányi of the Eötvös Loránd University, who has devoted more than a decade to investigating dogs’ cognitive abilities, has used anecdotes on many occasions to inspire more formal experimental investigations.

Con Slobodchikoff:
Thank you for your insightful comment, which I published on the dog behavior blog.  As you know, "anecdotal evidence" can sometimes be used as a convenient way to sweep inconvenient evidence under the proverbial rug. There surely has to be some kind of standard for how much anecdotal evidence constitutes proof, but I don't know of any standards. However, if you go back to the papers in biology journals of 60-70 years ago, much of what was presented there was in the form of anecdotal evidence. Where would science be today if we discounted such evidence?

Randall Johnson:
While writing my comment, I was pretty sure there was no standard for anecdotal evidence. You raised a good point about this kind of evidence having played an honorable role in science in the past.
Somewhere along the way, it was stripped of some of its original credibility and we need to get it back or devise some clear guidelines for its use as a source of evidence.

I am submitting a brief comment in response to the "Are We Unique?" essay on the Reconnect With Nature blog. (I'm surprised this one hasn't generated more responses.)

Con Slobodchikoff:
First, thanks for your comment on Are We Unique. As usual, a great comment! I have posted it, and I hope that it raises some discussion or at least some thoughts on the part of the readers.

For the anecdotal evidence part, it seems to me that biologists have bought into the idea that experimentation is the only way to obtain valid biological data. This is fine for lab studies, but experiments are often difficult to do in nature, under field conditions, and a lot of the time we are left with observations that cannot be quantified or even repeated. In my mind, this does not make them any the less valid. I think we need to come back to a point where a certain quantity of similar observations is accepted as proof of the validity of a particular behavior or biological circumstance.

Randall Johnson:
My question, at the beginning of this discussion, was: At what point does an observed behavior stop being treated as anecdotal evidence and be recognized as observational data? Most definitions I’ve looked at stated that anecdotal evidence is limited to one or a small number of cases, but no one, so far, has said, “Well, if you have 10 cases of “X” behavior, it’s anecdotal, but if you have 11 cases, it’s observational data.” Maybe it’s not all that important, but considering science strives for precision, there’s a big fuzzy area here that’s calling out for better definition.

However, the larger issue is still related to the credibility of anecdotal evidence. In the late 19th century, it fell under some heavy criticism, largely because George Romanes, Charles Darwin’s protégé and an eminent biologist in his own right, relied heavily on anecdotal methodology to develop his theory of comparative psychology and, to this day, there still seems to be a lingering ‘cloud of doubt’ hanging over it.

Then, the other day, I came across a news item from 2005 on the Science Week web site that I found particularly insightful. In discussing numbers and counting in chimpanzees, the writer pointed out that one problem with investigating non-human primates is that experiments can’t be done on large populations because such populations either don’t exist or are prohibitively expensive to maintain. The result is that reported experiments are often "anecdotal", i.e., experiments involving only a few or even a single animal subject. The writer then stated: “But anecdotal evidence can often be of great significance and have startling implications: a report, even in a single animal, of important abstract abilities, numeric or conceptual, is worthy of attention, if only because it may destroy old myths and point to new directions in methodology.”

The debate continues. Although I’m not a gambling man myself, I’m willing to bet that, in the end, anecdotal evidence will regain its former luster and that the academic / scientific community will universally recognize it as a valid—and valuable—source of data. Let’s face it. There are still lots of old myths out waiting to be slain.

Con Slobodchikoff:
Yes, it is true that many experiments cannot be done on multiple animals because of the cost or the lack of animals. I was talking just the other day to someone about Alex, the parrot (see earlier post: A Tribute To Washoe And Alex), and the person I was talking to pointed out that we can’t generalize from only one parrot to say that parrots might have language-like abilities or the cognitive skills to count and identify colors and shapes: Alex might have been a genius among parrots, a feathered Einstein. That might very well be true, and Irene Pepperberg is doing careful experiments with other parrots to test for this possibility. However, Pepperberg’s extensive experiments with Alex show that not all parrots are incapable of complex cognitive skills.

Unlike Pepperberg’s work with Alex, many observations cannot be repeated on command. Does that invalidate them? I guess it depends on the credibility of the observer and the relative novelty of the observation.  If I say that I saw my dog burying a bone in my back yard, no one would question this anecdotal observation. However, if I say that I saw my dog surfing the internet on my computer, lots of people would probably discount that as pure fantasy, because it does not fit the current dogma of what dogs are capable of doing and no one else has reported this (and no, my dog does not surf the internet).

But we have to be careful about discarding observations that do not fit our preconceived notions. If an observation comes from a credible source and can be documented, then that observation can have the seeds of changing our thinking about the world around us.

As William James once said, you need only one white crow to disprove the assertion that all crows are black.

Comments

[john]

Thanks for this article. :)

[Ann Vole]

To address the lack of numbers for test subjects and to allow a wide variety of people around the world to conduct independent research on the same conditions but different individual animals, I hope to develop computer interactions with animal species already known to communicate verbally namely the Richardson Ground Squirrel. I will raise these animals as pets in a fairly close-to-natural environment and set up for people to communicate with them via chat and computer interpretation software and computer generated calls and voice recognition. To prepare such a project, I will be studying what you have concluded about prairie dog language.

[Ben Cluff]

Dr. Slobodchikoff,
I don't know if you remember me, but I took Animal Behavior and Why Save the Prarie Dogs from you in about 2005. I was reading a article today about animal communication and remembered you ard your work with PD's. I googled you name and saw your blog. I just thought that I would drop you a line and say hello. Hope that all is going well for you in you research.
I remember having a conversation with you one day after the PD class and we were talking about hunt, specially shooting PD's. I was amazed at the passion that you had for the animals. It had never bothered me before to shot a PD, but out of respect for you and your studies I haven't shot a PD since that talk. I do however still hunt, and feel nothing is wrong with that, but the PD's can live on. I appreciate your passion for life. I believe that God has given us this earth to be stewards over it and the animals. Sometimes we can do a better job. I am so far right that is it good to have people with passion such as yourself to help me meet in the middle on equal ground.
I am currently starting my 3rd year of medical school and have a beautiful wife and 2 children. I hope the best for you.
Ben Cluff