I bet you have filled out questionnaires like we all had to do at one time: a paper form (or a digital one) with questions, yes/no, multiple choice or open questions, to be answered by you for other people to know more about you.
And sometimes there is no problem in doing that, the answers are easy and based on facts, like for athletes: what sports do you do, when did you start in your sport, what is you birthdate, your weight or your height? All reasonable easy to answer, but not completely accurate, take for instance your weight and height in your passport probably filled out a long time ago and often estimated instead of measured.
No, my problem is that I see more and more questionnaires being used as a “scientific” tool to derive useful information from individuals or groups.
Here are my objections: most questions aren’t about objective measurable information, like bodyweight or birthdate, but just a subjective self-estimation, in my case about your health or about stress.
And I see a huge problem here: first of all these questions are subject to be blurred by intentions: the intention of the questioner and the intention of the one who has to answer the questions. An example: the weird question we have to answer when travelling to the USA: it goes something like: “do you seek to espionage, sabotage, export violations, or any other illegal activity while in the US?” yes/no.
I wonder how many people have answered this question with: yes…….. (and if they did, why they did it).
Have you ever re-read your own curriculum since the last time you applied for a job? Is that the “real” you or did you skip, ignore, forget, or play down your less desirable qualities? And during a job interview you are asked whether you are, let’s say “stress-resilient” or a “team player”. Would you really have answered that you are a fragile as Chinese porcelain when it comes to stress, or that basically you are a nervous wreck that tends to crack under pressure….? Or that in reality you are a pure individualist who likes to work according to his/her own rules?
No, of course not, because of the intentions of the questions, you just give socially desirable answers.
And these are answers that you probably have enough (self-)knowledge about.
But what about the question: do you possess an adequate mental resilience? What is adequate and what is resilience? It depends on your definition and your frame of reference.
We even see questionnaires being used as a measure of heath and I cannot disagree more with that.
Our car comes with a manual and has a dashboard that shows us objectively what is happening with our car, the temperature of oil and water, the amount of petrol left, the RPM and the speed, etc.
Our organism just did not come with a manual, nor do we have a dashboard, so that makes it hard to tell if we have a headache from drinking too little coffee or if it is because of a brain tumor. We cannot judge if the pain on our left part of our upper body is because we slept on our left shoulder or if this is the sign of a heart attack. That is why the expression “listening to your body” is ridiculous, our body does not speak like a dashboard, but just mumbles at most, ”pain, pain” without the slightest indication of what the cause is. If our body could speak like a dashboard, we’d never have to take an X-ray, MRI, ECG or take blood…..
Even worse you can have a disease for a long time, even for years, like hypertension, diabetes or a tumor, without any noticeable sign or symptom.
That makes the filling out of a questionnaire for the purpose of measuring e.g. health a questionable activity.
A field in which questionnaires often are used is nutrition where dieticians make you write down what you ate over e.g. the last week. Here the differences between the results of the questionnaire and the real caloric intake by measurement are no less than shocking; there is up to 40% underreporting of calories!
You can see this phenomenon when you watch weight-loss programs on TV where a family with a weight-problem is visited by a weight-loss consultant who places cameras in the house and asks the people how much they ate that day. ”How many spoons of gravy did you take with your potatoes?” “One, I always take only one spoon”. Studying the recordings however show that this person took 5 spoons of gravy with his potatoes. He probably wasn’t lying to the consultant, he knew everything was recorded, still, his perception was very distorted.
Yes, of course, I can see the advantages of using questionnaires to get information: it’s cheap, it’s fast and it’s very efficient, but there is a huge price to pay in terms of value and reliability.
I don’t believe that somebody has a normal blood pressure because they wrote it down, I don’t believe that people are getting overweight from only drinking water, just because they filled it out on a form. I refuse to believe that people are sick or healthy, stressed or not, or that athletes are tired or “in good shape” just because they say so or filled out the questionnaire. Questionnaires are not a proper alternative for objective testing or measurements. It’s not that I believe that people intentionally lie in their answers, I believe they honestly don’t really know or their answers are tainted by their subjective perception or their beliefs.
Questionnaires: be very critical when using them.
Limited bibliography:
Scholler, D.A; Foreyt, J; Thomas, D; Self-report–based estimates of energy intake offer an inadequate basis for scientific conclusions; Am. J Clin. Nutr.Vol.97, No.6, 2-13, pg.1413-1415.
Kipnis, V; Midthune, D; Friedman, L: Bias in dietary-report instruments and its implications for nutritional epidemiology; Public Health Nutrition, Vol.5, Issue 6a, 2002, pg.915-923.
Schatzkin, A; Kipnis, V: A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study; Int.J.Epidemiology, Vol.32, 2003, pg.1054-1062.
