Quantitative Unease
A column dedicated to demystifying psychotherapy research – love it, hate it, or both…at least try to know what it’s all about!
(Matrix) Questions and Questionnaires
How would you answer this question: “Does it seem possible or does it seem impossible that group analysts never contributed to the extermination of persecuted people?”
Hmm. What kind of question is this, you may wonder. And what would it tell you about the matrix, if anything at all? You pause and think.
Is it a factual question that elicits objective information? No, because it relates more to your subjective experience, attitudes, opinions, feelings and beliefs than objective facts. It is subjective and you may hold a strong view about this issue and be hesitant to answer at all. Or you may not understand the question immediately. Be honest, how many times did you read the above question before you answered it, if you did? And if you answered, are you sure your response expresses your true belief?
If the question confuses you that may not be related to your unconscious but to the clumsy wording. How we phrase questions matters not just in therapy but in research, too. Surveys influence policy decisions and therefore questionnaire (survey) questions need to be carefully constructed. The secret of a good questionnaire lies in the matrix of questions and there are many types of questions:
- Open-ended or closed ones. Open-ended questions aren’t followed by a specified choice (yes or no) like closed questions but are, as the names suggests, open-ended (“How do you feel about …?”).
- Contingency questions are specific closed questions where the relevance of the second question is contingent on the previous ‘filter question’: “Is it the first time that you thought about group analysts’ involvement in torture?” (Yes or No.) If answered Yes: the contingency question is: “What happened as a consequence?” (1. Nothing. 2. I quit my profession. 3. I drank a bottle of Whisky.)
- Matrix questions aren’t about the network of relationships but are a method of organising a large set of rating questions that have the same response category (“I strongly agree; I agree; It depends”).
- Threatening questions are anxiety-arousing questions such as the ones posed here about persecution and torture. Frequently, they result in response bias (respondents either deny or under-report). To decrease bias, open-ended questions and a long introduction to the specific question man yield more accurate (valid) answers.
- Leading questions have nothing to do with leadership but are phrased in such a manner that certain answers are expected (“Would you say you’re not in favour of group analysts getting involved in torture?”). The respondent is expected to answer ‘yes’ here.
Just like in therapy, when constructing a questionnaire, the wording, format and sequence of questions are crucial. When asking questions, avoid ambiguous words and choose words that are easily understood (by the average 6th grader).
Different formats are possible. You can present questions and ask respondents to choose a category (strongly agree; agree; disagree strongly). Such a Likert rating scale is commonly used in questionnaires. But you can get creative and use ‘semantic differentials’ by contrasting adjectives at each end of a bipolar scale (bad-good; fair – unfair). Or you can rank answers. It yields information regarding the degree of importance (“Which characteristics of a group analyst are important?” Honesty. You can assign ranks 1 2 3 4; Reliability. Ranks 1 2 3 4).
You can use a ‘funnel sequence’, where each successive question is related to the previous one with a progressively narrower scope (“What do you think are the most important problems facing Group Analysis?” Followed by: “Of the problems you mentioned, which do you think is the most important one?”).
In an ‘inverted funnel sequence’, questions are narrow, followed by broader ones (“How many group analysts are conducting research?” Followed by: “How many do you think publish their results?” Followed by: “In general, how well is research carried out by group analysts”?).
Probably you all know that the order in which questions are presented affects the responses. Interestingly, answers to attitude questions can vary depending on the proceeding items in the questionnaire. Bear this in mind. The position of items also matters. The first items on a list are often chosen and receive higher ranks as well. And the first rank is more commonly assigned to the first category than the last, particularly, when asking about beliefs and attitudes.
You may now be totally confused, scratch your head and wonder what can be done to avoid these pitfalls? As it turns out, the answer to many questionnaire difficulties can be found in the mix of the matrix. Vary the wording and sequence of matrix questions. Acquaint your respondents with the list of items and change your question format to avoid a ‘response set’ (answers are in a specific direction regardless of their content).
Go and (re)search your questionnaire this Easter and let me know the outcome. Until the next time, Seasonal Greetings to all of you.
Susanne Vosmer
s.vosmer@gmail.com