Directly after the exposure to the matched-guise experiment, the participants’ perception of the recorded discussion is collected via a net-based questionnaire. The questionnaire is integrated in the package so that at the end of the matched-guise recording, the students will get access to it. The questionnaire focuses on relevant aspects of the perception of the discussants in the matched-guise design. Thus, an experiment set up in sociolinguistics may focus on conversational features such as the perception of the turn-taking and use of conversational features, whereas an experiment set up in personality psychology may focus on personality features such as the discussants' perceived personalities, estimated in relation to the theory of Big Five personality traits.
Group scores are then calculated automatically and later revealed. Individual data are anonymous and cannot be linked to an individual and are therefore not considered, an important ethical consideration in this context. The recorded differences between the groups serve as the starting point for the discussions in the debriefing session. Since the two groups have listened to exactly the same recording, the null hypothesis would predict no difference between the groups and, consequently, any result deviating from the null hypothesis is worthy of attention. For this reason the design of the perception test is important for final outcome. Ideally, the test should be able to record the stereotypical differences that the matched-guise experiment is set up to generate. Below we give examples of two different sets of perception tests.
Example of perception test - the case of sociolinguistics One university course in which we have used the RAVE method is English sociolinguistics with a gender focus. One of the aims of this course is to make students aware of the gendered patterns that exist in conversations and the expectations they imply. Thus we wanted to expose our students to their preconceived ideas of how men and women interact in conversations. The design of the perception test is illustrated in Figure 1 below.
Example 1. Perception test for conversational features using percentage of conversational features
A perception test such as that illustrated in Example 1, will be able to identify systematic differences between the two groups listening to Robin and Terry as men and women. Because the conversations are identical, there are measurable correct answers to questions 4 and 5. Thus, a systematic difference between the answers of the groups doing the matched-guise experiment would most plausibly be explained by stereotypical assumptions affecting the participants' perception of floor space and use of hedges. Such differences serve as the starting points in the debriefing, where the participants' are forced to come to grips with their own answers (as groups).
Example 1 illustrates the design of the perception test in the pilot study. One problem we discovered was that students found it difficult to concentrate on two individuals and also found it difficult to keep them apart. The gender neutral names (which we had chosen with care) contributed to this diffuculty. The participants also found it difficult to estimate features and floor space in terms of percentages.
For this reason we developed the set-up and the perception test in such a way that only one of the discussants was in focus. We also reformulated the questions in the perception test so that the particpants had to react to a statement in terms of agreement or disagreement in 7-point likert scale. The new design is illustrated in Example 2.
Example 2. Perception test for conversational features using a likert scale of agreement/disagreement
In Example 2, the participants only have to focus on their impression of one individual and react to statements. These statements are no longer relating to objective facts, which was the reference point in Example 1 where we had an actual objective figure, but on subjective impressions. However, any systematic difference between the two groups could still be related to how the participants' perceptions were affected by expectations on men and women in conversations, and be problematized in the debriefing session. Example of perception test - the case of Personality psychology
Another course in which RAVE has been implemented is Personality psychology. As in the case of sociolinguistics mentioned above, the aim was to raise students' awareness of how gender stereotypes may affect their analysis of personality traits. Because of the different nature of the context the perception test was designed differently. Here we relied on established tests, among them the Ten-Item Personality Inventory-(TIPI) (Gosling, Rentfrow and Swann Jr, 2003), to estimate the participants' perception of one of the participants. The TIPI design is illustrated in Example 3 below.
Example 3. Perception test for personality traits using the Ten-Item Personality Inventory