RUNNING PARTICIPANTS 101
Running participants is one of the most important parts of the scientific process! Good data is essential for valid scientific research. Collecting good data, especially in behavioral research, is the responsibility of the experimenter. This page discusses we can do to collect the best and most valid data possible.
- Ensuring all participants are run in a safe, ethical way
- Ensuring all participants have the same psychological experience during the experiment
- Ensuring all participants understand instructions
- Ensuring the experimenter is able to answer any questions the participant may have
- Being cognizant of deadlines, recruitment, any issues with the study, and staying on-task
- Debriefing participants
1. All participants must be run in a safe, ethical way
Experiments like the infamous Stanford Prison Experiment show us why ethics are so important
This is the cardinal rule of human subjects research. All participants must be run in accordance with the project or lab’s IRB guidelines. This is entirely the responsibility of the experimenter, NOT the participant. One mistake here can have serious consequences!
All experimenters must understand what is and isn't okay when it comes to ethical treatment of research participants. This is why CITI Training is required for all experimenters working with human subjects. CITI Training goes over the history of ethics in human subjects research, as well as consent protocols, and general ethical guidelines.
However, just because something is ethical in general or seems like it wouldn’t be dangerous, doesn’t mean it is covered under our IRB!!! Our IRB protocol only covers a subset of all the awesome stuff we can do in psychology research. Each study we run has been carefully designed and approved by the Dartmouth IRB, so ANY deviation from how this study is designed can be really problematic, not only for the study, but for the lab as a whole. It’s crucial to adhere EXACTLY to the protocol that your lead researcher has trained you in (whether you're using a script, or running the study from memory). If you have questions or something goes awry during the study, now is NOT the time to improvise! Ask the lead researcher!! Improvisation, while tempting, could not only be unintentionally a violation of our IRB, but also could compromise the integrity of the research, which brings us to our next point...
2. All participants must have the same psychological experience during the experiment
Don't make your participant the odd one out
When we study psychology and neuroscience, we’re really interested in how the mind and brain work in different contexts. One difficult aspect of this kind of research is eliminating confounding variables. This is one huge benefit to in-lab studies vs. real-world testing: the experimenters are able to eliminate many of these variables by giving participants all the same simulated experience in a lab setting.
However, this balance can be easily disrupted. When all participants in a study are not treated the same by each and every experimenter, we lose validity in the conclusions we are trying to draw with our research.
For example, let’s say someone is running a study on how sadness affects our ability to respond to different colors, and this study involves a mood manipulation in which participants are purposely made to feel sad. The study has two experimenters: if one experimenter is overly friendly to participants because they personally feel bad about making them feel sad, while the other one is strictly professional, half the participants will be having a different psychological experience than the others! The friendly RA’s kindness to the participants, although well-intended, could mitigate the effect of sadness altogether, yielding an inaccurate representation of the phenomenon, potentially non-significant findings, and also a waste of the lab’s time and precious funding resources.
This may seem a bit extreme, but this is the reality of the situation. To have valid findings, ALL participants must have the same psychological experience while participating. In order for this to happen, RAs must stick to study protocols EXACTLY, even if this includes reading from a script for each and every participant.
3. All participants must understand instructions and research materials
Keep an eye out for this face
This may seem obvious, but it’s so important! Participants need to all have the same understanding of the task in order for us to learn anything generalizable about a psychological phenomenon. That's why we have instructions both on the screen, AND read these same instructions to our participants.
One pitfall that experimenters may experience is the inclination to read all the instructions really fast. This can come from nerves, or just from being someone who speaks and reads quickly; but it’s important to be conscious of the speed at which participants are hearing instructions. This not only allows them to synthesize what they’re hearing better, but it gives space for them to ask any questions they have, which is REALLY important! We want to resolve any questions or confusion they may have BEFORE we start the study, so we can make sure all their responses to the stimuli are informed by a full understanding of what we’re expecting them to do.
Another really important strategy is to ask at every step of the process whether or not our participants have questions - often they might be shy to ask if not given the opportunity. The way we phrase this is REALLY important: we want the participant to understand it’s important that they ask any questions they may have, while also not assuming that they understand everything they are supposed to do. So, instead of asking participants “do you have any questions” which may assume that they don’t, some better questions to ask are “what questions do you have?” “is there anything confusing that I can clear up?” or “let’s reiterate these instructions, unless you feel you fully understand them.” It’s also important to ask if participants have any questions about the consent forms. We need all participants to understand consent materials and feel comfortable participating in the study.
If you are at a loss as to how to better explain part of the study, don’t just continue on with the study and hope the participant gets it eventually. It’s okay (and encouraged) to go find a senior lab member and ask for help! And luckily, everyone in our lab is really nice and totally willing to help out.
4. The experimenter must be able to answer any and all questions the participant may have
You as the experimenter
Being able to answer questions isn’t about always knowing all the answers! It’s about clearly and accurately communicating what you do know, and knowing when to seek out someone else to ask for help. You DO need to be familiar enough with the study that when a participant asks you what a scale is measuring or if you could give examples of a concept, you’ll be able to accurately answer their questions. But if a participant asks a question you don’t know the answer to, or aren’t totally sure, ASK A SENIOR LAB MEMBER. Ideally, ask the lead researcher of the study (usually a grad student or postdoc), but if they aren’t available, the lab manager is another really great resource!
The only question we DON’T want to answer (until the end of the experiment) is “what is this study trying to find” or “why are you having me do this?” In these situations (when the question isn’t about understanding directions or a specific task, but more about the hypotheses of the study), it’s okay to say “I’m sorry, I can’t answer that question now, but I’d be more than happy to answer that question at the end of the study.”
5. Keep your study running smoothly
I’ll split this into four sections.
Be aware of what kind of deadline your project has. If you have a strict deadline, make sure you have a steady stream of participants signed up for and participating in your study, and that you’re on track to run participants. This includes sending email and text reminders to participants so they don’t miss their appointments, flyering, sending info out to mailing lists, and recruiting in any other capacity that’s covered by the IRB (ask the lab manager if you’re unsure). If you have a more relaxed deadline, it’s still important to follow all of the above guidelines.
B. Active recruitment
As an experimenter, it's also your responsibility to make sure there are participants actively signing up for the study you are running. This involves ensuring all participants qualify for the study (are in the correct age range, meet all inclusion criteria, and don’t meet any exclusion criteria), have completed all necessary steps prior to participating (prescreeners, online surveys, submission of relevant information), and reducing the incidence of no-shows by following up with each participant to remind them to complete aspects of the study, or to remind them of their appointments.
C. Issues with the study
Issues come up in studies sometimes. There might be a bug in a MATLAB script, or a confused participant, or a little mishap. If anything out of the ordinary happens while running a participant, it is the experimenter's responsibility to make a note of what happened, in very clear detail. For example, if a script crashes and needs to be started over, this must be documented (usually on the study’s Google doc/participant log). Contact the lead researcher right after you finish running your participant, so the issue can be resolved as quickly as possible.
D. Staying on task
Running participants may not always be the most exciting thing, and often involves a lot of waiting around by yourself. It can be tempting to veer off track: checking social media sites, browsing the internet, etc., but your responsibility while running participants is just that - running participants! Being unfocused or distracted while running participants can yield a lot of mistakes, and these are totally avoidable! Your attention to your participants is 100% the most important thing - everything else can wait.
7. Debriefing participants
The first part of the debriefing process is ALWAYS trying to gauge the participant’s level of suspicion by asking questions. These questions will be the same for every participant, and will be listed on the experimenter script provided by your lead researcher. If you don’t have any information on debriefing your participants, ask your lead researcher! The reason we ask participants questions BEFORE we tell them about any deception is pretty intuitive: we don’t want participants to feel foolish that they believed our deception, and pretend they knew what we were doing all along.
The second part of the debriefing process is to let participants know we have deceived them, and to tell them a bit about the study they are participating in. This includes giving participants a debriefing form that also says that they have been deceived, and includes contact information for the IRB and for the lead researcher in case they have any concerns or questions after they leave.
We need to ensure all participants have a positive experience participating in research, so we have to provide them with the opportunity to express any thoughts or feelings they have about the study after they leave.
Hopefully this document has shed some light on why we run participants the way we do! Participants and RAs are the lifeblood of psychology, and without them, who knows where the field would be?! If you have any questions or are unsure about anything discussed here, please ask a senior lab member, and they’ll help clear up any confusion you may have!
...and now you can test your participant-running expertise with a quick quiz!
Choose the BEST answer for each question.
1. Drew is a new RA running his first participant! Drew opens MATLAB and types in the participant’s ID number to start the experiment. However, Drew gets an error saying that the participant’s ID number already exists, and the options are to cancel or to overwrite the file. What should Drew do?
a. Overwrite the file, it doesn’t look like there’s another participant with that subject ID, so it should be fine.
b. Cancel the experiment and start this participant over with a different ID number that doesn’t give an overwrite error.
c. Politely interrupt the lead researcher’s conversation with another grad student to ask what to do.
d. Just use the next sequential ID number, since there isn’t any data for the next participant yet.
2. A participant comes in for a study, and Taylor, the RA, observes that the participant looks like they might be under 18 years old. This study (and currently every SSNL study) requires participants to be over 18 years old. What does Taylor do?
a. Ridiculously asks this participant and every participant how old they are, even though Taylor is sure they are over 18.
b. Logically assume that the participant is at least 18 if they were able to sign up for a timeslot on SONA and don’t mention it.
c. After the participant fills out the consent form and completes several surveys and a demographics questionnaire, check the age listed on the demographic questionnaire and contact the lead researcher if the participant is younger than 18.
d. Ask the participant which year in school they’re in - if they are not a freshman, assume they’re over 18.
3. Jesse is an RA who has been at the lab for several quarters. Jesse is running participants during finals, and has a huge paper due in a few hours. What is Jesse’s best course of action?
a. Bring all the paper writing materials to the lab and work on the paper while the participant is preoccupied with different parts of the study.
b. Text Hadley, another RA working on the study, and ask if they can trade timeslots, since the paper is going to take up more cognitive resources than is ideal to run participants.
c. Cancel the participant and tell the participant an emergency came up and that they’ll have to reschedule.
d. Cancel the participant and pay them anyway for the inconvenience.
4. One of Sydney’s participants looks confused when they are explaining study instructions. What is the best thing for Sydney to say to the participant?
a. “Do you have any questions?”
b. “Does everything make sense?”
c. “Should I go over it again?”
d. “What questions do you have?”
5. Terri is an RA running a study on emotion. Part of the study involves a mood manipulation in which the participant is induced to feel sadness. Terri feels bad about the participant feeling sad, they want to be especially nice to this participant. What should Terri do?
a. Smile at the participant and be extra friendly so the participant doesn’t feel so bad
b. Tell the participant they are sorry, but that they can’t be friendly with them
c. Treat the participant just like they have treated everyone else in the study, and compensate with extra friendliness after debriefing if they still feel the need to
d. Give the participant extra money to make up for making them feel sad
6. Alex is running a participant who is a psych major and is curious about the hypotheses of the study. While Alex is explaining the instructions, the participant asks Alex what the study is trying to measure. What should Alex say?
a. “We’re examining how feeling different emotions affects how people process images, but I can’t tell you exactly what we expect to find.”
b. “I’m sorry, you’re not allowed to ask questions until the end.”
c. “We think people respond to images slower when they’re sad, but don’t let that alter your natural response to the images!”
d. “I’m sorry, I can’t answer that right now, but we can come back to that question when you’re done with the task!”
Answers: 1) c. 2) a. 3) b. 4) d. 5) c. 6) d.
(Info and quiz adapted from Zoe Kleiman)