Randomized controlled trial (RCT) studies—in which participants are randomized into either a treatment or control group—are a common practice in Western medicine and the pharmaceutical industry. Since the pandemic started, we have quickly had to become literate in public health measures, learn terminology like the “N-95” mask, and appreciate how long it takes to develop a vaccine. As many will have read or heard, a vaccine must go through several rigorous RCT phases with increasingly large populations before it can be deemed safe.
RCTs have also gained popularity in the social sciences. They are considered the “gold standard” of scientific methodology because they seek to reduce selection bias to the greatest extent possible by randomly assigning people. The goal of an RCT is to ensure that there are no significant differences between the groups being compared so that the outcomes can be attributed to the intervention.
But they can be controversial and challenging, especially when they involve people who are vulnerable, marginalized, or requiring services. Critics point to the ethics of randomizing vulnerable people into a control group and the harm that such an experience can cause. Researchers and community-based organizations have also discussed the challenges that emerge in applying this scientific standard to real-world contexts. These are important questions, and ones that the Making the Shift (MtS) Demonstration Lab has wrestled with over the last two and half years.
When the pandemic hit, these questions took on a new urgency. Recently I spoke with Alexandra Amiri, a former Research Assistant with the Making the Shift Demonstration Lab, to learn about how the pandemic shifted the context of research and how the lab was able to adapt the Housing First for Youth RCT design to respond to an unprecedented global health and economic crisis. This adaption was accomplished in large part because of the community-research partnerships the lab has formed and the collective impact approach it takes.
MB: Let’s start with RCTs. Why do researchers believe they are a gold standard?
Alex Amiri: Many argue that in order to affect change at a policy level, rigour is important, and as of now RCTs are considered the most rigorous in terms of the evidence they produce. The At Home/Chez Soi project was a very large RCT with over 2000 participants, so I can understand that when a similar intervention was piloted with youth experiencing homelessness that an RCT would be implemented.
MB: Tell me about the Housing First for Youth study. What do you think are the advantages of having an RCT to demonstrate the effectiveness of the Housing First for Youth program? What are the disadvantages?
AA: The Housing First for Youth study was developed as a way to address the developing needs of youth experiencing homelessness and meet some of the lessons learned from At Home – namely to incorporate more developmentally appropriate supports created for youth as a way to help them transition to adulthood and provide a permanent exit from homelessness. I really do see the value in using an RCT. However, when it starts to become the gold standard for social interventions I think it becomes even more important for researchers to think about what they are offering to one group and not the other, particularly if you work with vulnerable populations.
MB: What was your experience collecting data in this study and the randomization process?
AA: I really enjoyed being involved in this study and the journey we went through with the young people. The data collection process gave me insight into the lives of the youth who were in this study and not just by virtue of the data itself. In fact, meeting the young people on a consistent basis gave me an opportunity to hear about their lives and how the services in their lives, or lack thereof, were affecting them. Sometimes these nuances are not captured in the data.
As for the randomization process, it was difficult to say the least. All the youth who signed a consent form were told extensively about the study and the randomization process, but of course when it comes down to being told you’re in the control group understandably there were youth who were disappointed. I was so grateful for my team in those times. We debriefed with each other a lot and tried to find ways to make the process easier for one another and especially for the young person.
MB: You mentioned in previous conversations that there was a drastic change once the pandemic hit. Can you describe that shift? What did you notice about the youth in the control group who were receiving regular services in the community versus the youth who were receiving Housing First for Youth services after the pandemic?
AA: I think once we were in a global crisis, the differences really came to light. I can only speak to my experience with the youth I was in close contact with, but there was a stark contrast to me with youth who had a support system to fall back on and those who suddenly found themselves having to navigate a pandemic on their own. All of the changes and new information that came with the pandemic, including the introduction of CERB, decreased access to some services, having to understand new terminology, were and are difficult to navigate on your own as a young person experiencing homelessness. Many of my participants in the intervention group were very grateful for their case manager during this time.
"Before covid-19 I had a plethora of support systems at my disposal. For things like mental Health services, food banks, social and communal activities, etc. When covid-19 hit, I lost all of these supports. Both my mental well-being and livelihood have been at risk. I went from consistent mental Health services to not being able to see anyone. I went from being able to make sure that I have places to get food to not be able to go anywhere. As an artist I do a lot of showings and events where I'm able to sell my art. Those things offer communal support and financial support. As a result of not having anywhere to sell my art I don't have any money to buy supplies to make more art, I'm unable to use this free time to work on that.” --Young person in the control group in Toronto
MB: How did the team adapt the RCT to respond to this changed context?
AA: Our team decided to make our primary study end at 18 months as opposed to 24 months. One of the factors that led to this decision was the new circumstances in which youth receiving regular services in the community found themselves. I was lucky that I was able to be a part of this change and watch as the youth in the control group who were now past their 18 month time point in the study were moved to receive HF4Y services.
MB: This adaptation seems like an important innovation made by the lab. Do you think we’re entering a new paradigm for social sector research?
AA: I recognize that not all studies can offer the intervention to their control participants (although that would be great!), but I do believe that when we work with vulnerable populations like youth who experience homelessness, something should be set in place. Our research team was given an opportunity that I know may not be the case for other studies, but it definitely felt like the right thing to do.
I also remember very early on after the quarantine was in effect our research team put together a “living document” of resources that we sent out to all the youth in the study. We would try to update the document on a weekly, sometimes daily, basis; that’s how quickly new information was coming out. At the time it felt like such a small gesture on our part, but I remember some of our control participants mentioning they used the document to find free groceries when they didn’t have money to pay for food. It just goes to show that sometimes you don’t need to make big changes to have a big impact.