BetaGov Blog

Three Reasons To Do A Randomized Controlled Trial

Posted on   August 2, 2017     By Lauren Rilling

Woman thinking with light bulbs around her head

Wondering if a randomized controlled trial is the best way to test an idea at your agency? Wonder no more. An RCT is the gold standard for research, and research is worth doing right. Here are three reasons to run your own RCT.

1.  It's not the same as a pilot. It's better.

Pilot studies are great, but they're not RCTs. A pilot involves trying out a program or idea with a limited number of participants and seeing how it goes before launching the idea full-scale. Pilots are easier to get started than full programs, and they're less expensive. Plus, leaders can use their observations of the pilot to decide whether it should be expanded. It's a win for everyone.

However, simple pilots can't provide evidence on a program's effectiveness because they can't control for confounding factors. The enthusiasm surrounding a pilot, human error, or even the weather can affect outcomes, and only a well-conducted RCT can reveal if the idea being tested is the cause of any observed effect.

2.  Your intuition isn't always right.

Let's be honest: many people simply don't care about the quality or reliability of the data in a research study. If the results affirm their intuition, they'll tout the research findings regardless of the study's methodology. We get that not everyone is a research nerd (although it's hard for us to understand why), but we think everyone can value testing ideas in an unbiased way. The only way to remove this bias and get reliable, objective data is with a well-designed RCT.

3.  A small trial can make a big impact.

You might be thinking that your agency is too small or your idea is too insignificant to have an impact. This isn't the case! Not only can an RCT tell you if a policy idea is effective at your agency, but when the same trial is replicated elsewhere, trends can be identified. The combined wisdom of several small trials can help agencies around the country to improve their policies and increase their impact.

If you're on the fence about running a randomized controlled trial, contact a member of the BetaGov team and we'll help you think it through. You never know what results you might discover or the impact you might have.