Variation sets allow you to run very similar tests on different versions of a design, or multiple tests on the same design, and ensure that all participants across all tests in the set are unique. This ensures that participants have not seen the design before, so will not be biased by previous tests.
This is great for iterating on a design or testing variations to see which performs best.
How to create a Variation set
To create a variation set, head to any existing test you'd like to include in the set or create a new test. Click Make variation set on the top-right corner of your test to get started:
Next, you'll need to give your new variation set a name:
Your test will be added to the set automatically. You can now click Copy variation to create a duplicate of this test for quick editing:
You can also add any other existing tests to your set:
Sharing your variation set with your own audience
Similar to other tests, a variation set has a unique link you can share with your audience to direct them to take a test from your set. Toggle on and copy this link from the Recruit participants tab.
Recruits who visit this link will be taken to one random variation. We'll split them evenly across variations. Your recruited participants will only be allowed to participate in one of the tests in this set, whether they revisit the set's recruitment link or visit the recruitment link of any test in the set.
If you are on the Pro plan, you can request demographic information from your participants by turning on Request demographics from the Recruit participants tab of your set. Information collected will be available with results: age, country, gender, and education level.
Collecting unique panel responses with Variation Sets
You might be curious about whether you'll automatically see new participants in all your tests. It's important to note that by default, we don't automatically provide unique panelists for each test. However, Variation Sets come to the rescue, allowing you to make sure that all participants across all tests in the set are indeed unique. This means that each participant has not encountered the design before, removing the potential for biases from previous tests.
Placing panel orders using Variation Sets
Unlike recruiting from your own audience, when you order responses from the Lyssna panel you cannot place an order against the variation set to receive an equal split of unique participants across your tests. Instead, you will need to place individual response orders for each test within your variation set.
For example, if you want to survey a total of 100 unique participants across 4 tests equally in a variation set, you will need to place an order for 25 responses on each test. By placing your tests inside a variation set we ensure that each response comes from a unique panelist.
Please note that this restriction can affect order delivery times if you order many responses (or from niche demographics) for tests within a variation set. The estimated turnaround time for the order you place on your last test may be longer than your first test as those panelists have been removed from the pool of eligible participants.