Preference test sections help you choose between design variations by asking participants to select one based on specific criteria, or simply pick a design they prefer. When participating in preference tests, the participant will be shown your criteria along with all of the design options and then asked to choose one design. You can add several tasks to the same set of designs, and participants will complete them one after another.
Creating a preference test section
From your dashboard, click +Create study, and then Test or survey
Add a Preference test section. You can rename this section to help keep large tests organized if desired.
Upload two or more design alternatives. You can provide up to 6. All tasks in this section use these designs.
Under Task detail, write instructions for your first task. Instructions are optional.
To run another preference test on the same designs, click Add another task and repeat step 4.
You can then optionally add follow-up questions to each task. Participants answer them after they have completed the task and before moving on to the next one. To show a task's follow-up questions in a different order to each participant, tick Randomize follow-up questions below the questions. This setting applies to each task separately.
Adding multiple tasks to one set of designs
Each preference decision you want participants to make is a task. A Preference test section can contain up to 10 tasks, all shown against the same designs. For example, you could show the same three layouts and ask one task "Which layout feels most trustworthy?" and another "Which layout would you rather use every day?".
All tasks in a section share the same designs, device frame, and the option to randomize the order of the designs. Each task has its own instructions and follow-up questions.
Tasks are numbered in the builder (1.1, 1.2, 1.3), and each task's follow-up questions are numbered beneath it (1.1.1, 1.1.2).
Add a task: click Add another task below the last task.
Manage a task: use the ⋮ menu at the top right of the task to choose Duplicate, Move up, Move down, or Delete. Deleting a task also deletes its follow-up questions.
Minimum: a section needs at least one task.
Participants complete tasks in the order you set. For every task, participants choose from the designs again. They don't carry over their choice from the previous task.
Note: each task increases your test's estimated length, which affects the cost of recruiting from the Lyssna panel. The estimate updates in the builder as you add tasks.
Preference test results
Results show the popularity of each design and will calculate the statistical confidence of the winner on tests with 2 options (not yet available for tests with 3 or more designs).
If you used a Linear scale or Ranking question as a follow-up, you will see the mean of the responses in the bottom right corner of the section, rounded to two decimal places.
Results for multiple tasks
Each task in a Preference test section has its own results card beneath the section heading. Each card shows that task's instructions, how many participants chose each design, the average time taken, and the results of its follow-up questions. On a task with 2 designs, the card also shows the winner and its statistical confidence.
If you filter by participants who chose a particular design on one task, the filter narrows the response set across your whole test. If two tasks both have participants who chose the same design, you can still filter each task separately.
Individual responses and exports
Individual responses show each task in the order you set, with the design the participant chose, how long it took, any recording, and their follow-up answers. CSV exports group each task's columns together.
Exporting your results as a visualization
You can export a custom visualization of your preference test results by clicking the download arrow, then selecting Customize:
A pop-up will allow you to change details about the exported results visualization. You can change certain details like the thumbnails being shown or not, number of responses, image sizing, transparent background and more.
Once you have your chosen selections you can download the image by clicking the Download button or simply copy the image to your clipboard if you'd prefer.
Common Questions
What type of file can you upload?
The file must be a JPEG, PNG, GIF, MP4, or MPEG.
The maximum file size for images is 10MB, and the maximum height and width dimensions are 16000x16000 pixels.
The maximum file size for video and audio files is 100MB.
How are designs displayed to participants?
Images are displayed as a grid on one page.
Unmodified images are shown at native height and width
HiDPI images are shown at half native height and width
Framed images are shown within a device of your choice
In what order are designs displayed to participants?
By default, the order is automatically randomized for each participant. In a section with multiple tasks, the order is randomized again for each task, then stays the same while the participant is on that task. This randomization can be disabled by unchecking the checkbox shown below. When unchecked, designs will be ordered as they are arranged within the test builder, for every task.
What can you test with Preference test sections?
Preference test sections are great for choosing between variations of logos, color schemes, layouts, icons, interfaces, and video or audio files. We also see clever uses like testing copy preferences—email subject lines for example.
Can I use different designs for each task in the same section?
No, you can't use different designs for each task in the same section. All tasks in a Preference test section share the same designs. To test a different set of designs, add another Preference test section to your test.
Can I randomize the order of the tasks in a section?
No, you can't currently randomize the order of tasks. Tasks appear in the order you set them in the builder. Only the order of the designs within each task is randomized.
Can I randomize the order of follow-up questions in a task?
Yes, you can randomize the order of follow-up questions in a task. Tick Randomize follow-up questions below the task's questions. The setting applies to each task separately, so you can turn it on for some tasks and leave it off for others.
How many tasks can I add to one Preference test section?
You can add up to 10 tasks to one Preference test section. If you need more, add another Preference test section to your test.
Does a participant's choice carry over to the next task?
No, a participant's choice doesn't carry over to the next task. Each task starts with no design selected.
Does adding tasks change the cost of my test?
Yes, each task you add increases your test's estimated length, which can increase the cost of recruiting from the Lyssna panel. The estimate updates in the builder as you add tasks. Recruiting your own participants is always free.