A/B Testing Cover, Title and Book Description

What is an A/B test you might ask?

In product development and marketing, A/B testing describes a randomized experiment with two variants called A and B. Randomised means that participants (readers in our case) are arbitrarily split into two groups readers and in each group the readers see one version (variant) of the book. Group A gets to read Variant A, while group B is asked to read variant B. Assignment to each group is random and neither group is aware of the existence of the other group. The latter is important to avoid the results being skewed by users in Group A thinking they’d rather read the version in Group B and vice versa. For each group Jellybooks measures a range of variables such as completion rate, and recommendation factor. We measure if the variation produces a difference in outcome and if so which variant produces the better outcome? The goal of A/B testing is to replace subjective opinion with measurable, quantifiable data.

A/B data have limits though. They are well suited for specific well defined changes, such as title, cover, book description and similar. They are less well suited for testing more general or creative aspects, such as the interplay of many variables, though it is certainly possible to A/B test different endings for a book. Also A/B test require more test readers and thus the range of variables that can be tested is very much limited by the number of test readers that can be recruited.

For authors the top two items to AB test are:

• Book cover
• Book description

An A/B test can provide vital clues as to which version leads to readers picking a book, if the A/B testing is conducted in the context of a multi-title test (10-20 books form different authors available and the user has two choose two).

However, whether it is a multi-title or a single-title test, we can measure the impacts of on completion rate, recommendation factor, as well as download to read conversion ratio and other variables.

Another important variable you might wish to test is:

• Endings

This is particularly important for titles that are first in series, which are the entry points for readers to be exposed to a series. If they don’t enjoy the first book, then they are unlikely to buy the other books in the series. Thus it is particularly important to test and optimise books that are the first-in series, so you can optimise the chances that the book will be picked by reader and that they will convert into fans who buy the other books in the series.