Designing an AI feature for Spotify mobile

Spotify currently uses AI to recommend music to its users. However, the use of AI for music exploration is expanding and continuously improving so I'm setting out to discover if the services use of the technology can be taken further!

ROLE:

Strategy, Research, UX Design

MENTOR:

Julien Deveaux

PROJECT SCOPE & TOOLS:

80 hours: Pen & paper, Sketch, Invision

Secondary Research

Spotify does an excellent job of helping its users discover new music. Despite this, music taste is wildly specific to an individual which means that there is an exciting opportunity here to improve upon music exploration within the Spotify app. Before designing, I needed to understand specifically what opportunities there were and how we could deliver on those to offer a better product.

I discovered that AI is used in different ways by various streaming services. Some brands used AI to help users browse for music using voice search, others used it to recommend tracks. I then looked into how Spotify recommended music technically as this would give me greater insight into how the proposed feature could integrate into the existing infrastructure. I discovered that Spotify uses three recommendation models in combination to recommend music to users. These include collaborative filtering, Natural language Processing (NLP) and raw audio models.

View a full summary of Spotify recommendation models

Spotify Recommendation Models Diagram

A technical diagram of the Spotify recommendation system

Contextual Enquiry

At this point, I had a good understanding of the market but needed to understand the typical Spotify user and what their goals and constraints were. I conducted a contextual enquiry with three regular users of the Spotify Android app and learnt the following:

Spotify users rely heavily on AI suggested playlists

Environment impacts how the user uses the Spotify app (earphones in public, Bluetooth speakers in the car).

The type of music listened to is heavily influenced by the user's mood and activity. For example, one participant listened to upbeat music when tired and driving long distances.

It was then essential to define the user, the goals (user, business and technical) and the direction of the AI feature.

User Persona

The user persona painted the picture of a general Spotify mobile user who relied on the app to listen to music on the go.

User Persona

Goals

Research had revealed numerous user, business and technical goals and I set these out as they would all influence the new AI feature, both directly and indirectly.

User, Business and Technical Goals diagram

Ideation

I ideated on various AI features, making sure to design for user goals and constraints. I could see potential in using voice search to help users explore music but concluded that users tended to scroll through recommended playlists rather than search. Instead, I decided to expand on the music recommendation AI by adding real-time user data such as current location and accelerometer feedback. Practically, this would mean that music could be played/suggested to the user which is suitable for the activity (e.g. walking, working) and the environment (e.g. hot weather, car journey).

Rough sketches were drawn in the initial ideation process

Affinity Diagram

I considered various versions of the feature including an auto play button and a more integrated design.

Affinity diagram laying out potential implementations

Task Flow / diagram

I also considered how the feature would integrate technically into the existing Spotify recommendation models.

Invision Prototypes

I prototyped two different versions of the AI feature, designing the screens in Sketch and then building these into functioning prototypes with Invision. This workflow would allow ‘lifelike’ testing of the app feature on a physical mobile.

Prototype 1: Spotify AI feature

Prototype 2: Spotify AI feature

Testing with context

I then tested the prototypes with three participants on an Android mobile in the appropriate environment. It was very important to me that there was context so if a user tended to listen to Spotify in their car, that’s where the test would be. The format was relaxed and the participants were given space to explore the app as well as discuss their thoughts and actions.

Initial testing revealed that users could potentially find the feature useful and didn’t seem to have a problem with the use of their data. Spotify users trusted the brand and could see how the feature would be valuable. However, it became apparent that the current design blended in too much with the existing interface which essentially hid the feature.

A participant, Mary, using the prototype before a long car journey

Iteration

In a process of iteration, I took the testing feedback on board and designed / redesigned the following:

A popup that presents a customised playlist when a new environment / activity is detected

New iconography, graphics

The creation of new playlists to suit mood / environment rather than suggesting existing playlists

Iterated AI feature

Reflection

I discovered that Spotify users are trusting of the service and are happy for their data to be used in order to improve music suggestions. While users want to feel that music suggested on the app is fully personalised to them, they also want this process to happen almost automatically and don’t feel the need for further controls. For that reason, the proposed feature uses multiple sources of data to further personalise music suggestions in an expected and unobstrusive fashion.

Next Steps

It is worth bearing in mind that the research conducted was mostly of a qualitative nature. While it’s been very effective for discovering opportunities, quantitative data will be needed to prove the effectiveness of this feature and it’s design. To take this project further, I believe that a functioning prototype would need to be developed which could be released to active users via A/B testing. Over time, user data and feedback would give an accurate idea of the success of the AI feature and would be an ideal place to move forward.