Algorithmic experience: visualising the Instagram machine learning process for end-users

Abstract

Algorithmic experience (AX) is a Human Computer Interaction concept that applies to digital products where a significant part of the end-user experience is determined by the algorithms. In other words, it is not only the quality of the interface that is relevant, but also the algorithmic processes whose result is represented by the interface. Some examples of such software products are social media platforms like Facebook, Instagram, TikTok, YouTube, LinkedIn, and others. With the advancement of algorithms, machine learning and AI, the algorithmic experience that is delivered is increasingly personalised. Moreover, the tailored content means that the experience can be different for each user, depending on several factors. Digital product designers therefore face the challenge of researching with users about their algorithmic experience. However, when we speak of algorithms, we mean complex processes that are invisible to end-users. Typically, understanding algorithmic models and concepts also requires advanced mathematical or technical knowledge. So far, such research has been conducted by means of in-depth interviews, but hit many additional obstacles with, for example, the understanding of basic algorithmic vocabulary. During the thesis, it was proposed to overcome this barrier by using visualisation. Building a common ground between designers and end-users using visual language could deepen the quality of the interviews. This would enable UX researchers to provide more valuable insights to the data science team and also be responsible for shaping the algorithmic experience of the product. The popular social media platform Instagram was chosen as an example for visualisation. A series of images explored how to present the algorithmic process to non-experts. The process included not only image-making but also conversations with Instagram users in an iterative process: design – evaluation with five users during in-depth interviews – design – and next sessions with users. This made it possible to provide an interactive final visualisation that mainly focuses on inputs and outputs, elements in the algorithmic process that are familiar to users. Combined with Instagram’s familiar layout, this enabled discussion on multiple levels, not only referencing users’ own experiences of using the platform, but also learning how much and how users combine information in their mental model of the algorithm. The visual investigation also allowed for a broader consideration of privacy policies and data gathering by technology companies, and their real impact on users’ algorithmic experience. The illustrations opposite show the concepts tested during the design process. During the image-making process, an effort was attempted to combine, on the one hand, Instagram’s known layout for users and, on the other, to present what data is processed by machine learning and AI processes that determine the shape of the algorithmic experience. However, the main focus was on the input and output data in the input-black-box-output process.