Prague University of Economics and Business
Bachelor’s Thesis
2021 Lizaveta Bykava
Prague University of Economics and Business Faculty of Business Administration
Bachelors’ Field: Corporate Finance and Management
Title of the Bachelor’s Thesis:
How User Experience contributes to customer satisfaction
Author: Lizaveta Bykava
Supervisor: Ing. Ladislav Tyll, MBA, Ph.D.
D e c l a r a t i o n o f A u t h e n t i c i t y
I hereby declare that the Bachelor’s Thesis presented herein is my own work, or fully and specifically acknowledged wherever adapted from other sources.
This work has not been published or submitted elsewhere for the requirement
of a degree programme.
Acknowledgements:
I would like to express my sincere gratitude to my advisor Ing. Ladislav
Tyll, MBA, Ph.D. for his valuable inputs into the thesis, as well as
guidance and ongoing support throughout my bachelor’s degree study.
Title of the Bachelor’s Thesis:
How User Experience contributes to customer satisfaction Abstract:
This research paper is aimed at unfolding the significance of user experience in businesses as a facilitator to customer satisfaction. The thesis covers the definition of User Experience and User Interface as well as a historical timeline of the development of User Experience in business throughout the years. Furthermore, tools and methods in the sphere of User Experience, such as heatmapping, System Usability Scale by Brooke, efficiency and effectiveness equations were highlighted as they were used in the experiment held in the research paper. The sample size of 102 participants underwent an experiment with two contrasting designs on the basis of heuristic evaluation. The most crucial finding in this research paper is that a user-friendly User Experience and User Interface design facilitates customer satisfaction when completing a certain task on a web platform, which was proved through the System Usability Scale by Brooke tool.
Additionally, a user-friendly design is proved to be more effective and efficient in terms of task completion, nevertheless even for a well-designed web platform there is capacity for ongoing improvement according to user feedback and a necessity for more in-depth research tools. In conclusion, it is necessary for businesses to acknowledge the influence User Experience and User Interface has on their relationships with customers and draw attention towards this aspect especially during hectic COVID-19 times.
Key words:
User Experience, User Interface, customer satisfaction.
Table of Contents
1 INTRODUCTION ... 1
2 THEORETICAL PART ... 3
2.1 UX DEFINITION ... 3
2.2 UI DEFINITION ... 3
2.3 UXGOALS ... 4
2.4 LITERATURE REVIEW ... 5
2.4.1 Additional findings ... 10
2.5 QUANTITATIVE AND QUALITATIVE DATA ... 13
2.6 HEATMAPPING ... 14
2.7 HEURISTIC EVALUATION ... 15
2.8 UXMETRICS ... 17
2.9 SYSTEM USABILITY SCALE BY BROOKE TOOL ... 18
2.10 EFFICIENCY AND EFFECTIVENESS OF AN ONLINE PLATFORM ... 20
2.11 USABILITY HUB ... 21
3 METHODOLOGY ... 22
4 PRACTICAL PART ... 24
4.1 RESULTS ... 24
4.1.1 Design versions ... 25
4.1.2 Heatmaps ... 28
4.1.1 Effectiveness ... 30
4.1.2 Efficiency ... 31
4.1.3 SUS By Brooke ... 31
4.1.4 Data Evaluation ... 33
4.1.5 Insights ... 34
4.1.6 Summarization of the practical part ... 35
4.2 DISCUSSION ... 36
5 CONCLUSION ... 38
6 BIBLIOGRAPHY ... 41
1 Introduction
In recent years the terms “user experience” and “user interface” (UX/UI) on web and mobile platforms have gained immense popularity. They have touched upon all types of businesses and business processes. Specifically, this concerns businesses that possess an online web platform and/or a mobile application through which they interact with their customers. Taking into consideration the COVID-19 crisis that started in early 2020, most businesses were forced to transfer from on-site to online mode. Thus, creating an immense competition for attracting and retaining a customer online. Attracting and retaining a customer online is closely connected to customer experience and satisfaction with the brand, which brings us to the UX/UI design of the company’s web or app - the main touchpoint companies have with their customers virtually. The problem is that most businesses don’t acknowledge the power of UX/UI and how it can be revolutionary for their relationships with customers and the enterprise as a whole. This is why the author decided to conduct a research paper in the sphere of UX/UI and to highlight how useful it can be.
The leading motive for this particular topic of the thesis is author’s personal interest in this field. She believes that the world will no longer be the same after the COVID-19 crisis and a large amount of companies and consumers will continue to interact mostly online, thus putting user experience (UX) in the center of the online experience. The gained knowledge from this thesis will provide her with a strong theoretical foundation and invaluable practical experience in this sphere.
The aim of this thesis is to use modern tools and methods to find a correlation between good UX/UI and customer satisfaction when completing certain tasks by conducting an experiment with the Bachelor of Business Administration (BBA) Club’s website design and to investigate the behavior of users involved. The research question of this thesis is the following:
How user-friendly UX/UI facilitates customer satisfaction when completing a task on a web platform?
The research question can be answered through conducting, analyzing, summarizing and evaluating the results of the experiment through the tools and methods defined in the theoretical part of the thesis.
The main limitation of the thesis would be the narrow number of participants within the Faculty of Business Administration. The experiment will have a relatively small sample size that can be rather biased. A more sophisticated result would have been obtained with a larger sample size.
The thesis consists of two parts: theoretical and practical. In the theoretical part of the thesis the author will firstly introduce the main definitions and then the literature review will be evaluated. Furthermore, the reader will have a chance to delve into the tools and methods commonly used to examine user behavior, which afterwards will be applied in
the practical part of the thesis. So as to prove the usefulness of UX/UI, the author decided to conduct an experiment that would contrastingly depict the significant integration of UX/UI in web platforms and applications in the practical part of the thesis. The author designed herself two different versions of the BBA Club’s website that were sent to the participants from the sample, then she used the tools described in the theoretical part and applied them in the experiment. The results of the experiment were assessed and summarized. After mentioning the findings, complications, suggestions and limitations of the experiment, the research question will be answered. The conclusion crowns all the points discussed in a brief summary. The Appendices to this thesis include pdf versions of the designs, heatmapping, click tests and calculations made for the experiment.
The contribution of this research paper is for businesses to acknowledge the influence and potential benefits from a well-designed UX/UI on their business activities, yet the most vital potential contribution of the thesis is into the Prague University of Economics and Business, more specifically into the BBA faculty and its students. The BBA Club is a vital center for student communication, support and development. An advanced, user- friendly UX/UI of the future BBA Club should attract potential applicants and encourage current BBA students to actively participate in university events.
2 Theoretical part
In the beginning, it is vital for the reader to understand what UX/UI is, therefore the theoretical part of the thesis is structured in a peculiar way, so as to firstly introduce to the reader what user experience and user interface is, followed by a literature review relevant for this research paper. After the literature review the reader will encounter the introduction of tools and methods that are used in the UX sphere, some of which will be furthermore applied in the practical part of the thesis.
2.1 UX definition
To begin with, Jesse Garrett (2010, p.10) emphasized that “every product that is used by someone has a user experience: newspapers, ketchup bottles, reclining armchairs, cardigan sweaters.”. Thus, unconsciously this aspect is as old as the hills, but the first definition of “user experience” has been first introduced by a cognitive psychologist and designer Don Norman in 1993 (Nielsen, 2017). Yet, in our hectic everchanging world even Don Norman made a comment to his definition: “I invented the term because I thought human interface and usability were too narrow. I wanted to cover all aspects of the person’s experience with the system including industrial design, graphics, the interface, the physical interaction, and the manual. Since then, the term has spread widely, so much so that it is starting to lose its meaning... People use them often without having any idea why, what the word means, its origin, history, or what it’s about” (Merholz, 1998). His comment is absolutely valid as throughout the years with the development of technology the term “user experience” (UX) has been mixed up with alike terms such as
“User-centered design” (UCD) for instance. Among the years, it has been clarified that UCD is a design process where the end-users influence the outcome of the design (Abras, Maloney-Krichmar & Preece, 2004). The modern definition of UX by Don Norman and Jakob Nielsen (2014), the founders of the Nielsen Norman Group, is the following: “User experience encompasses all aspects of the end-user’s interaction with the company, its services, and its products”.
2.2 UI definition
Maioli (2018, p.116) defined User Interface (UI) as “the means by which a person interacts and controls a device, software, or application. This control can be done through buttons, menus, and any element that provides an interaction between the device and the user”. UX and UI cannot exist without each other since they enhance one another. “You can improve the user interface (UI) in order to create a better user experience (UX)” (Maioli, 2018, p.115). UI is concerned with the visuality of the website or app and gives the very first impression to the user, it works as a source of communication with the user because “typography, iconography, color scheme” (Maioli, 2018, p.116) are all part of the UI and at the same time can retain the user.
There are different types of hierarchies of the UI elements depending on the main goals of the website. This research paper is aimed at a website for a student club, where there won’t be a large number of texts or articles, thus the Z pattern is the most acceptable to stick to. The Z pattern “applies to pages that are not centered on the text”
(Maioli, 2018, p.123). In the figure 1 below we can imagine what will catch the user’s eye when first seeing the website in a Z pattern. First of all, the reader will check the top of the page as he/she is used to finding the most necessary information there, then users are most likely to go diagonally down to check information of secondary importance.
(Maioli, 2018).
Figure 1 Z-pattern hierarchy type
Source 1 Maioli L., 2018, p.123
Following the pattern Z as shown in the figure above isn’t compulsory, but definitely recommended so that the users don’t have to search for necessary information, but will find it at a known place.
2.3 UX Goals
In the UX book by Hartson and Pyla (2012, p. 361), the definition is as follows: “UX goals are high-level objectives for an interaction design, stated in terms of anticipated user experience”. The end UX goal is prioritized in every step of the design process and is aligned with the organization’s goals and/or mirror the direct usage or purpose of the end-product (Hartson & Pyla, 2012). “Examples of user experience goals include ease- of-use, power performance for experts, avoiding errors for intermittent users, safety for life-critical systems, high customer satisfaction, walk-up-and-use learnability for new users, and so on (Hartson & Pyla, 2012, p.361). For every UX design project the UX goal will differ depending on the purpose of the end-product. For instance, a different UX goal will be set for a gaming mobile app and a pharmaceutical app due to dissimilar grounds of the platforms.
Now the reader knows what UX and UI stand for, their complex fate in terms of definition and mix ups with other concepts, what UX goals are and the basic hierarchy types on web platforms. Knowing this, the reader can move on to the following point, which will cover the literature review in the sphere of UX in a historical timeline. The timeline will help to build the flow of development of UX throughout the years and the spread of its influence on business processes and customers.
2.4 Literature Review
After many years of defining and investigating the UX definition, the studies in the field of UX have become more involved in the psychological perspective of the term. The author found it significant to describe to the reader how researchers investigated deeper in how human emotions are interconnected with user experience. Sharp, Rogers &
Preece (2019) described the spur of emotions a user feels when doing simple everyday activities such as buying a product online. The whole decision-making process involves excitement, annoyance, frustration, relief, doubt. This is an emotional interaction of a user with the product or service given online. Obviously, companies want to provoke only positive emotions from their end-users and to be associated with a pleasant, not tiresome or boring experience. For instance, Don Norman (2005) contemplates that when people are in a happy state, they won’t be bothered as much by long loading times or other minor problems with the interface as opposed to people in an angry state. Then, to bring color into these words, an experiment was held and described in an article by Desmet, Porcelijn
& Van Dijk (2007) using telephone models as an example. Desmet, Porcelijn & Van Dijk (2007) contemplated about conceptualizing the “wow-factor” when it comes to human interaction with a product. They have tested 8 different telephone models that were already on the market and asked a focus group to give their opinion on it. Also, they held a quantitative research with the emotion measurement instrument. After gathering all the information from consumers, they attempted to make a prototype of a cellphone that would provoke the desired “wow-factor” and it did indeed. The result of this experiment is to show how a wow-factor can be conceptualized and designed to provoke a certain impressionable feeling from the end-user. Desmet, Porcelijn and Van Dijk wrote a great article in terms of investigating the connection between emotions and UX, nevertheless it doesn’t bridge these findings with the business processes and how they can benefit from it. The emotional research into user experience is an essential input into this sphere, so that designers could build such a design for a company so that the consumer will find it emotionally satisfying and will want to return.
For the purpose of this research paper, after covering an article by Desmet, Porcelijn &
Van Dijk about the very rise of user experience and first experiments with consumers, it is essential to dive deeper into how at last user experience started to be associated with business processes. Sward and Macarthur (2007) composed an article “UX as a business strategy”. In the beginning the authors stress, as once Don Norman did, the ambiguity and the broadness of “user experience”. It cannot have a specific definition, yet it is
highly dependent “on the author’s discipline” (Sward & Macarthur, 2007, p.35). Sward
& Macarthur (2007) mention the UCD concept, that was constantly mixed with UX as mentioned earlier in the definition of UX, as a way “to humanize our interaction with technology” (Sward & Macarthur., 2007, p.36). The relevance of the article to UX can be proved by a figure shown to depict the relationship between UX and UCD as shown below:
Figure 2 Relationship between UX and UCD
Source 2 Sward D. & Macarthur G., 2007, p.36
UCD is a part of the broad definition of UX. A very crucial comment from Sward and Macarthur (2007) would like to put an emphasis on: “Delivering a good experience requires a wide range of disciplines, such as marketing, ethnography, industrial design, graphics designers, human factors engineering, software engineering, hardware engineering, interaction designers, information architects, and business process analysts…” (Sward & Macarthur, 2007, p.36). To this day, it is not only the designer’s responsibility to provide a good user experience, but the whole company should be involved in this process to achieve this goal.
Moving on to the next point, in order to elaborate more on the UX definition, Sward and Macarthur (2007) propose to view it as a mixture of components: marketing and awareness; acquisition and installation; product or service use; product support;
removal or end of life. The limitation of the article is that all of these components are actually mentioned for a real-life product experience. Since the article was written in 2007, it is understandable that they put it in this way because e-commerce was not as popular and people were not used to ordering products online. Thus, it still has validity in its meaning, but it’s not up-to date. The figure below visually represents the interconnection between all the components around UX mentioned by Sward &
Macarthur.
Figure 3 User experience components
Source 3 Sward D. & Macarthur G., 2007, p.36
Another really unique part from Sward and Macarthur (2007) that is pivotal to acknowledge: “the market is calling for further action as we enter the experience economy” (Sward & Macarthur, 2007, p. 37). The experience economy is all about providing the customer not only the product, but also passing on the experience, feelings and emotions that the company wants to evoke from the consumer. It started in the early 2000s, when the article was issued. For instance, a customer pays a larger sum for coffee at Starbucks not exclusively for the coffee itself, but for the working atmosphere, where people can work and create, for the aesthetics of the place. (Sward & Macarthur, 2007).
Finally, Sward and Macarthur (2007) discuss the interconnected strategies that should be integrated into a business, so as to work towards the vision of providing a memorable user experience for the customer. The three strategies are: linking UX to the bottom line of the firm; implementing a User Experience program; managing the results and effectiveness of UX (Sward & Macarthur, 2007).
To crown this article by Sward and Macarthur (2007), the most vital aspects that should be recognized is the role UX plays in every business process, how it is a business strategy and its significance in the experience economy.
The most recent and relevant article for this research paper is a study from Chen, Peng, Jing, Wu, Yang & Cong (2020), since it is connected with user experience on online educational platforms in China during the COVID-19 outbreak. All educational institutions were faced with a challenge to transfer from on-site teaching to full online teaching mode. Surely enough, before the pandemic online platforms, such as Zoom, were used, but only for tutoring lessons or very small one-on-one sessions, whilst holding professional classes online is a completely different challenge. When talking
about online teaching methods, one seldom thinks about the effect UX has on it. “UX is widely used in the evaluation and optimization research of commercial platforms, but rarely in online education.” (Chen et al., 2020, p.4). Chen et al (2020) unfold the core issues that hold back proper teaching nowadays, such as bad video connection or unfamiliar hard features that most professors do not know how to use, thus wasting time on technical formalities instead of education. (Chen et al., 2020). The study held an experiment evaluating the “change of user experience focus by analyzing the data before and after the outbreak of COVID-19” (Chen et al., 2020, p.28) using for their evaluation famous in China platforms such as Zoom Cloud, Tencent Meeting, DingTalk, MOOC, TIM, WeChatWork and Chaoxing learning. Interestingly enough, the results were:
“before the outbreak of the pandemic, users were concerned about the access speed, reliability, and timeliness of video information transmission of the platform, and the user experience of the Zoom Cloud platform was the best and after the outbreak of the pandemic, users mainly focused on course management, communication and interaction, learning and technical support services of the platform, and the user experience of the platform was the most important.” (Chen et al., 2020, p.28). Although this study by Chen et al (2020) is highly relevant to the topic of this thesis, it does not unfold whether parameters mentioned above will contribute to the satisfaction of the customer, in this case students, after improving or enhancing these features on an educational online platform. The author believes that Chen et al (2020) wrote the most accurate and up-to-date study for this thesis as it is touching upon the topic of online education and user experience.
To crown the literature review part of the thesis, it is written as a historical guide throughout the main checkpoints of UX. The first would be when UX took a psychological path in the history, which is also known as emotional design. The paper by Desmet et al.
(2007) was aimed at provoking a certain emotion from the consumer, more specifically the “wow-factor” in an experiment with different telephone types. A paper by Sward and Macarthur (2007) pointed out how even approximately 60 years later after the first definition of UX there is still no clarity to what it is exactly, yet the importance of this article is how it has been linked to the business processes. That UX is interconnected with all the essential business components that result in the end product for the customer. The most crucial statement is that UX should be integrated in the strategy of the company in order to arouse true memorable emotions from the customer, provide him with the best possible experience and to retain. The main limitation of the study by Sward and Macarthur (2007) is that it is written before the e-commerce era. That is why the follow-up, the most relevant to this day article, is about how the COVID-19 outbreak affected the online teaching, emphasizing the significance of UX in the quality of the teaching provided by Chen et al (2020). The study provided quite captivating results of the experiment being the shift in UX focus before and after the COVID-19 outbreak, whilst UX being of utmost importance nowadays, yet the study didn’t investigate whether the parameters would truly increase customer satisfaction with online educational platforms in the end.
Therefore, it can be concluded that there is still no proper research made on how good UX actually facilitates customer satisfaction. In fact, a study by Luther, Tiberius& Brem (2020) evidently stresses out the lack of proper amount of UX research made in the sphere of business. The figure below shows the annual number of UX publications made:
Figure 4 Annual number of UX publications
Source 4 Luther L. et al. (2020, p.4)
In the figure above we can see that throughout the 1980s to early 2000s there was as little as one article on the topic of UX was published. It is most likely due to the unpopularity of the UX as a concept. Since 2012 the number of publications started steadily rising, the author’s assumption would be that smartphones started to be more popular at the time and the focus started steadily shifting to online mode. With smartphones, people have easier access to the internet and could come across first signs of e-commerce. “The articles from 2012 to 2019 represent 85% of the total literature data set and are the most productive years.” (Luther et al., 2020, p.4). More detailed, in the figure 5 below we can see the division of all UX publications by discipline, where Business Economics takes 3rd place with a total of 68 records published throughout the years in this sphere.
Figure 5 Number of UX publications per discipline
Source 5 Luther L. 2020 p.5
Thus, conducting a research paper that touches upon how good UX/UI can actually facilitate customer satisfaction when completing a task on a web platform is a relevant input into this sphere as it is so narrowly investigated.
At this point, the reader can not only differentiate between user experience and user interface, but also have an idea of how and in which spheres UX/UI has developed overtime. As a recap: UX/UI was investigated from a psychological perspective, then the concept was integrated as a business strategy and the final research paper covers significance of UX/UI in educational platforms. Knowing this, the flow of the research paper follows by which data can be gathered in the field of UX/UI, methods, tools and the way to measure the success rate of a design on a web platform or application.
2.4.1 Additional findings
“A study at University of London demonstrates that 90% of users confirmed that they stopped using an app because of poor performance, and 86% of them deleted or uninstalled an app because they found problems with its functionality or design.”
(Maioli, 2018, p.11).
Online businesses are hit the worst when it comes to bad user experience. A study from Baymard Institute (2016) has been tracking for 11 years average cart abandonment rate from more than 100 leading e-commerce sites. The most recent data about the average cart abandonment rate stands at 68.8%. If you think about it, approximately 2 out of 3 users drop-out of the site and do not end up ordering products. This has an immense impact on the business, as they’re losing potential revenue.
The figure below from Baymard Institute (2016) shows their most recent quantitative study on the topic “Reasons for Abandonments During Checkout”. The target audience
was mostly adults from the United States and below you can observe the percentage of respondents for each reason shown on the left side of the graph.
Figure 6 Reasons for Abandonments during checkout
Source 6 Baymard Institute
After analyzing the figure above, the author noticed that most of the reasons mentioned above can be solved or at least diminished if the user experience was enhanced. For instance, the 3rd reason “too long/complicated checkout process” can be simplified if a right approach to user experience has been integrated. According to new research, “the average checkout flow contains 23.48 form elements and 14.88 form fields (when purchasing as a new non-account customer), yet a fully optimized checkout flow can be as short as 7 form fields, with a total of just 12 form elements.” (Holst C., 2016, para.14).
Another captivating example from a research by Holst (2019) is about how return policies affect the customer retention rate on web and mobile platform. More than half of the online shopping users have returned an item in the year 2018 (Holst, 2019). In a quantitative research study by Baymard Institute it has been “found that 7% of users would never again purchase from the last site they returned an item to, solely due to the site’s returns experience, while an additional 13% of users would be unlikely to purchase from that site again (Holst, 2019, para. 7). In the same study, it has been found that from 42 largest e-commerce sites in the world – more than half (54%) have “significant UX performance issues within their ‘Order Returns’ flow (Holst, 2019, para. 9). The sites
don’t realize the potential rise in customer retention rate and satisfaction if they have invested in enhancement of UX performance in this part of the website. Assumingly, according to the research, 20% of customers who commonly return products, can be retained and bring even more sales back, since they would be assured that if they don’t like the product it can still be easily returned.
Below you will see a figure that shows the correlation (the red line) between good UX performance and the number of form elements the users first see on the website during checkout process. On the Y-axis you can see checkout usability performance that is calculated as the number of users that have successfully completed the checkout process and on the X-axis you can see the total amount of fields (both optional and required) that users saw during the checkout process.
Figure 7 Checkout Usability Performance/ Number of Form Fields
Source 7 Baymard Institute (2016)
As we can see from the figure 7 above, that the fewer fields we have, the better the UX performance is. Thus, users are more successful in the checkout process and in placing an order on the e-commerce website.
Another study from Medallia Analysis mentions that “customers who had the best past experiences spend 140% more compared to those who had the poorest past experience.” (Kriss P., 2014, para 4).
To crown it all, the author decided to add these findings in the literature review so as to enhance and persuade the reader how UX/UI can influence businesses processes and their relationships with customers. These are just few examples and statistics taken from
UX research and are widely read by UX designers, so as to get a better understanding of the user.
2.5 Quantitative and qualitative data
Quantitative data is any data that can be “quantified” and used statistically
(Hay, 2017). As shown in figure 8, it is concerned with discovering facts, not opinions, about the design. Usually it is presented in numerical comparison format and answers the question “what?” is happening with the website or web application. (Hay, 2017).
Whereas the qualitative data can answer “why” something is happening with the website or web application. It Is about understanding the behavior and motivations of the user and is reported in the form of the language of the users.
Figure 8 The difference between quantitative and qualitative data
Source 8 Hay L., 2017, p.8
The figure above provides a very good and understandable comparison between the two methods of data gathering in the sphere of UX. The quantitative data should give an insight on “how much of a behavior is present in a group” or in other words, how many users have completed some sort of action with the website or web application. The qualitative data is mostly concerned with determining “what factors determine the behavior in a group”, giving a reasoning behind why and how the person acts and feels about the design.
Luke Hay (2017, p.9) enumerates the most common methods used for quantitative and qualitative data gathering.
Quantitative methods:
• Web analytics
• Form analytics
• Heatmaps
• A/B testing
• Eye tracking
• Click testing
• Surveys (can be quantitative and qualitative)
For this research paper the heatmapping method will be used and the derived version click mapping. The reason the author chose this method is because it is accessible and understandable for a broad audience. Additionally, it is integrated in the online platform that will be used in the experiment. The platform will be described further in the thesis.
Heatmapping will be described in the next point separately. Furthermore, Luke Hay (2017, p.10) introduced the following qualitative methods:
Qualitative methods:
• Usability testing
• Ethnographic/field studies
• Focus groups
• Diary studies
• Screen/session recording
• User feedback
• Stakeholder interviews
• Immersive research
• Design workshops
• Heuristic assessments
For this research paper in terms of qualitative data user feedback for insights will be used. The author chose to take into consideration the heuristic assessments when building the designs of the web platform., which can then provide a clear differentiation in designs.
The most effective way to gather analytics about UX on a website or web application is to combine, evaluate and summarize both methods. Then, depending on the results there can be alterations made to data gathering so as to obtain a more in-depth analysis of the design.
The following point will describe one of the quantitative methods that will be used in the experiment: heatmapping.
2.6 Heatmapping
Heatmapping is one of many analytical tools used. “Heatmapping tools use colors to show graphical representations – primarily of where users have clicked on a page.” (Hay, 2017, p.15). The figure below is an example of a screenshot from a website.
Figure 9 Heatmapping example
Source 9 Hay L. 2017 p.16
The figure above shows in color where users have clicked the most. The red color indicates that users click here more often. Thus, the brighter the color (from green to red) the more users have clicked on the button. Heatmapping are useful for analyzing which type of content users see and how they get to the content (Hay, 2017). Click mapping is exactly the same as heatmapping, but instead of colors it presents the click of a user in individual circles.
In the following point, the reader will be familiarized with a method commonly used in the sphere of UX/UI as a checklist for a splendidly designed web platform or web application.
2.7 Heuristic Evaluation
How can one actually differentiate a good design from a bad one? Isn’t this really subjective? Lisandra Maioli (2018, p.21) has defined a poor user experience “as anything that prevents the users from accomplishing the task on a site, app, service, or product”.
On the other hand, a good user experience cannot be defined in just one sentence. It is a congestion of many features that should come together in one end-product.
A usability consultant Jakob Nielsen with Rolf Molich in 1990 have developed a methodology named “heuristic evaluation”. “Heuristic evaluation is a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the "heuristics").” (Nielsen and Molich, 1990). It is crucial to mention that usability “is concerned with creating systems that are easy to learn and use” (Maioli, 2018, p.186). There is a total of 10 “heuristics” that will roughly help to differentiate a good UX/UI from a bad one. A good UX/UI will fulfill most of the
“heuristics” that are listed below with brief explanations of each. Nielsen and Molich (1990) have defined the following principles:
1. Visibility of system status. The system in the app or site must “talk” to the user.
Let the user know what is happening, either the page is loading or processing a payment as an example. Also, the system must give instant feedback to his actions. An example can be seen in figure 10, we can see a design of how a button should react to customer’s actions mentioned on the left side of the figure. The top icon shows how a button behaves when it is at the normal or passive stage. The second icon shows what will happen when a person “touches” with the cursor the button or holds it. The third icon is how the button will react when it is tapped. The fourth icon shows an inactive state of the button
2. Relation between the system interface and the real world. The system must be as accessible to the user as possible. Integrating within the system objects that are familiar to the user. For instance, in figure 9 below we can see icons that can be used in any app or site and the user will know exactly what to do.
In figure 11, we can see the first icon meaning
“history” to the user. As the icon itself can be interpreted “back in time”. When pressing on this icon, the user will expect to see the history of orders made before, for example. The second icon is the calendar. The third icon is to call. The fourth icon is for settings. All of these images resemble a relation to real-life.
3. Freedom and control of the user. In unexpected situations the user must always have the option for an emergency exit. Meaning he can undo an action or return to a previous point.
4. Consistency. Definitely there is no need in UX/UI design to invent something new in terms of the language or icons used. A user is used to an icon meaning “history” as shown in figure 11, thus if the designer decides to change this and make it a “timer”
icon, then the user might get frustrated with the system as he has to learn again how to use it. This should be minimized so that the user can perform the desired actions as quickly as possible – the most valuable thing in the 21st century is time.
Figure 10 Visibility of system status
Source 10 Author
5. Prevention of errors. This point in UX/UI is crucial none the least. For example, before deleting something the app or site should ask for your confirmation of the action in case the user pressed an icon by mistake. This is also known as a “slip”
of the user.
6. Recognition instead of remembrance. The efforts of the user must be at a minimum. Meaning the user shouldn’t memorize where the contacts can be found, but instead it is always recognizable by the icons on the screen.
7. Flexibility and efficiency of use. The design should be made to cover a large target audience of a company: both experienced and inexperienced users must have the opportunity to use the web or app to their convenience. For instance, a short tutorial should be available for first-time users, but long-term users won’t need this extra option on the screen.
8. Aesthetics and minimalist design. The user doesn’t need to be bombarded with unnecessary information and icons on the screen. Keeping an interface minimalistic means providing for the users only the most essential icons and information, so that the main goals of the app/web design are supported.
9. Help users recognize, diagnose, and fix errors. This point is about most users, who do not know what separate error codes indicate and thus, for the convenience of the user there must be provided an explanation of the error in simple language and preferably a solution to the problem.
10. Help and documentation. The best app and websites should be designed in such a way, that the users won’t require any additional help. Nevertheless, the interface might be inevitably complex, then the user must have easy access to documentation on how to work with the app in a simplistic form, for instance bullet points, steps or videos.
These 10 principles represent usability and design evaluation of the website or web application. So as to “quantify” the usability the System Usability Scale by Brooke tool can be used, which will be explained in paragraph 2.8.
2.8 UX Metrics
In this paragraph, the reader will acknowledge the UX metrics applicable specifically for the UX/UI design sphere.
Figure 11 Relation between the system interface and the real world
Source 11 Author
Albert and Tullis (2013, p.6) mention that “a metric is a way of measuring or evaluating a particular phenomenon or thing”. Metrics are used in our daily lives, yet they might be modified or interpreted differently according to the field (Albert & Tullis, 2013). After the UX/UI designer completes a project, they want to be assured that the hard work is paid off and the design is a success or also gather feedback, so as to improve the mobile or web platform. The peculiarity about a UX metric is that it “reveals something about the interaction between the user and the product: some aspect of effectiveness (being able to complete a task), efficiency (the amount of effort required to complete the task), or satisfaction (the degree to which the user with his or her experience while performing the task) (Albert & Tullis, 2013, p. 7).
In the following paragraphs, the reader will recognize in-depth explanation and measurement of the three core UX metrics: effectiveness, efficiency and satisfaction.
2.9 System Usability Scale by Brooke tool
“The System Usability Scale (SUS) was developed by Brooke (1996) as a “quick and dirty” survey scale that would allow the usability practitioner to quickly and easily assess the usability of a given product or service” (Bangor, Kortum & Miller, 2008, p.574). Even though that to this day there are many other alternatives offered to test usability, specifically SUS is the most preferred one as, according to Bangor et al. (2008) it can:
a) Grasp a wider range of interface technologies used;
b) Is easily read and understood without any in-depth scientific knowledge;
c) This survey is an unpatented invention, making it cost effective.
“The original SUS instrument (Brooke, 1996) is composed of 10 statements that are scored on a 5-point scale of strength of agreement. Final scores for the SUS can range from 0 to 100, where higher scores indicate better usability” (Bangor et al., 2008, p.576).
Basically, the questionnaire is aimed at indirectly “checking” the 10 “heuristics”
mentioned in the previous point. The survey is structured in such a way that the user has to contemplate on each question, since they follow a positive-negative sequence.
(Brooke, 1996).
Bangor et al. (2008, p. 576) have thoroughly evaluated the SUS instrument and they have found out that around 10% of the participants “had a question about the word
“cumbersome” in Statement 8-“I found the system very cumbersome to use […] We eventually settled on “awkward” because “awkward” is a much more commonly used word in English than “cumbersome” (Oxford University Computing Service, 2001) and most test administrators reported using “awkward” in their explanation of the definition of “cumbersome” to a participant, with good success”. According to this statement, the author will also use a modified version of the SUS tool offered by Bangor et al. (2008) so as to ensure higher success of it.
Eventually, the final version of the questions in the SUS survey will look like in the figure below:
Table 1 SUS By Brooke (formatted version)
Question # Strongly
Disagree
Strongly agree 1. I think that I would like to
use this system frequently. 1 2 3 4 5
2. I found the system unnecessarily complex
1 2 3 4 5
3. I thought the system was easy to use.
1 2 3 4 5
4. I think that I would need the support of a technical person to be able to use this system
1 2 3 4 5
5. I found the various functions in this system were well integrated.
1 2 3 4 5
6. I thought there was too much inconsistency in this system.
1 2 3 4 5
7. I would imagine that most people would learn to use this system very quickly.
1 2 3 4 5
8. I found the system very
awkward to use. 1 2 3 4 5
9. I felt very confident using
the system. 1 2 3 4 5
10. I needed to learn a lot of things before I could get going with this system.
1 2 3 4 5
Source 12 Brooke J. (1996) p.192 & Bangor A. et al. (2008) p.577
The SUS is usually given to the tester after evaluating the design and before the discussion. He/She isn’t expected to think over every question for a long time, vice-verse, it is the first impression, so he/she should reply as he truly feels about the design.
“The SUS is scored using the following formula:
((Q1-1)+(Q3-1)+(Q5-1)+(Q7-1)+(Q9-1)+(5-Q2)+(5-Q4)+(5-Q6)+(5-Q8)+(5-Q10)) x 2.5” (Xiong, Acemyan & Kortum, 2020, p.136). It is crucial to mention that “each item’s score contribution ranges from 0 to 4” (Escanillan-Galera & Vilela-Malabanan, 2019, p.
1229). Questions formulated negatively (questions 2,4,6,8,10), the contribution is 5 minus the chosen by user number. Positively formulated questions (questions 1,3,5,7,9), the contribution is the chosen variant by user minus 1. The whole sum from the questions is multiplied by 2.5. (Escanillan-Galera et al., 2019, p. 1229)
As for the interpretation of the SUS results, Bangor, Kortum & Miller (2009) have written an article about an adjective rating scale. In this article, they have undergone many SUS results and defined certain benchmarks to interpret the result. The figure of the interpretation, inspired by the article from Bangor et al (2009) can be seen below:
Table 2 SUS Score interpretation
Score Interpretation
Below 36 Unusable system
Below 51 Poor system
Between 51 and 68 Fair system
Between 68 and 80 OK system, but could be improved
Over 80 Very good, highly usable system
Source 13 Bangor A. et al (2009)
This instrument is widely known and is proved to be efficient due to the construction of questions as well as the easy interpretation of the results that can help make fast design decisions based on testing of the design by users. SUS is crucial in this research paper as it is the main tool that will measure user satisfaction and user experience.
(Escanillan-Galera et al., 2019)
In the next point, the author will describe the statistical usability metrics that will be used in the experiment to obtain additional information on user behavior in the experiment.
2.10 Efficiency and effectiveness of an online platform
According to the International Organization for Standardization (ISO, 2018) it is recommended to use effectiveness, efficiency and satisfaction as core usability metrics.
(Escanillan-Galera et al., 2019). “Effectiveness is accuracy and completeness with which users achieve specified goals” (ISO, 2018, p.3.1.12) Effectiveness can be calculated as
number of tasks completed successfully over total number of tasks multiplied by 100%
as shown in equation 1 below:
Equation 1 Effectiveness
𝐸𝑓𝑓𝑒𝑐𝑡𝑖𝑣𝑒𝑛𝑒𝑠𝑠 =!"#$%& () *+,-, .(#/0%*%1 ,"..%,,)"002
3(*+0 4"#$%& () *+,-, ∗ 100%
Source 14 Escanillan-Galera K.M.P. et al., 2019, p.1228
“Efficiency is resources used in relation to the results achieved” (ISO, 2018, p. 3.1.13).
Efficiency should be calculated as the overall time spent by the user on completion of a task on a website or web application (Escanillan-Galera et al., 2019). Equation 2 shows the calculation of efficiency through time base:
Equation 2 Efficiency
𝐸𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 =∑ 6789 ∑ !789
!"#
$"#
!6
Source 15 Escanillan-Galera K.M.P. et al., 2019, p.1228
Where:
R: the number of users N: the number of tasks
nij: result for tasks (i) by the user (j).
If the task is completed, then nij=1, else nij=0 tij: time spent by the user (j) to complete the task (i).
If the user did not complete the tasks, time will be measured until the moment the user gives up the task. (Escanillan-Galera et al., 2019, p.1228)
The user’s satisfaction will be measured by SUS as the third component as mentioned in the previous point.
In the following point, the reader will be familiarized with an online platform that combines different types of testing tools to gather quantitative and qualitative data that will be used in tools mentioned earlier in the theoretical part.
2.11Usability Hub
Usability Hub (2021) is an online platform where a variety of different analytical tools for UX can be found to help make design decisions. In total, they provide 5 different testing tools, which provide both: quantitative and qualitative data. The “five testing tools available: five second test, click test, question test, navigation test and preference test. The quantitative tests are the click, navigation and preference tests.” (Hay, 2017, p.21).
The click tests function like a heatmapping analytical tool, it records where users click, but instead of a fully functioning website screenshots are used. It also records how long users spent on a certain screenshot before pressing a button, thus recording the user’s
“decision-making” time (Hay, 2017).
The main difference of click tests from navigation tests is that the navigation test has multiple screenshots, which record how the user “navigates” through the design draft to accomplish some certain task which is mentioned for the tester in the very beginning.
For instance, a user receives a task, then the first screenshot appears and the tester chooses where he/she would press, their decision leads them to another screenshot and so on (Hay, 2017). This website provides the necessary analytical tools that provide enough information to conduct an experiment and evaluate its results. The limitation of this platform is that it doesn’t show screen recording – a tool to record a users’ journey throughout all the screenshots. Moreover, each screenshot should have a “hit zones”, which will be marked in green.
Moving on to the next point which will describe the methodology used in the research paper.
3 Methodology
So as to solve the research question “how user-friendly UX/UI facilitates customer satisfaction when completing a task on a web platform?” an experiment was conducted.
First and foremost, for the experiment we needed a sample to work with. The target audience for the themed website for the BBA Club were students of Prague University of Economics and Business. The sampling process was done by the issuance of a google forms survey, which gathered general information about the person and their contact information. More specifically, the form consisted of the following points:
• Name;
• Age;
• Represented faculty at VŠE;
• Email.
After the form was issued, a total of 102 responses were gathered, giving 102 in sample size. The group was split evenly without any criteria: 51 (group A) and 51 (group B).
Group A was given a model of the BBA website without user-friendly UX/UI and were asked to subscribe to the BBA Club’s Events Calendar send-outs. Group 2 was given a model of the BBA website that was designed with a friendly UX/UI and were asked to register for the upcoming event.
This experiment was fully conducted online, thus the tools used were aimed at depicting user’s behavior throughout the given task on an online web platform. Usability Hub is the remote user research platform within which the navigation testing was used.
Navigation testing is performed by defining a task for users to carry out by clicking
through a series of screens. After completion of the task, users were asked to undergo a questionnaire. A specific questionnaire for user testing is a separate tool and it is the System Usability Scale by John Brooke. The questionnaire consisted of the following questions:
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very awkward to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
The responses range from Strongly Disagree to Strongly Agree on a scale from 1 to 5.
What is more, after the users had undergone the navigation test, the tool also recorded heatmapping of the users along with click mapping and total time of task duration of each user whilst completing the defined task. These variables helped calculate efficiency and effectiveness of the two models of the BBA Club website.
Finally, all the tools combined depicted a vivid comparison between the two models.
4 Practical Part
To verify the tools and methods that have been tested and explored by various academics described in the theoretical part of thesis, the author decided to conduct an experiment to define whether a good UX/UI facilitates customer satisfaction. The experiment will consist of the following: the author made up two contrasting designs for the BBA Club’s website with dissimilar UX/UI characteristics on the basis of heuristic evaluation. The heuristic evaluation, as the reader may recall, allows to differentiate a good UX/UI from a poor one. The first design follows most of the heuristic assessments to make the design user-friendly, the other design neglects most of the heuristic assessments making it anti-user-friendly.
The sample was gathered from a target audience – a total of 102 BBA students were approached to participate in the experiment.
Those who signed up were entered into Appendix 4. The sample size of all students was a total of 102. So as to be unbiased, the sample size was divided in half: 51 and 51 students that will be further known as “Group A” and “Group B” respectively. Group A and Group B both received an email with the following context:
Hello there!
Thank you for agreeing to be a part of a research experiment in the sphere of UX/UI. Below you will find a link to mock-up design of the future website for the BBA Club. When you click on the link, you can start the test. At the beginning, you will be given a task to complete on the website. Then, you will see screenshots of the BBA Club’s website where you can click anywhere you feel is necessary to accomplish the task. After, you will find 10 questions on the design you have experienced. Please reply without much thinking, rather focus on how you felt. Thank you for your participation.
The difference between the two emails will be in the given link: Group A will receive a link with a poorly designed website, whereas Group B will receive a good-designed website. The task for group A was “How would you subscribe to the BBA Club’s Events Calendar send-outs?” and the task for group B was “How would you register for the upcoming event?’. Furthermore, a couple of participants were willing to provide feedback on the designs, so that the author could obtain insights on this matter.
4.1 Results
In this point there will be shown results of the experiment both: visually and descriptively. Firstly, the two versions of the design will be shown and their primary differences according to the heuristics as well as task duration comparison in terms of task completion as an average from all users in the following paragraph 4.1.1.
Afterwards, the screenshots of heatmapping will be evaluated followed by calculation of effectiveness and efficiency on the basis of click mapping. The most significant part is
the SUS score comparison of both designs, which defines user satisfaction. Last but not least, the author will share the insights from user feedback and data evaluation method of efficiency and task duration.
4.1.1 Design versions
Appendix 1 presents the two versions of the design for the BBA Club’s website. Figure 1-4 represents the mock-ups for a poorly designed UX/UI website and Figure 5-6 represent the mock-ups for a well-designed UX/UI website. The criteria for the author to make designs in such a way was based on the heuristics described in point 2.6.
The reader firstly can vividly notice that there are twice as much screenshots required to complete the task than that of the good UX/UI design. That’s the first difference – it will take the user twice as many actions to perform a task on a badly designed website in comparison with a well- designed website. Does this mean it would take twice as much time?
Appendix 4 can be referred to with the calculation on task duration for both designs.
Usability Hub provides the opportunity to download the whole time spent by the user whilst completing a task, but in milliseconds. Thus, the author calculated the seconds of task completion by the user for convenience and understanding of the reader. Sheet
“BAD_UX_TaskDuration” in Appendix 4 points out that users spent on average 45.5 seconds completing the whole task. Whereas in sheet “GOOD_UX_TaskDuration” the calculation is that a user spent on average 16.3 seconds completing the task. Thus, proving that a poorly designed website took up 2.8 times more for the user to complete a task (see Appendix 4, sheet “End_Results”). Table 3 below shows the task duration of all 51 one participants from Group A. On the Y axis is the total amount of seconds it took a user on the X axis to complete a task. It can be seen in the table that most users on average performed within the 45 second average mentioned earlier. Some users were quicker by a few seconds and others were faster, but there were 4 users who took an exceptionally long time to perform the task, they can be seen by the long bars in the range of 80-90 seconds. The assumption the author made is that these users might have spent a long time studying the 3 part of the design (see Appendix 1, Figure 3). On the other hand, users 6 and 8 performed the task in 20 seconds and under. Although it doesn’t guarantee that they correctly performed the task.
Table 3 Task duration for bad UX/UI
Source 16 Author, Appendix 4
Table 4 below depicts the task duration in a good UX/UI design. It is similar to Table 3 above, but on the Y axis the maximum value is 40 instead of 100. In a good UX/UI design it is seen below that most users averaged the 16 seconds mentioned above, yet there were also users 11 and 26 who took up a long time to complete the task, but it is still smaller than the average task duration in a bad UX/UI design. The author assumes it happened because the user was unsure whether he should choose the event on the left first or signed up straight away on the right (see Appendix 1, Figure 6). The reason for this assumption is in the heatmaps which are described in the following point 4.1.2.
0 10 20 30 40 50 60 70 80 90 100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
Number of seconds
User #
Task duration in seconds
Table 4 Task duration in seconds in a good UX/UI design
Source 17 Author - Appendix 4
The main goal of task duration calculation was to acknowledge that a poorly designed web platform can force the user to complete a task much longer than he/she could have in a well-designed website. In the real world, users are likely to drop out of their task if it takes too long (Nielsen, 1993).
The second difference concerns the aesthetics and minimalist design, which is also one of the heuristics. Figures 1-4 do not comply with this aspect since they have an unnecessary background picture, which creates “noise” for the users – too many things to pay attention to, as well as there is extra useless information for the user such as the date in the top right corner shown in Figure 1 Appendix 1. A good UX/UI provides for the end-user necessary information or makes it clear what possible actions can be performed. That is why in Figure 5 we can see that the design does not have extra “noise”
in the background. It is rather minimalistic and in the top right corner in Figure 5 the reader can already see the menu visible. This way the end-user can start performing the desired action quicker.
On the other hand, the bad UX/UI design does possess good heuristics. The similarity between the two designs will be in the UI that is designed in a Z pattern, which is described in point 2.2.
The second similarity would concern the heuristic “visibility of system status”. It should let the user know that the action was understood. In both of the Figures 2-3 the reader can find a response of the system to the user – when the user presses “Events”
then he/she will be redirected to the corresponding page and the user will see on which
0 5 10 15 20 25 30 35 40
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51
Number of seconds
User #
Task duration in seconds
page he/she is on the top right corner in Appendix 1 Figure 3. At the same time, the reader can notice that Figures 5-6 show a reaction to the user that when he/she presses on the “Events” button it becomes bold. It also lets the user know on which page he/she is at the same position.
The bad UX/UI design has recognition for the user in terms of the “burger” menu button shown in Appendix 1 Figure 1 in the top right corner. This is a widely accepted icon used in many websites, the user understands that if he/she presses on this icon they will find the menu of the website. This will affect the experiment.
4.1.2 Heatmaps
Appendix 2 presents heatmaps for both designs. Figure 1-4 shows the heatmaps for bad UX/UI design, Figure 5-6 shows for good UX/UI. The green zone on the screenshots is the area defined by the author where the user is allowed to click to continue to the next slide in Usability Hub – known as hit zones described in the theoretical part. The author decided to make the whole zone green to gather as many results in all the screenshots of the mock-up design as possible. Otherwise, the navigation test works in such a way, that if the author defined different hit zones, for example only the correct ones, then the users would have been done with the experiment the moment they pressed on an area out of the green zone, meaning they will not have a chance to react on the following screenshots. In bright red color we can see where the users clicked the most.
In Appendix 2 Figure 1 the reader can see that the user recognized the burger button and that most of the users pressed there. A green spot in the slogan of the website means that some users thought the slogan would lead them to a desired page. A couple of blue spots are individuals who might have been completely confused on where they are supposed to click. To summarize Appendix 2 Figure 1, most users pressed on the “burger”
icon recognizing it as a menu button, which means that one of the heuristics – recognition – worked for this design. The green spot means that a smaller number of users mistook the slogan for a button, the author’s assumption would be that it is because of the strange position of the slogan at the bottom of the page. It is placed in an unusual spot for the user, who might have pressed it out of curiosity.
In Appendix 2 Figure 2 it can be seen that most of the users, according to the defined task, pressed on the “Events” button. It is also visible that few users tried pressing
“About”, “Contacts” and the slogan. The reason for this may be the distraction of the user on too many objects. These distractions happened because the design does not correlate with a minimalistic, aesthetical style – a heuristic principle.
Appendix 2 Figure 3, which depicts full confusion of the user. The takeaway is that the button leading to subscription to calendar events is unnoticeable at the very bottom of a page in form of an arrow pointing upwards (see Appendix 1, Figure 3). We can see that there were users who found it, but it definitely was not the first place where they looked for it. By the bright red color, we can see that the users paid most attention to the big
“Sign up” button in bold, but it was misleading and distracted the user from the end goal.
