Customer perspective: Usability Testing

Maurice was one of the first participants of the first usability testing for Lido Flight 4D during the Lido Flight 4D User Group in November, 2017. In 2020, Maurice, together with his colleagues, participated in 6 sessions. KLM is one of 10 airlines that participated in 41 individual sessions.

Usability testing is one of the user experience research techniques used to discover problems with the design of the product. The technique is also used to evaluate the usability of the developed system. The User Researchers from the Lido Flight 4D Product Research, Design and Training Team regularly conduct usability tests when working on the Lido Flight 4D application.

In the recent studies, we mainly tested the prototypes (designs in InVision) and the developed Product Increments (PIs) of the Lido Flight 4D IFS on the Sandbox. During the session, the facilitator asks the participants to perform some tasks, observes human-computer interaction and asks additional questions. The observations are later analysed and interpreted. The findings are reviewed by the Product Owners and other specialist, such as Requirements Engineers, Business Analysts, UX and User Interface Designers. The Product Owner prioritises the findings, improvements are planned, and then implemented during the design and development.

Let’s talk to Maurice and see what this looks like from his point of view.

Lufthansa Systems (LSY): What is it like participating in the usability testing sessions?
Maurice: We experience this as a positive development. The biggest advantages are that you get to see the new IFS in a very early stage and that our input is taken in to account, which we notice when the PIs are rolled out. The sessions take place in a friendly atmosphere and there is room for open discussion, which we appreciate.

LSY: What does it require from your side to participate?
Maurice: From our side we always try to involve our Functional Application Manager, a Flight Dispatcher and myself. I act as an SME subject-matter expert on both the Flight Dispatcher tasks and management tasks due my background. If needed, additional disciplines are invited as well (e.g. the Senior Data Specialist or other people like Analysts). We try to have a few people in the same location so we can have internal discussions if needed, but these sessions can also easily be attended via remote location.

LSY: Where do you see the value in participating in the usability testing sessions?
Maurice: The biggest value of these sessions for us is the involvement, and the possibility to give our opinion and ideas on new functionalities and requirements directly to the Lufthansa Systems specialists. We have noticed that if we make suggestions or give ideas over new functions or functionalities, these are seriously considered and if possible, they will be implemented in future PIs. This hopefully will result in a product which has a high level of user acceptance, which fits in our organization and that we have a system that meets all our standards and requirements when all is finalized.

LSY: You have been participating in these sessions for a few years already. Has your perception of the usability testing changed over time? If yes, how?
Maurice: My perception has indeed changed. Before we started these sessions, I was a little bit sceptical and expected that this would be a session where we would be presented a (nearly) finished product or functionality and that the purpose of the test session would be to see if we - as an end user - were able to use it. However, we noticed that the functionality or products that were presented and used in a session were far from ready, and that our input and opinion was asked during the (early) development (or even design) phase. We noticed that our input was used; it is much more taken into account than I had expected. I think that this is a very good development because now I expect there will be an end product which is much more user friendly and will have a higher degree of user acceptance.

LSY: Is there anything you would like to be changed or improved in these sessions.
Maurice: No, for now I can’t think of any improvements for these sessions. What we would like to see is an overview of the planning of the PIs and which functionality will be in which PI so that we can decide what the important PIs for us are. Overall, we are happy with the way LSY is involving us as a customer in the development of the new IFS. It gives us insights into the new IFS from a very early stage and this can be used for our own benefit. The cooperation with the LSY employees we experience as pleasant, and they are open for suggestions and ideas, but still they keep a good overview of the direction of the end product.

LSY: Thank you for the interview and your support!