We decided to conduct our user testing in two stages to allow for iterative improvements between cycles. We used an interview style and ‘think aloud’ approach to gather information and feedback naturally on how the user interacted with our system. Each interview started by asking a user questions about their background and experience to gather their level of modelling expertise. The purpose of this was to understand how different abilities interacted with the UI. Following this, the users conducted specific exercises and scenarios whilst talking us through their thought process, actions, and the reasons for them. This allowed us to understand why a user took certain actions, to evidence how intuitive the system is.

The user testing provided a range of positive and negative feedback which highlighted definite consistencies in some of the users’ thought processes. We used this to form a consolidated list of improvements which was prioritised and labelled according to the MoSCoW approach as “must have”, “should have”, “could have” and “won’t have” changes. This prioritisation was essential to help keep the team focused and ensure we had the most impactful changes in place before the second round of user testing took place.

Before the second round of user testing, our UI developers at Oxford Computer Consultants worked hard to implement as many of those changes as possible. By implementing the changes suggested from the first round of user testing prior to the second round, we had the opportunity to see the value of our improvements. This proved to be a useful approach as it meant less repetitive feedback for the development team and resulted in a more positive and focussed second round.

A new testing cohort was recruited for the second round, as fresh pairs of eyes to judge and test the system as a ‘new’ user to keep the two tests fair. Using the same cohort may have resulted in repetitive testing results from users who focus on the changes rather than the current system they are reviewing.

Due to the significant improvements to the user journeys and basic UI, the second round resulted in feedback focussed on modelling needs and language. This in-depth feedback has given us ideas on how to make the system more usable and helpful to the day-to-day users. The next step is for us to repeat our process of consolidating the feedback, prioritising and then implementing the changes before our Beta release.