Usability Testing Log

Sunday, 4/10/2022

Usability Test 1

Time: 3:00pm - 4:00pm

Consultant: Ivona Gorgioski

Logger(s): Audrey Schmitt, Foenix Szot

Developer(s): Kevin Cornell, Austin Sopha

Participant 1

Log:

• Is using a laptop
• Session 1:

• Scaling on buttons and font for menu and overview were really small compared to background of screen
• Not sure where to start, seems confused and overwhelmed
• Assumes needs to connect different nodes
• Connects railway to demand node
• Visible confusion as to what the connections mean
• Figured out what the names of the nodes are, but still confused on what they do
• Connects supply node to road
• Tried to connect the roadway to the railway (doesn’t work but user is not sure why)
• Not sure when the game ends, or what the goal of it is

• Session 2:

• User figures out how to disconnect nodes
• After reading instructions, is able to remove all connections
• Tries to disconnect node in incorrect order (destination to source)
• No easy way to clear all selections
• Vehicles still move around ‘road’ after disconnection

• Session 3:

• Connects distribution node to demand nodes without supply node involved
• Still doesn’t entirely understand what the game does

Interview Questions:

• What is your first impression of the website?

• User understood he had to connect nodes, but wasn't sure how to do it. Eventually figured it out but it was unclear to start.

• Did this impression change over the course of testing the site?

• Yes, the help section was helpful in clarifying the rules and node selections.

• Is there anything you would change in regards to how the simulator functioned?

• "I don't believe so, it seemed to function for what it needed to do."

• What was the best part of this experience?

• "Best part was once I figured it out it was easy to use."

• What was the more frustrating part of this experience?

• "Worst part was figuring out if nodes were connected or not.”

• Do you feel you learned something about supply chain while navigating this application?

• “I am not sure.”

Usability Test 2

Time: 4:00pm - 5:00pm

Consultant: Ivona Gorgioski

Logger(s): Kevin Cornell, Foenix Szot

Developer(s): Audrey Schmitt

Participant 2

Log:

• Session 1:

• Reads supply node description
• Is noticeably confused and asks developers for help at first
• Connects supply node to road
• It is not apparent to user what connects to what still
• Pauses and analyzes the screen
• Connects road to railroad, then to demand node
• Tries to connect two demand nodes but can’t
• Connects railroad to another demand node
• Facial expression is very confused after ‘ending’, “I don’t think I did it right.”
• Can’t disconnect and clear screen, has trouble figuring it out

• Session 2:

• Analyzing the ‘How to Play’ menu
• Misunderstands clearing selection (could reword it)
• Manages to clear screen after rereading help screen
• Visual bug in budget and emissions
• Tries to disconnect between road and demand, but doesn’t because connection was between railroad and demand
• Has trouble with selecting nodes and connecting them even after reading instructions, but is able to clear them
• Still misunderstand on valid connections (add further help in help menu)
• Not really apparent how many nodes can connect to another node (have help screen point out selection menu box)
• Goes back to the ‘How to Play’ menu after a long while of trying to figure out how to ‘complete’ the game
• User is possibly reading the nodes

• Session 3:

• Is faster at clearing the screen
• Notices where the total budget is
• Reconsults ‘How to Play’ menu for clarification
• Is still unsure if they have ‘finished’ the simulation or reached the goal

Interview Questions:

• What is your first impression of the website?

• “It looked like a normal game you would see on any website.”

• Did this impression change over the course of testing the site?

• No

• Is there anything you would change in regards to how the simulator functioned?

• “Make a way to clear all the connections at once, that would be helpful.”

• What was the best part of this experience?

• “Best part was that it was fun.”

• What was the more frustrating part of this experience?

• Wished the help menu was more clear

• Do you feel you learned something about supply chain while navigating this application?

• Yes, somewhat

• Could you identify the four kinds of nodes?

• Identified: road, train track, supply was a tree or bush (?), demand was a minecart (?), noticed vehicles between connections as trucks

Usability Test 3

Time: 5:00pm - 6:00pm

Consultant: Ivona Gorgioski

Logger(s): Kevin Cornell, Audrey Schmitt

Developer(s): (unknown)

Participant 3

Log:

• Session 1:

• Knows to click ‘play’ to start the simulation
• Was expecting a tutorial, confused at the start
• Clicking on things to orientate themselves
• Visibly confused as the what the clicks mean and unsure of what things do
• Expresses desire for tutorial
• User is trying to connect demand nodes to distribution nodes (incorrect order)
• Finds the roald being brawn are odd
• Not sure what the ‘goal’ is or what to do when finished
• May have figured out how to delete connections on their own

• Session 2:

• Doesn’t know where help section is
• User looks under ‘overview’ for help page but wasn’t sure this was the right place to look
• Thinks it may be in the gameplay screen, that it should be available in the game screen
• User noticies ‘total budget/total emissions’ bug
• Budget bug is confusing
• User states that they are doing the same thing as the previous scenario
• Seems to find it easier to connect nodes
• User expresses a desire for a ‘goal’, not sure of what to strive for
• User notices that even though connections are deselected, trucks stay moving
• The participant is able to easily connect and disconnect nodes

• Session 3:

• Given goal seems to help
• Achieves goal quickly

Interview Questions:

• What is your first impression of the website?

• Looked interesting and classic. Initial design was good on the home screen but game screen could use work. Liked the name of the game and ‘lore’.

• Did this impression change over the course of testing the site?

• “A little bit, I felt there could have been more challenges. If you’re going to continue with this and develop it more, you could add levels to it. That would be better and would challenge players a bit more. For a basic game, it was pretty good.”

• Is there anything you would change in regards to how the simulator functioned?

• Add levels, change lines to “flow better”

• What was the best part of this experience?

• “The best part of it was the overall design and concept, those kinds of games are really popular and if you’re teaching someone about supply and demand, that could be a useful tool.”

• What was the more frustrating part of this experience?

• “The most frustrating part was that it was so easy. I felt there could’ve been a greater challenge to it. There’s a lot more you could do with that.”

• Do you feel you learned something about supply chain while navigating this application?

• Yes

• Could you identify the four kinds of nodes?

• Identified supply nodes (forest), truck node (road), railway node (railroad), appreciated that the sprite was a road and not a truck. Identified demand node (not sure what the demand node was supposed to be, thought it looked like rocks).