Usability Test Report

Jamie Berger

Team 5: Traffic Counters

Table of Contents

Introduction 2

Test Plans 3

Results 7

Conclusions 16

Appendix A: Team 5 Attendance 17

Appendix B: Bug Report 18

Introduction

Description of UI

The user interface allows the user to act as a data collector (student) or a data analyzer (professor). As a student, the user may record which intersection and the time interval they must observe cars. Time intervals may be an hour or longer. On the next page, there is a birds-eye view of the intersection with 3 buttons on each of the North, South, East, and West ends of the intersection to indicate a straight, left or right turn through the intersection. On the same page, there is a start session button that the user must press in order to start the counter. If the user miscounts during collection, they may press the undo button in the center of the screen. Once the user is done counting, either time will run out or they may select the end session button. After this, the next page has a summary of the data gathered. The other view as a data analyzer involves downloading a csv file from the application to the user’s device by pushing a download button. The user may also view the data by a date range or by the selected time interval.

Usability Test Goals

The usability test for this application has the goal of being effective for a traffic engineering course. The professor must be able to use it to analyze traffic data, and the students must find ease with the application in order to record accurate data. This means that buttons must be convenient to select because traffic can get busy. Also, data must be easily downloadable and organized in a fashion that is useful for research and interpretation.

Description of Tests

There were 2 sets of tests. The first test was a simple confirmation that the counter works and that the overall concept and design of the application is user-friendly. The second test was split into 2 scenarios where the participant acted as a student and as a professor. In the student scenario, the participant had to go outside to an intersection and use the application. Before any recording took place participants were asked to fill a consent form and pre-test. After the scenarios the participants were asked to answer some dialogue questions about the application, and finally followed a post-test. During the testing, the facilitators recorded quantitative and qualitative information.

Test Plans

This section includes the testing schedules for each test as well as the testing scenarios.

Test 1

There were 2 tests taking place on April 17, 2018 from 6 - 8 pm, a half-hour each. View below for the breakdown of what to expect during that half-hour.

Pre-Test & Set-Up 5 minutes

The facilitator will provide written test description and the task list that will be carried out by the participant. The facilitator will allow for questions and then pass out the consent form. Once the consent form is signed, the participant will begin answering the Pre-Test questions.

Scenario #1 10 minutes

The facilitator will read aloud the description, then assist the participant in downloading and opening the application. (The participant will perform login ONLY before the intersection). The facilitator will then walk to the participant to the intersection of Macinnes Drive and Townsend Drive. The facilitator will help the participant find the correct orientation of cars to analyze, and then the participant will carry on steps 1-5 test scenario #1. While the participant is using the application the facilitators are keeping track of quantitative and qualitative data elements.

Dialogue Questions 10 minutes

The facilitator and participant will discuss, through oral dialogue, the evaluation of the scenario. Results may be temporarily recorded.

Post-Test 5 minutes

The facilitator will pass out written post-test questions that ask about an overall evaluation of the application.

Test 2

There were 5 tests taking place on April 21, 2018 from 12:30 - 1:30 pm. View below for the breakdown of what to expect during that hour.

Pre-Test & Set-Up 15 minutes

The facilitator will provide written test descriptions and the task lists that will be carried out by the participant. The facilitator will allow for questions and then pass out the consent form. Once the consent form is signed, the participant will begin answering the Pre-Test questions.

Scenario #1 20 minutes

The facilitator will read aloud the description, then assist the participant in downloading and opening the application. (The participant will perform login ONLY before the intersection). The facilitator will then walk to the participant to the intersection of Macinnes Drive and Townsend Drive. The facilitator will help the participant find the correct orientation of cars to analyze, and then the participant will carry on test scenario #1. While the participant is using the application the facilitators are keeping track of quantitative and qualitative data elements.

Scenario #2 5 minutes

The facilitator will read aloud the description, then assist the participant in switching to a professor perspective and carry out scenario #2. While the participant is using the application the facilitators are keeping track of quantitative and qualitative data elements.

Dialogue Questions 10 minutes

The facilitator and participants will discuss, through oral dialogue, the evaluation of the scenario. Results may be temporarily recorded.

Post-Test 5 minutes

The facilitator will pass out written post-test questions that ask about an overall evaluation of the application.

Scenario #1

The Student Perspective

Goals

1. Participants can easily and accurately log and count automobiles in real-time from each intersection (using the mobile version)

2. Participants can correct their errors

**All of the below information was also provided to the participant

You are an undergraduate student in Dr. Zhang’s traffic engineering task. Your most recent assignment is to go to the ___________________bound end of the Macinnes and Townsend Drive intersection and count how many cars and semi-trucks pass through. You will use the Traffic Counter application to record the results. Make sure to count the cars moving in the direction of your assigned post only. You will carry out 2 rounds in different posts of the interestion.

Scenario #1 Task List

1. Login // don’t begin step 2 until we are outside

2. Select Intersection

3. Select Duration 5 Minutes

4. Start Counting/Session

   1. Note: not many semi-trucks pass through this area. We will substitute red cars for semi-trucks in this test

5. Count from Assigned (North, South, East, West) Orientation

6. Start another session from another orientation

7. Select Intersection

8. Select Duration 10 Minutes

9. Repeat 4-5

10. End Session sometime between (4 - 7 minutes)

11. Answer Dialogue Questions

Scenario #2

The Professor Perspective

Goals

1. Users can easily access, download, and read the data (using the desktop version)

**All of the below information was also provided to the participant

Description of Scenario

You are the professor of the Traffic Engineering course (Dr. Zhang) and you would like to interpret the results that the students collected for your research. This means that the data should be readable and easy to parse.

Scenario Task List

1. Login

2. View Data

3. Select Date Range

4. Read Data and Think About Interpretation

5. Answer Dialogue Questions

Results

Pre-Test Results

All participants were aged 18-23 and had various majors. The majors include computer science, computer engineering, business, mechanical engineering technology, chemical engineering, and one no response. Figure 1 shows the spread of each participant’s year of study.

The participants asked if they had any visual impairments and what device, if any, they use to correct it. Everyone claimed to have no visual impairment or that they wore glasses.

The participants were asked how often they use their phone outside so we can determine the adaptability. For example, those who use their phone outside more frequently would know to turn up their brightness. The other two measurements were rarely and never, but those were not selected by any participant.

Surprisingly, two out of the seven participants have counted cars at an intersection before.

We also asked participants how many years they’ve used a smartphone to gauge their comfortability levels as well as how they would describe their experience using mobile phones for the first time. Here are the results.

Dialogue Question Results

Participants in Test 2 were asked questions in a focus group. After listening to the recordings, here is a summary of the results.

If you made any errors, were you able to correct them? Was it easy to correct?

Users knew about the undo button when told. Those who knew about it found it very helpful, those who didn’t know about it would use a method to error correct: they would remember when they have over-counted and then skipped a count for the next coming car. [Some noticed the button and didn’t know what it was.]

Rate the level of difficulty of interpreting the start auto-toggle button (1 easy - 5 hard) and explain.

The auto-toggle button was rated a 2. It’s not very intuitive and the color makes it unsure.

Rate the level of difficulty of interpreting the start session button (1 easy - 5 hard) and explain.

The start button was rated a 1, but some felt that it was unnecessary to have 2 start counting buttons.

Rate the level of difficulty of interpreting the start end button (1 easy - 5 hard) and explain.

The end session button is convenient, but the bug should be fixed.

Explain your experience/thoughts of interpreting the csv file results .

It’s not easy to determine cardinal directions on the csv file. Also, the 0 for a car and 1 for a truck is not quickly intuitive. The dates and times are easy to read.

Extra Comment It would be helpful to have some sort of haptic feedback. Looking at the count number isn’t helpful.

Post-Test Results

Participants were asked if they enjoyed using the application. No one said no.

Most participants found that the application would be effective or very effective for counting cars at an intersection and all participations found that the application would be helpful or very helpful for the professor of the traffic engineering course. No one felt that the application would be ineffective or not helpful.

Common Usability Question Results

We had a goal to test each direction at least twice.

We measured the time at an intersection which is what we consider to be when the user starts to enter the intersection information while they are outside until the time they are walking back inside after recording data. The average time at an intersection was around 21 minutes.

In an attempt to measure the accuracy of the counter, we attempted to keep count by-hand to compare the results. Unfortunately, the testing day was very busy so it was difficult for the facilitators to keep count by-hand. Therefore, these results are not very helpful.

Facilitators were asked to watch the level of difficulty the participants were having while navigating the application. No user had a difficult time navigating the application.

Facilitators were also asked to record qualitative measurements about the participants such as their visual emotions and body language. Here is a summary of the results:

Most participants had content visual emotions while using the application. There was some confusion during the set-up. One participant missed the end session button and felt a little panicked.

The body language of the users were very attentive to the automobiles and relaxed while counting. No one seemed flustered.

Conclusions

The conclusions include a list of changes to be made on the application as well as a list of usability problems to consider.

Changes to be made

1. Format the csv file correctly

2. Have an indication to let the user know to increase the brightness

3. Add some type of orientation information

4. Make full screen capabilities better known and move the horizontal phone orientation notification to before the start session and after the intersection data form

5. Make the function of undo button more obvious and make it more easily seen

6. Add a help page that guides users how to use the application

7. Change the colors of the auto-toggle so that it turns red when trucks are being counted

Usability Problems

Time Input Problem

The application doesn’t confirm that a time is input, so it will have an infinite session

Intersection Counting Problem

The application doesn’t confirm a chosen intersection, but will count anyway

CSV Problem

There is no csv file download button

By Interval Problem

You can not view data by the time interval

Network Problem

If the user has Sprint, the application runs slow

Appendix A: Team 5 Attendance

Test 1

Caden Sumner, Connor Przybyla, Richy Carpenter, Noah de Longpre’, Ethan Marnin, Dustin Rens, Qu Shuhao

Test 2

Caden Sumner, Connor Przybyla, Richy Carpenter, Noah de Longpre’, Ethan Marnin, Dustin Rens, Qu Shuhao, Eamonn Daley

Appendix B: Bug Report