top of page

UNC Campus Recreation
Usability Study 

A semester-long usability project of the UNC Campus Recreation Website covering all aspects of designing and conducting a usability study including defining users, initial heuristic evaluation, designing tasks, creating a methodology, conducting a study with participants, and creating recommendations.

Campus Rec Usability Readout Title Slide.png

Introduction

Campus Recreation Desktop Website: campusrec.unc.edu

​

The website is used for:

  • Campus facility information

  • Campus-sponsored events

  • Signing up for fitness, health, and outdoor activities and excursions

* Note: Sign-up requirements changed halfway through the semester.

​

Why did we choose to evaluate this website?

  • Accessing Campus Rec services is a large part of the student experience at UNC-Chapel Hill

    • Students must go through this website to participate in any recreational activity from intramural sports to group fitness to signing up for a time to use the normal gym facilities

​

Evaluation Goals

  • To determine usability problems with signing up for gym time slots and group fitness​

  • To determine usability problems with signing up for sponsored activities

  • To determine usability problems with finding information on facilities

  • We chose these goals because:

    • Due to COVID-19, gym time slots had to be reserved, an unfamiliar process for many students

    • Many students participate in group fitness

    • Gym facility information is important for finding policy information, location, etc.

Methodology

Test Design

  • Within-subjects study design

    • All participants were tasked to complete all of the tasks​

  • Think-aloud​

    • Participants were asked to talk aloud about their reasoning and thought process when conducting the tasks​

  • Mixed-method questionnaires

    • Pre and post evaluation questionnaires​

  • Post-task questions​

    • Two scalar questions were asked after each task​

  • 30-minute evaluation time​ per participant​​

  • Structured so that pages did not change from user to user

    • Consistent data collection​

​

Test Environment, Equipment, and Materials

  • Environment

    • Reserved study rooms, provided classroom space

    • Participants were seated in front of a laptop with Zoom screen recording software

  • Equipment and Materials

    • Laptops for participant and note-takers​

    • Zoom installed on the participant laptop

    • Chairs for participant, moderator, and note-takers

    • Google forms

      • Questionnaires​

      • Consent Form

    • Printed moderator script​

    • Printed task Instructions

    • Printed post-task questions

Task Descriptions and Motivations

Our group conducted a heuristic evaluation of the website and decided on tasks for the evaluation. These tasks were evaluated to determine the necessary steps for completion. If usability problems were extremely obvious for tasks or tasks were easy to complete then they were not chosen for use in the study.

​

Task 1: Find out if workout reservations are required to use the SRC gyms

  • Task 1 provides insight into the ability to participate in the primary function of on-campus gyms; workouts

  • Initially planned to ask participants to sign up for a workout reservation but policy changed halfway through the semester so that sign-ups were no longer required

    • This task still had value in showing how well the website provided changing policy information in this chaotic time period​

​

Task 2: Find an Intro to Climbing Course on the upcoming Wednesday at 8:00PM

  • Task 2 provides insight into the usability of the system to find and sign up for campus-sponsored events

​

Task 3: Find a Yoga Session on Wednesday from 7:00PM to 8:00PM at Rams Head

  • Task 3 provides insight into the usability of finding and signing up for regular group fitness activities

  • The difference between tasks 2 and 3 are:

    • Task 2 is a campus-sponsored event which means it is not necessarily a regularly held event and (in most cases) requires payment to participate

    • Task 3 is a group fitness activity which is included in student fees and is usually a regularly scheduled activity

​

Task 4: Find a Swimming Pool close to the UNC Pit

  • Task 4 provides insight into the ability of the interface to help users locate recreational facilities

Data Collection

Data Collection

  1. Consent forms, pre-evaluation questionnaires, and post evaluation questionnaires were filled out via Google Forms

    1. The pre-evaluation questionnaire asked demographic questions and if the participant had previous experience using the Campus Rec website​

    2. The post-evaluation questionnaire asked scalar questions regarding the participant's experience completing the task as well as free response questions regarding their thoughts on the website, its issues, and what improvements could be made

  2. Post-task scale questions​

    1. After each tasks participants were given two 5-point Likert scale questions about the ease of use of completing the task and if the steps to complete the task made sense​

  3. Tasks and think-aloud were recorded via Zoom screen recording and the built-in laptop microphone

  4. Notes were taken by note-takers during the study and while watching the recordings and were later aggregated in a shared team document

Campus Rec Usability Eval Evaluation Metrics.png

Data Analysis and Results

Our pre-evaluation questionnaire revealed that 75% (3 out of 4) of our participants had at least some experience using the Campus Rec website previously.

Campus Rec Usability Pre Results.png

We decided to represent our number of errors in table form to better illustrate the discrepancy between error rates for different tasks. Tasks 1 and 4 had relatively low and consistent numbers of errors, however Tasks 2 and 3 had both more and more varied numbers of errors between participants which indicates that these tasks are specific points of interest.

Campus Rec Usability Errors Results.png

Consistent with the results shown in the previous table, Tasks 2 and 3 had both many more and extremely varied numbers of clicks between participants. This indicates both that participants seemed to have more problems with these tasks, and that these tasks are designed in such a way that users cannot consistently reach a point of completion with a similar number of clicks so there is some confusion in the process of completing these tasks.

Campus Rec Usability Clicks Results.png

While all of the tasks here varies somewhat in terms of their times of completion, it is also once again clear that there is particularly more variation in the time to complete tasks 2 and 3.

Campus Rec Usability Time Results.png

This question was asked after each task was finished on a 5-point Likert scale with 1 meaning very easy and 5 meaning very difficult. The bars are the mean. The error bars represent a confidence interval of 95%. We can see that Task 1 was considered to be relatively easy. The bars themselves make it seem like the other tasks were considered to be between 2.5 and 3, which one might think is not too difficult, but the error bar shows that this is not the case. For Task 2 in particular the answers varied widely so the confidence in the mean is not very high.

Campus Rec Usability Post Task Results.png

This question was asked after each task was finished on a 5-point Likert scale with 1 meaning very clear and 5 meaning not very clear. Once again, we can see that Task 1 seemed to be relatively clear with the means for the others not being overly unclear, but Task 2 has a very large error bar indicating variance and low confidence in the mean.

Campus Rec Usability Post Task 2.png

We asked 2 scale questions in our post-evaluation questionnaire about whether the participants liked the website and if they would recommend it to friends and family. 1 would indicate that they do not like it while 5 would indicate they like it very much. 1 would indicate that they do not recommend it and 5 would recommend it that they recommend it very much. The mean for likeability is around 3 so participants were relatively neutral on this front. Recommendability seemed higher than expected, but this is also the only website and only way to access these functions so participants do not have a great alternative to this website.

Campus Rec Usability Post eval results.png

These are some of the insights we gained from the free response questions on the post-evaluation questoinnaire with some quotations.

Campus Rec Usability Post Eval 2.png

As we saw in our data, Task 2 represented the greatest struggle and variance among participants. Participants generally took between 3-16 clicks which is a very high degree of variance considering the actual minimum number required starting from the homepage without any magic links was 3 clicks. One of the biggest challenges particpiants ran into was that they followed the hyperlink for Climbing in programs but that only takes the user to a page describing the climbing facility rather than any place they can go to actually see or book a spot for the event. This is confusing for the user given the label of Program for the category while another place tab named Facilities makes more sense, and this is indeed where the link takes you from the Programs page.

Campus Rec Usability Task 2 struggles.png

Task 3 was the other major task where there was a lot of challenge and variance among participants. One of the greatest challenges participants encountered was that the navigation to sign up for group fitness classes takes one to this calendar page. The calendar itself is information overload with small text and no immediate clues to what the user is looking for. Also, the way forward is not clear. Many participants clicked the calendar icon but this just downloads a .ics file to add the event to your device's calendar. The actual thing the user is supposed to do is click the black Description text which then has a tiny link in a popup box to the sign up portal.

Campus Rec Usability Task 3 Struggles.png

Finally, for Task 3 there was a usability issue as users had to eyeball this as the map did not show proximity and clicking on the names below the map takes one to the facility description pages that do not show any map or location information. This did not prove overly challenging for participants, however. The other major problem was that participants were expected to choose the Bowman Gray Memorial Pool as the pool closest to the UNC Pit (though it was not considered wrong if they chose otherwise) but all chose Kessing Outdoor Pool instead. The reason for this is that, despite the name showing under the calendar and the Bowman Gray Memorial Pool showing up during our initial study design process, it no longer appeared on the calendar during the two weeks we conducted our study. It reappeared during our data analysis. Because of the inconsistency of display on this map, participants were not even given a real chance to find the expected answer.

Campus Rec Usability Task 4 Struggles.png

Recommendations

We created final recommendations based on the insight from our study.

​

  1. Easier Navigation 

    • “I would simplify it. especially the navigation. I would have fewer pages.” ​

  2. Clearer menu heading structure

    • Users were consistently confused by distinctions between “Facilities,” “Programs,” and “Events,” as the pages listed within each of these were not intuitive​

  3. ​Calendar should be more simplified and easier to glance at to find events

    • “grid was a little overwhelming”​

  4. Concise and consistent pathway to booking​

    • Many events/classes that required booking had many different ways to get to the sign-up portal​

  5. ​Consistency of look and layout between pages

    • “I think the graphics and site look nice, but that is at the cost of good usability. Many pages also have very different styling.”​

*This project was completed as part of a group of 4 people and information pulled from our reports and final submission

**All tables and graphs were created by our team using Microsoft Excel and Microsoft Word.

**Slide template was created Slidesgo, icons by Flaticon, and infographics and images on slides by Freepik.

bottom of page