Usability

Testing Report
Evaluation of the UNF
Department of English
website
Prepared by:
Jocelyn Tolbert, Rebekah
Goodrich, and Hannah Othmer
Table of
Contents
Executive Summary ..........................................................3

Introduction ..........................................................................4

Executive
Methodology ........................................................................5
Participants ...........................................................................5

Summary
Participant Data Summary ..........................................6
Procedure ...............................................................................7
Tasks ..........................................................................................8
Task Completion ................................................................9
Usability Metrics .................................................................9
Scenario Completion Time ..........................................9

Findings and Recommendations ...........................10

Conclusion ............................................................................15

Appendices ..........................................................................16

References ............................................................................18

2
Executive Summary

Executive
Summary
This report details a usability study conducted by three graduate student
evaluators of the University of North Florida (UNF) Department of English (DOE)
website. Within the study we have developed a series of surveys and tasks for our
participants to use on the Department of English website. The contents of the
study below will explain how we found the recommendations for changes to the
website, unless the finding was positive. For example, our first finding showed
that the website is compatible with the three-click rule.

In preliminary testing we found that graduate students might be an under-

evaluated group of users. After identifying participants, we conducted the study
in person and on Zoom. Each participant answered preliminary questions and
signed a consent form. After the completion of the tests, they completed a post-
study questionnaire and a survey on the study itself.

We evaluated how easily participants can navigate the website, trying to find
options that relate to: class descriptions, faculty bios, as well as club and event
information. These areas were evaluated on the basis of the three-click rule, video
recording and having the participants speak aloud their thoughts.

All of the testing was done at unf.edu/coas/english/index.html, and will be

referred to as UNF DOE throughout the paper.
3
Introduction

Introduction

The purpose of this usability study is to guide further development of the

Department of English website for the University of North Florida. Preliminary
research suggested that students should be our usability test participants, but
faculty were also consulted on the quality of the website. In order to examine
the accessibility and ease of use of the website, we observed content across the
website including faculty bios, course description, and student organization
information.

Our participants including graduate students enrolled in the English Graduate

program since their opinions and program-relevant content seemed
underrepresented. The lack of graduate content on the website did not
encourage students to take part in the extracurricular experiences that are
available to them. Our findings suggested that while the content on the website
was highly accessible, the navigational tools were slightly confusing.

4
Methodology

Methodology

Participants
The study consisted of three participants who are
graduate level students at UNF in the English program.
Preliminary research suggested that students were most
likely to use the UNF English Department website, but
the graduate students’ unique needs and capabilities
might have been overlooked.

Participants were UNF students that are enrolled in the

English Masters program and were not previously aware
of the project. The students were asked to participate
through email recruitment. Basic technological skills and
ability to navigate a website were required for the
involvement in the study. Participants may have had
prior experience with the UNF English website, but
experience with the specific tasks were not expected or
required. 5
Methodology

"[The DOE site] is

Participant Data Summary
more for
prospective
All of our participants were UNF English graduate students."
students who were currently enrolled or had been
within the last year. The graph below shows the
range of occupations from our participants and
their comfort level with the website before going
through the tasks.

All test participants identified as female.

One participant is visually impaired.
One participant indicated she had never used the DOE website since
enrolling; one said she did not use it often.

Comfort level
with UNF DOE
Participant Age Occupation
website before
test

Public high
Participant 1 55 5
school teacher

Graduate
assistant/
Participant 2 23 3
Writing center
consultant

Graduate
Participant 3 23 2
student 

Table 1. Participant age, occupation and comfort level with the website (1-5, 5
being the most comfortable), gathered during participant recruitment phase.

6
Methodology

Procedure

The participants took part in the usability test at a location on the UNF campus
and via Zoom. A desktop or laptop with the website and supporting software was
used in the participant's environment.

The participant’s interaction with the website was monitored by the facilitators
seated in the same area, or on the same Zoom call. Note takers and a data logger
monitored the sessions in the environment, or connected by video camera feed
on Zoom. Each test session was recorded and documented.

Testing Procedures At a Glance

Before testing began, participants Each participant also signed a

completed a background questionnaire.
consent form that
At the start of each task, the participant
was read aloud the task description from acknowledges the following:
the printed copy and then go through Participation is voluntary, that
the tasks.
participation can end at any
The facilitator instructed the participant
to "think aloud" so that a verbal record time, and that the session
exists of their interaction with the would be recorded but their
website.
identity will be kept
Zoom participants were asked to share
their screens so that the test proctor anonymous.
could observe how they go about the
tasks.
The facilitator observed and recorded
user behavior, user comments, and
system actions.
After testing was complete, participants
were asked to complete a final survey
about their experiences.

7
Methodology

Tasks

Task 1 Task 2
Find the phone number or email Locate the class meeting times of a
address of Dr. Kailan Sindelar, a given undergraduate course.
professor in the English department.
This task tests whether the current
This task tests the usability of the layout of the "Course Descriptions" tab
“Faculty & Administration” tab of the is useful and functional for students to
site. Currently, there are two ways to find out about class offerings.
find this information on the site, one of
which lists only English professors, and Note: During test planning, course
the other lists the entire staff of the descriptions for the next semester were
College of Arts and Sciences. not yet available on the DOE website,
so a course was chosen at random by
the proctor before test time.

Task 3 Task 4
Find out when the first English Choose an upcoming event that you
Graduate Organization (EGO) would like to attend and find out when
conference was. and where it will be held.

This task tests whether a user can find This task tests whether the current
information that is not prominently layout of the "Events" tab. Currently, this
displayed on the site. We hope to tab is populated from the bottom up, so
evaluate whether a tab for the English the farther down a site visitor scrolls, the
Graduate Organization (EGO) is needed. older the events (most of which have
already passed) they will see.

8
Methodology

Task Completion
Each task requires that the participant obtains data that would be used in the course of
a typical graduate student’s semester.

The task is completed when the task's goal has been obtained, the participant shares
that they cannot complete the task, or the participant requests to end the test.

Usability Metrics
Subjective Evaluations
Usability metrics refers to user
performance measured against specific A subjective evaluation survey
performance goals necessary to satisfy regarding ease of use and
usability requirements. We have measured satisfaction was collected via a
the outcome of each task on four main Likert rating scales questionnaire
factors: at the conclusion of the session.
The questionnaire included the
following questions:
The time which it takes the participants
to complete each task (See Appendix 2) I thought the website was easy
How many errors they make to use.
I think I would need the
How often they recover from errors
support of a technical person to
How many participants can complete be able to use this system.
the task successfully I found the tasks easy to
complete.
Additionally, our subjective evaluation I would imagine that most
survey outlined at right provides more people would learn to use this
insight into the participants’ attitudes system very quickly.
toward the site and its usability.
Additionally, participants were
provided a portion of time in which
to share verbally any additional
open-ended observations or
feelings. Verbal responses were
recorded. See Appendix 1.

9
 ENC 6721

Findings &
Recommendations
Detailed findings by task, as well as our recommendations for actionable steps (if
applicable), are explored in this section. Please see the Appendices section for
more information about our findings.

Number of Clicks Needed

According to uxdworld.com, "The 3-Click Rule states

that users should be able to perform a task in 3
clicks[.]" An evaluator familiar with the DOE site
tested each task for the least amount of clicks
possible to arrive at the intended result.

Finding 1
Task 1:
All tested information on the UNF DOE
As intended: 1 click
website is accessible within 3 clicks.
With COAS nav bar: 2 clicks
Task 2: 1 click
Task 3: 3 clicks Recommendation 1
Task 4: 1 click Ensure that all information, such as how
to apply for the English Graduate
Program, also adhere to the 3-Click Rule.
10
ENC 6721

Finding 2
Users were confused by the College
of Arts and Sciences navigation bar.
Upon being asked to complete the
first task, a majority of participants
started with the prominent
navigation bar at the top of the
page. That bar is not specific to the
Department of English; rather, it
displays information for the entire
College of Arts and Sciences (COAS).

Instead of using the “Faculty &

Administration” tab on the bottom
left for Task 1, which asked
participants to find the phone
number or email address of English
professor Dr. Kailan Sindelar,
participants instead went for the
“Faculty and Staff” drop-down
menu along the top. Fig. 1. The drop-down menu shown when a
user engages with the COAS navigation bar.
While that link does allow the
participants to complete the task,
two problems arise: one more click
is required to find the actual list of
faculty (confusingly labeled "Faculty
Bios," though no bios are actually
contained on the page it returns),
and the page is populated with the
contact information of every “I didn’t realize there
professor in the COAS. This was were this many
surprising to users — they found professors in the
much more information than they English department.”
expected, and a lot more scrolling
was required to find the information
they were seeking.
“I think the bio
thing is really
helpful.”

11
ENC 6721

Findings & Recommendations

Fig. 2. We recommend the main navigation bar for the

English Department be moved to a more useful position.

Recommendation 2
Move the department’s navigation bar to a more prominent position.

When users navigate to the Department of English site, the first thing they see is the
most prominent navigation bar across the top of the page. However, this bar is not
specific to the DOE but rather contains links that help navigate the sections of the
COAS, and thus when clicked they take users away from the DOE site. Our findings
indicate that visitors to the site often attempt to use that bar to navigate the DOE site.
This issue can be frustrating for users who are looking for specific information.

Currently, the navigation bar for the DOE is located farther down the page, and
depending on screen size and orientation, may not even be visible when users first
arrive. We recommend that the main navigation bar be moved to the top of the page,
where the COAS navigation currently resides. This change would allow users to
navigate the site more easily.

More user testing is needed to confirm a more useful location for the COAS links. 12
ENC 6721

Findings &
Recommendations
Task 3
Trying to get involved with other
graduate students is difficult.

Trying to find the English Graduate

Organization (EGO) information was
difficult for all three participants. It
would be in everyone's best interest to
make it easier to find the information
for graduate students. Since graduate
students were our main focus, trying
to work with issues they each had an
common.

Recommendation
Put the EGO info in the main
navigation area.

EGO was difficult to find, and then

when you do find it, it is not updated
or included in the website. Most
newly admitted graduate students
did not know EGO existed until they
receive an email saying they had
been invited to join. Allowing the
website to add EGO into it would
benefit graduate students. Adding
the information would be beneficial
so other students can get involved.
(Continued on next page.)

Fig. 3 (right). Information about the EGO is

not listed in the side navigation bar. It is
located at the very bottom of the English
Graduate Program page, in the Programs
and Organizations tabbed box. One must
find it and select it to learn about the
organization.

13
ENC 6721

Findings &
Recommendations
Recommendation 3 (continued)
We also recommend an updated navigation chart, including more
tabs or a different color font to make the words stand out, on the side
of the website; its very bland and doesn't offer anything to catch the
eye if someone were looking for something for specific.

Fig. 4. A closer view of the

EGO information available
on the DOE website,
including the information
none of our participants
were able to locate during
the test.

Fig. 5. We recommend to
add a direct EGO link to
the main sidebar
navigation. We believe
making the EGO more
prominent on the site will
increase engagement
with the organization.

14
Conclusion

Conclusion

The UNF DOE website has some improvements to make, the navigation bar for example,
but it has several strengths. Pages within the site adhere quite well to the three-click
rule, allowing users to find most information easily. The overall organization of the site
works well, the main sections are labeled and important sections are found within three
clicks.

Our recommendations are based upon three one-on-one usability tests and our
evaluation of the data gathered. We believe the recommendations we make within this
report, if taken up by the university leadership, will improve current and prospective
graduate students' engagement levels with the site, the English Department and the
English Graduate Organization.

For the overall goal, the study found that the site adheres to the three-click rule from the
first participant. The second finding showed the confusion with the navigation bar, and
how moving it could be more benefitial to help students looking for what they need. The
third recommendation, is to help make graduate clubs easier to find so graduate
students can become more involved.

15
ENC 6721

Appendices

Appendix 1 - Subjective Evaluation Survey Data

A subjective evaluation survey regarding ease of use and satisfaction was collected via a
questionnaire at the conclusion of the session. The questionnaires utilized Likert rating
scales and yielded the results found in table 2.

An answer of "1" means the participant completely disagreed with the statement. "5"
indicates complete agreement. An answer of "3" is a neutral/medium response.

Statement Participant 1 Participant 2 Participant 3 Average

I thought the website was

4 3 4 3.66
easy to use.

I think I would need the

support of a technical
1 1 1 1
person to be able to use this
system.

I found the tasks easy to

3 3 3 3
complete.

I would imagine that most

people would learn to use 3 5 4 4
this system very quickly.

Table 2. Evaluation survey feedback with calculated average ratings.

16
ENC 6721

Appendices
Appendix 2 - Time on Task

The time to complete each task, not including subjective evaluation survey time, was
recorded. No participant was able to complete Task 3: "Find out when the first English
Graduate Organization (EGO) conference was." We have included the time each
participant attempted to complete it before giving up (critical error).

Fig. 6. Task times organized by seconds and task. Viewed this way, it's clear
each participant had trouble with Task 3, while Tasks 1, 2 and 4 were relatively
simple for them to complete.

17
ENC 6721

References
Goodman, E., Kuniavsky, M., & Moed, A. (2012). Observing the User Experience.
Morgan Kaufmann.

University of North Florida. (n.d.). Department of English. UNF Department of

English. Retrieved April 17, 2023, from
https://www.unf.edu/coas/english/index.html

UX World. (2021, May 28). 3-click rule and usability - UX design world. UX Design
World - Stories about UX, usability and product design. Retrieved April 16, 2023,
from https://uxdworld.com/2020/01/28/3-click-rule-and-usability/

18