education

sciences
Article
Integrated Outreach: Increasing Engagement in
Computer Science and Cybersecurity
Shaya Wolf 1 , Andrea Carneal Burrows 2 , Mike Borowczak 1, * , Mason Johnson 1 ,
Rafer Cooley 1 and Kyle Mogenson 2
1 Computer Science Department, University of Wyoming, Laramie, WY 82071, USA; [email protected] (S.W.);
[email protected] (M.J.); [email protected] (R.C.)
2 School of Teacher Education, University of Wyoming, Laramie, WY 82071, USA;
[email protected] (A.C.B.); [email protected] (K.M.)
* Correspondence: [email protected]




Received: 26 October 2020; Accepted: 18 November 2020; Published: 26 November 2020


Abstract: Research on innovative, integrated outreach programs guided three separate week-long
outreach camps held across two summers (2018 and 2019). These camps introduced computer science
through real-world applications and hands-on activities, each dealing with cybersecurity principles.
The camps utilized low-cost hardware and free software to provide a total of 84 students (aged
10 to 18 years) a unique learning experience. Based on feedback from the 2018 camp, a new pre/post
survey was developed to assess changes in participant knowledge and interest. Student participants
in the 2019 iteration showed drastic changes in their cybersecurity content recall (33% pre vs. 96%
post), cybersecurity concept identification within real-world scenarios, and exhibited an increased
ability to recognize potential cybersecurity threats in their every-day lives (22% pre vs. 69% post).
Finally, students’ self-reported interest-level before and after the camp show a positive increase across
all student participants, with the number of students who where highly interested in cybersecurity
more than doubling from 31% pre-camp to 65% post-camp. Implications for educators are large as
these activities and experiences can be interwoven into traditional schooling as well as less formal
camps as pure computer science or through integrated STEM.

Keywords: computer science; cybersecurity; outreach; effective practices; innovative activities;

K-12 students; K-12 teachers; summer camp; informal learning; integrated STEM

1. Introduction and Context

Computer science topics have spread into many science, technology, engineering, and mathematics
(STEM) fields as related skills have become a prerequisite for solving technical problems. In fact, by 2029,
employment in computer and information technology occupations is projected to grow 11% while
creating over half a million new jobs [1]. This has prompted a growing demand for computer science
literacy, creating a gap between current education offerings and industry/labor demands. While many
educators equate coding and computer science, and while coding is fundamental to many computer
science courses, computer science also involves computational thinking, algorithm and program
design, and problem-solving processes [2]. Simply stated, computer science and programming are
not synonymous [3]. Many current courses focus solely on programming instead of the broader and
more transferable concepts and content found within computer science and computation thinking spaces.
In fact, there is little agreement of what belongs in pre-collegiate computer science curricula and what
does not [4], making curricula generation difficult for even experienced educators.
Furthermore, rapidly advancing technology has introduced numerous cybersecurity vulnerabilities,
underscoring a need for cybersecurity professionals and, by extension, substantial cybersecurity

Educ. Sci. 2020, 10, 353; doi:10.3390/educsci10120353 www.mdpi.com/journal/education

Educ. Sci. 2020, 10, 353 2 of 23

education, a broad topic well situated within the blossoming arena of computer science. Many schools
struggle, for a variety of reasons, to offer quality education in these subjects [5]. Outreach opportunities,
after school or in the summer, may help to support traditional education efforts, while providing unique
experiences for students of all ages. This research study specifically focused on how a cybersecurity
outreach program, implemented as three free “open to all” week-long summer camps, impacted
students’ (aged 10–18 years) content learning and interest in cybersecurity.
In addition to a lack of basic computer science and cybersecurity content knowledge, soft skills which
include effective communication, teamwork, and conflict resolution are important for students as future
STEM professionals [6,7]. Information and technology advances have changed how people communicate,
socialize, and co-exist [8,9] and soft-skills are as desirable in working STEM professionals as core technical
skills [10]. Currently, many college graduates lack leadership, organizational, communication, and critical
thinking skills [11]; however, prior work has shown that, with explicit instruction and guidance, students
can understand and apply soft skills [6]. There is an opportunity for university students to engage in
outreach events to expand their opportunities in professional skill sets of communication, leadership
through mentoring, and gaining confidence in their field of study [12–14]. There should be a focus on
these skill sets in STEM professions such as flexibility, multidisciplinary problem solving, teamwork,
conflict resolution, and communication [9].
With all of these considerations, educators have different ideas of what STEM education should
include [15]. Computer science can (and the authors argue should be) integrated into the STEM
fields and courses to prepare students in developing solutions to today’s interdisciplinary problems
(e.g., tracking infectious diseases, modeling animal behavior, investigating water quality, developing
new materials). The STEM disciplines can intertwine with each other, strengthening each other as
one large field rather than four distinct fields by interacting with real-world, authentic problems and
by demonstrating the connections of ideas and skills in all of the disciplines [16,17]. Although not
the focus of this work, utilizing an integrated STEM framework [16,18] could offer insight into using
real world applications, building skill sets, and engaging collaborative teams. Computer science
offers a space for these skills focused on real-world problems as well as intelligent solutions to
issues facing the STEM community. Previous research has shown the importance of integrating
computer science into STEM [17] and that computer science concepts can be taught embedded within
traditional STEM lessons in pre-collegiate settings [3]. Students should be able to recall/apply what
they have learned to solve real-world problems [19] and utilize problem solving exercises while
synthesizing and developing core concepts [20]. These types of skills are enhanced with hands-on
student-centered learning that increases conceptual understanding rather than educator-centered
lectures which focus on recall [21]. Female students in particular benefit from student-centered
learning and hands-on activities [22–25]. Often, student experiences with integrated STEM subject
matter promotes engagement [16]. The outreach camps created, implemented, and studied by authors
Burrows and Borowczak in 2018 and 2019 build on this research, capitalizing on computer science
integration, teaching students to solve problems in creative ways, and showcasing STEM as one
interdisciplinary field where problems span multiple disciplines.
The three outreach camps provided week-long computer science exposure through educator
activities focused on effective teaching methods, and student activities full of hands-on labs and
real-world applications. This type of outreach experience provides computer science exposure to
students who may not have the opportunity to learn effective computer science skills otherwise.
Availability of these types of outreach activities gives students the opportunity to start building skills
that can be applied daily in evolving science and engineering fields. For example, data scientists are
growing in demand as Industry 4.0 [26] advancements require large amounts of data to be analyzed.
Additionally, cybersecurity landscapes have evolved to include more complex adversaries which
necessitates cybersecurity specialists for system protection. The summer camps described in this article
introduced pre-collegiate students, age 10–18, to necessary problem-solving skills required across
various fields, and sparked increased interest in continuing with a cybersecurity career trajectory.
Educ. Sci. 2020, 10, 353 3 of 23

2. Literature Review
Following a review of the current literature regarding STEM outreach, STEM education,
and computer science/cybersecurity teaching methods, three themes emerged that are discussed
in more detail after they are initially presented:
• The Need for Outreach Programs and Open-Source Activities: Computer science literacy is
lacking or lagging behind and many educators are struggling to provide quality computer science
education, making outreach programs and quality open-source activities crucial for meeting the
growing demand for computer science professionals.
• Outreach Experiences Support Continued Learning: Outreach programs support continued
learning through instilling confidence and motivation, employing hands-on and student-centered
activities, and promoting technical skills desirable by future employers.
• Sensible Learning Experiences for Diverse Students: Constructive outreach promotes sensible,
novel, and cost-effective activities that fit the needs of many diverse students and promote
their engagement.

2.1. The Need for Outreach Programs and Open-Source Activities

Regardless of the pressing need for computer science literacy and integrated STEM education
as described earlier, countries around the world are struggling to provide adequate opportunities for
students to master crucial skills in these areas. For example, a study of Australian educators found
that only 37% of educators felt they could promote a career in computer science and only 26% of
educators reported that they were able to host industry visitors to promote computer science [27].
While experienced STEM university faculty could offer assistance and create broader outreach efforts
for pre-collegiate students, these faculty activities are often regarded as an inferior mission, subsidiary
to university teaching and research obligations, and thus they are often not explored in depth [28].
In the United States (U.S.), the same trends are true, as computer science literacy is lacking and
outreach activities/programs which increase motivation could offer a potential solution until school
infrastructure has been revisited [9]. Increasing student interest and motivation is critical for fulfilling
the growing demand for computer science professionals. In 2016, the U.S. Bureau of Labor Statistics
predicted that one of every two STEM jobs would require computer science and three of four require
some computer science skills [29]. Only a year later in 2017, computer occupations already made up
45% of STEM jobs [30], nearly reaching the previous five year prediction in a single year. By 2018,
the U.S. Bureau of Labor statistics found that computer and mathematical occupations accounted for six
of the top 30 fastest growing occupations and, of those six occupations, information security analysts
was projected to grow the fastest by 2028 [31]. A report in 2019 found that computer and information
scientists are specifically projected to see a 16% increase in employment by 2028, which is significantly
higher than the average growth rate across all occupations of 5% [32]. Furthermore, although interest
in a few STEM fields has grown, interest in computing and related fields has declined [33], regardless
of the growing industry demand. Thus, at least a temporary intervention is required to promote
computer science with all students, as outreach opportunities for students must be provided to combat
the decreased interest in STEM [4]. Reassuringly, this focus has spurred outreach activities in the U.S.
as well as other countries [34], and more are needed.
Educators are asked to do an extraordinary number of tasks, and learning a content areas outside
of their realm of expertise is often not possible. However, just searching the internet for help is not
enough. Research shows that, when educators with limited computer science content knowledge
search for resources, their search phrases are too generic and then yield unproductive results [2].
As an extension, students do not get that additional support at home either, as parents searching for
computer science opportunities for their children are generally unsuccessful [35].

2.2. Outreach Experiences Support Continued Learning

Computer science education enables students to solve problems creatively, recognize opportunities
for technical innovations, and forge career paths in numerous different fields [29]. Outreach activities
Educ. Sci. 2020, 10, 353 4 of 23

make this continued involvement possible. For example, professional development focused on simple
computer programming approaches can propel pre-collegiate educators to integrate new, challenging
computer-controlled robotics into their instruction [36]. Importantly, all educators can access lessons with
computer science or cybersecurity components (e.g., TeachEngineering.org, csunplugged.org) and find
professional development and workshop opportunities (e.g., TeachEngineering.org) [37], but finding the
time to invest in learning these concepts and activities is difficult.
Outreach activities support continued student achievement through increased motivation [24],
especially since many students who participate in outreach activities do not have much background
in STEM fields. In fact, low motivation has been linked to low achievement in individuals with
non-science backgrounds attempting to learn science topics [24,38]. There is a universal need to feel
connected to others and a desire to experience integration and freedom [39], particularly among young
women [40]. This implies that students are motivated by a sense of belonging to a community as
well as a sense of agency over their learning. Outreach programs, including summer camps, are one
potential avenue to achieve this level of connection. Keeping students motivated requires building their
confidence and providing them with a more realistic baseline when assessing their own self-efficacy [41].
Outreach programs can foster achievement, confidence, and self-efficacy, through the creation of
inspiring learning environments that demonstrate the need for computer science and encourage
continuous learning [42]. Furthermore, establishing long-term student interest requires support from
educators [43,44], and educators should be involved in maintaining the learning environment that
houses outreach events that spark motivation [34].
Student commitment to learning increases when they have a sense of respect from instructors,
such as expressed admiration or using examples that students identify with [45]. This allows for
outreach staff to serve as role models for students [45,46] and helps students form practical perceptions
of future professions. Additionally, student commitment is increased as images of professionals
become more accurate and attainable [45,47]. Furthermore, outreach camps can provide student spaces
to break free of stereotypical images of STEM professionals that can broaden student involvement,
supporting diversity.

2.3. Sensible Learning Experiences for Diverse Students

Outreach activities should be sensible and reasonable. This means that the materials and
methods chosen should properly support learning. In STEM fields, particularly computer science
and cybersecurity, this means considering current trends in a quickly changing technology landscape
and creating novel approaches for teaching new concepts. Many STEM fields benefit from the use
of relatively expensive materials; regardless, cost-effective approaches should be taken to provide
outreach activities. This ensures that they are maintainable, repeatable, and extensible. In addition to
monetary restrictions, outreach availability is largely constrained by activities that are only offered
in a single location [48]. Some outreach opportunities are established through university programs,
and they are often limited to the university’s city. While outreach activities are vital to supporting local
interest in STEM, they do little to reach many diverse students.
Outreach programs should also create conducive learning environments for diverse students [4].
Simply exposing students to STEM topics does not guarantee an impact on the diversity of STEM
fields [49]. Therefore, students’ interests and motivating factors must be considered to provide
conducive learning environments. For example, activities should be suitable for a wide age range,
maintaining flexibility while ensuring core concepts are conveyed. By gathering students across
wide age ranges, outreach opportunities provide unique experiences for students who are normally
surrounded by others close to their age. Furthermore, studies have shown that students develop
attitudes towards STEM professions early in their lives. Young girls start making decisions about
future careers before entering even middle school (around age 11) [20,50,51]. Always being surrounded
by similar classmates limits the opportunity for students with different backgrounds to learn from
each other. For example, older students are robbed of the benefits of leading younger students and,
Educ. Sci. 2020, 10, 353 5 of 23

more broadly, establishing early leadership roles. Therefore, outreach opportunities have implications
in supporting diversity in STEM fields. In particular, women are underrepresented in STEM fields and
care should be taken to research gender differences in learning patterns [49] before making important
decisions about the activities included in an outreach program. For example, research shows that
hands-on activities are particularly effective for female students [24].

2.4. Research Questions and Instruments

The identification of the lack of pre-collegiate computer science instruction, along with possible
outreach solutions, led the authors of this article to address current gaps in understanding pre-collegiate
students’ interest, awareness, and knowledge of cybersecurity. The researchers investigated three
research questions during the week long summer camps:
1. How does a week-long summer outreach program impact student interest in cybersecurity?
2. How do summer outreach activities influence student security awareness and privacy practices?
3. How do hands-on activities and student-centered labs impact student content knowledge after
voluntary cybersecurity summer camps?

3. Methods and Context

During the summers of 2018 and 2019, three free summer camps were offered for pre-collegiate
students in the Rocky Mountain region of the U.S. These camps provided an introduction to computer
science with a particular focus on cybersecurity. Topics were presented to student participants through
real-world applications, cross-disciplinary approaches, and hands-on activities. All of the camps were
led by by experts in computer science, cybersecurity, and education as well as collegiate camp staff.
These camp staff members were undergraduates majoring in computer science and graduate students
with expertise in computer science and cybersecurity.

3.1. Methods
The authors of this article utilized field notes (qualitative methodology) and surveys (quantitative
methodology) for the three summer camps. After the first two summer camps in 2018 [52], the authors
used the field notes to evaluate the strengths and weaknesses of each and improvements were made for
the third summer camp in 2019. One of the most relevant assessment changes for 2019 was the creation
of a new 10 question pre/post survey (see Appendix A.1 with nine content questions and one student
interest question) which the team utilized in conjunction with demographic survey data to answer
the research questions presented earlier. Furthermore, the 10 question survey instrument captured a
high-level assessment of students’ cybersecurity interests, practices, and knowledge. Beyond gauging
baseline interest in the field, the survey asked students to identify important cybersecurity concepts,
addressed security and privacy practices through real-world scenarios, and tested for specific content
knowledge, all in a series of multi-select and ordering questions. Thus, in 2019, two student surveys
(demographic and 10 question content/interest) and field notes (taken by the article’s authors) were the
main data collection methods.
The demographic survey was completed by the middle and high school students’ parents during
the initial registration process, while the 10 question survey was completed by the student participants
once before the week-long camp and again during the final hour of the week-long camp. On the
reverse side of the 2019 pre-camp survey (before camp activities began), the students wrote a question
about computer science or cybersecurity. Throughout the camp, instructors touched on each of these
questions/topics through conversations, real-world examples, and hands-on activities. At the end of
the camp, the students were given the 2019 post-camp survey with the same content questions. On the
back of the post-camp survey (end of the week), the students rated the outreach week experience from
one to five (five being the best) and they wrote a comment explaining why they gave the camp that
rating. There were a total of 24 paired surveys in which the students completed both the pre-camp
survey and the post-camp survey.
Educ. Sci. 2020, 10, 353 6 of 23

3.2. Demographics and Context

The 2018 and 2019 summer camp demographics (n = 84), summarized in Tables 1 and 2, show an
average female to male gender breakdown of 27% to 73% and an average age of about 13 years
old, with 49% of the students in grades 7 and 8 (ages 12–14). While the advertised age range
for the camps was set at 13–18, several younger students (n = 7) ultimately participated as well.
In addition, though the percentage of females participation in the camps does not reach parity with
male participation, the percentage of female participation is 10% higher than the average U.S. computer
science program.

Table 1. Gender breakdown of student camp participants.

# Participants # Females # Males

Camp 1 23 5 18
Camp 2 31 13 18
Camp 3 30 5 25
Total 84 23 61

Table 2. Grade and age breakdown of student camp participants.

U.S. Grade Level 5th 6th 7th 8th 9th 10th 11th 12th
Age Range 10–11 11–12 12–13 13–14 14–15 15–16 16–17 17–18
Camp 1 0 2 4 3 3 2 7 2
Camp 2 1 4 11 4 4 1 3 3
Camp 3 0 0 10 9 5 4 1 1
Total 1 6 25 16 12 7 11 6

These camps utilized a set of best-known practices, grounded in literature, and activities for informal
education tailored to participant demographics. These practices included the use of a structured and
predictable schedule which featured hands-on unplugged activities in the morning and engaged inquiry
and computer-based programming lab activities in the afternoon (see Appendix B). The unplugged
activities served to introduce basic computer science and cybersecurity concepts which scaffolded student
knowledge in order for them to be successful during the hands-on programming of Micro:Bit devices.
There were multiple opportunities for collaboration daily and throughout the week. For example,
during the programming labs, student participants were grouped into smaller staff led/supported
problem solving teams. As students worked towards solving real-world cybersecurity problems
(through the application of computer science concepts in the programming of Micro:Bits), the camp staff
supported the student teams in their problem solving activities by asking questions, providing advice,
offering potential solutions, and sharing insight into potential security vulnerabilities of solutions.

4. Results
The results from the three camps were categorized into themes from field note observations and
survey results. The first result set explains the field note observations that were made during the camps,
how each successive camp was improved based on these findings, and how those improvements
affected future camps. The second result set details the effectiveness of the camps based on the survey
(given before the 2019 camp and again after the week-long 2019 camp).

4.1. Qualitative Results: Field Note Observations and Improvements by Theme for 2018 Summer Camps

4.1.1. Theme 1: Adjusting for Initial Learning Curves

Many of the students attending the 2018 summer camps had never been exposed to programming
or computer science basics. This created a bit of a learning curve during the first few activities since
Educ. Sci. 2020, 10, 353 7 of 23

the students were introduced to new hardware, new software, programming, and computer science
topics all at once. While some students could liken the Micro:Bit to a Raspberry Pi or Arduino,
other students were not familiar with similar technology. For students without this prior exposure,
the bridge between programming and actually implementing code on hardware was a completely new
concept. Furthermore, since programming benefits from logical yet creative thinking, some students
struggled with the initial code. When creativity outweighed logical reasoning, the student had many
ideas on how to solve certain real-world problems (e.g., how to advance their robot), but struggled
with implementing those solutions programmatically. When logical reasoning outweighed creativity,
students were able to create new functionality for their devices, but many of these upgrades were not
particularly useful in solving the given problems.
While the introductory activities in the 2018 summer camps taught students how to use the
programming environment, it did little to sharpen their problem solving skills. Therefore, introductory
Micro:Bit 101 activities were introduced in the summer 2019 camp to address the issues regarding
baseline programming knowledge. Although the Micro:Bit website has online tutorials that work with
students to create interesting projects, the Micro:Bit 101 activities designed by the researchers were
more beneficial to the students than the online tutorials since they were built towards specific learning
goals (called level-up guides). Specifically, the tutorials showed the students how to program the
devices (e.g., showing which blocks go where in block programming). This approach gives a quick
answer and can appease students, but stifles creativity, does not help the students build problem solving
skills, and robs them of the opportunity to discuss their solution differences with others. Furthermore,
while the tutorials helped students achieve functionality on their Micro:Bits, it did not help students
understand the logistics of a given problem, internalize a solution, and synthesize that solution into a
program. Since all of the answers were functionally the same, students were unable to use soft skills
and communicate to others what made their code unique. Conversely, by providing students with
the hand crafted Micro:Bit 101 exercises in the summer 2019 camp, students were able to build their
problem solving skills (while still learning the programming environment and Micro:Bit basics).

4.1.2. Theme 2: Maintaining Attention and Engagement

The unplugged activities in the 2018 camps had less engagement than the programming activities.
To raise this engagement in 2019, methods were used such as likening activities to real-world
applications and keeping the students actively engaged. The students were grouped into two sections
based on their self-reported experience level (either novice/intermediate or intermediate/advanced).
Each group was instructed simultaneously by a different instructor in a separate room, and then the
groups switched rooms. This raised student attentiveness by requiring students to focus for shorter
periods of time, provided less competition for open discussions, and allowed the outreach staff to
interact with them in more specialized conversations.
Furthermore, although the basic curriculum remained the same for the unplugged activities,
it became more interactive in 2019 to stimulate engagement. For example, in 2018 camps, the “telephone
game” was played in order to demonstrate how a message could degrade as it was passed from one
person to the next, showing a lack of integrity (a principle of computer science information theory).
This game was well received as many students had played the game before, and it was a familiar
concept to them (pass the message from the front of the line to the back of the line). This introduced
an important cybersecurity concept: What if a message is deliberately changed? While opening the
door to conversations about a “man-in-the-middle” attack, the conversation was cut short. In 2019,
this activity was lengthened and given more focus. Additionally, a lesson on Wi-Fi was developed to
further solidify the concepts and relate them back to students’ daily lives.

4.1.3. Theme 3: Providing Explicit Real-World Applications

The lessons and activities were all planned around common cybersecurity concepts and computer
science challenges facing professionals today, with specific care taken towards relating the topics
to student experiences. However, in the 2018 labs, while there were strong parallels between
Educ. Sci. 2020, 10, 353 8 of 23

topics and students, the burden for drawing those parallels rested in the hands of the students.
For example, one activity involved beacons that sent messages over radio waves and receivers that read
those messages, mimicking current beacon technology (used largely for advertising and marketing).
While this parallel existed, students were not able to draw this connection on their own. Without
properly forming these connections to the real-world, the activity lost quality as students struggled to
find ways to make sense of why these beacons degrade security. To solve this issue, in the 2019 labs,
camp staff made specific connections to the students’ previous knowledge. While the students were
all familiar with robotics in general, they were unaware of many of the security concerns associated
with this technology. Simply explaining the security vulnerabilities was not sufficient in grounding
their knowledge in real-world applications. To fully relate robotics security, the staff explicitly drew
comparisons to the real-world. Instead of likening the activity to robotics in general, the staff gave
explanations on precisely how they are similar. For instance, the students quickly learned that it was
not particularly difficult to control their friend’s robot. This demonstrated a common attack vector
in unmanned aerial vehicles (UAV) technology or self-driving cars. However, rather than trusting
the students to draw these comparisons, specifics were given that directly mapped the activity to
these types of issues. Questions were posed such as “How does this relate to self-driving cars?” and
“What would happen if this was an actual car and not just a robot?”
Being explicit about the real-world implications of the labs solidified student learning as well as gave
students a place to connect the lesson to their life. In other words, clearly explaining connections between
lessons and the real-world gave students familiar topics with which to associate the new knowledge.
Lastly, consider password strengths. Many people today do not understand why their password should
be long and contain different types of characters. To these people, a five character password made up
of lowercase letters should be hard to crack since there are 265 = 11,881,376 total possible passwords
and 11,881,376 is a big number. However, 11,881,376 is actually quite a small number considering how
fast a computer can generate and test the passwords. Accordingly, when the lessons were grounded in
real-world applications (like password strengths), not only was the concept clearly linked to students’
existing knowledge, but the proper scaffolding was included to ensure retention.

4.1.4. Theme 4: Enhancing Incentives and Motivation

During all three camps, the students were able to track their progress using their individual
badge, and the badges were fit with a ring of light emitting diodes (LEDs) that demonstrated their
achievements. This gave the students incentive to work through the labs and stretch to achieve goals
that might have seemed out of reach. The students were not required to complete the most difficult
levels of the labs or create the best code; instead, they were motivated to solve problems to the best of
their ability.
While the badges provided a low-level of motivation in the 2018 camps, the potential incentive was
stifled by the inconsistency of the reward. In 2018, the camp staff leveled-up the badges (or increased
the score) when they felt the students had demonstrated mastery over a new concept. In 2019,
this system was standardized such that activities had specific checkpoints (or level-up guides) that,
when reached, granted a student to level-up their badge. This created a direct expectation for levels and
better motivated the students by giving them clear goals tied to specific incentives. Furthermore, in the
2018 labs, the students worked to level-up their badges throughout the week (no daily differentiation).
However, in 2019, each day marked a new opportunity to level-up their badge. At the end of each
day, the students’ level was reset to zero meaning that each day they had the opportunity for success
regardless of what happened previously in the camp. This better motivated students because they
were not as discouraged by earlier mistakes.

4.2. Quantitative Results: Descriptive Statistics for the 2019 Summer Camp
The three research questions presented earlier form the basis for the 10 question pre/post survey
(see Appendix A.1) that was administered to all the students. For the purpose of this study, the survey
Educ. Sci. 2020, 10, 353 9 of 23

is segmented into four distinct areas of “Confidentiality, Integrity and Availability” recall (Q1) and
applications (Q2–4), followed by online security (Q9), and finally camp interest (Q10).

4.2.1. Q1: CIA Recall

Student participants gained knowledge and confidence in being able to correctly identify the
components of the CIA triad—namely confidentiality, integrity, and availability. The CIA triad is
integral to cybersecurity and drives many of the decisions made in the field. The CIA triad was
the first concept taught to the students and each lesson related back to these three core principles.
Likewise, the first question on the survey was based on recalling this most basic cybersecurity lesson
and asked: When talking about information security, what do the following letters represent, C, I,
and A? The question asked the students to “select all that apply”. When asked this question before
the camp, almost half of the students selected that C, I, and A stood for Central Intelligence Agency
and only eight students (33%) selected the correct answer (Confidentiality, Integrity, and Availability).
After the camp, however, the wrong answer was only selected three times and the correct answer was
selected 23 times (96%). These results can be seen graphically in Figure 1—with only two students
answering more than one answer (in both the pre-camp survey and the post-camp survey).

Figure 1. Student responses to the first question on the pre-camp and post-camp survey. The question
asked: When talking about information security, what do the following letters represent, C, I, and A?
The correct answer is Confidentiality, Integrity, and Availability. Before the camp, approximately 33% of the
students answered this question correctly; however, after the camp, approximately 96% answered correctly.

4.2.2. Q2–4: CIA Applications

Student participants identified real-world applications for each component of the CIA triad—with
drastic improvement in understanding availability and integrity. Additionally, students generally
understood applications of confidentiality and availability at higher rates than the concept of integrity.
Questions 2–4 of the survey focus on the application of the CIA concepts. During the camp, each of
these concepts were related to real-world issues. For example, confidentiality was linked to media
streaming platforms. Many people share their passwords to media streaming platforms. Average users
feel safe doing this because they trust the person with whom they are sharing the password. However,
many people also use similar passwords across all of their accounts. This means that they are trusting
that person to use that password appropriately, as well as trusting them to keep that password a secret at
the risk of multiple accounts being compromised. This underscores a significant breach in confidentiality
because, while people may share passwords with someone they trust, it becomes difficult to maintain
privacy. Students understood this example since they use passwords on a daily basis and are familiar
with media streaming platforms. To relate integrity to the real-world, the camp staff asked the students
Educ. Sci. 2020, 10, 353 10 of 23

if they had social media accounts and what they would think if they logged on one day to find their
account filled with random information. This demonstrated the importance of protecting data from being
changed by unwelcome parties. This was also solidified through a lab activity involving the Micro:bots.
The students were not told to pick a unique channel for their robot to receive control messages on.
This meant that the students were sending control commands to other students’ robots and the robots
acted erratically. The students learned quickly that ensuring valid data was integral to controlling their
robot, and in turn they learned how integrity impacts their day-to-day lives. To relate availability to
the students, camp staff asked the students if they played games online, a common pastime for most
students. Online games require a connection to a server. If the device (e.g., computer, gaming console)
cannot connect to the Internet or the server cannot handle all of the traffic, the game becomes unavailable.
Games that are continuously unavailable, or devices that consistently lack functionality, lose value over
time and the companies providing them are less likely to be successful. While students did not seem to
particularly care about the business aspect, they were able to identify that when services they want are
not available, they go somewhere else (e.g., they pick a different game).
To determine the effectiveness of relating the CIA triad to real-world examples, the surveys
included three questions that gave an example of a security failure, then asked the students to identify
the lapse in security. They were given five multiple choice options and were asked to “select all that
apply.” The questions were also shuffled such that the answers were not in order with the CIA acronym.
If the student selected the correct answer (even if they also selected other answers), the answer was
categorized as correct. If the student did not select the correct answer, their answer was classified as
incorrect. The results are shown in Figure 2.

Figure 2. Number of correct and incorrect answers for the CIA application questions. More students
answered each of these three questions correctly after the camp than before the camp. This demonstrated
that relating the CIA concepts to real-world scenarios helped the students identify basic security concepts
when given example problems.

The students demonstrated the greatest improvement in the availability question with 10 (42%)
answering correctly before the camp and 22 (92%) answering correctly after the camp. In other words,
the number of students who were able to identify availability as a security concern (when given a
real-world example) more than doubled during the camp. The number of students who answered
the integrity question correctly also increased dramatically. Although only 15 (63%) students selected
the correct answer after the camp, only three (13%) students answered correctly before the camp,
meaning five times as many students answered correctly. The difference in the pre-camp surveys
and post-camp surveys for availability and integrity both increased by 12 correct answers; however,
since so few had answered the integrity question before the camp, the students showed more class-wide
improvement on the integrity question, but a higher level of class-wide mastery on the availability
Educ. Sci. 2020, 10, 353 11 of 23

question. The question concerning confidentiality, a less foreign concept to the students, demonstrated
the least improvement between the pre-camp and post-camp surveys. Specifically, 18 (75%) of the
students answered correctly before the camp and 20 (83%) answered correctly after. Finally, notice
that more students answered the availability question correctly after the camp than the confidentiality
question, even though the confidentiality question had much better pre-camp results. This likely
means that the activities relating to availability were particularly relatable to the students.
While the first question indicated that students were able to recall the CIA triad and what each
letter stands for, the next three questions showed that the students were also able to apply those
concepts to real-world examples.

4.2.3. Q9: Online Security

Student participants recognized the implications of sharing information online, with a majority
successfully identifying the risks associated with certain seemly benign social media activities.
All students determined that sharing information has an impact. Other topics beyond the basic
CIA triad concepts were also considered. During one activity, the students were shown a common
“What _____ character are you?” meme. These memes assign popular characters to people based
on some identifying information. For example, it may determine what popular movie character
each person would be based on their birthday, the first letter of their middle name, and the name
of their first pet. The students reviewed the memes and, when asked what the point of the meme
was, most answered that the meme was supposed to be funny. However, these memes are usually
created using common security questions and other personal information. When someone posts their
character, they are posting the answers to their security questions and breaking confidentiality best
practices. During the pre-camp and post-camp surveys, students were asked: Online surveys and
“What _____ character are you?” memes are... (See Appendix A.1 for potential answers). Before the
camp, the most popular answer was that the memes were funny with 10 (42%) students selecting
that answer and seven (30%) students selecting each of the other answers. After the camp, the most
popular answer was the correct answer that the memes contain security questions, with 18 (75%)
students answering correctly. These results can be seen in Figure 3. This shows that the students
understood that the memes contained security questions, but also understood that they are not meant
to be funny (they are meant to discover personal information) and that the memes do have an impact
on unsuspecting people.

4.2.4. Q10: Cybersecurity Interest

Overall, student interest level increased by the end of the week long outreach camp, with all
students having a medium to high interest level and with the percentage of students highly interested
doubling. Recall that one of the main goals of the outreach activities was to increase interest in
computer science and cybersecurity, and the students were asked how interested they were in computer
science and cybersecurity both before and after the camp. Before the camp, four (17%) students said
that they were only minimally interested in cybersecurity and indicated that they were unsure of
cybersecurity. Eleven (46%) students indicated that they were moderately interested in cybersecurity
and seven (30%) students reported that they were highly interested in computer science/cybersecurity,
finding it fun and exciting. This showed that before the camps, the students were generally unsure of
what computer science and cybersecurity entails, but, after the camp, none of the students reported
being minimally interested in computer science and cybersecurity. The number of students highly
interested in cybersecurity increased to 15 (63%), more than doubling, and seven were moderately
(30%) interested. In total, 22 students (92%) were interested in cybersecurity. These results indicate that
the camps increased the students interest in cybersecurity and computer science and they are shown
in Figure 4.
Educ. Sci. 2020, 10, 353 12 of 23

Figure 3. Number of students selecting each answer for the information security question on the
pre-camp and post-camp surveys. In the pre-camp surveys, 7 of the 31 selections were the correct
answer (23%). In the post-camp surveys, 18 of the of the 26 selections were the correct answer (70%).

Figure 4. Student interest level in cybersecurity. Student interest increased over the course of the
camps, with the number of students highly interested in cybersecurity more than doubling. None of
the students reported that they were not interested at all, and the students that indicated that they were
only minimally interested in cybersecurity showed an increase in interest.

4.2.5. Camp Rating

During their post-camp survey, in addition to indicating their interest in cybersecurity and
computer science, the students were asked to rate their camp experience and write a comment
explaining their camp rating. None of the students ranked the camp as a “one” out of five or “two”
out of five, indicating that all of the students enjoyed the camp to some extent. Only three of the
students (13%) ranked the camp as a “three” out of five. Of these students, one student noted that they
would still recommend the camp to others. Six students (25%) rated the camp as a “four” out of five
and 15 students (63%) rated the camp as a “five” out of five. These results indicate that the outreach
week was engaging and the activities held the interest of the students. They also indicate that some
modifications could be made to improve camps in the future for some students. These results are also
shown in Figure 5 and the comments are listed in Table 3.
Educ. Sci. 2020, 10, 353 13 of 23

Table 3. Comments from the students along with the rating they gave the camp.

Rating Comment (Spelling Errors Corrected)

Longer lunch.
Too much hand drawings. Not enough programming antivirus or firewalls.
3
I don’t think that I would do another one of these camps but it was definitely an experience that I would recommend to others if they haven’t done something
like it already.
I enjoyed the camp but felt there was a lack of focus on firewalls. What they are and how they work, and viruses, how to avoid them, and how to remove them
before a complete wipe.
This camp was really interesting and fun and I really liked it but there were some boring parts like the hand, but the programming and hacking were fun.
4 It was fun.
This camp was very fun.
Because it was fun but some minor adjustments would be better.
It was fun hacking and programming.
The camp was interesting, the activities were mostly really fun. I also gained knowledge and skill that will help me in the real-world.
I chose this due to some lack of explanation on somethings ex ciphers, overall it was great.
Because I would want to come back to this camp if I had the opportunity to and it teaches you about online services and how to protect your accounts.
Because it was fun to hack other peoples microbots.
I had a lot of fun and met friends.
It was exciting and I would totally do this again and it boosted my interest in cybersecurity.
5 I had amazing leaders who helped me and encouraged me.
Because I now am more interested and knowledgeable than I was before I started this camp.
I want to do this camp again to gain more knowledge.
Fun and interesting.
very fun/interesting and would like to do it again.
It was a blast and I learned so much. I learned a lot but it did not feel like school and I enjoyed it. I liked programming the Micro:Bits and the robots.
Learning how to code was a ton of fun and it was very enjoyable. I would like to go to another camp like this because it had the best activities and was
a nice way to spend a week in the summer.
We learned important information about safety and security, explored coding and hacking, and were informed about potential employment.
I loved this camp, it was super fun and informative. It had great projects.
It was interesting to me because I like computers.
Educ. Sci. 2020, 10, 353 14 of 23

The comments provided a qualitative context in support of the quantitative assessment, and they
clarified which components of the camps were well received and which ones were not (from themes
discovered in the 2018 camps’ field notes).

Figure 5. Student ratings of the camp as a whole. Nearly two-thirds of the students rated the camp as a
five out of five and none of the students reported that the camps should be rated a one or two of five.

5. Discussion
The results, implications, and limitations are discussed in this section in relation to the three main
research questions. The authors highlight connections to current literature and propose future research
questions. A list of suggestions are also provided for those creating student outreach opportunities.

5.1. Research Question Discussion

Related work in this field demonstrated a need for outreach programs and open-source activities
in response to growing demands for computer science and cybersecurity professionals [32]. Outreach
programs can increase interest in these fields and motivate students to become not only good
programmers, but good computer scientists [9]. This central issue guided the first research question:
How does a week-long outreach program impact student interest in cybersecurity? The responses
from the 10 question interest survey revealed that the number of students who reported being highly
interested in cybersecurity more than doubled between the start and end of the camp. This indicates
that week-long outreach programs can greatly increase interest in cybersecurity. In addition, all of the
students who indicated that they were only minimally interested in cybersecurity at the start of the
camps reported being more interested by the end of the camp. This demonstrated that the camps did
not just make the already-interested students more interested, but also increased interest across the
group regardless of their ability level.
Recalling that educators should aim to teach more than just programming to their students when
teaching computer science [2], they should teach their students through real-world problems that apply
to their everyday lives [19]. This idea guided the investigation into the second research question: How
do summer outreach activities influence student awareness of their security and privacy practices?
One survey question asked the students about online memes that can compromise the privacy of
unsuspecting users. While most of the students did not realize the security implications of some of
their actions online before the camp, 18 of the 24 students recognized the given situation as a potential
breach of their confidentiality. This indicated that the camp activities focused on teaching students to
be mindful of their online actions were successful. This result also shows that outreach activities can
influence how students think about their regular behavior, conditioning them to be cognizant of the
information they share.
Lastly, it is well known that outreach programs should provide hands-on, student-centered
activities [19–24]. Students responded positively to the hands-on activities and laboratory projects
Educ. Sci. 2020, 10, 353 15 of 23

created for this outreach program. This gave students the opportunity to discover cybersecurity lessons
through exercises led by university camp staff as well as industry professionals. This engagement enabled
the team to address the third research question: How do hands-on activities and student-centered labs
impact content knowledge in voluntary summer camps? Data from the first four questions in the
survey confirm that students were able to recall important security information as well as apply basic
security concepts to the real-world. When asked about the CIA concepts after the camps, 23 of 24 students
were able to recall what the acronym stood for, 20 were able to identify a breach of confidentiality, 15 were
able to identify a breach in integrity, and 22 were able to identify a breach in availability. This indicates
that the hands-on activities and student-centered labs were successful in teaching the students basic
security concepts and, by extension, that hands-on activities and student-centered labs had a positive
impact on information retention in the voluntary summer camps.

5.2. Implications
Educators and those creating outreach experiences for students can glean insight from the authors’
lessons learned during the three, week-long summer camps. Recall that related work in the field also
showed that educators are currently struggling to find helpful resources for teaching computer science
and cybersecurity [2,35]. The camp staff focused on creating quality activities that used computer
science problem solving and programming to solve real-world problems. To support the growth
of usable activities, each of the Micro:Bit activities was made available to educators along with the
“level-up guides” at http://www.wycyberstars.com. This supports educators and parents looking
to provide students with cybersecurity lessons. Each of the activities has flexible time constraints to
fit both short classes as well as longer classes. In addition, since the outreach program welcomed
students across a wide age range, the activities were built to suit students with varying backgrounds.
The authors created level-up guides with activities (or milestones) that “grow incrementally” from
step to step. The authors believe that while novice/intermediate students learned more by going
through the first one or two level-up stages, the intermediate/advanced students were able to expand
their learning and make meaningful connections throughout the levels while sometimes engaging in
extensions beyond the level-up guide.
Other related works stressed the importance of student motivation [24,38] and strong leadership
from outreach staff [41,42,45,46]. The camps focused on motivating and engaging students through the
badge leveling system. In addition, the camp staff expressed genuine excitement for students when
their solutions worked and supported the students when their solutions did not work as students
learned through failure. The camp staff shared their experiences in computer science and cybersecurity
with the students, modeling that professions in STEM fields are attainable. These mentors came from
many different backgrounds including educational professionals, computer science and cybersecurity
educators, professors with industry experience, and collegiate camp staff. By providing a variety
of models (e.g., female doctoral student, Hispanic female masters student, African-American male
undergraduates student, female high school student), students were able to benefit from the mentors
in a variety of ways.
Related work also shows that outreach programs can promote the growth of soft skills such as
teamwork and communication [8,9,53]. In these outreach camps, students were encouraged to
communicate their thoughts and ideas to each other as they worked together in groups to create solutions
to open problems. The intermediate/advanced students were often paired up with novice/intermediate
students and given the opportunity to lead and mentor them. This created a chain of support for the
students. The intermediate/advanced student participants supported the novice/intermediate student
participants, while the high school, undergraduate, and graduate students supported all of the student
participants while the camp leaders supported the whole team. This promoted leadership across the
entire group as well as cooperation and teamwork.
Additional related works were concerned with the sensibility and effectiveness of outreach
activities [4,49]. Constantly improving lessons and upgrading laboratory activities helped maintain
Educ. Sci. 2020, 10, 353 16 of 23

the camps, keeping them up-to-date with current cybersecurity issues facing the field. The lessons
were based on these real-world issues and camp staff explicitly tied the lab back to these applications.
Related works were also concerned about the accessibility of outreach programs [48]. The low cost
of the lab materials supported the maintainability of these outreach activities as well as encouraged
the use of the materials by educators looking for cost-effective ways to include computer science and
cybersecurity in their classrooms. Furthermore, these camps were held in numerous rural locations to
provide outreach opportunities to a wide range of students.
Lastly, these camps aimed to align with other related works which highlighted the need to be
flexible for a diverse set of students [20,24,49–51]. The camps focused on creating lessons that could
be taught in multiple ways, making it more efficient for camp staff to pivot when students lost focus
or needed assistance in approaching the problem in a different way. In addition, because the 2019
camps provided space for smaller groups of students, camp staff were able to better tailor lessons and
methods to students learning needs. The considerations made in the pursuit of providing an effective
week-long summer outreach program were successful. This is demonstrated through field notes showing
high student engagement and in the high ratings the students gave the camp during their post-camp
survey (15 students giving the camps a “five” out of five rating and another six giving a “four” out of
five rating).

5.3. Limitations
Despite the improvements made to the outreach experience labs, they were limited by the amount
of time students had to work on some projects. To provide a relative structure to the day, regular
breaks were built in to accommodate the students’ attention spans. Some of the breaks came at
inopportune times throughout the day and disrupted a project’s flow. Some activities held student
attention much longer than anticipated, forcing hard decisions about how to structure the rest of the
day to provide enough time for each activity. This also inhibited deeper dives into subjects with some
students becoming restless (and some students becoming more engaged). In addition, these camps
may not be indicative of outreach opportunities and are thus not generalizable, and the scope of the
results is limited to week-long, free, voluntary, student summer camps. Many of the decisions made
during these particular outreach camps influenced the results since the camp was only a week long.
Furthermore, these results only give overall perceptions of the students’ content knowledge, interest
level, and cybersecurity awareness. While these perceptions give an impression of how the outreach
activities influenced the students, they do not prove that specific activities or outreach decisions were
responsible for the discussed findings.

6. Conclusions
This research study was situated at the crossroads of three challenges currently facing computer
science educators: (1) need for outreach programs and open-source activities as educators report
being under-prepared to provide quality computer science education to meet the growing demand for
computer science professionals; (2) need for outreach considerations that support continued learning
and include instilling confidence and motivation, employing hands-on and student-centered activities,
and promoting technical skills desirable by future employers; and (3) need for sensible learning
experiences for diverse students that can be promoted through novel, cost-effective outreach activities.
These camps created many open-source activities with staged levels of difficulty (level-up
guides) to provide a wide-range of educator resources for computer science classrooms. In addition,
these activities are integrated with other subject-matters, connecting different lessons through common
themes, although they are focused on cybersecurity. To support continued learning, high school
students and collegiate camp staff were employed to create working relationships and give student
participants the chance to work with others and engage in an array of complementary strengths.
Additionally, unplugged activities gave students a platform for computational thinking and practice
for formulating solutions beyond their current programming abilities. Finally, these outreach activities
 Educ. Sci. 2020, 10, 353 17 of 23

focused on novel approaches to real-world problems through cost-effective labs (i.e., Micro:bits) that
educators can easily integrate into their existing curriculum.
Observational field notes were made to promote improvements for the summer camps. Then,
the effectiveness of the outreach camps was assessed through the use of pre-camp and post-camp
surveys. Finally, the field note observations and results were discussed in the context of the
previous research, explicitly outlining the contribution to the field as well as implications and
limitations. The camp surveys showed that outreach programs can positively impact student interest
in cybersecurity, raising both awareness and curiosity for cybersecurity. The results indicated that
the number of students who reported being highly interested in cybersecurity more than doubled
from the start to the end of the camp. Furthermore, the results did not solely indicate that students
who were already interested in cybersecurity became more interested; instead, all of the students who
were only minimally interested in cybersecurity at the beginning of the camp reported being more
interested after the camp. In addition, students showed increased awareness for the implications
of their online behavior, recognizing the potential vulnerabilities associated with common online
activities. Enforcing cyber-safe practices (e.g., being cognizant of sharing personal information) equips
students to protect themselves in an advancing technological world. Lastly, these camp surveys
indicated that the hands-on activities and student-centered labs were successful in teaching basic
cybersecurity concepts. Specifically, students were able to identify breaches in confidentiality, integrity,
and availability posed in real-world scenarios. Holistically, this research concludes that outreach
programs centered around cybersecurity can have a positive impact on student perceptions through
the use of hands-on labs and student-centered activities, teaching them the importance of cybersecurity
and cultivating curiosity in exploring cybersecurity topics.
Based on the results of these three week-long camps, suggestions for educators include:

1. Set the stage with baseline information in short (less than 20 minutes) of the topic;
2. Connect the topic to the real-world and give explicit and varied examples;
3. Have students create a real-world question (from the problem) to investigate;
4. Allow students flexibility to change their questions as evidence is collected;
5. Provide opportunities for students to practice and fail in a safe space and continue changing their
question and collecting evidence;
6. Encourage students to use the data collected and analyze it in different ways;
7. Promote students’ ideas for improving designs, data collection, and analysis;
8. Provide a space for students to share outcomes, lessons learned, potential impacts on society,
career connections to the work, applications to other fields, and more;
9. Give students choices of topics and sample questions as well as non-examples if they get stuck;
10. Promote student engagement by allowing students to find a place of connection with the computer
science (or any other) work;
11. When possible and appropriate, foster student engagement in real-world scenarios through
connections with experts in the field (such as industry professionals;
12. Provide computer science activities that rely on computational thinking rather than only
programming and stress the importance of responsible code use (from shared libraries, code banks,
and widespread tool-kits);
13. Be explicit about the real-world implications of each activity to solidify learning and encourage
trying again or trying differently when student failure occurs.

The authors are planning future student outreach activities and educator professional development
opportunities for upcoming summers. Activities are extended and new activities will be introduced
based on the collected field notes and survey results. The future camps reach out to new rural areas
through in-person and virtual scenarios. In addition, new research studies could focus on current issues
facing STEM fields and cybersecurity professionals, as well as current methods being used to provide
outreach opportunities. Research into the industry might reveal new concepts and problems to pose to
students, while the research into other outreach offerings might drive enhancements on the outreach
Educ. Sci. 2020, 10, 353 18 of 23

logistics. Future research is needed concerning the current status of computer science, cybersecurity,
and their relation to integrated STEM education, sustaining interest in STEM fields, and employing
outreach programs like summer camps.

Author Contributions: Conceptualization, A.C.B. and M.B.; methodology, A.C.B. and M.B.; software, S.W., R.C.,
and M.J.; validation, A.C.B. and M.B.; materials creation M.J., S.W., and R.C.; activity leadership, A.C.B., M.B.,
M.J., S.W., R.C., and K.M.; formal analysis, S.W.; investigation, A.C.B., M.B., and S.W.; resources, M.B. and A.C.B.;
data curation, A.C.B. and M.B.; writing—original draft preparation, S.W.; writing—review and editing, S.W.,
M.B., A.C.B., R.C., M.J., and K.M.; supervision, A.C.B. and M.B.; project administration, M.B. and A.C.B.; funding
acquisition, M.B. and A.C.B. All authors have read and agreed to the published version of the manuscript.
Funding: This work was supported and funded by (1) The National Science Foundation (NSF) and National
Security Agency (NSA) GenCyber Award #H98230-18-1-0095 (called GenCyber:COWPOKES) and (2) The NSF
Noyce Grant No 1339853 (called SWARMS). Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF or the NSA.
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the
study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to
publish the results.

Appendix A. Instruments

Appendix A.1. GenCyber Wyoming Pre/Post Assessment Survey

Q1. When talking about information security, what do the following letters represent C, I, and A:

[] Command, Instruct, Affirm

[] Conspiracy of Illegal Activity
[] Central Intelligence Agency
[] Confidentiality, Integrity, Availability
[] Cooperation, Interrogation, Activation
[] None of the above
Q2. You’re about to play your favorite game but you get the message “host cannot connect to server...”
This is an example of poor:

[] Command
[] Cooperation
[] Instruction
[] Integrity
[] Availability
[] None of the above
Q3. You were finally able to connect to your favorite game, but when you start playing you realize
that your profile contains random information. This is an example of poor:

[] Integrity
[] Information
[] Instruction
[] Confidentiality
[] Cooperation
[] Command
[] None of the above
Q4. Your friend asks you for your Hulu/Netflix/Amazon Prime/Spotify password and you share it.
This is an example of poor:

[] Integrity
[] Cooperation
[] Activation
[] Confidentiality
[] Instruction
[] Availability
[] None of the above
Educ. Sci. 2020, 10, 353 19 of 23

Q5. When using the internet, it is possible to:

[] Do or say anything you want

[] Remain completely anonymous
[] Hurt others or their property
[] Transmit information secretly
[] None of the above
Q6. A castle is an example of:

[] Layering
[] Defense in Depth
[] Principal of Least Privilege
[] M-O-A-T
[] Technology over Strength
[] None of the above
Q7. Cybersecurity deals with:

[] Computers/Programming
[] People/Social Interactions
[] Money/Finance
[] Power/Greed
[] Politics/Laws & Regulations
[] None of the Above
Q8. Rank these defense strategies for creating secure products (games, computers, buildings, etc.)
from 1 to 5, (1 being the best, 5 being the worst):

[] Keep everything secret (obscurity)

[] Pretend nothing will happen (denial)
[] Build “Fort Knox” (only security matters)
[] Know something will happen (acceptance)
[] Design for Security (built in protection)
Q9. Online surveys and “what _____ character are you” memes:

[] Are funny
[] Contain security questions
[] Have no impact on you
[] Share your data with the government
Q10. Your interest in cybersecurity is:

[] Non-existent (my parents made me do this)

[] Minimal (I don’t know what this is about)
[] Medium (It could be interesting)
[] High (This is interesting/fun/exciting)

Appendix B. Camp Materials

Appendix B.1. Micro:Bit Labs

The outreach activities utilized various different types of hardware. The main staple of the hardware
collection was the BBC Micro:Bit. The Micro:Bit is a small computer, much like a Raspberry Pi or
Arduino, which costs less than twenty dollars. The low-cost (yet high functionality) of the Micro:Bit
meant that each student could take home a Micro:Bit after the camp and continue learning through the
extensive tutorials and functionality of the boards. They are equipped with radio/Bluetooth, two buttons,
a compass, an accelerometer, a temperature sensor, a light sensor, and 25 programmable LED lights.
The Micro:Bit can be programmed using block programming, JavaScript, or Python. This allowed
for flexibility in the labs and provided student participants with implementation options to suit their
interests. The rest of the hardware was built around the Micro:Bit and the student participants were
able to see every component of the construction. This transparency in hardware design demonstrated
Educ. Sci. 2020, 10, 353 20 of 23

to students that components work together and even complex systems can be broken down into easily
understood parts.
All of the programming took place using the online Micro:Bit web tools. This meant that there was
no need to install software and the activities could take place in any lab. This ensured that the lessons
could be applied in classrooms everywhere as well as students’ homes. The actual programming of
the boards was done by downloading the project directly from the editor then clicking and dragging
the file onto the board.
Like other conferences and camps, these outreach camps utilized an electronics name badge.
Each badge used a Micro:Bit and was built to be easily disassembled and re-assembled. They were
used to track learning progress through a ring of LED lights that demonstrated student achievements.
This incentivized students to complete each challenge and lab.

Appendix B.1.1. Micro:LockBox

Camp staff also created Micro:Bit-powered lock boxes that simulated smart locks. From the outside,
they looked like simple cardboard boxes with a slide that was either green to indicate open or red
to indicate closed. These locks showed that, even without knowledge of what was inside the box,
an adversary can gain access into a system or physical area. These stations had different difficulty
levels and gave students a firm and tangible goal to program towards. By giving different inputs to the
Micro:Bit, they could infer and learn for themselves what the lock was trying to do. After the activity
was completed, the students were shown the inside of the box and discussed ways to secure the system.

Appendix B.1.2. Micro:Bots

Student participants also programmed small robots, dubbed Micro:bots. The Micro:bots used
a Micro:Bit and an extension to the Micro:Bit (called a Moto:bit) to control two motors that drove
wheels on each side of the robot. The wheels could either turn forward or backwards at a given speed.
This allowed students to fine-tune their robots until they could be successfully piloted through a maze
(taped on the floor). They were also tasked with programming a controller for their robot using a
second Micro:Bit. Since they were required to send messages using radio communications, they were
[unknowingly] tasked with creating a security vulnerability for their robots, allowing camp saff to
hack their robots and cause them to act unexpectedly.

Appendix B.2. Unplugged Activities

Aside from the Micro:Bit activities, unplugged activities used materials that can be found in
most classrooms (paper, markers, scissors, and so on). Unplugged activities were designed to allow
students the opportunity to launch into computer science topics through computational thinking
without having to learn programming first [54]. For example, one of the unplugged activities involved
creating a Caesar cipher and then using it to encode secret messages to their neighbor. The staff printed
two wheels, one smaller than the other, each with the alphabet written around the edge. The students
then cut the wheels out and pinned them together, the smaller on top of the larger, so that the smaller
wheel could be rotated and the letters around the larger wheel would align with the encoded letter.
These unplugged activities were integral in teaching cybersecurity and computer science topics than
other courses simply use pre-built tools for. To expand, there are programming tools that allow
students to use a Caesar cipher without having to build the encoding method themselves; it simply
does the encoding for them. However, using this type of tool without demonstrating what is going on
behind the scenes removes the opportunity to understand how their code works. This programming
habit is responsible for introducing countless security vulnerabilities in programs that do not work as
intended when the programmer does not know how the tool reacts in certain situations.
Educ. Sci. 2020, 10, 353 21 of 23

References
1. Computer and Information Technology Occupations: Occupational Outlook Handbook; The Bureau of Labor
Statistics: Washington, DC, USA, 2020.
2. Falkner, K.; Vivian, R. A review of computer science resources for learning and teaching with K-12 computing
curricula: An Australian case study. Comput. Sci. Educ. 2015, 25, 390–429. [CrossRef]
3. Borowczak, M.; Burrows, A.C. Ants Go Marching—Integrating Computer Science into Teacher Professional
Development with NetLogo. Educ. Sci. 2019, 9, 66. [CrossRef]
4. Nelson, T.H.; Lesseig, K.; Slavit, D. Making Sense of STEM Education in a K-12 Context. In Proceedings of
the 2016 NARST International Conference, Baltimore, MD, USA, 14–17 April 2016.
5. Webb, M.; Davis, N.; Bell, T.; Katz, Y.J.; Reynolds, N.; Chambers, D.P.; Sysło, M.M. Computer science in K-12
school curricula of the 2lst century: Why, what and when? Educ. Inf. Technol. 2017, 22, 445–468. [CrossRef]
6. Burrows, A.C.; Borowczak, M. Computer science and engineering: Utilizing action research and lesson
study. Educ. Action Res. 2019, 27, 631–646. [CrossRef]
7. Burrows, A.C.; Borowczak, M. Hardening Freshman Engineering Student Soft Skills. In Proceedings of the
2017 FYEE Conference, Daytona Beach, FL, USA, 6–8 August 2017.
8. Binkley, M.; Erstad, O.; Herman, J.; Raizen, S.; Ripley, M.; Miller-Ricci, M.; Rumble, M. Defining twenty-first
century skills. In Assessment and Teaching of 21st Century Skills; Springer: Dordrecht, The Netherlands, 2012;
pp. 17–66.
9. Vennix, J.; Den Brok, P.; Taconis, R. Perceptions of STEM-based outreach learning activities in secondary
education. Learn. Environ. Res. 2017, 20, 21–46. [CrossRef]
10. Nelson, K.; Sabel, J.; Forbes, C.; Grandgenett, N.; Tapprich, W.; Cutucache, C. How do undergraduate STEM
mentors reflect upon their mentoring experiences in an outreach program engaging K-8 youth? Int. J. Stem Educ.
2017, 4, 3. [CrossRef] [PubMed]
11. Dostis, M. Degree Alone Not Enough to Prepare Grads for Workforce. USA Today College. Available online:
http://www.usatoday.com/story/news/nation/2013/10/31/more-than-a-collegedegree/3324303/ (accessed
on 20 November 2014).
12. Caldwell, K.; McCoy, J.; Albers, L.; Smith, A.; Parry, E. The Impact Of K 12 Outreach Programs On Graduate
In addition, Undergraduate Experiences. In Proceedings of the 2007 Annual Conference & Exposition,
Honolulu, HI, USA, 24–27 June 2007; pp. 12–1430.
13. Albers, L.; Smith, A.; Caldwell, K.; McCoy, J.; Bottomley, L.; Parry, E. The impact of out-of-school time (OST)
math and science clubs on elementary and middle school students, teachers, schools and the undergraduate
and graduate fellows that facilitate them. In Proceedings of the American Society for Engineering Education
Annual Conference, Pittsburgh, PA, USA, 22–25 June 2008.
14. Albers, L.A. The Creation and Evolution of an Energy Engineering and Education Outreach Model.
Strateg. Plan. Energy Environ. 2015, 34, 63–71. [CrossRef]
15. Lamberg, T.; Trzynadlowski, N. How STEM Academy Teachers Conceptualize and Implement STEM
Education. J. Res. Stem Educ. 2015, 1, 45–58.
16. Johnson, C.C.; Mohr-Schroeder, M.J.; Moore, T.J.; English, L.D. Handbook of Research on STEM Education;
Routledge: Abingdon upon Thames, UK, 2020.
17. Burrows, A.C.; Lockwood, M.; Borowczak, M.; Janak, E.; Barber, B. Integrated STEM: Focus on Informal
Education and Community Collaboration through Engineering. Edu. Sci. 2018, 8, 4. [CrossRef]
18. Burrows, A.C.; Slater, T. A proposed integrated STEM framework for contemporary teacher preparation.
Teach. Educ. Pract. 2015, 28, 318–330.
19. Michael, J.; Modell, H.I. Active Learning in Secondary and College Science Classrooms: A Working Model for
Helping the Learner to Learn; Routledge: Abingdon upon Thames, UK, 2003.
20. Fogg-Rogers, L.; Lewis, F.; Edmonds, J. Paired peer learning through engineering education outreach. Eur. J.
Eng. Educ. 2017, 42, 75–90. [CrossRef]
21. Langheinrich, J.; Bogner, F.X. Computer-related self-concept: The impact on cognitive achievement.
Stud. Educ. Eval. 2016, 50, 46–52. [CrossRef]
22. Lee, V.E.; Burkam, D.T. Gender differences in middle grade science achievement: Subject domain, ability
level, and course emphasis. Sci. Educ. 1996, 80, 613–650. [CrossRef]
Educ. Sci. 2020, 10, 353 22 of 23

23. Burkam, D.T.; Lee, V.E.; Smerdon, B.A. Gender and science learning early in high school: Subject matter and
laboratory experiences. Am. Educ. Res. J. 1997, 34, 297–331. [CrossRef]
24. Goldschmidt, M.; Bogner, F.X. Learning about genetic engineering in an outreach laboratory: Influence of
motivation and gender on students’ cognitive achievement. Int. J. Sci. Educ. B 2016, 6, 166–187. [CrossRef]
25. Schwortz, A.; Burrows, A.C. Authentic science experiences with STEM datasets: Post-secondary results and
potential gender influences. Res. Sci. Tech. Educ. 2020, 40, 1–21. [CrossRef]
26. Lasi, H.; Fettke, P.; Kemper, H.G.; Feld, T.; Hoffmann, M. Industry 4.0. Bus. Inf. Syst. Eng. 2014, 6, 239–242.
[CrossRef]
27. Michell, D.; Szabo, C.; Falkner, K.; Szorenyi, A. Towards a socio-ecological framework to address gender
inequity in computer science. Comput. Educ. 2018, 126, 324–333. [CrossRef]
28. Gumaelius, L.; Almqvist, M.; Árnadóttir, A.; Axelsson, A.; Conejero, J.A.; García-Sabater, J.P.; Klitgaard, L.;
Kozma, C.; Maheut, J.; Marin-Garcia, J.; et al. Outreach initiatives operated by universities for increasing
interest in science and technology. Eur. J. Eng. Educ. 2016, 41, 589–622. [CrossRef]
29. Giannakos, M.N.; Pappas, I.O.; Jaccheri, L.; Sampson, D.G. Understanding student retention in computer science
education: The role of environment, gains, barriers and usefulness. Educ. Inf. Technol. 2017, 22, 2365–2382.
[CrossRef]
30. Fayer, S.; Lacey, A.; Watson, A. STEM Occupations: Past, Present, In Addition, Future; Spotlight on Statistics
from US Bureau of Labor Statistics; The Bureau of Labor Statistics: Washington, DC, USA, 2017.
31. Dubina, K.S.; Morisi, T.L.; Wagoner, A.B. Projections overview and highlights, 2018–28. Mon. Labor Rev. 2019.
[CrossRef]
32. U.S. Bureau of Labor Statistics. Computer and Information Research Scientists: Occupational Outlook
Handbook. 2019. Available online: https://www.bls.gov/ooh/computer-and-information-technology/
computer-and-information-research-scientists.htm (accessed on 20 November 2020).
33. Landry, J.P.; Barnett, H.L.; Chapman, D.L.; McCullough, R. Four Strategies for Driving a University
Pre-College Computing Outreach Program. J. Inf. Syst. Educ. 2019, 30, 191–201.
34. Aslam, F.; Adefila, A.; Bagiya, Y. STEM outreach activities: An approach to teachers’ professional development.
J. Educ. Teach. 2018, 44, 58–70. [CrossRef]
35. Disalvo, B.; Reid, C.; Roshan, P. They can’t find us: The search for informal CS education. In Proceedings of
the SIGCSE 2014—45th ACM Technical Symposium on Computer Science Education, Atlanta, GA, USA,
5–8 March 2014; pp. 487–492.
36. Burrows, A.C.; Borowczak, M.; Slater, T.F.; Haynes, C.J. Teaching computer science & engineering through
robotics: Science & art form. Prob. Edu. 21st Century 2012, 47, 6–15.
37. Forbes, M.H.; Sullivan, J.F.; Carlson, D.W. Ascertaining the Impact of P: 12 Engineering Education Initiatives:
Student Impact through Teacher Impact. In Proceedings of the 2017 ASEE Annual Conference & Exposition,
Columbus, OH, USA, 25–28 June 2017.
38. Glynn, S.M.; Taasoobshirazi, G.; Brickman, P. Science motivation questionnaire: Construct validation with
nonscience majors. J. Res. Sci. Teach. 2009, 46, 127–146. [CrossRef]
39. Deci, E.L.; Ryan, R.M. The “what” and “why” of goal pursuits: Human needs and the self-determination of
behavior. Psychol. Inq. 2000, 11, 227–268. [CrossRef]
40. Nesiba, N.; Dana-Farley, J.; Muhyi, N.; Chen, J.; Ray, N.; Pontelli, E. Young Women in Computing: Creating
a successful and sustainable pipeline. In Proceedings of the 2015 IEEE Frontiers in Education Conference
(FIE), El Paso, TX, USA, 21–24 October 2015; pp. 1–9.
41. Pajares, F. Self-efficacy beliefs in academic settings. Rev. Educ. Res. 1996, 66, 543–578. [CrossRef]
42. Turner, P.R.; Fowler, K.; Wick, D.; Ramsdell, M.; Gotham, G.; Glasgow, E.; French, C. BOCES-University
Partnership as a model for Educational Outreach: K-16 STEM Professional Development. In Proceedings of
the Math and Science Symposium, Knoxville, TN, USA, October 2007.
43. Lakanen, A.J.; Isomöttönen, V. Computer science outreach workshop and interest development: A longitudinal
study. Inform. Educ. 2018, 17, 341–361. [CrossRef]
44. Renninger, K.A.; Hidi, S. Revisiting the conceptualization, measurement, and generation of interest.
Educ. Psychol. 2011, 46, 168–184. [CrossRef]
45. Vennix, J.; Den Brok, P.; Taconis, R. Do outreach activities in secondary STEM education motivate students
and improve their attitudes towards STEM? Int. J. Sci. Educ. 2018, 40, 1263–1283. [CrossRef]
Educ. Sci. 2020, 10, 353 23 of 23

46. Sjaastad, J. Measuring the ways significant persons influence attitudes towards science and mathematics.
Int. J. Sci. Educ. 2013, 35, 192–212. [CrossRef]
47. Woods-Townsend, K.; Christodoulou, A.; Rietdijk, W.; Byrne, J.; Griffiths, J.B.; Grace, M.M. Meet the scientist:
The value of short interactions between scientists and students. Int. J. Sci. Educ. 2016, 6, 89–113. [CrossRef]
48. Straw, S.; Macleod, S. Evaluation of STEMNET’s Operations and Impacts 2011–2015: Summary Report; NFER:
Slough, UK, 2015.
49. Krogstad, J.; Tyler, K.; Johnson-Glauch, N.; Dean, L. Promoting Positive Outcomes in K-12 Outreach through
Design. JOM 2019, 71. [CrossRef]
50. Archer, L.; DeWitt, J.; Osborne, J.; Dillon, J.; Willis, B.; Wong, B. “Balancing acts”: Elementary school girls’
negotiations of femininity, achievement, and science. Sci. Educ. 2012, 96, 967–989. [CrossRef]
51. Murphy, P.; Whitelegg, E. Girls in the Physics Classroom: A Review of the Research on the Participation of Girls in
Physics; Institute of Physics: London, UK, 2006.
52. Wolf, S.; Moss, F.P.; Manandhar, R.; Cooley, M.; Cooley, R.; Burrows, A.C.; Borowczak, M. Building
Collaboration and Securing Interest in Computer Science Education through Outreach Opportunities.
In Proceedings of the 2019 ASEE Annual Conference & Exposition, Tampa, Florida, USA, 16–19 June 2019.
53. Laursen, S.; Liston, C.; Thiry, H.; Graf, J. What good is a scientist in the classroom? Participant outcomes and
program design features for a short-duration science outreach intervention in K–12 classrooms. CBE Life
Sci. Educ. 2007, 6, 49–64. [CrossRef] [PubMed]
54. Computational Thinking and CS Unplugged. Available online: https://csunplugged.org/en/computational-
thinking/ (accessed on 20 November 2020).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional
affiliations.

c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).