See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/373166024

Generative artificial intelligence: University student awareness, experience, and

confidence in use across disciplines

Article in Journal of University Teaching and Learning Practice · August 2023

DOI: 10.53761/1.20.6.12

CITATIONS READS

31 1,494

3 authors, including:

Miriam Sullivan
Edith Cowan University
20 PUBLICATIONS 472 CITATIONS

SEE PROFILE

All content following this page was uploaded by Miriam Sullivan on 17 August 2023.

The user has requested enhancement of the downloaded file.

 Journal of University Teaching & Learning Practice

Volume 20 Article 12
Issue 6 Quarterly Issue 3

2023

Generative artificial intelligence: University student awareness, experience,

and confidence in use across disciplines
Andrew Kelly
Edith Cowan University, Australia, [email protected]

Miriam Sullivan
Edith Cowan University, Australia, [email protected]

Katrina Strampel
Edith Cowan University, Australia, [email protected]

Follow this and additional works at: https://ro.uow.edu.au/jutlp

Recommended Citation
Kelly, A., Sullivan, M., & Strampel, K. (2023). Generative artificial intelligence: University student
awareness, experience, and confidence in use across disciplines. Journal of University Teaching &
Learning Practice, 20(6). https://doi.org/10.53761/1.20.6.12

Research Online is the open access institutional repository for the University of Wollongong. For further information
contact the UOW Library: [email protected]
Generative artificial intelligence: University student awareness, experience, and
confidence in use across disciplines

Abstract
The global higher education sector has been significantly disrupted by the proliferation of generative
artificial intelligence tools such as ChatGPT, especially in relation to its implications for assessment.
However, few studies to date have explored student perspectives on these tools. This article reports on
one of the first large-scale quantitative studies of student views on generative artificial intelligence at an
Australian university (n = 1,135). When the survey was conducted, most students had low knowledge,
experience, and confidence in using these tools. These results varied across disciplines and across some
student sub-groups, such as mature-age students and international students. Confidence appeared to
increase with experience, although the data also revealed a portion of students that have never used
these tools yet still felt confident in using them. In exploring these results, this article aims to shed new
light on this fast-evolving landscape and inform the future direction of supporting students to engage with
generative artificial intelligence tools appropriately.

Practitioner Notes
1. Students need to be explicitly taught how to use generative artificial intelligence tools
appropriately.
2. Learning activities that build student skills in using generative artificial intelligence should
be embedded into curricula.
3. The ways in which students learn how to use generative artificial intelligence will need to
vary based on the needs of each disciplinary area.
4. Student reports of self-confidence in using generative artificial intelligence may be
overstated.
5. Assessment tasks need to be redesigned to reduce the academic integrity risks
associated with using generative artificial intelligence.

Keywords
Generative artificial intelligence, ChatGPT, students, higher education

This article is available in Journal of University Teaching & Learning Practice: https://ro.uow.edu.au/jutlp/vol20/iss6/
12
 Kelly et al.: Generative artificial Intelligence

Introduction
The open-access release of generative artificial intelligence (GenAI) tools in late 2022 and early
2023 prompted a major disruption to the teaching and learning practices of universities worldwide.
ChatGPT, and similar GenAI tools, create sophisticated bespoke content that can pass many
traditional forms of university assessment. For example, U.S. company OpenAI’s most advanced
GenAI tool, GPT-4, has already comfortably passed some of the most challenging professional
accreditation exams, including the Uniform Bar Exam for legal practice and the Certified Public
Accountant Exam for accounting practice (Gaetano, 2023; Koetsier, 2023). Some academic
responses to GenAI have been largely positive, noting these tools’ capability to enhance student
learning and accessibility (Lyerly, 2023; Sullivan et al., 2023). Others have correctly pointed out
that GenAI tools also have several weaknesses, including the possibility of generating inaccurate
information and fabricating references (Farrokhnia et al., 2023; Rudolph et al., 2023).

Universities worldwide, in turn, have also been varied in their responses to GenAI. These
responses have ranged from completely banning the use of these tools to allowing use with
appropriate acknowledgement (Sullivan et al., 2023). Increasingly, however, universities are
starting to adopt policies that allow students to use GenAI in their studies (Xiao et al., 2023). In
the Australian context, GenAI has been a catalyst for significant national discussion about the
future of teaching, learning, and assessment in higher education institutions across the country.
Peter Coaldrake, the Chief Commissioner for Australia’s Tertiary Education and Quality
Standards Agency (TEQSA), suggested that the rapid development of GenAI requires a “deep
rethink” of how universities assess students. “This is a challenge facing the whole sector”,
Coaldrake said in a TEQSA webinar with Deakin University’s Centre for Research in Assessment
and Digital Learning in February 2023, noting that GenAI “presents significant opportunities to
support learning” but must be balanced against the risks that it poses to academic integrity
(TEQSA, 2023)

There is certainly an emerging exploration of the implications of GenAI for learning and
assessment design. Academic staff views on GenAI tools on teaching, learning, research, and
policy have already been collected from a range of multidisciplinary perspectives (Dwivedi et al,
2023), and it is highly likely that this body of scholarly
literature will continue to expand. However, at the time of Academic Editors
writing, there has been no equivalent study to date that
Section: Educational Technology
explores student views and compares findings across
Editor-in-Chief: Dr Joseph Crawford
disciplinary boundaries. Gaining an understanding of the Senior Editor: A/Prof Michael Cowling
student's perspective is crucial for effectively supporting
Publication
their development in using GenAI tools and how to
minimize academic integrity risks. To respond to this Received: 19 June 2023
current gap in the literature, this article reports on a cross- Revision: 26 July 2023
Accepted: 14 August 2023
disciplinary study conducted at an Australian university in Published: 16 August 2023
March 2023 that surveyed 1,135 students about their
Copyright: © by the authors, in its year of first
views on GenAI tools. The data reveals a mix of relative
publication. This publication is an open
awareness, experience, and confidence in using these access publication under the Creative
tools across disciplines and sub-cohorts. Framed in this Commons Attribution CC BY-ND 4.0 license.

1
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

context, this article provides a new basis for advancing our broad understanding of how these
tools will impact students as they study at university.

Literature
While the availability and degree of sophistication of GenAI is a relatively new phenomenon,
researchers have been interested in the affordances of artificial intelligence in education for
decades. Previous studies have shown how the use of artificial intelligence can be used in
meaningful ways to improve assessment feedback and make administrative duties more efficient
(Brown et al., 1978; Crompton & Bruke, 2023; Garito, 1991; Popenici & Kerr, 2017). Prior to the
public release of ChatGPT and similar GenAI tools in late 2022, UNESCO also published
recommendations on the ethical use of artificial intelligence. Key principles included maintaining
safety, human oversight and transparency in use, ensuring fairness and non-discrimination,
respecting a right to privacy and data protection, building awareness, and collaborating with
stakeholders (UNESCO, 2022).

The reality of the current context, however, is much different than previous forms of artificial
intelligence. The ability of GenAI to create complex bespoke content on demand is transforming
multiple industries at a rapid pace (Forbes, 2023). It also prompted the Russell Group, a coalition
of twenty-four leading universities in the United Kingdom, to recommend new principles on the
use of GenAI in higher education. These principles mirrored some of the principles previously
recommended by UNESCO but advocated specifically for building university staff and student
GenAI literacy, adapting teaching learning and practices to incorporate ethical use, ensuring
academic integrity is upheld, and encouraging best practice to be shared collaboratively across
the tertiary sector (Russell Group, 2023). One recent study has even begun to consider the use
of GenAI in marking student work (Kasneci et al 2023), though such use needs to be handled
cautiously given student privacy implications and the propensity of these tools to generate
incorrect information. This will undoubtedly have an impact on how students progress through
tertiary studies and how well they are prepared to secure meaningful employment upon
graduation. In turn, universities need to consider how they prepare students to use these tools
ethically in their disciplinary contexts.

To that end, the most immediate concerns about GenAI explored in the current literature are the
implications for assessment and supporting academic integrity. Studies to date reflect the need
to rethink the typical types of university assessment tasks which could now easily be completed
by GenAI tools (Cotton et al, 2023, Crawford et al 2023; Farrokhnia et al, 2023; Perkins 2023;
Rudolph et al, 2023). These principally relate to summative written assessments such as essays
and reports, though as Susnjak (2022) pointed out, the ubiquity of GenAI tools also calls into
question the integrity of online exam tasks. Beyond text-based tasks, GPT-4 is also capable of
producing outputs based on images (Bubeck et al., 2023). This capability limits the usefulness of
using images to mitigate the risk of students using GenAI to complete assessment tasks. In short,
articles published on this topic so far outline a very clear picture: the ways in which universities
and their respective academic staff set assessment tasks needs to transform rapidly. Traditional
university assessments have typically relied on students to produce artefacts to infer that learning
has occurred. Now that GenAI tools such as ChatGPT can produce these artefacts to a

https://ro.uow.edu.au/jutlp/vol20/iss6/12 2
 Kelly et al.: Generative artificial Intelligence

reasonable degree of quality, universities need to rethink the association between learning and
performance (Lodge, 2023).

There has also been a positive discourse surrounding GenAI tools in university teaching and
learning contexts, especially in relation to student support. These tools can demystify challenging
academic concepts in simple language and improve inclusion with communication disabilities
(Hemsley et al., 2023; Lyerly, 2023; Starcevic, 2023). With appropriately used inputs, they can
also produce structures for written assessment tasks, give grammatical feedback, and develop
sample practice quiz questions for test preparation (Sullivan et al., 2023). These affordances can
make learning at university more accessible to diverse learners, especially for students from
cultural and linguistic backgrounds that differ from the expected conventions of the university in
which they are studying.

A key challenge with the use of GenAI is the need for strong critical thinking and digital literacy
skills. For over two decades, there has been a contested scholarly debate about whether the
current generations of students are “digital natives”, meaning that they have grown up using
technology and are thereby well prepared to adapt to the use of emerging technologies (Evans &
Robertson, 2020; Prensky, 2001). Many university students—especially those born in Western
countries during the 21st century—fall into this category and may already have the requisite digital
literacy tools to engage successfully with GenAI tools in such a way that will provide useful outputs
and be able to critically evaluate them (Willems et al., 2019). To some degree, student
experiences of learning remotely during the COVID-19 pandemic accelerated the use of learning
technologies and the digital literacy skills required to engage with them (Butarbutar et al., 2021;
Udeogalanya, 2022; Yu, 2022). However, it is important to note that growing up surrounded by
technology and personal experiences of learning online during the COVID-19 pandemic does not
necessarily mean that a student will be automatically prepared to adapt well to the use of
emerging GenAI technologies in a formal learning environment. Even when students have good
general digital literacy they may still feel limited in specific areas (Zhao et al., 2021) or follow
rituals they have developed over time rather than thinking about new situations critically (Bhatt &
MacKenzie, 2019). For example, early research on ChatGPT suggests that people underestimate
how much the information influences their judgements (Krugal et al., 2023).

Nonetheless, students that can analyse GenAI outputs critically will be in a more advantageous
position to leverage the benefits of these tools and safeguard against its risks (Hess, 2023). Digital
literacy (or digital competence) means that students should be able to use technology confidently
and critically in their education and workplace (Zhao et al., 2021). Moving forward, universities
will need to embed the teaching of using GenAI tools appropriately in student learning programs
(García-Peñalvo, 2023). Specifically in relation to artificial intelligence literacy, appropriate use
entails understanding broadly how GenAI works, evaluating outputs, effectively interacting with
these tools to solve real-life problems (Kong et al., 2021), and communicating the results to others
effectively (McCoy et al., 2020). Previous research on digital literacy shows that students rely
heavily on the guidance of their lecturers when it comes to finding and using information (Bhatt &
MacKenzie, 2019). This means that in order to foster meaningful use for everyday life, it is
important that digital literacy is embedded in discipline-specific ways throughout all programs, and
not as ad-hoc or optional activities (Smith & Storrs, 2023).

3
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

The current literature has not yet explored student perspectives on GenAI at-scale. Some student
views have been explored in social media posts, though the current sample sizes are reasonably
small and do not explore different perspectives across disciplines (Haensch, 2023; Tlili et al.,
2023). Firat (2023) gathered a small number of student perspectives on the implications of
ChatGPT for universities, though these responses were from postgraduate students enrolled in
doctoral programs. Shoufan (2023) gathered student views from a computer engineering
program—most respondents saw the benefits of ChatGPT but remained cautious about the
inaccuracy of some of its outputs. Limna et al. (2023) also surveyed a small number of students
(n = 15) and found a generally positive perception of GenAI tools, yet students also raised
concerns about data privacy and storing of personal information. Overall, this limited exploration
of student perspectives on GenAI tools by mid-2023 contributed to the need for the study reported
in this article.

Method
Research Context and Scope

This student survey was undertaken in March 2023 at a mid-size Australian university with over
25,000 students enrolled from a diverse range of linguistic, cultural, and socioeconomic
backgrounds. Participants were all students enrolled in on-campus or online study at the time of
the survey. The survey was sent to all students to understand their awareness, perceptions, and
use of GenAI. As part of the University’s response to GenAI, students were allowed to use these
tools but were expected to acknowledge their use if they did. The survey responses were sought
to inform the University’s response to integrating GenAI into curriculum, assessment, and
teaching. The University’s human research ethics office considered a waiver of explicit consent
for the research and granted ethics approval (REMS number: 2023-04278). For the purposes of
this article, only the quantitative results have been included; qualitative results will be published
in a future article.

Survey Design

The project followed a mixed-methods approach using an anonymous student survey. The ten-
item survey was designed with a mix of closed-ended questions, open-ended questions, and
Likert questions clustered into five categories: demographics; knowledge, experience,
confidence, and perceptions about GenAI. As Table 1 outlines, all demographic questions were
closed-ended except for participants being able to add their discipline if it was not included in the
list.

There were two knowledge-based questions. The first was a Likert question that quantified how
much the participants had heard of GenAI. The second asked where the participant had first heard
of GenAI; this was closed-ended but provided a text field for a different option. If the participant
selected “nothing/negligible” to the first knowledge question, the survey skipped to the second
knowledge question and the experience questions. Next, there were two specific questions about
how much the participant had used GenAI, and then an open-ended question that asked the
participant how they have used GenAI tools. Skip logic was used to skip the second experience
question if the participant selected “not at all” to the first experience question. There was one

https://ro.uow.edu.au/jutlp/vol20/iss6/12 4
 Kelly et al.: Generative artificial Intelligence

Likert question that quantified how confident each participant felt about using GenAI ethically for
university study. Finally, there was one open-ended question about student perceptions of GenAI.

Table 1

Overview of the survey categories and its associated description

Category Description
Demographics Participants were asked questions related to relevant education information
(discipline of study, level of study, international or domestic) and personal
characteristics (age).
Knowledge of Participants were asked how much they had heard of GenAI and where they
GenAI had heard of it. Participants could choose multiple options on this question.
Use of GenAI Participants were asked if they had used GenAI tools, with examples
provided (ChatGPT, Dalle-E, Chatsonic, etc.) and how they had used the
tools. Both were measured on five-point Likert scales, with options ranging
from “not at all” ranging to “a lot (I have used Generative AI tools on a daily
or weekly basis)”.
Confidence Participants were asked how confident they feel that they can use GenAI
using GenAI ethically in their university studies, on a five-point Likert scale from “Not at all
confident” to “Very confident”, with an additional option of “Unable to judge”.
Perceptions Participants were asked to share their thoughts about GenAI.
about GenAI

Survey Distribution

Qualtrics was used as the platform to design and administer the survey. An email was sent from
the Deputy Vice-Chancellor (Education) to all undergraduate and postgraduate students (26,686)
in week two of Semester One 2023 (2 March). Within twenty-four hours, 7,574 students had
opened the email (28.4%) and 56 (0.21%) students had accessed the survey. After six days,
9,100 students had opened the email (34.1%) and 77 (0.29%) had accessed the survey. A follow-
up SMS was sent to all students on 16 March prompting completion of the survey. Within twenty-
four hours of the SMS being sent, just over 1,000 students had responded to the survey. The
survey closed on 23 March, with 1,135 total responses gathered.

Data Analysis

The quantitative data was analysed in SPSS (version 29). Non-parametric statistics were used
since the data was primarily nominal and ordinal (Pallant, 2016). For group comparisons on Likert
scales, the Mann-Whitney U test was used when comparing international and domestic student
responses. All other groups were analysed with the Kruskal-Wallis test, and the Dunn-Bonferroni
approach was applied for pairwise comparisons. For confidence compared to use, students who
answered they knew “nothing” about GenAI were combined in a group with those who said they
had not used it, and a correlation analysis was conducted using Spearman’s rho. Comparisons
between sources of information were performed using Chi-squared tests. International students
and sources of information are presented with a Yate’s Correction for Continuity and the phi

5
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

coefficient, while schools and sources of information are reported with Cramer’s V to account for
the larger table (Pallant, 2016).

Unfinished and incomplete responses were excluded from the data analysis (n = 450). Students
were slightly more likely to fail to finish the survey if they answered the first question as they know
“nothing” about GenAI (p = .048). For the analysis of discipline-based data, the University’s
Preparation Course (n = 37) and the School of Performing Arts (n = 28) were excluded from the
analysis due to low sample size. The University Preparation course students were also excluded
from the analysis of course level. Written responses for course enrollment and where students
had heard about GenAI were manually checked and reassigned to the correct category where
appropriate, for example, if a student had written “blog” into the “other” response for where they
heard about GenAI, it was reassigned to social media.

Results
At the time of administering the survey, most students had either heard nothing (13%) or very little
(28%) about ChatGPT and other GenAI tools. For students that had heard of it, even fewer had
used GenAI (see Figure 1). Of the 959 respondents who had heard about GenAI, most of them
(64%) had found out about it through social media, followed by news media (41%), other students
(32%), and work (20%). Students were able to provide other sources of information, in which they
mostly listed university staff and communications (n = 78), friends and family (n = 53), or
unspecified (n = 34).

Figure 1

Student Awareness and Use of GenAI Tools

https://ro.uow.edu.au/jutlp/vol20/iss6/12 6
 Kelly et al.: Generative artificial Intelligence

Use and Confidence

Participants’ confidence in their ability to use GenAI ethically increased with experience using
GenAI (p = .001, rho = .494, Figure 2). About 15% of students who had not used GenAI (n = 686)
rated themselves as “slightly confident” or “very confident”, compared to 91% of students who
had used GenAI “a lot” (n = 56).

Figure 2

Student Confidence in their Ability to Use GenAI Ethically

Disciplinary Area

Figure 3 shows significant differences between disciplines in how much their students had heard
about GenAI (p = .001). These have been grouped into each of the University’s Schools. Pairwise
comparisons revealed that Nursing students (n = 113) had heard about GenAI significantly less
than students in all other disciplinary areas (p > .05) except Medical and Health Sciences (n =
121, p = .347). Science students (n = 199) had heard more about GenAI than Medical and Health
Sciences (p < .000), Business and Law (n = 167, p = .001), and Arts and Humanities (n = 201, p
= .005).

There were also differences between disciplines in use of GenAI (p = .001). Engineering (n = 91)
was also significantly higher than Nursing (p = .004), Medical and Health (p = .011) and Arts (p =
.003). Science was higher than Nursing (p < .000), Medical and Health (p < .000), Arts (p < .000),
Business and Law (p = .004) and Education (p = .003). Differences in confidence in using AI
ethically (p = .001) were similar. Engineering was higher than Nursing (p = .001) Medical and
Health (p = .005), and Arts (p = .006); while Science was significantly higher than Nursing (p <
.000), Medical and Health (p = .001), and Arts (p = .001).

7
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

Figure 3

Proportion of students who stated that they had a moderate or high level of awareness, use and
confidence with GenAI by disciplinary area (School)

Students from different Schools had significantly different sources of information about GenAI for
all sources, such as social media (p = .001, Cramer’s V = .179), news (p = .001, Cramer’s V =
.142), other students (p = .003, Cramer’s V = .137), and work (p = .001, Cramer’s V = .147). As
seen in Figure 4, Nursing students used all information sources less than students enrolled in
other Schools.

Figure 4

Source of information used to learn about GenAI by disciplinary area (School)

https://ro.uow.edu.au/jutlp/vol20/iss6/12 8
 Kelly et al.: Generative artificial Intelligence

International and Domestic Status

International students (n = 239) had less awareness of GenAI (p = <.001, U = 88148, z = 4.573,
r = .135), with 72% of international students saying that they had heard “nothing” or “a little” about
GenAI compared to 56% of domestic students. However, there were no significant differences in
their use (p = .08) or confidence (p = .113).

The difference in awareness of GenAI may be related to where students are sourcing information.
International students were equally likely to find out about GenAI from social media (p = .178) and
other students (p = .418), but less likely to use news media (p = .001, phi = .197; 39% of domestic
students compared to 16% of international students) or work (p = .001, phi = .146; 20% of
domestic students compared to 7% of international students).

Study Level

There was no difference in how much students had heard about GenAI based on their level of
study (p = .391). However, there were small differences in whether they had used GenAI (p =
.007), with 67% of higher degree by research student respondents using GenAI (n = 53) compared
to 41% of postgraduate students (n = 296, p = .007) and nearly significant compared to 48% of
undergraduates (n = 746, p = .55). When it came to confidence, postgraduate students were more
confident than undergraduates (p = .033).

Age

There was no difference between age groups in having heard about GenAI (p = .443). However,
there was a difference in who had used GenAI (p = .004), with pairwise comparisons showing that
over 40s (n = 228, 31% used) were less likely to have used GenAI than under 25s (p = .002, 44%
used). This trend was more pronounced in confidence using GenAI (p = .001), with only 21% of
over 40s being slightly or very confident compared to 37% of under 25s (n = 489, p = .001) and
36% of 25-29s (n = 156, p = .012).

Discussion
A key finding from this study was that students had relatively low knowledge, experience, and
confidence with using GenAI. Given the rapid pace at which these tools became available in late
2022 and early 2023, it is understandable why so many students were unfamiliar with them at the
time. This timeline limited opportunities for academic teaching staff to consider deeply the
emerging challenges and risks associated with GenAI and how to incorporate these tools into
their respective teaching and learning practices. The researchers suspect that future studies that
explore student perspectives will find an increased level of awareness of these tools, especially
as they start to become integrated into other commonly used student platforms such as the
Microsoft suite and Grammarly (Spataro, 2023). The future degree of experience and confidence,
however, is less clear, as exposure to using these tools appropriately in a university setting will
largely be shaped by the policy positions, education initiatives, and assessment applications
adopted by each institution.

The results suggested that student confidence in using GenAI ethically increased with experience.
This, again, was another logical and predictable outcome, although it should be noted that

9
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

exposure to technology does not necessarily lead to understanding (Murray & Perez, 2014).
However, an interesting finding was a proportion of students (15%) that indicated they had never
used GenAI tools but nonetheless still felt confident in their use. There may be several
explanations for this finding. Firstly, as the survey was conducted during the early weeks of the
first semester that GenAI tools had been widely available, many students may have heard much
about the tools but had not yet had opportunities to engage with these tools for the purposes of
learning and assessment. Another possible explanation is that these specific student responses
were typical examples of Prensky’s (2001) “digital native” students, insofar as they had grown up
using technology and consequently felt confident to use new ones successfully even if they had
not directly used them yet. However, students are known to overrate their abilities in multiple
areas of digital literacy (Smith & Storrs, 2023), and results may also be a manifestation of the
Dunning-Kruger effect (Dunning, 2011). Bhatt and Mackenzie (2019) argue that most students
are relatively passive consumers of online content, and do not truly understand how online
information is curated, especially in new platforms that are not transparent or accountable in their
design architecture. Students with low GenAI experience, in short, may be overestimating their
abilities to adapt successfully to this new form of technology.

There were noticeable variations in student responses across disciplines. As Figure 3 highlights,
students from science and engineering disciplines generally reported higher awareness,
experience, and confidence in using GenAI, whereas the healthcare disciplines generally reported
the lowest rates. To some extent, this trend may be connected to relative interests in the use of
technologies for different disciplines. Despite the possibility of occasionally producing incorrect
information, GenAI’s ability to produce code, solve mathematical equations, and design scientific
experiments may seem more directly practical for students in those disciplines. Conversely,
students studying in disciplines that focus on human care may not draw the same immediate
connections as to how tools such as ChatGPT may be applied in healthcare settings (Cascella et
al., 2023). Student levels of awareness and use may also have been shaped by the extent to
which each disciplinary area specifically discussed GenAI tools directly with students. This is
similar to a study by Smith and Storrs (2023), which found that students in communications and
health science disciplines had higher digital literacy when using social media because it was
integrated into their professional standards curricula.

Similarly, there were some demographic differences across different student cohorts.
International students, for example, reported lower awareness of GenAI and were less likely than
domestic students to learn about GenAI through work or the news. There are several likely factors
that influence this difference. For instance, international students in Australia have reported
challenges in building local social connections upon arrival (Khanal & Gaulee, 2019), which may
limit the possibilities to learn about GenAI through friends or student peers. Another variation was
between age groups; students forty years old and over, for example, reported they were less likely
to have used GenAI and were less confident about using it ethically compared with younger
students. This cohort had generally less opportunities to engage with technology growing up and
in previous study than younger generations (Prenksy, 2001), which may contribute to the
explanation as to why experience and confidence rates were lower in this cohort. However, most
students lacked confidence with GenAI regardless of age and the differences were not as large

https://ro.uow.edu.au/jutlp/vol20/iss6/12 10
 Kelly et al.: Generative artificial Intelligence

as those between Schools, confirming Smith and Storrs’ (2020) argument that digital literacy
should focus on professional disciplinary competencies rather than age stereotypes.

A final reflection from these results are the various ways in which students are learning about
GenAI. Ideally, students would learn about GenAI in appropriate and ethical ways via their
institution, both within and outside the curriculum. However, the results of this study suggested
that students were more likely to learn about GenAI through social media. There are some studies
that have begun to explore social media data on GenAI (Haensch, 2023; Tlili et al 2023), and the
ways in which these tools are socialized in social media platforms will be important to investigate
further. For instance, social norms can impact the likelihood of student cheating (Hutton, 2006),
so universities and its respective teaching staff must do all they can to communicate clearly the
ways in which GenAI can be used appropriately, including outlining the academic integrity risks if
its use is not acknowledged or referenced properly.

Limitations and Future Research

There are some limitations to this research that need to be acknowledged. While this study is one
of the first published that explores student perspectives on GenAI, the student sample was drawn
from only one Australian university and focused only on the quantitative results. Student
awareness and use of GenAI is likely to vary considerably depending on how much their
university, local news media, and social circles have engaged with the technology.

Not all students that started the survey completed it, and students who indicated that they knew
nothing about GenAI were less likely to complete the survey. This limits the student insights that
can be drawn from those that may be the most unprepared for the impact of GenAI on their
respective university studies. Similarly, students across all disciplinary areas did not respond at
equal rates. This may also skew the results towards disciplinary groups that completed the survey
at higher rates than those that did not.

The methodology used a conservative method of calculating significance for pairwise

comparisons, which may mean that differences between small groups (e.g., higher degree by
research students) and small differences between large groups may be underestimated. The
survey also did not explicitly seek close-ended responses from students about whether teaching
staff, friends or family were the sources of learning about GenAI tools, which means the data on
these may also be underestimated. Finally, the authors acknowledge the general limitations of
online surveys with respect to self-selection bias and potentially limited internet access of target
participants during the time in which the survey was conducted (Bethlehem, 2010).

Some of the limitations in this study, however, also pave the way for future research to build upon
its findings. Future research could explore student perspectives on GenAI in other international
contexts and track how awareness and usage changes over time, especially across different
disciplines. Focus groups and semi-structured interviews are other ways in which qualitative data
on GenAI could be gathered in more detail, including practical examples of how these tools have
been used by students for learning and assessment. Gathering more student perspectives on
GenAI and its academic integrity implications would add significant value to the current literature.

11
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

Conclusion
GenAI tools have significantly disrupted teaching and learning practices in universities worldwide.
While there are reported positive benefits for GenAI tools in enhancing student learning and
accessibility, the lack of research into student perspectives of these tools to date limits the
conclusions that can be drawn about how students will engage in practice. By shedding light on
student perspectives, this study contributes to our broad understanding of how GenAI tools will
impact students during their university studies. As this research has explored, most students
surveyed in March 2023 had low knowledge, experience, and confidence with GenAI. Student
confidence increased with experience, although these rates also vary across disciplines and
across some student sub-groups.

As universities navigate this new landscape, it is crucial to consider both the potential benefits
and limitations of GenAI tools, ensuring that they are used ethically and appropriately to support
student learning and academic integrity. Continuous monitoring, evaluation, and adaptation of
teaching and assessment practices will be necessary to integrate GenAI tools effectively into
educational contexts while addressing the concerns raised by students and academic staff alike.

Conflict of Interest
The authors disclose that they have no actual or perceived conflicts of interest and have not
received any funding for this manuscript. They have also produced this manuscript without
artificial intelligence support.

References
Bethlehem, J. (2010). Selection bias in web surveys. International Statistical Review, 78(2), 161-
188. https://doi.org/10.1111/j.1751-5823.2010.00112.x

Bhatt, I., & MacKenzie, A. (2019). Just Google it! Digital literacy and the epistemology of
ignorance. Teaching in Higher Education, 24(3), 302-317.
https://doi.org/10.1080/13562517.2018.1547276

Brown, J. S., Collins, A., & Harris, G. (1978). Artificial intelligence and learning strategies. In F.
O. Harold (Ed.), Learning strategies (pp. 107-139). Academic Press.
https://doi.org/10.1016/B978-0-12-526650-5.50010-1

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., ... & Zhang, Y.
(2023). Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint
arXiv:2303.12712. https://doi.org/10.48550/arXiv.2303.12712

Butarbutar, R., Sauhenda, A. F., Marnina, H. R., & Wahyuniar, S. M. R. L. (2021). Challenges
and opportunities of accelerated digital literacy during the COVID-19 pandemic. Hong
Kong Journal of Social Sciences, 57, 160-168.
http://hkjoss.com/index.php/journal/article/view/424/420

https://ro.uow.edu.au/jutlp/vol20/iss6/12 12
 Kelly et al.: Generative artificial Intelligence

Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023). Evaluating the feasibility of ChatGPT
in healthcare: an analysis of multiple clinical and research scenarios. Journal of Medical
Systems, 47(1), 33. https://doi.org/10.1007%2Fs10916-023-01925-4

Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic
integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1-
12. https://doi.org/10.1080/14703297.2023.2190148

Crawford, J., Cowling, M., & Allen, K. A. (2023). Leadership is needed for ethical ChatGPT:
Character, assessment, and learning using artificial intelligence (AI). Journal of University
Teaching & Learning Practice, 20(3), Article 2. https://doi.org/10.53761/1.20.3.02

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: the state of the field.
International Journal of Educational Technology in Higher Education, 20(1), 1-22.
https://doi.org/10.1186/s41239-023-00392-8

Dunning, D. (2011). The Dunning–Kruger effect: On being ignorant of one's own ignorance.
In Advances in experimental social psychology (vol. 44, pp. 247-296). Academic Press.
https://doi.org/10.1016/B978-0-12-385522-0.00005-6

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... & Wright, R. (2023).
“So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges
and implications of generative conversational AI for research, practice and policy.
International Journal of Information Management, 71, 102642.
https://doi.org/10.1016/j.ijinfomgt.2023.102642

Evans, C., & Robertson, W. (2020). The four phases of the digital natives debate. Human
Behavior and Emerging Technologies, 2(3), 269-277. https://doi.org/10.1002/hbe2.196

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT:
Implications for educational practice and research. Innovations in Education and Teaching
International, 1-15. https://doi.org/10.1080/14703297.2023.2195846

Firat, M. (2023). What ChatGPT means for universities: Perceptions of scholars and students.
Journal of Applied Learning and Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.22

Forbes. (2023). What is ChatGPT? How AI is transforming multiple industries.

https://www.forbes.com/sites/qai/2023/02/01/what-is-chatgpt-how-ai-is-transforming-
multiple-industries/?sh=69b38fe6728e

Gaetano, C. (2023, May 22). ChatGPT passes CPA exam on V.4.0. Accounting Today.
https://www.accountingtoday.com/news/chatgpt-retakes-cpa-exam-as-v4-0-passes

García-Peñalvo, F. J. (2023). The perception of Artificial Intelligence in educational contexts after

the launch of ChatGPT: Disruption or panic? Education in the Knowledge Society, 24, 1-
9. https://doi.org/10.14201/eks.31279

Garito, M. A. (1991). Artificial intelligence in education: evolution of the teaching—learning

relationship. British Journal of Educational Technology, 22(1), 41-47. https://
doi.org/10.1111/j.1467-8535.1991.tb00050.x

13
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

Haensch, A. C., Ball, S., Herklotz, M., & Kreuter, F. (2023). Seeing ChatGPT Through Students'
Eyes: An Analysis of TikTok Data. arXiv preprint arXiv:2303.05349.
https://doi.org/10.48550/arXiv.2303.05349

Hess, F. (2023, February 8). Will ChatGPT be a blow to learning, or a boom? We’ll decide. Forbes.
https://www.forbes.com/sites/frederickhess/2023/02/08/will-chatgpt-be-a-blow-to-
learning-or-a-boon-well-decide/?sh=29824ba66651

Hutton, P. A. (2006). Understanding student cheating and what educators can do about it. College
Teaching, 54(1), 171-176. https://www.jstor.org/stable/27559254

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci,
G. (2023). ChatGPT for good? On opportunities and challenges of large language models
for education. Learning and Individual Differences, 103, 102274.
https://doi.org/10.1016/j.lindif.2023.102274

Khanal, J., & Gaulee, U. (2019). Challenges of international students from pre-departure to post-
study: A literature review. Journal of International Students, 9(2), 560-581.
https://doi.org/10.32674/jis.v9i2.673

Koetsier, J. (2023, March 14). GPT-4 beats 90% of lawyers trying to pass the bar. Forbes.
https://www.forbes.com/sites/johnkoetsier/2023/03/14/gpt-4-beats-90-of-lawyers-trying-
to-pass-the-bar/?sh=76617c930279

Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy
course for university students with diverse study backgrounds. Computers and Education:
Artificial Intelligence, 2, 100026. https://doi.org/10.1016/j.caeai.2021.100026

Krügel, S., Ostermaier, A., & Uhl, M. (2023). ChatGPT’s inconsistent moral advice influences
users’ judgment. Scientific Reports, 13(1), 4569. https://doi.org/10.1038/s41598-023-
31341-0

Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence literacy in
higher and adult education: A scoping literature review. Computers and Education:
Artificial Intelligence, 100101. https://doi.org/10.1016/j.caeai.2022.100101

Limna, P., Kraiwanit, T., Jangjarat, K., Klayklung, P., & Chocksathaporn, P. (2023). The use of
ChatGPT in the digital era: Perspectives on chatbot implementation. Journal of Applied
Learning and Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.32

Lodge, J. (2023, January 30). ChatGPT consistently fails (most parts of) the assessment tasks I
assign my students. Here’s why. LinkedIn. https://www.linkedin.com/pulse/chatgpt-
consistently-fails-most-parts-assessment-tasks-jason-m-
lodge/?trackingId=VMJksVXH6rIlXAycwVqZOQ%3D%3D

Lyerly, E. (2023). Utilizing ChatGPT to help students with disabilities. Disability Compliance for
Higher Education, 28(9), 2-7. https://onlinelibrary.wiley.com/doi/pdf/10.1002/dhe.31479

https://ro.uow.edu.au/jutlp/vol20/iss6/12 14
 Kelly et al.: Generative artificial Intelligence

McCoy, L. G., Nagaraj, S., Morgado, F., Harish, V., Das, S., & Celi, L. A. (2020). What do medical
students actually need to know about artificial intelligence? NPJ Digital Medicine, 3(1), 86.
https://www.nature.com/articles/s41746-020-0294-7

Murray, M. C., & Pérez, J. (2014). Unraveling the digital literacy paradox: How higher education
fails at the fourth literacy. Issues in Informing Science and Information Technology, 11,
85-100. http://iisit.org/Vol11/IISITv11p085-100Murray0507.pdf

Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using IBM SPSS.
McGraw-Hill Education (UK).

Perkins, M. (2023). Academic integrity considerations of AI Large Language Models in the post-
pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice,
20(2), Article 7. https://doi.org/10.53761/1.20.02.07

Popenici, S. A., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and
learning in higher education. Research and Practice in Technology Enhanced Learning,
12(1), 1-13. https://doi.org/10.1186/s41039-017-0062-8

Prensky, M. (2001). Digital natives, digital immigrants part 1. On the Horizon, 9(5), 1–6.
https://doi.org/10.1108/10748120110424816

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional
assessments in higher education? Journal of Applied Learning and Teaching, 6(1).
https://doi.org/10.37074/jalt.2023.6.1.9

Russell Group. (2023, 4 July). New principles on use of AI in education. Russell Group. Accessed
25 July 2023. https://russellgroup.ac.uk/news/new-principles-on-use-of-ai-in-education/

Shoufan, A. (2023). Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-
up Survey. IEEE Access. https://doi.org/10.1109/ACCESS.2023.3268224

Spataro, J. (2023, March 16). Introducing Microsoft 365 Copilot – your copilot for work. Microsoft.
https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-
copilot-for-work/

Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations
for academic integrity and student learning. Journal of Applied Learning and Teaching,
6(1). https://doi.org/10.37074/jalt.2023.6.1.17

Smith, E. E., & Storrs, H. (2023). Digital literacies, social media, and undergraduate learning: what
do students think they need to know? International Journal of Educational Technology in
Higher Education, 20(1), 1-19. https://doi.org/10.1186/s41239-023-00398-2

Smith, E. E., Kahlke, R., & Judd, T. (2020). Not just digital natives: Integrating technologies in
professional education contexts. Australasian Journal of Educational Technology, 36(3),
1-14. https://doi.org/10.14742/ajet.5689

Susnjak, T. (2022). ChatGPT: The end of online exam integrity? arXiv preprint arXiv:2212.09292.
https://doi.org/10.48550/arXiv.2212.09292

15
 Journal of University Teaching & Learning Practice, Vol. 20 [2023], Iss. 6, Art. 12

TEQSA. (2023, 17 February). TEQSA and Deakin University AI webinars launched. Tertiary
Education and Quality Standards Agency. https://www.teqsa.gov.au/about-us/news-and-
events/latest-news/teqsa-and-deakin-university-ai-webinars-launched

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B.
(2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots
in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-
023-00237-x

Udeogalanya, V. (2022). Aligning digital literacy and student academic success: Lessons learned
from COVID-19 pandemic. International Journal of Higher Education Management, 8(2).
https://doi.org/10.24052/IJHEM/V08N02/ART-4
UNESCO. (2022). Recommendations on the ethics of artificial intelligence. UNESCO. Accessed
25 July 2023. https://www.unesco.org/en/artificial-intelligence/recommendation-ethics

Willems, J., Farley, H., & Campbell, C. (2019). The increasing significance of digital equity in
higher education: An introduction to the Digital Equity Special Issue. Australasian Journal
of Educational Technology, 35(6), 1–8. https://doi.org/10.14742/ajet.5996

Xiao, P., Chen, Y., & Bao, W. (2023). Waiting, Banning, and Embracing: An Empirical Analysis
of Adapting Policies for Generative AI in Higher Education. arXiv preprint
arXiv:2305.18617. https://doi.org/10.48550/arXiv.2305.18617

Yu, Z. (2022). Sustaining student roles, digital literacy, learning achievements, and motivation in
online learning environments during the COVID-19 pandemic. Sustainability, 14(8), 4388.
https://doi.org/10.3390/su14084388

Zhao, Y., Llorente, A. M. P., & Gómez, M. C. S. (2021). Digital competence in higher education
research: A systematic literature review. Computers & Education, 168, 104212.
https://doi.org/10.1016/j.compedu.2021.104212

https://ro.uow.edu.au/jutlp/vol20/iss6/12 16
View publication stats