Journal of Teaching of English
Volume 8 No. 4
e-ISSN: 2548-6810
Exploring The Impact of ChatGPT on English Education Department
Student’s Motivation and Performance
Mohammad Yusfi Afkarin1, Candra Hadi Asmara2
[email protected]1,2 Universitas Muhammadiyah Gresik, Indonesia
ABSTRACT
This study investigates the impact of the AI tool ChatGPT on the motivation and academic
performance of higher education students. Anchored in the theoretical framework of Self-
Determination Theory (SDT), the research hypothesizes that the use of ChatGPT would
positively influence student learning outcomes by fulfilling the psychological needs of
autonomy, competence, and relatedness. Utilizing a mixed-methods approach, the study
involved a quantitative analysis comparing the motivation scores and GPAs of ChatGPT
users (n = 25) with non-users (n = 10), and a qualitative analysis exploring students'
experiences and perceptions of using ChatGPT. The participant was 35 students of 7th
semester of English Education Department in University of Muhammadiyah Gresik. The
results revealed that students who utilized ChatGPT demonstrated significantly higher
motivation and academic performance, with mean motivation scores of 44.88 for users versus
17.60 for non-users, and mean GPAs of 3.63 for users compared to 3.06 for non-users.
Qualitative findings further illuminated these results, with themes such as 'Enhanced
Academic Efficiency and Productivity' and 'Better Understanding and Independent Learning'
emerging, underscoring how ChatGPT facilitated more efficient and in-depth learning
experiences. The study confirms the positive impact of ChatGPT on student motivation and
performance, aligning with and expanding upon the principles of SDT. It highlights the
potential of integrating AI tools in educational strategies, while also acknowledging the need
for cautious interpretation due to limitations like the violation of homoscedasticity
assumptions. This research opens new avenues for the strategic use of AI in education and
sets a foundation for future exploration in this evolving field.
Keywords:
ChatGPT, Student Motivation, Student Performance
1. Introduction
In recent years, artificial intelligence (AI) has emerged as a transformative force
across various sectors, including education. Traditional teaching methods, which have
long dominated educational environments, have undergone a paradigm shift in the
21st century due to the integration of AI-driven tools. This shift represents a move
toward a more inclusive, flexible, and effective learning environment.
The application of AI in education has shown promising results, offering
personalized and dynamic learning experiences. Early uses of AI in education were
observed in computer-based training systems, but it has since evolved into a key
Journal of Teaching of English Vol. 8 No. 4
element in modern educational contexts. For instance, the study conducted by Neo et
al. (2022) highlighted the potential of AI, particularly chatbots like MERLIN, to
enhance student learning and motivation. AI's impact on education is not confined to
one geographical location. Dr. Padma C and Dr. Rama C's study (2022) in the Indian
education industry emphasized AI's disruptive methods in personalizing learning
experiences. Moreover, AI has extended its influence to medical education, with Nagi
et al.'s (2023) scoping review revealing its potential to improve individualized
learning experiences. Tiwari (2023) conducted a comprehensive literature review on
AI and machine learning in education, underlining AI's potential to personalize and
enhance students' educational experiences. However, the review also stressed the
need for further research to understand AI's promise and limitations in education
fully.
Motivation plays a pivotal role in student engagement and performance.
Psychological factors such as autonomy, competence, and relatedness, as highlighted
by Ryan & Deci (2000), significantly influence motivation. Another aspect impacting
motivation includes engaging activities, exemplary behavior demonstration,
enhancing enthusiasm, offering affirmative feedback, and fostering strong
relationships (Agus et al., 2022). Additionally, there is a strong association between
student performance and their motivation (Juliana et al., 2022). Furthermore, student
performance influenced by various factors, including individual characteristics,
family background, institutional support, and teacher-student interactions, as noted
by Jou et al. (2022).
In conventional teaching methods, sustaining student interest can be a
challenge. However, AI tools, like ChatGPT, offer a potential solution by providing
personalized, adaptive learning experiences. Previous research has shown the positive
effects of AI on understanding challenging subjects and improving problem-solving
abilities. For instance, studies conducted by Jiao et al. (2022), Bressane et al. (2022)
,Jokhan et al. (2022), and Huang (2022) have highlighted these benefits. Furthermore,
Corsten & Skousen (2023) have emphasized the advantages of AI-generated questions
in evaluations.
The integration of AI in education has the potential to enhance both student
performance and motivation. Various AI-driven tools and techniques cater to diverse
learning needs. Research indicates the positive impact of interactive learning methods
(Trajkovik et al., 2018) and the importance of fostering productive online learning
communites (Ho et al., 2023). ChatGPT, developed by OpenAI, stands out for its
conversational abilities and adaptability. Its applications span various fields,
including medical and agriculture, making it a suitable choice for educational contexts
(Williams, 2023). ChatGPT is recognized for its conversational skill. In contrast to
conventional AI, it communicates via chat, responding to questions and even pointing
out errors, making it suitable for a variety of uses, particularly in education. Its
application in medical, helping with ophthalmic jobs (Ting et al., 2023) and in
agriculture, for decision-making and production optimization (Siche & Siche, 2023),
both serve as examples of its adaptability. The real-time feedback and conversational
aspect of ChatGPT in education suggest a revolutionary potential in improving
learning experiences.
384
Journal of Teaching of English Vol. 8 No. 4
While AI integration in education shows promise, there are limitations and
gaps in current research. Many studies have been conducted in controlled settings,
potentially limiting their applicability to real classrooms. Additionally, most research
has focused on short-term outcomes, raising questions about the long-term
effectiveness of AI-driven educational tools. The lack of diversity in research settings
is another concern, as studies have often been conducted in well-resourced
institutions.
This research addresses these gaps by conducting a comparative study between
users and non-users of AI, such as ChatGPT, in the English Education Department at
the University of Muhammadiyah Gresik. This inclusive approach ensures a more
equitable and representative examination of ChatGPT's impact on a diverse range of
student populations and socioeconomic levels, contributing to a comprehensive
understanding of AI's role in education.
2. Methods
A mixed-method research design was used in this study to investigate
ChatGPT's impact on student performance and motivation. To collect data at one
specific moment in time, a cross-sectional survey methodology was utilized. Both
students that use and did not use ChatGPT for academic purposes represented the
target population. The participants comprised students from the English Education
Department at the University of Muhammadiyah Gresik, specifically those in their 7th
semester. As it stated in Table 1, total of 35 students were selected to ensure a
representative sample. The interview by open-ended questions were done by
randomly selected.
Table 1 Frequency of Participants' Gender
Frequency Percent Valid Percent Cumulative Percent
Male 4 11.4 11.4 11.4
Female 31 88.6 88.6 100
Total 35 100 100
Open-ended questions provided qualitative insights, while scaled items
measured motivation on ChatGPT. Demographic information is also included in the
questionnaire. This study employed a structured questionnaire grounded in the Self-
Determination Theory (SDT) framework by Ryan and Deci (2000, 2017, 2019) to
investigate ChatGPT's impact on student motivation. The questionnaire in Table 2 was
thoughtfully adapted to align with SDT's core elements: Autonomy, Competence, and
Relatedness. It encompassed demographic inquiries for comparative analysis,
assessed ChatGPT's academic usage frequency, explored autonomy through
questions like "I feel that using ChatGPT allows me to study topics that I am
personally interested in," examined competence via items like "ChatGPT contributes
to my understanding of complex topics," and probed relatedness and motivation. To
investigate student performance, researcher gathered their grade point average (GPA)
for the whole semester, this aimed to get an overview of the learning outcomes that
385
Journal of Teaching of English Vol. 8 No. 4
students had achieved during college. Additionally, open-ended queries gathered
qualitative insights (table 3).
Table 2 Questionnaire Details
No Section Question Items
1 Q1 – Q3 Demographic Information
2 Q4 – Q5 General Usage of ChatGPT
3 Q6 – Q8 Autonomy
4 Q9 – Q11 Competence
5 Q12 – Q14 Relatedness
6 Q15 – Q17 Impact on Motivation
Table 3 Open-Ended Questions Details
No Question Details
1 Q1 What types of academic tasks do you use ChatGPT for?
2 Q2 What do you like most about using ChatGPT for your studies?
3 Q3 How could ChatGPT improve or support your academic motivation and
performance?
The questionnaires' validity and reliability were ensured through expert
reviews, and being tested by using Cronbach's alpha test.
Table 4 Reliability Test for Questionnaire
Reliability Statistics
Cronbach's N of
Alpha Items
0.976 16
In Table 4, Cronbach Alpha for this questionnaire was calculated as 0,976 which
means it has high score. By this score, it could be concluded that this questionnaire
was reliable and could be used to conduct this research.
Descriptive statistics was employed to analyze quantitative data from closed-
ended questions in order to gather demographic data and response patterns. To assess
the impact of ChatGPT hypotheses, the users and non-users of ChatGPT were
compared with their total result in motivation questionnaire and their grade point
average (GPA) by using individual sample T-Test. Moreover, Thematic analysis, by
identifying themes and patterns, was used to the qualitative data obtained from open-
ended questions.
3. Result
General usage of ChatGPT
Table 5 The Use of ChatGPT
Frequency Percent Valid Percent Cumulative Percent
Yes 25 71.4 71.4 71.4
No 10 28.6 28.6 100
386
Journal of Teaching of English Vol. 8 No. 4
Total 35 100 100
Table 6 Frequency of ChatGPT Usage
Frequency Percent Valid Percent Cumulative Percent
Never 10 28.6 28.6 28.6
Once in a While 6 17.1 17.1 45.7
Sometimes 11 31.4 31.4 77.1
Almost Always 8 22.9 22.9 100
Total 35 100 100
Table 5 clearly demonstrates that the percentage of students utilizing ChatGPT
(25 students) exceeds those who did not use it (10 students). The data presented in
Table 6 shows the varying frequency of ChatGPT usage across students. It has been
observed that a majority of the students occasionally utilize ChatGPT. Evidence
indicates that 10 students who did not use ChatGPT is significantly higher numbers
in comparison to 8 students who almost always and 6 students once in while utilize
ChatGPT. Nevertheless, when considering the whole data, users of ChatGPT show a
higher frequency compared to students who did not use it.
Impact on Student Motivation
Table 7 Group Statistic in Student Motivation
Students N Mean Std. Deviation Std. Error Mean
QMotivation Users 25 44.88 4.604 .921
Non-users 10 17.6 1.265 .400
Table 8 T-Test on Student Motivation
Levene's t-test for Equality of Means
Test for
Equality of
Variances
F Sig. t df Sig. Mean Std. 95%
(2- Differ Error Confidence
tailed) ence Differ Interval of the
ence Difference
Lower Upper
QMotivation 12.5 <.001 27.175 30.972 <.001 27.28 1.004 25.233 29.327
06
In Table 7, it can be observed that the average scores on the motivation
questionnaire for ChatGPT users and non-users are quite different, with a mean of
44.88 for users and 17.60 for non-users.
However, it's important to note that the results of Levene's test in Table 8
indicate that the data does not meet the assumption of homogeneity (unequal
variances), with a significance value (Sig.) of 0.001. This suggests that the variances
387
Journal of Teaching of English Vol. 8 No. 4
between the groups of ChatGPT users and non-users are significantly different, which
is a violation of the assumption for the t-test.
The results of the T-test show that the p-value is 0.001 (p < 0.05). A p-value
smaller than the predetermined significance level (α = 0.05) indicates that the
difference in mean scores on the motivation questionnaire between ChatGPT users
and non-users is statistically significant.
Therefore, we can conclude that ChatGPT users have significantly higher
motivation scores compared to non-users. This finding suggests that the use of
ChatGPT may positively influence student motivation.
However, it's important to consider the violation of the equal variances
assumption as a limitation of the analysis. This violation may impact the interpretation
of the T-test results, and it's advisable to exercise caution when generalizing these
findings. Additionally, further research could explore the underlying reasons for the
observed differences in motivation scores between the two groups.
Impact on Student Performance
Table 9 Group Statistic in Student Performance
Students N Mean Std. Deviation Std. Error Mean
QPerformance Users 25 3.63 .089 .017
Non-users 10 3.06 .513 .162
Table 10 T-Test on Student Performance
Levene's t-test for Equality of Means
Test for
Equality of
Variances
F Sig. t df Sig. Mean Std. 95%
(2- Differ Error Confidence
tailed) ence Differ Interval of the
ence Difference
Lower Upper
QPerformance 20.3 <.001 5.479 33 <.001 .57180 .10435 .35949 .78411
62
In Table 9, it can be observed that the average GPA of ChatGPT users (3.63) is
higher than that of non-ChatGPT users (3.06). This difference is quite significant, with
ChatGPT users having an average GPA that is approximately 0.57 points higher than
non-ChatGPT users.
However, it should be noted that the results of the Levene's test in Table 10
indicate that the data does not meet the assumption of homogeneity (unequal
variances), with a significance value (Sig.) of 0.001. This implies that the variances
between the ChatGPT user and non-ChatGPT user groups are significantly different.
Therefore, caution is warranted when interpreting the results of the T-test.
The T-test results indicate that the p-value is 0.001 (p < 0.05). A p-value smaller
than the predetermined significance level (α = 0.05) suggests that the difference in
388
Journal of Teaching of English Vol. 8 No. 4
average GPA between the ChatGPT user and non-ChatGPT user groups is statistically
significant.
Consequently, we can conclude that ChatGPT users have a higher average GPA
compared to non-ChatGPT users, and this difference is not merely due to chance.
These findings may have practical implications, such as indicating that the use of
ChatGPT may positively contribute to students' academic performance.
However, it is important to remember that these results are based on data that
may exhibit unequal variances, which is a limitation in the analysis. Therefore, caution
should be exercised when generalizing these findings, and it is essential to
acknowledge the presence of other factors that may influence students' GPA but were
not considered in this study.
Qualitative Insight from Open-Ended Questions
The thematic analysis of interviews conducted with all respondents revealed
that there are several points that refer to the main theme "Impact of ChatGPT on
Students' Motivation and Academic Performance." There are approximately 7 sub-
themes that support the main theme, which are:
1. Enhanced Academic Efficiency and Productivity
Many respondents cited the use of ChatGPT as a means to enhance efficiency
and productivity in academic tasks, such as research, essay writing, and concept
comprehension.
2. Improved Access and Quality of Information
ChatGPT is credited with providing instant access to diverse and relevant
information, which is seen as a key factor in strengthening academic performance.
3. Support in Time Management and Organization
Some respondents indicated that ChatGPT assists them in better time management
and organization of study materials, directly supporting an improvement in academic
performance.
4. Better Understanding and Independent Learning
ChatGPT is identified as a tool that supports better comprehension of complex
material and encourages independent learning.
5. Influence on Language Skills and Communication
ChatGPT is also acknowledged as a useful tool in enhancing language skills and
communication, especially for those learning a new language or needing assistance in
writing.
6. Increased Motivation for Learning
Responses indicate that ChatGPT has played a role in enhancing learning motivation
through more dynamic engagement with subject matter and broader accessibility to
learning resources.
7. Resistance to Technology Use
A number of respondents expressed resistance to using technology like ChatGPT,
relying on traditional learning methods and not reporting any change in motivation
or academic performance.
4. Discussion
389
Journal of Teaching of English Vol. 8 No. 4
This study provides a detailed exploration into the impact of ChatGPT on
higher education students' motivation and academic performance, integrating
quantitative findings with rich qualitative insights and established theoretical
frameworks.
Consistent with the Self-Determination Theory (SDT) by Ryan and Deci (2000),
the quantitative data demonstrated a significant positive correlation between the use
of ChatGPT and student motivation, with ChatGPT users scoring notably higher
(mean: 44.88) compared to non-users (mean: 17.60). This was mirrored in academic
performance, with ChatGPT users achieving higher GPA scores (mean: 3.63) than non-
users (mean: 3.06), aligning with studies by Neo et al. (2022)and Dr. Padma C and Dr.
Rama C (2022).
The qualitative data revealed several key themes that explain the quantitative
findings. Participants frequently cited ‘Enhanced Academic Efficiency and
Productivity’ as a significant benefit of using ChatGPT, noting that it facilitated
quicker and more effective research and assignment completion. Additionally, the
theme of ‘Better Understanding and Independent Learning’ emerged, with students
reporting that ChatGPT helped clarify complex concepts, fostered independent
inquiry, and personalized their learning experiences. These qualitative insights align
with the SDT's emphasis on autonomy and competence, suggesting that ChatGPT not
only aids in information acquisition but also in cultivating a more engaging and self-
driven learning environment.
Combining these qualitative narratives with the quantitative results offers a
comprehensive picture of ChatGPT's impact. The higher motivation and GPA scores
among ChatGPT users can be partly attributed to the tool’s ability to enhance learning
efficiency and promote deeper understanding, as described by the study participants.
Echoing Tiwari (2023), Jokhan et al. (2022), and Huang (2022), this study
underscores the practical utility of integrating AI tools like ChatGPT in educational
settings. However, the noted unequal variances in the data, a concern raised by
Corsten & Skousen (2023), suggests that these findings should be interpreted with
some caution.
Acknowledging the limitations, including statistical concerns, the study
highlights the need for further research. Future studies could focus on exploring these
findings in diverse educational settings and over longer periods to fully understand
the long-term impact of AI tools in education.
5. Conclusion
This research began with the hypothesis, rooted in the principles of Self-
Determination Theory (SDT), that the integration of AI tools like ChatGPT in higher
education would significantly enhance student motivation and academic
performance. This hypothesis was anchored in the belief that ChatGPT could satisfy
the key psychological needs identified by SDT – autonomy, competence, and
relatedness – thus positively influencing student learning outcomes.
The findings that is presented have not only validated this hypothesis but also
provided a deeper understanding of the dynamic role of AI in education. The
empirical data revealed that students using ChatGPT showed significantly higher
motivation scores and GPAs compared to their counterparts who did not use the tool.
390
Journal of Teaching of English Vol. 8 No. 4
Furthermore, the qualitative data, with themes like 'Enhanced Academic Efficiency
and Productivity' and 'Better Understanding and Independent Learning', illustrated
how ChatGPT contributes to a more efficient, personalized, and in-depth educational
experience. These outcomes demonstrate a remarkable convergence between our
theoretical expectations and the practical realities observed in the study.
As we look to the future, the potential for extending this research and applying
its findings is vast. Future studies could explore the long-term impact of AI tools like
ChatGPT in various educational contexts, examining their influence across different
academic disciplines and student populations. The practical application of these
findings also presents exciting prospects. Integrating AI into different facets of
education, from curriculum development to teacher training and student support,
could significantly enhance the educational landscape.
References
Agus, W., Pratiwi, E., Sapan, Y. T., & Agustina, S. (2022). The Effect of SEA- Teacher
Mentoring on The Students ’ Intrinsic Motivation in Teaching Practice. Journal of
Teaching of English, 7(4).
Bressane, A., Spalding, M., Zwirn, D., Loureiro, A. I. S., Bankole, A. O., Negri, R. G.,
de Brito Junior, I., Formiga, J. K. S., Medeiros, L. C. de C., Pampuch Bortolozo, L.
A., & Moruzzi, R. (2022). Fuzzy Artificial Intelligence—Based Model Proposal to
Forecast Student Performance and Retention Risk in Engineering Education: An
Alternative for Handling with Small Data. Sustainability (Switzerland), 14(21).
https://doi.org/10.3390/su142114071
C, D. P., & C, D. R. (2022). A Study of Artificial Intelligence in Education System &
Role of AI in Indian Education Sector. Interantional Journal of Scientific Research in
Engineering and Management, 06(03), 1–9. https://doi.org/10.55041/ijsrem11854
Corsten, D., & Skousen, B. (2023). ‘Get AI Answer?’ – Exploring the Effects of
Embedding ChatGPT in Exams to Improve Student Learning Outcomes. SSRN
Electronic Journal, March, 1–12. https://doi.org/10.2139/ssrn.4409035
Ho, V. T., Tran, V. D., & De Nguyen, V. (2023). Examining the factor structure of the
teachers’ sense of efficacy scale in the Vietnamese educational context.
International Journal of Education and Practice, 11(1).
https://doi.org/10.18488/61.v11i1.3257
Huang, L. (2022). The Establishment of College Student Employment Guidance
System Integrating Artificial Intelligence and Civic Education. Mathematical
Problems in Engineering, 2022. https://doi.org/10.1155/2022/3934381
Jiao, P., Ouyang, F., Zhang, Q., & Alavi, A. H. (2022). Artificial intelligence-enabled
prediction model of student academic performance in online engineering
education. Artificial Intelligence Review, 55(8), 6321–6344.
https://doi.org/10.1007/s10462-022-10155-y
Jokhan, A., Chand, A. A., Singh, V., & Mamun, K. A. (2022). Increased Digital
391
Journal of Teaching of English Vol. 8 No. 4
Resource Consumption in Higher Educational Institutions and the Artificial
Intelligence Role in Informing Decisions Related to Student Performance.
Sustainability (Switzerland), 14(4). https://doi.org/10.3390/su14042377
Jou, Y. T., Mariñas, K. A., & Saflor, C. S. (2022). Assessing Cognitive Factors of
Modular Distance Learning of K‐12 Students Amidst the COVID‐19 Pandemic
towards Academic Achievements and Satisfaction. Behavioral Sciences, 12(7).
https://doi.org/10.3390/bs12070200
Juliana, R., Rohmana, & Agustina, S. (2022). The Corelation between Students
Motivation and their English Learning Achievement during COVID-19
Pandemic. Journal of Teaching of English, 7(4), 70–77.
Nagi, F., Salih, R., Alzubaidi, M., Shah, H., Alam, T., Shah, Z., & Househ, M. (2023).
Applications of Artificial Intelligence (AI) in Medical Education: A Scoping
Review. Studies in Health Technology and Informatics, 305.
https://doi.org/10.3233/SHTI230581
Neo, M., Lee, C. P., Tan, H. Y. J., Neo, T. K., Tan, Y. X., Mahendru, N., & Ismat, Z.
(2022). Enhancing Students’ Online Learning Experiences with Artificial
Intelligence (AI): The MERLIN Project. International Journal of Technology, 13(5).
https://doi.org/10.14716/ijtech.v13i5.5843
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and Extrinsic Motivations: Classic
Definitions and New Directions. Contemporary Educational Psychology, 25(1).
https://doi.org/10.1006/ceps.1999.1020
Siche, R., & Siche, N. (2023). The language model based on sensitive artificial
intelligence - ChatGPT: Bibliometric analysis and possible uses in agriculture
and livestock. Scientia Agropecuaria, 14(1).
https://doi.org/10.17268/sci.agropecu.2023.010
Ting, D. S. J., Tan, T. F., & Ting, D. S. W. (2023). ChatGPT in ophthalmology: the
dawn of a new era? In Eye (Basingstoke). https://doi.org/10.1038/s41433-023-
02619-4
Tiwari, R. (2023). The integration of AI and machine learning in education and its
potential to personalize and improve student learning experiences.
INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING
AND MANAGEMENT, 07(02). https://doi.org/10.55041/ijsrem17645
Trajkovik, V., Malinovski, T., Vasileva-Stojanovska, T., & Vasileva, M. (2018).
Traditional games in elementary school: Relationships of student’s personality
traits, motivation and experience with learning outcomes. PLoS ONE, 13(8).
https://doi.org/10.1371/journal.pone.0202172
Williams, A. E. (2023). Has OpenAI Achieved Artificial General Intelligence in
ChatGPT? Artificial Intelligence and Applications, February, 1–15.
https://doi.org/10.47852/bonviewaia3202751
392
