Hyesun Cho

DOI: Vol.26(No.2) 9-32, 2023

In this study, ChatGPT was asked to determine the likeliness of a given sentence being written by a native speaker of English. A total of 2,000 prompt sentences were provided: 1,000 from an English native-speaker corpus (LOCNESS) and 1,000 from a Korean EFL learner corpus (GLC). It was found that ChatGPT determined the sentences from the native corpus to be more nativelike than those from the nonnative corpus. The classification accuracy (proportion of native sentences classified as native and nonnative sentences classified as nonnative) was 62.75%. Most of the responses were simple phrases, such as “very likely” or “not very likely,” but in some cases, ChatGPT also provided reasons for a judgment or suggestions for correction. ChatGPT’s answers show that it considers grammaticality and plausibility as the criteria for determining the nativelikeness of a sentence, conforming to the literature. Following the initial testing, ChatGPT was fine-tuned with 1,600 sentences from the prompt data and tested with the remaining 400 sentences. After fine-tuning, the classification accuracy increased to 96.75%. This result showed that the fine-tuning of ChatGPT can enhance accuracy. Finally, the study briefly discussed the characteristics of the answers provided by GPT-4 in comparison with those provided by GPT-3.5.

ChatGPT, AI, conversational AI, nativelikeness, EFL writing, fine-tuning, grammaticality, plausibility

Xiaoran Wang

DOI: Vol.26(No.2) 33-58, 2023

This study examined Chinese college English learners’ perspectives of the automated writing feedback generated by Pigai, the most widely used automatic writing evaluation (AWE) tool in China. Additionally, this study investigated the differences in Chinese college learners’ perspectives of Pigai feedback according to their proficiency levels. Student participants included 70 students from two classes who took the same college English course at one Chinese university. The students in class A were higher level EFL learners, while students in class B were lower level EFL learners. In both classes, Pigai was incorporated into the process-based writing practice along with teacher feedback. Students’ perspectives of the automated writing feedback generated by Pigai were investigated via survey after the course. The findings revealed that all participants demonstrated highly positive perspectives. Besides, both higher- and lower-proficiency groups exhibited similar perspectives regardless of their proficiency levels. Moreover, most participants expressed a preference for receiving both Pigai feedback and teacher feedback integrated into their writing process. Practical implications were also discussed for writing instruction using AWE technology in EFL writing classes such as AWE tools can be used with teacher feedback together to assist writing for classes with students of various proficiency levels.

automated writing evaluation, AWE, Pigai, students’ perspectives, feedback

박휴용

DOI: Vol.26(No.2) 59-83, 2023

This paper discusses the key technologies and limitations of AI-based chatbots, as well as their types and functions, and their contributions to English language learning. Two specific research questions were posed: first, what are the background and key technologies of AI-based language learning, and what are the technological limitations? Second, what are the conversation processing technologies and types of AI-based chatbots, their roles in English language learning, and pedagogical challenges? Based on the discussion, the main conclusions are as follows: first, the core principles of AI chatbots are human-robot interaction (HRI) technology and natural language processing (NLP) technology, which are evolving into various interactive and game-like models, tailored to the diversity and individuality of learners. Second, natural language processing technology has overcome initial linguistic, technical, and practical challenges and is increasingly being utilized for English language learning through the emergence of ChatGPT. Third, AI personal assistant tools can become effective lifestyle English learning tools as natural language processing technology continues to advance. Fourth, AI-based English learning chatbots are effective in pattern matching, conversation database, knowledge base, and language generation models, in that order. Finally, the role and pedagogical challenges of AI chatbots as English language learning tools are discussed.

AI, chatbot, natural language processing, English learning, chatGPT

이대기 Lee Dae-ki , 성민창 Sung Min-chang

DOI: Vol.26(No.2) 84-103, 2023

This study investigated the usage of wh-questions by a chatbot for English education, within the framework of the 2022 revised national curriculum, which highlights the use of AI technology and edutech in English language teaching and learning. Initially, we analyzed both the lexical and structural features of the example sentences at the elementary English level in the curriculum and identified ten types of wh-questions that would be suitable for elementary English. With reference to the ten types of wh-questions, we explored the conversational experiences provided by AI Pengtalk, a chatbot widely used by elementary English learners. In particular, we examined the formal and functional characteristics of 185 wh-questions in the chatbot’s speaking task. Notable parallels and divergences were identified between the chatbot and elementary English textbooks. For example, both frequently used what-questions, while only the chatbot regularly employed why-questions. Furthermore, the chatbot was found to incorporate at least five different types of wh-questions across all conversation topics, providing many opportunities to practice various roles in meaningful contexts. These findings may contribute to improving chatbots for foreign language learners and advancing edutech-based English education programs.

chatbot, edutech, wh-question, 2022 revised national curriculum of English, elementary English

최윤희 Choe Yoonhee

DOI: Vol.26(No.2) 104-132, 2023

This study investigates the influence of ChatGPT on Korean pre-service English teachers' summary writing skills, utilizing a qualitative research approach. The participants included 25 pre-service English teachers, each assigned the task of engaging with ChatGPT during their summary writing revision. Data were collected through participants’ initial drafts, revised summaries, reflective journals, in-depth interviews, and surveys. Thematic analysis was employed to scrutinize the influence of ChatGPT on participants’ summary writing. The results identified key themes categorized into pedagogical, technical and affective aspects. On the pedagogical aspect, the utilization of ChatGPT led to improved lexical choices, enhanced organizational revisions, and heightened awareness of content inclusion and exclusion. Additionally, participants benefited from continuous, individualized, and technically immediate feedback. These pedagogical and technical benefits subsequently influenced participants’ affective perspectives towards ChatGPT, with reports of reduced writing anxiety, increased confidence in summary writing, and a more positive attitude towards the willingness to use of ChatGPT. This study emphasizes the value of ChatGPT as a tool for enhancing summary writing competencies and fostering positive pedagogical and affective attitudes among pre-service English teachers. It underlines the potential for integrating ChatGPT into English writing teaching methodologies.

ChatGPT, English summary writing, process-based writing, pre-service English teachers