Student Performance in Writing Prompts for Text-based GenAI tools in a Research Methodology Course
Palabras clave:
Prompts, Artificial Intelligence, Interactions, Generic CompetenciesResumen
Prompts are essential for obtaining high-quality results in interactions with Generative Artificial Intelligence (GenAI). The precision and clarity of a prompt determine the relevance and usefulness of the generated response. This article explores how undergraduate students write prompts, using text-based tools to be assisted on their research proposals (n= 36). The results show that the group of students only reaches the knowledge level of Bloom's taxonomy in their interactions with GenAi. It is concluded that having access to is not enough. On the contrary, it is necessary to demonstrate various generic competencies, such as analysis, decision-making, and critical thinking. Moreover, the ability to formulate effective prompts is considered fundamental for maximizing the potential of GenAI in educational contexts, suggesting the need for specific training in the creation of clear and precise prompts to improve academic performance and research quality.
Citas
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152
Bozkurt, A. (2024). GenAI et al.: Cocreation, authorship, ownership, academic ethics and integrity in a time of generative AI. Open Praxis, 16(1). DOI: https://doi.org/10.55982/openpraxis.16.1.654
Bozkurt, A., & Sharma, R. C. (2023). Generative AI and prompt engineering: The art of whispering to let the genie out of the algorithmic world. Asian Journal of Distance Education, 18(2), i–vii. DOI: https://doi.org/10.5281/zenodo.8174941
Calma, A., & Davies, M. (2020). Critical thinking in business education: current outlook and future prospects. Studies in Higher Education, 46(11), 2279–2295. https://doi.org/10.1080/03075079.2020.1716324
Carucci, R. (2024). Critical Thinking Is More Needed Than Ever. Forbes. https://www.forbes.com/sites/roncarucci/2024/02/06/in-the-age-of-ai-critical-thinking-is-more-needed-than-ever/
Grassini, S. (2023). Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings. Education Sciences. 13(7):692. https://doi.org/10.3390/educsci13070692
Haque, M.U., Dharmadasa, I., Sworna, Z.T., Rajapakse, R.N., & Ahmad, H. (2022). "I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data. ArXiv, abs/2212.05856. https://ar5iv.labs.arxiv.org/html/2212.05856
Kojima, T., Gu, S., Reid, M., Matsuo, Y. & and Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in neural information processing systems, 35, 22199–22213. https://arxiv.org/abs/2205.11916
Liu, M., Ren, Y., Nyagoga, L.M., Stonier, F., Wu, Z. & Liang Yu, L. (2023). Future of education in the era of generative artificial intelligence: Consensus among Chinese scholars on applications of ChatGPT in schools. Future Educ. Res. 1,72–101. https://onlinelibrary.wiley.com/doi/epdf/10.1002/fer3.10
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H. & Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 1–35, 2023. https://dl.acm.org/doi/10.1145/3560815
Marr, B. (2023). A Short History Of ChatGPT: How We Got To Where We Are Today. Forbes. https://www.forbes.com/sites/bernardmarr/2023/05/19/a-short-history-of-chatgpt-how-we-got-to-where-we-are-today/
OpenAI. (2024). ChatGPT (June 15 version) [Large language model]. https://chat.openai.com
Rusdin, D., Mukminatien, N., Suryati, N., Laksmi, E. D., & Marzuki. (2023). Critical thinking in the AI era: An exploration of EFL students’ perceptions, benefits, and limitations. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2023.2290342
Schoepp, K. (2017). The state of course learning outcomes at leading universities. Studies in Higher Education, 44(4), 615–627. https://doi.org/10.1080/03075079.2017.1392500
Silva, A. de O., & Janes, D. dos S. (2022). The Emergence of ChatGPT and its Implications for Education and Academic Research in the 21st Century. Review of Artificial Intelligence in Education, 3(00), e06. https://doi.org/10.37497/rev.artif.intell.educ.v3i00.6
Sutton J, Austin Z. (2015). Qualitative Research: Data Collection, Analysis, and Management. Can J Hosp Pharm, 68(3),226-31. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4485510/
Thiga, M. M. (2024). Generative AI and the Development of Critical Thinking Skills. IRE Journals, 7(9), 83-90. https://www.irejournals.com/formatedpaper/1705580.pdf
Vera, F. (2020). Concepciones de docentes universitarios chilenos sobre el pensamiento crítico. Transformar, 1(1), 20–41. https://revistatransformar.cl/index.php/transformar/article/view/14
Vera, F. (2023a). Integración de la Inteligencia Artificial en la Educación superior: Desafíos y oportunidades. Transformar, 4(1), 17–34. https://www.revistatransformar.cl/index.php/transformar/article/view/84
Vera, F. (2023b). Integrating Artificial Intelligence (AI) in the EFL Classroom: Benefits and Challenges. Transformar, 4(2), 66–77. Recuperado a partir de https://revistatransformar.cl/index.php/transformar/article/view/93
Vera, F. (2024). Interacciones de Estudiantes de Grado con la Inteligencia Artificial Generativa: Estudio de Caso en un Tecnológico Mexicano. Transformar, 4(4), 5–19. de https://revistatransformar.cl/index.php/transformar/article/view/106
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L. & Sun, H. (2023). Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Vol. 1, 2717–2739, Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.153.pdf
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V. & Zhou, D.(2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824–24837. https://arxiv.org/abs/2201.11903
Zohrabi, M. (2013). Mixed Method Research: Instruments, Validity, Reliability and Reporting Findings. Theory and Practice in Language Studies, 3(2), 254–262. https://www.academypublication.com/issues/past/tpls/vol03/02/06.pdf
