Student Performance in Writing Prompts for Text-based GenAI tools in a Research Methodology Course

Autores/as

Palabras clave:

Prompts, Artificial Intelligence, Interactions, Generic Competencies

Resumen

Prompts are essential for obtaining high-quality results in interactions with Generative Artificial Intelligence (GenAI). The precision and clarity of a prompt determine the relevance and usefulness of the generated response. This article explores how undergraduate students write prompts, using text-based tools to be assisted on their research proposals (n= 36). The results show that the group of students only reaches the knowledge level of Bloom's taxonomy in their interactions with GenAi. It is concluded that having access to is not enough. On the contrary, it is necessary to demonstrate various generic competencies, such as analysis, decision-making, and critical thinking. Moreover, the ability to formulate effective prompts is considered fundamental for maximizing the potential of GenAI in educational contexts, suggesting the need for specific training in the creation of clear and precise prompts to improve academic performance and research quality.

Citas

Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152

Bozkurt, A. (2024). GenAI et al.: Cocreation, authorship, ownership, academic ethics and integrity in a time of generative AI. Open Praxis, 16(1). DOI: https://doi.org/10.55982/openpraxis.16.1.654

Bozkurt, A., & Sharma, R. C. (2023). Generative AI and prompt engineering: The art of whispering to let the genie out of the algorithmic world. Asian Journal of Distance Education, 18(2), i–vii. DOI: https://doi.org/10.5281/zenodo.8174941

Calma, A., & Davies, M. (2020). Critical thinking in business education: current outlook and future prospects. Studies in Higher Education, 46(11), 2279–2295. https://doi.org/10.1080/03075079.2020.1716324

Carucci, R. (2024). Critical Thinking Is More Needed Than Ever. Forbes. https://www.forbes.com/sites/roncarucci/2024/02/06/in-the-age-of-ai-critical-thinking-is-more-needed-than-ever/

Grassini, S. (2023). Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings. Education Sciences. 13(7):692. https://doi.org/10.3390/educsci13070692

Haque, M.U., Dharmadasa, I., Sworna, Z.T., Rajapakse, R.N., & Ahmad, H. (2022). "I think this is the most disruptive technology": Exploring Sentiments of ChatGPT Early Adopters using Twitter Data. ArXiv, abs/2212.05856. https://ar5iv.labs.arxiv.org/html/2212.05856

Kojima, T., Gu, S., Reid, M., Matsuo, Y. & and Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in neural information processing systems, 35, 22199–22213. https://arxiv.org/abs/2205.11916

Liu, M., Ren, Y., Nyagoga, L.M., Stonier, F., Wu, Z. & Liang Yu, L. (2023). Future of education in the era of generative artificial intelligence: Consensus among Chinese scholars on applications of ChatGPT in schools. Future Educ. Res. 1,72–101. https://onlinelibrary.wiley.com/doi/epdf/10.1002/fer3.10

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H. & Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 1–35, 2023. https://dl.acm.org/doi/10.1145/3560815

Marr, B. (2023). A Short History Of ChatGPT: How We Got To Where We Are Today. Forbes. https://www.forbes.com/sites/bernardmarr/2023/05/19/a-short-history-of-chatgpt-how-we-got-to-where-we-are-today/

OpenAI. (2024). ChatGPT (June 15 version) [Large language model]. https://chat.openai.com

Rusdin, D., Mukminatien, N., Suryati, N., Laksmi, E. D., & Marzuki. (2023). Critical thinking in the AI era: An exploration of EFL students’ perceptions, benefits, and limitations. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2023.2290342

Schoepp, K. (2017). The state of course learning outcomes at leading universities. Studies in Higher Education, 44(4), 615–627. https://doi.org/10.1080/03075079.2017.1392500

Silva, A. de O., & Janes, D. dos S. (2022). The Emergence of ChatGPT and its Implications for Education and Academic Research in the 21st Century. Review of Artificial Intelligence in Education, 3(00), e06. https://doi.org/10.37497/rev.artif.intell.educ.v3i00.6

Sutton J, Austin Z. (2015). Qualitative Research: Data Collection, Analysis, and Management. Can J Hosp Pharm, 68(3),226-31. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4485510/

Thiga, M. M. (2024). Generative AI and the Development of Critical Thinking Skills. IRE Journals, 7(9), 83-90. https://www.irejournals.com/formatedpaper/1705580.pdf

Vera, F. (2020). Concepciones de docentes universitarios chilenos sobre el pensamiento crítico. Transformar, 1(1), 20–41. https://revistatransformar.cl/index.php/transformar/article/view/14

Vera, F. (2023a). Integración de la Inteligencia Artificial en la Educación superior: Desafíos y oportunidades. Transformar, 4(1), 17–34. https://www.revistatransformar.cl/index.php/transformar/article/view/84

Vera, F. (2023b). Integrating Artificial Intelligence (AI) in the EFL Classroom: Benefits and Challenges. Transformar, 4(2), 66–77. Recuperado a partir de https://revistatransformar.cl/index.php/transformar/article/view/93

Vera, F. (2024). Interacciones de Estudiantes de Grado con la Inteligencia Artificial Generativa: Estudio de Caso en un Tecnológico Mexicano. Transformar, 4(4), 5–19. de https://revistatransformar.cl/index.php/transformar/article/view/106

Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L. & Sun, H. (2023). Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Vol. 1, 2717–2739, Association for Computational Linguistics. https://aclanthology.org/2023.acl-long.153.pdf

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V. & Zhou, D.(2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824–24837. https://arxiv.org/abs/2201.11903

Zohrabi, M. (2013). Mixed Method Research: Instruments, Validity, Reliability and Reporting Findings. Theory and Practice in Language Studies, 3(2), 254–262. https://www.academypublication.com/issues/past/tpls/vol03/02/06.pdf

Descargas

Publicado

07-08-2024

Cómo citar

Vera, F. (2024). Student Performance in Writing Prompts for Text-based GenAI tools in a Research Methodology Course. Transformar, 5(2), 71–90. Recuperado a partir de https://revistatransformar.cl/index.php/transformar/article/view/129

Artículos más leídos del mismo autor/a

1 2 3 4 > >>