Enhancing Mixed Methods Research with your generative AI assistant – A Tutorial and Evaluation for Scholars

Enhancing Mixed Methods Research with your generative AI assistant – A Tutorial and Evaluation for Scholars | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Enhancing Mixed Methods Research with your generative AI assistant – A Tutorial and Evaluation for Scholars Celeste Combrinck This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4176435/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Scholars from all disciplines can benefit from learning how to use generative Artificial Intelligence (GAI or AI) for data analysis. The current article used real data to demonstrate the analysis and synthesis of mixed methods research (MMR) data with generative AI. I also explore how reliable and valid data outputs are and how to improve the use of generative AI for research. The demonstration data used emanated from a study done in South Africa, with a quantitative sample size of 969 first-year engineering students and, for the qualitative part, 14 second-year students. In the current article, I compare my original analysis to ChatGPT results. Generative AI is a mind tool that is ideal when utilised with human insight to check the cohesion, consistency, and accuracy of the data analysis. The current content is geared towards enhancing methodological application regardless of field or discipline and includes access to a prompt library and examples of using outputs. For the qualitative analysis, I found that ChatGPT could detect similar themes but missed some, and its write-up was shallower than our human version. The quantitative analysis was accurate for the descriptive statistics, but the researcher had to use best judgment to select the correct inferential analysis. A quantitative and qualitative analysis should be conducted separately in generative AI before asking the bot for help with mixed methods research. I give guidelines and a tutorial on how to use chatbots in an ethically responsible and scientifically sound manner for research in social and human sciences. Generative Artificial Intelligence (AI or GAI) Chat Generative Pre-Trained Transformer (ChatGPT) also known as Chad Mixed Methods Research (MMR) Large-Language Models (LLMs) Chatbots data analysis techniques Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Introduction Recent advances in generative Artificial Intelligence (AI) are changing how we conduct scholarly research. We are challenged to use new tools in all educational disciplines [ 1 – 4 ]. The rapidly shifting technological landscape requires us to experiment with new tools, share what we have learned to support academics and students and engage in intellectual curiosity [ 5 , 6 ]. Therefore, I showcase in this article how to use a generative AI platform to analyse quantitative and qualitative data and integrate findings for mixed methods research (MMR) using generative AI as an assistant. Chat Generative Pre-Trained Transformer (ChatGPT) has emerged as a dominant role player in the generative AI field, and higher institutions are reporting that students and academics are using the platform increasingly [ 7 – 9 ]. Some researchers [ 10 ] focus on the positive role that generative AI can play in higher education, for example, aiding in managing large classes and online learning. Other researchers focus on how generative AI will impact assessment [ 11 ], how it can be used to promote student learning and motivation [ 12 ] and which frameworks to use when integrating Chatbots into teaching and learning [ 13 ]. Most articles conclude that we need more research and guidelines on using generative AI in higher education [ 13 , 14 ]. The current article is not the first exploration of how to use generative AI in higher education [ 15 ]; many good publications exist [ 16 , 17 ]. The current article adds a new approach by demonstrating the application of generative AI in mixed methods research (MMR) and providing prompts and guidelines for use. My work here is intended to be a resource for students and the academy, showing how generative AI can feed into the research process for academic advancement [ 17 ]. Consequently, I use the following terms, which are distinct but related: generative Artificial Intelligence (AI or GAI), Chatbots and Large-Language Models (LLMs). By generative Artificial Intelligence (AI), I mean AI systems trained to produce new content based on but divergent from their training data [ 18 ]. Chatbots interface AI and humans by applying LLMs to interact through text or voice [ 19 ]. LLMs serve as the basis for many types of generative AI and were designed to understand and function through human language [ 20 ]. Conceptualising AI in terms of human interactions is functional and was the creators' intention when they relied on language as the medium of interaction. LLMs were designed to mimic natural human interactions, making it seem like you are speaking with someone [ 21 ]. I take a similar approach in this article, treating ChatGPT as an assistant and telling the story of how to use its functionalities in a more approachable, humanistic way [ 22 ]. Generative AI continues to evolve; for example, ChatGPT-3.5 is a text-to-text model, and ChatGPT-4 is a data-to-text model [ 23 ]. More exploration is needed to understand artificial intelligence's role in higher education, especially in research [ 24 ]. The usefulness of the current article is in its practical application to broaden our understanding of generative AI in higher education, and I present a tutorial which can be used by professors and students [ 25 ]. I used mixed methods research (MMR) for the tutorial and application because MMR offers us a robust framework for addressing many multifaceted issues in higher education. MMR explores complex human phenomena [ 26 , 27 ]. Moreover, by analysing mixed methods data in ChatGPT, I create sections on quantitative and qualitative analysis that researchers from these disciplines could use even if they are not interested in MMR. The prompts and exercises in the current paper were piloted on over 100 academics in a South African university. Members of the academy from all Faculties and most departments at my university attended the training and provided feedback. I subsequently refined the tutorial based on the pilot. Technological revolutions Before the 2000s, software was syntax-based, and researchers had to write instructions [ 28 – 30 ]. This often led to considerable effort to fix and manage mistakes in writing syntax, apart from the knowledge and skills needed to conduct and interpret the data analysis [ 31 ]. The Graphical User Interface (GUI, ‘gooey’) revolution brought about easier use of statistics, where pointing and clicking was all the programming knowledge required [ 32 ]. At each phase of software evolution, the researcher needs to understand what they are doing and the statistical choices [ 33 ] or qualitative methods they employ [ 34 ]. The roles and responsibilities remain; the researcher must be well grounded in their understanding of data analysis theory and application. However, the researcher can now use the Chatbot as an assistant to suggest further analyses, conduct some of the analysis, and obtain help with interpreting outputs. At each step of the evolutionary process, there is less pressure on the researcher to know the underlying nuts and bolts of software and more opportunity to focus on the data. Researchers are only beginning to explore all the options that generative Artificial Intelligence offers [ 35 ]. Understandably, significant shifts are frightening, and the generative AI revolution in the academic world is no different. Luca Mari summarises this experience eloquently (2023, p.29): Copernicus showed us our cosmological non-centrality Darwin showed us our biological non-originality Chatbots are showing us our cognitive non-uniqueness Some of our shared fears include the idea that AI will fundamentally replace us, for example, being able to do work for which we were once considered crucial [ 36 ]. Another fear is that AI will reduce creativity, innovation and original work, replacing our humanity [ 37 ]. Hinks [ 38 ] found that negative expectations of artificial intelligence's role in our society are associated with reduced life satisfaction. Students are worried about the impact generative AI will have on their development, and the academy is concerned about academic integrity and the impact on assessment [ 39 ]. The Council on Higher Education [ 40 ] suggests that treating generative AI as a "crime" or plagiarism outright will negatively impact teaching and learning. How should we deal with GAI in higher education? To navigate these existential crises and our trepidation, we need to engage with our fears reasonably, as Chatbots are here to stay [ 41 ]. We must find ethically sound ways to use the new tools to advance human knowledge and development. Our friends from the philosophy of science can help us to think critically about the latest existential crises. Heidegger was worried that technology would strip us of our humanity (forgetting to "be"), Idhe was concerned with its role in our daily lives, and Stiegler focused on how technology could disrupt our cultural memories [ 42 , 43 ]. Hui [ 43 ] merges these ideas and sees technology as the conduit through which we understand our world (he calls this “cosmotechnics”). Baker and Hui are linked to the idea from Ihde [ 44 ] that science is the conceptual side of a technological society and that scientists should reimagine the role of technology and its ethical use. Metaphysics, cultural shifts and geopolitical implications cannot be ignored when significant technological changes occur [ 45 ]. New technologies shape the world around us, reshape how we see ourselves and lead to new traditions, potentially replacing our cultural heritages and requiring profound pluralism from us [ 43 , 46 ]. In the research world, we measure what we value, pushing us to question what is valuable. While I focus on the practicalities in the current paper, I am also proposing a person-centred, meaning-making approach to scholarship and integrating artificial intelligence into our research [ 47 ]. I explore this idea further by examining how researchers can ethically apply the new GAI tools. Ethical and safe use of generative AI in research There is nothing inherently wrong with using generative AI during data analysis as long as the researcher does so responsibly [ 48 ]. For the reader's purposes, I recommend using generally established ethical guidelines for generative AI use in research [ 49 , 50 ]: being transparent in your use of generative AI during your research processes being responsible for the inputs and how you use the outputs protecting the data privacy of your participants using AI for specific purposes in the research process and reporting the use as one would with any software [ 51 ] considering and limiting potential bias from the AI platform emphasis on the humanisation and centrality of participants in research As a researcher, you need to understand that Chatbots are tools. The user still needs to have the required foundational knowledge to use the instrument for the intended purpose in the same way you need to know why you select statistical tests when using GUI software. Designers of Chatbots add guardrails to prevent offensive, inflammatory or dangerous outputs in attempts to protect the users [ 52 , 53 ]. However, jailbreaking still occurs; some individuals can find ways around the guardrails [ 54 ]. Jailbreaking is a complex issue and can be part of the innovative strategies of users. Guardrails have downsides, such as over-correction and replacing one type of bias with another – users need to be aware of this when employing Chatbots [ 55 ]. While the designers of generative AI try to protect users with guardrails, the researcher needs to understand settings to ensure safer use. To enhance the ethical use of ChatGPT and your data, you can go to privacy settings, choose “Settings” – “Data control” – “Chat History and Training”, then switch off the history and training [ 23 ]. In Fig. 1 , I show the guidelines for the ethical use of AI and Chatbots in MMR and data analysis. I advise that you refrain from uploading any sensitive data. Ensure you remove any identifying information from data sets before using generative AI to analyse the data. Use resources such as ChatGPT responsibly by rephrasing, repurposing, and reintegrating responses, just as you would information from other sources [ 56 , 57 ]. Critically evaluate all the information you receive from AI platforms and apply your interpretation. Be transparent about how and when you used artificial intelligence in your writing, analysis, and presentation. Cite the Large-Language Model (LLM) you used. Chatbots are sophisticated but only sometimes reliable, explainable and never accountable for what they produce [ 58 ]. Therefore, the researcher is responsible for fair and ethical use. Research questions How can generative Artificial Intelligence assist with data analysis of Mixed Methods Research (MMR)? To what extent does generative Artificial Intelligence yield reliable and valid findings for MMR data analysis? Materials and Methods The data used as a demonstration in the current article comes from a project that focused on enhancing first-year engineering education in South Africa. The data were collected as part of a concurrent triangulation research design, and the findings were reported in various publications (Author & Author, 2020; Author et al., 2022). We wanted to help first year engineering students by understanding student motivation, expectations, and perspectives and how these align with lecturer insights and curriculum design. Instruments The quantitative data were collected using the Academic Pathways of People Learning Engineering Survey (APPLES), which contains sections on learning expectations, beneficial and enjoyable educational experiences, skills and abilities required to become an engineer, motivation for studying engineering and post-graduate expectations [ 59 , 60 ]. The APPLES survey was adapted to have forced-choice options, a decision for which evidence is presented in Author (2024). The qualitative data were collected using in-depth interviews for which the researchers designed a guide. The survey contained two open-ended questions where additional qualitative data were collected. In the open-ended questions, students were asked to explain why they chose engineering and provided any other information they deemed appropriate. Sample The quantitative data collection yielded a sample of 969 first-year engineering students, who were primarily men (71%), Caucasian (58%) or Black African (26%) and came from middle-class or affluent homes (81%). Students were, on average, 19 years old. The students who participated in the in-depth interviews (14) were all Black African by the study's design, predominantly male (9 out of 14 participants) and came from varied socio-economic backgrounds. Data Analysis In the original analysis, I used the Rasch dichotomous model via Winsteps© 5.4.0.0 [ 61 ] and SPSS (Statistical Package for the Social Sciences) to generate descriptive and inferential statistics [ 62 ]. The qualitative interview data were transcribed and then analysed by two research psychologists using reflexive thematic analysis (TA) (Author et al., 2022). After that, the specialist inspection was conducted by two engineering educators, and the final themes and write-up were the result of a consensus being reached among the team of four researchers [ 63 – 65 ]. For the current article and demonstration, I reran the analysis through ChatGPT’s Advanced Data Analysis bot , and my custom settings are shown Table 1 . Custom ChatGPTs are available, and instructions can be customised [ 23 ]. I recommend that the user try out custom GPTs for themselves; the current tutorial utilises the Advanced Data Analyst , one of the original customs ChatGPTs [ 23 ]. Setting up custom instructions by clicking on your profile is recommended, and many good resources are available on the internet for custom instructions [ 66 ]. To use the custom instruction shown in Table 1 , click on your profile icon in ChatGPT (bottom left-hand corner), choose the Custom instructions option, and paste in the text. The text provided is a generic guideline, and can be adjusted to suit specific individual preferences and should reflect the type of research you are conducting [ 67 ]. Table 1 Custom instructions for mixed methods analysis in ChatGPT Question Heading Specification What would you like ChatGPT to know about you to provide better responses? Key Responsibilities Integration of qualitative and quantitative data analysis Coding, thematic analysis, and statistical analysis Synthesis of findings from both data types to generate comprehensive insights Knowledge or expertise Expertise in qualitative coding thematic analysis, and quantitative statistical methods Familiarity with integrating data from different sources to form cohesive conclusions. Proficiency in software tools for both qualitative (e.g., NVivo, ATLAS.ti) and quantitative analysis (e.g., SPSS, R) Current Projects [Add your projects which reflect mixed methods] Jargon or Terminology Qualitative coding, Reflexive Thematic Analysis, mixed methods integration, statistical analysis, multimodal data analysis, Rasch Analysis Goals and Objectives To create rich, multidimensional interpretations from diverse data sets, consider context, and apply interdisciplinary knowledge to enrich interpretations. Create meaningful interpretations from textual data, consider context, apply interdisciplinary knowledge to interpretations, limit bias Interactions Engage in both social science and statistical data analysis Narrative and analytical writing for diverse audiences, including academic, policy, and public sectors Educational and practical application of mixed methods research findings How would you like ChatGPT to respond? Tone Professional, empathetic, understanding, and clear Level of detail Provide comprehensive narratives that integrate findings from both qualitative and quantitative data, supported by quotes and statistical evidence Preferred References Scientific papers, psychological and educational frameworks, academic journals, primary sources across both qualitative and quantitative fields Examples or analogies Use examples from journal articles, research reports, or thesis chapters that effectively demonstrate the integration of mixed methods Avoidance of Ambiguity Present clear, analytical findings that seamlessly blend insights from both qualitative and quantitative research, grounded in theory and empirical evidence. Resources links Access to interdisciplinary data science libraries, mixed methods research papers, and online courses focusing on integrated data analysis techniques Follow up questions Research impact, future studies Problem-solving method Apply a systematic approach to mixed methods data analysis, detailing the process of qualitative coding, thematic generation, and quantitative statistical analysis, including the integration phase, to draw overarching conclusions Confidence of answers Include a confidence level with each piece of advice or information provided, especially regarding integrating data and interpreting mixed methods analysis results. Ethical considerations The present study was approved by the ethics committee of the Faculty of Engineering, Built Environment & IT (EBIT) at the University of Pretoria (EBIT/46/2020). The study was performed in accordance with the ethical guidelines of the University of Pretoria. All participants in this work have provided freely given, informed consent to participants in the study. The author affirms that the participants also provided informed consent for their data to be used for publication purposes. Tutorial and Results – Application Here, I present the roles that generative AI can play in the various types of data analysis. I give prompt examples, and I evaluate the reliability and validity of using ChatGPT for MMR. When asking ChatGPT to conduct analysis, I recommend that you give it frameworks, evaluation criteria or any other standards that it could use to improve outputs [ 68 ]. Ask ChatGPT to be brutally honest, to look for problems and lack of cohesion, to evaluate your writing critically, and to find ways to overcome positivity bias. ChatGPT can act as a research assistant who is available, teachable, and inexpensive. Meet your artificial assistant, ChatGPT, also referred to here as Chad (the only good Chad you will ever meet). The Chad is a naïve but well-meaning intern working for $20 monthly. Chad needs much supervision and makes silly mistakes but can add value when managed well. Chad is a well-mannered Chatbot (thanks to the guardrails) with much potential. Unfortunately, he is a people-pleaser, which can sometimes be annoying. Qualitative data analysis with ChatGPT ChatGPT and other similar generative AI platforms have immense capacity to analyse multimodal data [ 16 ]. ChatGPT can play several roles in qualitative analysis, including coding, where you ask ChatGPT to generate codes for textual data based on frameworks or theories. In addition to textual analysis, ChatGPT can detect sentiment, such as the emotional tone of the text [ 69 ]. ChatGPT can also generate themes and evaluate qualitative analysis for consistency by acting as a specialist inspector. The possibility of generating visualisations and tables with ChatGPT-4 is also helpful in quantifying or summarising aspects of the qualitative data. You can paste a limited amount of qualitative text into ChatGPT-3.5. In ChatGPT-4, the Advanced Data Analysis module makes it possible to upload data, ask for Excel codes per line, do more checks and comparisons, and create figures. Consider ChatGPT as an additional coder or a research assistant in this process. However, crucially, do your analysis and not rely solely on the Chatbot as it may miss rich aspects of your data or misinterpret the text. In my experience, inductive coding does not work well in generative Artificial Intelligence settings, and I advise you to avoid asking the Chatbot to cold code. Instead, ChatGPT should be given both an analytical framework and a theoretical or conceptual framework to guide the coding and theme generation, as shown in Fig. 2 (see also the Prompt Library in the appendix). Examples of analytical frameworks to consider using include thematic, narrative, content, and discourse analysis. Another suggestion is to code the first few lines or paragraphs to show ChatGPT as examples. ChatGPT assigns generic codes such as "other" when unsure or struggling to interpret the text. Such codes and themes require additional investigation as complex and valuable information is often submerged into these commonly named themes. When facing token restrictions, analyse a few interviews at a time; after that, ask ChatGPT to adjust themes based on additional interviews. As can be seen in Fig. 2 , Chad provides well-written themes and quotes the participants as requested. The theoretical framework has also been woven into his answers. Figure 3 demonstrates the prompt used to ask ChatGPT to code open-ended survey items. I uploaded the Excel file with the responses and labelled the columns where I wanted the bot to add the codes per line. In addition to the downloadable Excel file Chad produced, reasonable themes were also written, as seen in Fig. 3 . Checking the Excel file to see how every line was coded was very useful, and many vague statements had been given the code "other", and I could compare this to the human coding, which was more nuanced and consequential. Quantitative data analysis with ChatGPT Generative Artificial Intelligence (GAI) fulfils additional roles that GUI software did to a limited extent or not at all. ChatGPT can assist with cleaning data, for example, finding out-of-range values in data files. The Advanced Data Analysis custom ChatGPT can also scan your data file and recommend analysis. Here, I suggest sharing a codebook with the Chatbot with variables, labels, categories, and any required explanations, such as the Level of measurement. Use the suggestions from the Chatbot with care, as ChatGPT might suggest analysis that cannot or should not be done with the type of data presented. Chad can quickly generate descriptive statistics, as I will demonstrate here, as well as a paragraph with initial interpretation for the researcher to consider using. Commonly used inferential statistics can also be conducted in ChatGPT, with the bonus of interpretation. Currently, generative AI is fine for run-of-the-mill statistical analysis. Using generative AI to run more sophisticated statistical models requires further exploration. For example, I tried to conduct some structural equation modelling (SEM) with ChatGPT, but Chatbot has not yet been able to handle such complex analysis. In Fig. 4 , I show the prompt asking for Chad's recommended data analysis. As can be seen in Fig. 4 , Chad recommends many different types of analysis for my specific data set. However, not all of them are feasible or applicable. Nonetheless, this is a good starting point and ChatGPT may suggest types of analysis you had yet to consider. Figure 5 is where I asked ChatGPT to produce descriptive statistics in a specified table format. I also asked for an interpretation and write-up of the table. While I did not fully agree with its interpretation, this again provides a good departure point to help the researcher start the writing process. In Fig. 6 , I illustrate the prompt for asking for inferential statistics from the Chad. Instead of asking for specific analysis, I asked ChatGPT to explore various relationships and to report statistical significance in an interpretative paragraph. As can be seen from the response, the GAI accurately interprets the p -values, though it fails to report the effect sizes. I continued prompting beyond this point to obtain all the information I needed. Remember that just like a real assistant, you should keep speaking with the bot until you receive everything you need. See the appendix for the other prompts used. Mixed methods data analysis with ChatGPT A wide range of MMR analyses can be aided by generative AI, including concurrent triangulation, sequential explanatory and exploratory analysis, transformative designs (here, you must supply the paradigm) and multi-stage evaluations. ChatGPT-4 can also produce more complex figures and tables based on its own or your analysis, for example, side-by-side joint displays, statistics displayed by theme or vice versa and interview questions joint displays [ 70 ]. I have also created some infographics with ChatGPT-4, but these still require much input from the researcher, and it may be easier just to create your own in a different software environment. In ChatGPT-3.5, you can paste your findings from both the quantitative and qualitative analysis and work step by step with the bot to integrate analysis and output. To successfully use ChatGPT for mixed methods analysis, I recommend that you separately analyse the quantitative and qualitative data and combine the findings into a single document before asking the bot to assist with MMR analysis. In Fig. 7 , I prompt ChatGPT to use my combined quantitative and qualitative findings document and conduct a sequential explanatory analysis using this study's theoretical framework. I specified both the analytical and the theoretical (or conceptual) frameworks as this yields the best results. Researchers should use comprehensive frameworks when conducting and analysing MMR data, as shown by Corrigan and Onwuegbuzie [ 71 ]. The output received in Fig. 7 was sparse, but further prompting into aspects reported by Chad led to more valuable outputs. Based on the findings, I asked ChatGPT to create a joint table (see Fig. 8 ). While I would not use the table in its current format in a publication, I would use this summary to guide my writing and inform the creation of other graphics. I recommend using generative AI to create initial tables, figures and text based on the quantitative and qualitative data. The final integration should be based on the aims of the study. When asking ChatGPT to create side by side type of tables, check this against your own version to make sure everything relevant is included. Write your interpretations based on your research aims and objectives; do not rely solely on the bot. Reliability and validity of generative AI application In Table 2 , I compare the original analysis done by the research team and the outputs received from ChatGPT-4's Advanced Data Analysis Chatbot. Table 2 Comparison of original quantitative and qualitative analysis with ChatGPT Type Original (Researchers) ChatGPT version Qualitative interviews Theme: The academic program Theme: Academic and Personal Growth Qualitative interviews Theme: Student support Theme: Transition Challenges Qualitative interviews Theme: The role of peers in student success Theme: Importance of Social Connections Qualitative interviews Theme: Behavioral and attitudinal factors Quantitative data analysis Demographic profile (Table & Text) Demographic profile (Table & Text) Quantitative data analysis Recommended non-parametric analysis Recommended non-parametric but also other not applicable analysis Quantitative data analysis Non-parametric analysis and interpretation Non-parametric analysis done but less detail was provided Open-ended survey questions Theme: Intrinsic Psychological Motivation Theme: Personal Interest and Passion Open-ended survey questions Theme: Engineering as Social Good Open-ended survey questions Theme: Focused on career opportunities Theme: Career aspirations Open-ended survey questions Theme: Intrinsic behavioural motivation Theme: Influence of Technology and Innovation Open-ended survey questions Theme: Curiosity and Problem-Solving Comparing the original qualitative analysis done by the team of researchers to the ChatGPT themes, most were similar. The content of the themes is also consistent, but the most significant difference was related to the extent of coverage. ChatGPT found most of the same themes but gave shorter, less rich write-ups. ChatGPT also missed a theme we identified (behavioural and attitudinal factors). In terms of the quantitative analysis, the Chatbot suggested creating demographic profiles (for example, gender, age, and ethnicity), mean distributions (for continuous variables) and summaries for the categorical variables. Other suggestions included cross-tabulations, non-parametric analysis, factor analysis (for constructs in the survey) and regression analysis to estimate predictors of the continuous variables. All the former suggestions I considered myself and are reasonable. Further suggestions, which I had yet to consider, included cluster analysis to group the students (this was not feasible with the data set), creating profiles within the engineering student population, and outcomes based on educational preferences. These last suggestions were exciting but irrelevant for my data set, once again pointing to the fact that the scholar needs to understand their data and the statistical possibilities. The demographic tables, figures, and interpretations provided by ChatGPT were of high quality and a good match for our own demographic profiles. When I asked ChatGPT to suggest inferential analysis, it did suggest the same non-parametric tests that I initially used, but it also suggested less useful and sometimes irrelevant analysis. While descriptive statistics can quickly and reliably be done using ChatGPT, researchers should use their knowledge and judgment of statistics when deciding which tests to run in generative AI. The outputs are deemed acceptable and accurate and could be used in a research report or journal article, with the caveat that the text interpretation should be rewritten. Look for inaccuracies or exaggerations in the Chatbot’s output; for example, in my outputs, I found the claim that the sample is diverse to be an exaggeration. The open-ended survey questions revealed most of the same themes, but again, ChatGPT's descriptions and unpacking of the themes were shallow when compared to human analysis. ChatGPT also missed a theme when compared to the human coders. At the same time, Chad had a theme that the humans did not identify. This comparison between human and artificial analysis can create avenues for identifying additional themes. ChatGPT contributes to analysis, but human judgement and insight should also be present throughout the process. Troubleshooting As is the case with all technology, ChatGPT can experience technical difficulties. Troubleshooting for ChatGPT includes clearing the history, cookies, and cache for the "all-time" option. Next, you can restart the session. Lastly, use different browsers (for example, Firefox works well) or disable all the Chrome extensions. Discussion and key findings Generative AI is here to stay, and we must find ways to use it ethically and responsibly [ 3 ]. This is why the current study demonstrates how to conduct MMR analysis and integration using an artificial intelligence platform, ChatGPT. I also acknowledge that all types of technology have drawbacks, and a lack of transparency could call into question our research's methodological integrity, one of the biggest threats we face with GAI [ 72 ]. In this paper, I provide guidelines on using generative AI for mixed methods data analysis while balancing potential pitfalls. The main findings and recommendations from the current article are shown in Fig. 9 . In the quantitative section, I demonstrate how ChatGPT can quickly provide suggestions for data analysis techniques. Most of the suggestions are valid, and the bot has also improved since I used it last year, as it can now more accurately identify parametric from non-parametric statistical models. Providing the Chatbot with a codebook facilitates the data analysis, like setting up variables in software such as SPSS. Generative AI can provide accurate descriptive and inferential statistical outputs, but it’s ability to conduct more sophisticated analysis such as structural equation modelling (SEM) has yet to be tested. In the qualitative section, the Chatbot should be provided with an analytical and a theoretical model to improve the analysis. ChatGPT does not handle inductive coding well, and I suggest providing the bot with examples of how you want it to code. You could also specify if you want it to act as a splitter or lumper. For more informative text, such as interviews, ChatGPT produces coherent narrative themes and can quote participants. It was less useful for shallow qualitative data, such as answers to open-ended questions. I suspect the latter is due to the inherent vagueness of the responses, which even human coders find challenging. Mixed methods research (MMR) analysis works best if the quantitative and qualitative analyses are done separately before asking the bot to help integrate the findings. When the QUAL and QUANT are neatly combined in a document with clear headings, a prompt can be provided to the GAI wherein one asks it to help synthesise the findings. Here, I recommend specifying which MMR analytical approach you want it to use. The side-by-side tables it produces are also helpful, but as always, the researcher must add their own insights. The reliability and validity of straightforward tasks, such as producing descriptive statistics, are perfect in the generative AI environment. More interpretative outputs, such as narratives derived from qualitative data, are less reliable and valid when using ChatGPT. While it can detect most of the same themes, it sometimes misses them. The outputs provided by the GAI are also less rich and tend to be shallower summaries. Therefore, the researcher can consider the GAI an assistant in coding but should still create their own themes and write-ups to compare to bot outputs. Overall, the reliability and validity of using generative AI in analysing MMR data is acceptable. The researcher needs to have the required foundational knowledge and be discerning, rephrasing, repurposing, and reintegrating the outputs before final publication. One of the main advantages of using generative AI is the quick turnaround time for data analysis. Speedier analysis is most definitely desirable in a world where rapid publication is required in higher education [ 34 ]. Lastly, we may ask ourselves why we would create new knowledge in a world where artificial intelligence can produce equivalent artefacts. Yet there have always been others doing something similar, and who could do it better than we could. Excellent poets, writers, scholars, artists, and every type of creator have existed for millennia. We continue to create despite others doing the same work because everyone has a unique point of view. Different from another person or Chatbot. We create new knowledge as scholars because we need to contextualise what we do for our settings. We add value through our humanity, and others crave that human connection. Human production of creative work will become even more desirable in the future when machines make so much. Creativity is a natural part of who we are – we are our most authentic selves when we create. Authentic self-expression through knowledge creation can fulfil an identity need to feel connected to ourselves and our world through adding new insights and intellectual artefacts. Significance of the findings The current paper demonstrates innovative ways to use GAI for quantitative, qualitative, and mixed methods data analysis. I also showcase new ways to integrate findings with the assistance of generative AI to improve the interpretation of mixed methods data. The reader is provided with guidelines on using GAI for efficiency and automation to make handling large quantities of data easier. Generative AI misses themes sometimes but can also reveal themes and patterns humans overlook. Therefore, I state the case for combining human insight with generative artificial intelligence to obtain results beyond the abilities of either; the sum of the parts is indeed greater. My study can be used across many disciplines or for cross-disciplinary work where mixed methods are prominent. I have detailed my steps and provided a prompt library for easy replicability and application. The current paper serves as capacity building for researchers globally to advance their use and understanding of AI tools in research methodology. I explicitly addressed ethical considerations in this paper to clarify privacy, consent, and practical implications for researchers who want to use generative AI. My findings here are meant to strengthen digital literacy in a fast-developing technological landscape. Limitations of the study The tutorial and application were limited to an available data set, and the example was drawn from higher education. I only used one platform, ChatGPT, and I used the paid version. Using AI in data analysis has its limitations, as discussed in the paper. The quality of the results depends significantly on the knowledge and skills of the researcher. The platform user should have the requisite knowledge of mixed methods data analysis, synthesis, and application. Chatbots cannot replace human insight. Furthermore, the fact that bias and unfairness are built into artificial generative intelligence is well known [ 73 ]. Therefore, the researcher must actively manage the inputs and how the outputs are used, aware that there may be systemic, computational and human biases embedded in chats [ 74 ]. The landscape of generative AI is also changing rapidly [ 75 ], and the current paper is limited to the time and space in which it was written. ChatGPT is not the only platform available to researchers, and future researchers could look at platforms such as Perplexity (see https://www.perplexity.ai/ ), Jenni (see https://jenni.ai/ ) and Consensus ( https://consensus.app/ ). Declarations Funding statement No funding was received for the current work. Conflicts of interest There is no conflict of interest. Data Availability The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request. Code availability Not applicable. Consent statement All participants in this work have provided freely-given informed consent to participants in the study. The author affirms that the participants also provided informed consent for their data to be used for publication purposes. Author Contribution I am the only author of this paper; I confirm the single authorship. Acknowledgement The Living Lab for Innovative Teaching at the University of Pretoria (LLITUP) supported the development of a workshop to test the ideas presented in this paper. I am very grateful to them for their time, inputs and congeniality. References Howard, J., Artificial intelligence: Implications for the future of work. American Journal of Industrial Medicine, 2019. 62 (11): p. 917-926. Dwivedi, Y.K., et al., Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 2021. 57 : p. 101994. Ågerfalk, P.J., Artificial intelligence as digital agency. European Journal of Information Systems, 2020. 29 (1): p. 1-8. Ooi, K.-B., et al., The potential of generative artificial intelligence across disciplines: Perspectives and future directions. Journal of Computer Information Systems, 2023: p. 1-32. Dewey, J., M.C. Pautz, and M.K. Diede, How do we Address Faculty Burnout? Start by Exploring Faculty Motivation. Innovative Higher Education, 2023. Strzelecki, A., Students’ Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. Innovative Higher Education, 2023. Bahroun, Z., et al., Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability, 2023. 15 (17): p. 12983. Wikipedia. ChatGPT . 2024 [cited 2024, March 4; Available from: https://en.wikipedia.org/wiki/ChatGPT. Rawas, S., ChatGPT: Empowering lifelong learning in the digital age of higher education. Education and Information Technologies, 2023. Ruiz-Rojas, L.I., et al., Empowering education with generative artificial intelligence tools: Approach with an instructional design matrix. Sustainability, 2023. 15 (15): p. 11524. Ratten, V. and P. Jones, Generative artificial intelligence (ChatGPT): Implications for management educators. The International Journal of Management Education, 2023. 21 (3): p. 100857. Hmoud, M., et al., Higher Education Students’ Task Motivation in the Generative Artificial Intelligence Context: The Case of ChatGPT. Information, 2024. 15 (1): p. 33. Koh, E. and S. Doroudi, Learning, teaching, and assessment with generative artificial intelligence: towards a plateau of productivity. Learning: Research and Practice, 2023. 9 (2): p. 109-116. Dempere, J., et al., The impact of ChatGPT on higher education. Dempere J, Modugu K, Hesham A and Ramasamy LK (2023) The impact of ChatGPT on higher education. Front. Educ, 2023. 8 : p. 1206936. Islam, I. and M.N. Islam, Exploring the opportunities and challenges of ChatGPT in academia. Discover Education, 2024. 3 (31). Davidson, T., Start generating: Harnessing generative artificial intelligence for sociological research. Retrieved from osf. io/preprints/socarxiv/u9nft, 2023. Pack, A. and J. Maloney, Using Generative Artificial Intelligence for Language Education Research: Insights from Using OpenAI's ChatGPT. TESOL Quarterly, 2023. 57 (4): p. 1571-1582. Goodfellow, I., et al., Generative adversarial networks. Communications of the ACM, 2020. 63 (11): p. 139-144. Adamopoulou, E. and L. Moussiades. An overview of chatbot technology . in IFIP international conference on artificial intelligence applications and innovations . 2020. Springer. Hadi, M.U., et al., A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints, 2023. Korteling, J., et al., Human-versus artificial intelligence. Frontiers in artificial intelligence, 2021. 4 : p. 622364. Cooms, S. and V. Saunders, Poetic inquiry: a tool for decolonising qualitative research. Qualitative Research Journal, 2024. 24 (1): p. 3-22. OpenAI. ChatGPT (September 25 Version) [Large language model]. 2024; Available from: https://chat.openai.com. Barros, A., A. Prasad, and M. Śliwa, Generative artificial intelligence and academia: Implication for research, teaching and service. Management Learning, 2023. 54 (5): p. 597-604. Costello, G.R., K.R. Davis, and O.S. Crocco, Learning by Doing: Student & Faculty Reflections on a Collaborative Model for Conducting and Publishing Mixed Methods Research in a Graduate Course. Innovative Higher Education, 2022. 47 (6): p. 1067-1084. Malina, M.A., H.S. Nørreklit, and F.H. Selto, Lessons learned: advantages and disadvantages of mixed method research. Qualitative Research in Accounting & Management, 2011. 8 (1): p. 59-71. Onwuegbuzie, A.J. and C. Poth, Special Issue: Mixed Methods. International Journal of Qualitative Methods, 2015. 14 (2): p. 122-125. Coolidge, F.L., Statistics: A gentle introduction . 2020: Sage Publications. Chambers, J.M., Statistical Computing: History and Trends. The American Statistician, 1980. 283 . Abatan, S.M. and M. Olayemi, The Role of Statistical Software in Data Analysis International Journal of Applied Research and Studies (iJARS) 2014. 3 (8). Klinke, S. Statistical user interfaces . Center for Applied Statistics and Economics (CASE) REport, 2004. 35 . Martinez, W.L., Graphical user interfaces. Wiley Interdisciplinary Reviews: Computational Statistics, 2011. 3 (2): p. 119-133. Field, A.P., Discovering statistics using IBM SPSS statistics . 2024: Sage publications limited. Vindrola-Padros, C. and G.A. Johnson, Rapid Techniques in Qualitative Research: A Critical Review of the Literature. Qualitative Health Research, 2020. 30 (10): p. 1596-1604. Kantor, J., Best practices for implementing ChatGPT, large language models, and artificial intelligence in qualitative and survey-based research. JAAD international, 2024. 14 : p. 22-23. Cave, S. and K. Dihal, Hopes and fears for intelligent machines in fiction and reality. Nature machine intelligence, 2019. 1 (2): p. 74-78. McClure, P.K., “You’re fired,” says the robot: The rise of automation in the workplace, technophobes, and fears of unemployment. Social Science Computer Review, 2018. 36 (2): p. 139-156. Hinks, T., Artificial Intelligence Perceptions and Life Satisfaction. Journal of Happiness Studies, 2024. 25 (1): p. 1-14. Smolansky, A., et al. Educator and student perspectives on the impact of generative AI on assessments in higher education . in Proceedings of the tenth ACM conference on Learning@ Scale . 2023. Council on Higher Education, C. A Closer Look at Plagiarism within the Context of Student Academic Writing in an Era of Generative Artificial Intelligence . Briefly Speaking, 2024. 27 . Atkinson, R., Don't fear AI (Volume 2) . 2019: European Investment Bank. Baker, M., The Western Mathematic and the Ontological Turn: Ethnomathematics and Cosmotechnics for the Pluriverse , in Indigenous Knowledge and Ethnomathematics . 2023, Springer. p. 243-276. Hui, Y., For a cosmotechnical event: In honor of Don Ihde and Bernard Stiegler. Reimagining Philosophy and Technology, Reinventing Ihde, 2020: p. 87-102. Ihde, D., Technology and the lifeworld: From garden to earth. 1990. Fisher, W.P., Meaning and Method in the Social Sciences1. Human Studies, 2004. 27 (4): p. 429-454. Roth, P.A., Meaning and method in the social sciences: A case for methodological pluralism . 2019: Cornell University Press. Fisher Jr, W.P. and S.J. Cano, Ideas and Methods in Person-Centered Outcome Metrology. Person-Centered Outcome Metrology, 2023: p. 1. Schlagwein, D. and L. Willcocks, ‘ChatGPT et al.’: The ethics of using (generative) artificial intelligence in research and science. Journal of Information Technology, 2023. 38 (3): p. 232-238. Rothenberger, L., B. Fabian, and E. Arunov. Relevance of Ethical Guidelines for Artificial Intelligence-a Survey and Evaluation . in ECIS . 2019. Skorburg, J.A., K. O'Doherty, and P. Friesen, Persons or data points? Ethics, artificial intelligence, and the participatory turn in mental health research. American Psychologist, 2024. 79 (1): p. 137. O’Kane, P., A. Smith, and M.P. Lerman, Building transparency and trustworthiness in inductive research through computer-aided qualitative data analysis software. Organizational Research Methods, 2021. 24 (1): p. 104-139. Wang, Y. and L. Singh, Adding guardrails to advanced chatbots. arXiv preprint arXiv:2306.07500, 2023. Abdelkader, H., et al., ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study. arXiv preprint arXiv:2401.06513, 2024. Boxleitner, A., Pushing Boundaries or Crossing Lines? The Complex Ethics of ChatGPT Jailbreaking. The Complex Ethics of ChatGPT Jailbreaking (October 17, 2023), 2023. Kamalov, F., D. Santandreu Calonge, and I. Gurrib, New era of artificial intelligence in education: Towards a sustainable multifaceted revolution. Sustainability, 2023. 15 (16): p. 12451. Cheng, M.W.T. and I.H.Y. Yim, Examining the use of ChatGPT in public universities in Hong Kong: a case study of restricted access areas. Discover Education, 2024. 3 (1). Daniela, F.-T. Academic writing and ChatGPT: Students transitioning into college in the shadow of the COVID-19 pandemic . Discover Education, 2024. 3 , 1-10 DOI: 10.1007/s44217-023-00076-5. Mari, L., Chatbots: facing a cultural revolution and trying to understand it (a non-technical perspective) , B. Seminar, Editor. 2023: UC Berkeley. Donaldson, K.M., et al., Scaling up: Taking the Academic Pathways of People Learning Engineering Survey (APPLES) national , in 2008 IEEE Frontiers in Education Conference . 2008, IEEE Computer Society. p. F4H-6-F4H-11. Sheppard, S.D., et al., Exploring the Engineering Student Experience: Findings from the Academic Pathways of People Learning Engineering Survey (APPLES) . 2010, Center for the Advancement of Engineering Education Technical Report CAEE-TR-10-01. Linacre, J.M., Winsteps® (Version 5.4.0.0) 2023: Portland, Oregon: Winsteps.com. IBM, IBM SPSS Statistics for Windows (Version 28.0) . 2023, IBM Corp. Braun and Clarke, One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 2021. 18 (3): p. 328-352. Braun, V. and V. Clarke, Thematic analysis : a practical guide . 2022, London ;: SAGE. Braun, V. and V. Clarke, Using thematic analysis in psychology. Qualitative Research in Psychology, 2006. 3 (2): p. 77-101. Surach, D., AI foundations: Learn to leverage market moving AI tools . 2024, YouTube: YouTube. Surach, D., 5 ChatGPT Hacks | Take ChatGPT to the NEXT Level! , in AI Foundations . 2023: YouTube. Stapleton, A., How To Write An A+ Essay Using AI in 3 Simple Steps . 2023: YouTube. Owoahene Acheampong, K. and M. Nyaaba, Review of Qualitative Research in the Era of Generative Artificial Intelligence. Matthew, Review of Qualitative Research in the Era of Generative Artificial Intelligence (January 7, 2024), 2024. Guetterman, T.C., M.D. Fetters, and J.W. Creswell, Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. Ann Fam Med, 2015. 13 (6): p. 554-61. Corrigan, J.A. and A.J. Onwuegbuzie, Toward a meta-framework for conducting mixed methods representation analyses to optimize meta-inferences. 2020. Grimes, M., et al., From scarcity to abundance: Scholars and scholarship in an age of generative artificial intelligence. Academy of Management Journal, 2023. 66 (6): p. 1617-1624. Schwartz, R., et al., Towards a standard for identifying and managing bias in artificial intelligence. NIST special publication, 2022. 1270 (10.6028). Hwang, G.-J., et al., Vision, challenges, roles and research issues of Artificial Intelligence in Education . 2020, Elsevier. p. 100001. Nishant, R., M. Kennedy, and J. Corbett, Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda. International Journal of Information Management, 2020. 53 . Appendix A Appendix A –Prompt Library Prompts for Qualitative Analysis Square brackets indicate aspects you should modify to suit your need. Prompt 1 – You are a qualitative researcher. Use [Reflexive thematic analysis], according to Braun and Clark, to analyse the interview and generate themes. Use the [Self-Determination Theory] to interpret the data. The themes should be written as narrative findings in paragraphs, give each theme a name that reflects the main finding of that theme. Use quotes from the interviewee to support your themes. Only analyse the interviewee responses. Prompt 2 – Attached are open-ended responses. Code and classify the responses according to [theory] so that you create themes, write up the themes as narratives with quotes from respondents to support your argument. Cite the frequencies of how many times a theme was mentioned. Make the excel sheet available with the codes completed. Prompts for Quantitative Analysis Prompt 3 – I have cross-sectional survey data, please recommend analyses I could do with my data set. The variables include [demographic questions, such as gender, age, and race]. The constructs measured in the questionnaire include [motivation to attend university and long-term career goals]. [Codebook pasted with prompt]. Prompt 4 – Summarise the constructs [add list here] in the attached data set to identify trends. Create a table and write a paragraph to illustrate your findings. Prompt 5 – Please create demographic profiles for my data. Create a table in [APA 7] style as the output with a paragraph which briefly describes the student profile. Prompt 6 – Redo the analysis, consider that most of the demographic variables are categorial in nature. Add the categories to the table to make it easier for the reader to understand. Prompt 7 - Based on the previous data, run inferential statistical analysis and identify any items that are significant. Include p-values and effect sizes. Show the results in tables with a summary of your interpretation. Prompts for Mixed Method Analysis Prompt 8 – I have findings from [a questionnaire], and I also conducted [in-depth interviews]. The results from the quantitative and qualitative analyses are attached. Please conduct a [sequential explanatory analysis] and help me to identify patterns, achieve triangulation and integrate the results. Use the [Incentive Theory] to interpret the results. Write two pages based on your sequential explanatory analysis. Prompt 9 – Based on the findings from the previous prompt, create a statistics-by-themes joint table display of the findings. Make the table a downloadable Excel. OR Create a side-by-side joint display of the quantitative and qualitative findings. Make the display downloadable. Prompt 10 – Produce a two-page document of your findings and ensure your report clearly distinguishes between quantitative and qualitative findings and how they integrate. Based on your findings, suggest practical or theoretical implications. Reflect on the effectiveness of the mixed-methods approach for the data collected. Prompts for obtaining more from Chatbots Prompt 11 – Please give me more information about your analysis and findings Prompt 12 - You are the student's supervisor. Evaluate the problem statement according to the guidelines. Identify the short-comings and list them. Give the student valuable tips to improve the problem statement. Problem statement guidelines: a) What do we already know about the problem? (Use recent and relevant studies to substantiate, preferably studies no older than 10 years) b) What do we need to know about the problem? (Gap in literature/scholarly field) c) Why does it matter? (importance of the study) d) Conclude with how your study will address the problem. Prompt 13 - I want you to evaluate the writing. Act as a critic; be ruthless. Analyse the text and tell me where it can be better. Prompt 14 - Please write a review of the attached document and highlight both the strengths and the weaknesses of the submission. Please be as constructive and specific as possible when offering recommendations. Use the following guidelines when examining the document [insert journal guidelines or evaluation criteria here] Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 13 May, 2024 Reviews received at journal 02 May, 2024 Reviewers agreed at journal 01 May, 2024 Reviews received at journal 29 Apr, 2024 Reviewers agreed at journal 28 Apr, 2024 Reviewers agreed at journal 26 Apr, 2024 Reviewers invited by journal 19 Apr, 2024 Editor assigned by journal 19 Apr, 2024 Submission checks completed at journal 15 Apr, 2024 First submitted to journal 27 Mar, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4176435","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":291352943,"identity":"e18012f3-a8a0-4f4a-b614-260ea6df31ce","order_by":0,"name":"Celeste Combrinck","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDUlEQVRIiWNgGAWjYFAC5gYYi42BwYCBQYK9AbdiCGAEqkhA1sJzgCQtQCAhkYBfA397Y+PDnz9sgIzutMcVBXfkJWe+MXvAUGPHIN+O3TqJMwebjXkS0oCMs9sNzxg8M5wtnWNuwHAsmYGxB7t1BhKJbdIMCYeBjNxtkg0GhxPkpHPMJBjYDjAwM+DQIv+w/eePhP9IWiTPALX8O8DAxv8Ahy2MbQw8CQcQWqQleMyAggcYeHCEg8SZxGZpnrRkHrBfgFoMZ/aklRsk9gFFJLDbwt9++ODHHzZ2cvztvdseNvw5LC9x/PC2Bx++2cnJ92O3BQZ4kDlsII/z4FCJHbCRpHoUjIJRMAqGPQAA02BWVsrylNwAAAAASUVORK5CYII=","orcid":"","institution":"University of Pretoria","correspondingAuthor":true,"prefix":"","firstName":"Celeste","middleName":"","lastName":"Combrinck","suffix":""}],"badges":[],"createdAt":"2024-03-27 13:29:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4176435/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4176435/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":54989195,"identity":"faf51c35-6bf8-47df-b334-af8efbe71817","added_by":"auto","created_at":"2024-04-19 16:38:01","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":149804,"visible":true,"origin":"","legend":"\u003cp\u003eA framework for the ethical use of artificial intelligence in mixed method data analysis\u003c/p\u003e","description":"","filename":"image1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/37472bb99c62245a613f9731.jpeg"},{"id":54989196,"identity":"3481b13b-1c2b-4907-baac-4ef7d513e74d","added_by":"auto","created_at":"2024-04-19 16:38:01","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":452292,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 1 and response from ChatGPT to demonstrate how to ask for theme generation\u003c/p\u003e","description":"","filename":"image2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/7623473acc8784acb88a2ef5.jpg"},{"id":54989198,"identity":"e0f0c54e-ba16-42e8-8428-a94bd98211a0","added_by":"auto","created_at":"2024-04-19 16:38:02","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":396703,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 2 and response from ChatGPT to show the coding of open-ended survey questions\u003c/p\u003e","description":"","filename":"image3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/e8c7c1a42ae905886f477fcf.jpeg"},{"id":54989197,"identity":"e986a917-396a-4817-9df9-ebdfa4fbeed4","added_by":"auto","created_at":"2024-04-19 16:38:02","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":399019,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 3 to ask for suggestions on analysing a quantitative data set\u003c/p\u003e","description":"","filename":"image4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/1aa0e7b241170b04cc5f8c49.jpeg"},{"id":54989199,"identity":"010aa4db-18e4-4cc8-9820-4b057a345263","added_by":"auto","created_at":"2024-04-19 16:38:02","extension":"jpeg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":286298,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 4 descriptive statistical table output and interpretation from ChatGPT\u003c/p\u003e","description":"","filename":"image5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/337bb7a15b2738caf3e1f4c7.jpeg"},{"id":54989202,"identity":"05e09627-bad7-4881-b291-9120bc6cfdf5","added_by":"auto","created_at":"2024-04-19 16:38:02","extension":"jpeg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":307640,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 7 Inferential statistics and interpretation from ChatGPT\u003c/p\u003e","description":"","filename":"image6.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/2aedee13a7e732a3b249334a.jpeg"},{"id":54989586,"identity":"b895bd3b-90a5-45f9-b695-d77106c7c753","added_by":"auto","created_at":"2024-04-19 16:46:02","extension":"jpeg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":270975,"visible":true,"origin":"","legend":"\u003cp\u003ePrompt 8 – request integration of qualitative and quantitative data for MMR analysis\u003c/p\u003e","description":"","filename":"image7.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/9b90d093ffd30eec902d46aa.jpeg"},{"id":54989200,"identity":"a4d9f28d-024a-49d1-8aac-3b2f3f9ed8f2","added_by":"auto","created_at":"2024-04-19 16:38:02","extension":"jpeg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":324321,"visible":true,"origin":"","legend":"\u003cp\u003eStatistics-by-themes joint display generated by ChatGPT\u003c/p\u003e","description":"","filename":"image8.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/059c0b44fcbdb81c8cb4a8e8.jpeg"},{"id":54989585,"identity":"ad301874-5584-48c8-b49b-984b43e5e31d","added_by":"auto","created_at":"2024-04-19 16:46:02","extension":"jpeg","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":339242,"visible":true,"origin":"","legend":"\u003cp\u003eSummary of recommendations for generative AI mixed methods data analysis use\u003c/p\u003e","description":"","filename":"image9.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/df92fb53c8788164e7a7f7ac.jpeg"},{"id":54989981,"identity":"bb3c295f-5ccc-494e-bb63-7c70b92e50b4","added_by":"auto","created_at":"2024-04-19 16:54:02","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1863188,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4176435/v1/18d84ea1-8f27-4ae8-acd1-d39a3c318eb2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Enhancing Mixed Methods Research with your generative AI assistant – A Tutorial and Evaluation for Scholars","fulltext":[{"header":"Introduction","content":"\u003cp\u003eRecent advances in generative Artificial Intelligence (AI) are changing how we conduct scholarly research. We are challenged to use new tools in all educational disciplines [\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e]. The rapidly shifting technological landscape requires us to experiment with new tools, share what we have learned to support academics and students and engage in intellectual curiosity [\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e]. Therefore, I showcase in this article how to use a generative AI platform to analyse quantitative and qualitative data and integrate findings for mixed methods research (MMR) using generative AI as an assistant. Chat Generative Pre-Trained Transformer (ChatGPT) has emerged as a dominant role player in the generative AI field, and higher institutions are reporting that students and academics are using the platform increasingly [\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e]. Some researchers [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e] focus on the positive role that generative AI can play in higher education, for example, aiding in managing large classes and online learning. Other researchers focus on how generative AI will impact assessment [\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e], how it can be used to promote student learning and motivation [\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e] and which frameworks to use when integrating Chatbots into teaching and learning [\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e]. Most articles conclude that we need more research and guidelines on using generative AI in higher education [\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eThe current article is not the first exploration of how to use generative AI in higher education [\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e]; many good publications exist [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e]. The current article adds a new approach by demonstrating the application of generative AI in mixed methods research (MMR) and providing prompts and guidelines for use. My work here is intended to be a resource for students and the academy, showing how generative AI can feed into the research process for academic advancement [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e]. Consequently, I use the following terms, which are distinct but related: generative Artificial Intelligence (AI or GAI), Chatbots and Large-Language Models (LLMs). By generative Artificial Intelligence (AI), I mean AI systems trained to produce new content based on but divergent from their training data [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e]. Chatbots interface AI and humans by applying LLMs to interact through text or voice [\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. LLMs serve as the basis for many types of generative AI and were designed to understand and function through human language [\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e]. Conceptualising AI in terms of human interactions is functional and was the creators' intention when they relied on language as the medium of interaction. LLMs were designed to mimic natural human interactions, making it seem like you are speaking with someone [\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e]. I take a similar approach in this article, treating ChatGPT as an assistant and telling the story of how to use its functionalities in a more approachable, humanistic way [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. Generative AI continues to evolve; for example, ChatGPT-3.5 is a text-to-text model, and ChatGPT-4 is a data-to-text model [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. More exploration is needed to understand artificial intelligence's role in higher education, especially in research [\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e]. The usefulness of the current article is in its practical application to broaden our understanding of generative AI in higher education, and I present a tutorial which can be used by professors and students [\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e]. I used mixed methods research (MMR) for the tutorial and application because MMR offers us a robust framework for addressing many multifaceted issues in higher education. MMR explores complex human phenomena [\u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e]. Moreover, by analysing mixed methods data in ChatGPT, I create sections on quantitative and qualitative analysis that researchers from these disciplines could use even if they are not interested in MMR. The prompts and exercises in the current paper were piloted on over 100 academics in a South African university. Members of the academy from all Faculties and most departments at my university attended the training and provided feedback. I subsequently refined the tutorial based on the pilot.\u003c/p\u003e\n\u003ch3\u003eTechnological revolutions\u003c/h3\u003e\n\u003cp\u003eBefore the 2000s, software was syntax-based, and researchers had to write instructions [\u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e30\u003c/span\u003e]. This often led to considerable effort to fix and manage mistakes in writing syntax, apart from the knowledge and skills needed to conduct and interpret the data analysis [\u003cspan class=\"CitationRef\"\u003e31\u003c/span\u003e]. The Graphical User Interface (GUI, \u0026lsquo;gooey\u0026rsquo;) revolution brought about easier use of statistics, where pointing and clicking was all the programming knowledge required [\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e]. At each phase of software evolution, the researcher needs to understand what they are doing and the statistical choices [\u003cspan class=\"CitationRef\"\u003e33\u003c/span\u003e] or qualitative methods they employ [\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e]. The roles and responsibilities remain; the researcher must be well grounded in their understanding of data analysis theory and application. However, the researcher can now use the Chatbot as an assistant to suggest further analyses, conduct some of the analysis, and obtain help with interpreting outputs. At each step of the evolutionary process, there is less pressure on the researcher to know the underlying nuts and bolts of software and more opportunity to focus on the data. Researchers are only beginning to explore all the options that generative Artificial Intelligence offers [\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e]. Understandably, significant shifts are frightening, and the generative AI revolution in the academic world is no different. Luca Mari summarises this experience eloquently (2023, p.29):\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eCopernicus showed us our cosmological non-centrality\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eDarwin showed us our biological non-originality\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eChatbots are showing us our cognitive non-uniqueness\u003c/em\u003e\u003c/p\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n\u003cdiv id=\"Sec5\" class=\"Section3\"\u003e\n\u003cp\u003eSome of our shared fears include the idea that AI will fundamentally replace us, for example, being able to do work for which we were once considered crucial [\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e]. Another fear is that AI will reduce creativity, innovation and original work, replacing our humanity [\u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e]. Hinks [\u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e] found that negative expectations of artificial intelligence's role in our society are associated with reduced life satisfaction. Students are worried about the impact generative AI will have on their development, and the academy is concerned about academic integrity and the impact on assessment [\u003cspan class=\"CitationRef\"\u003e39\u003c/span\u003e]. The Council on Higher Education [\u003cspan class=\"CitationRef\"\u003e40\u003c/span\u003e] suggests that treating generative AI as a \"crime\" or plagiarism outright will negatively impact teaching and learning.\u003c/p\u003e\n\u003cp\u003eHow should we deal with GAI in higher education? To navigate these existential crises and our trepidation, we need to engage with our fears reasonably, as Chatbots are here to stay [\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e]. We must find ethically sound ways to use the new tools to advance human knowledge and development. Our friends from the philosophy of science can help us to think critically about the latest existential crises. Heidegger was worried that technology would strip us of our humanity (forgetting to \"be\"), Idhe was concerned with its role in our daily lives, and Stiegler focused on how technology could disrupt our cultural memories [\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e]. Hui [\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e] merges these ideas and sees technology as the conduit through which we understand our world (he calls this \u0026ldquo;cosmotechnics\u0026rdquo;). Baker and Hui are linked to the idea from Ihde [\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e] that science is the conceptual side of a technological society and that scientists should reimagine the role of technology and its ethical use. Metaphysics, cultural shifts and geopolitical implications cannot be ignored when significant technological changes occur [\u003cspan class=\"CitationRef\"\u003e45\u003c/span\u003e]. New technologies shape the world around us, reshape how we see ourselves and lead to new traditions, potentially replacing our cultural heritages and requiring profound pluralism from us [\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e46\u003c/span\u003e]. In the research world, we measure what we value, pushing us to question what is valuable. While I focus on the practicalities in the current paper, I am also proposing a person-centred, meaning-making approach to scholarship and integrating artificial intelligence into our research [\u003cspan class=\"CitationRef\"\u003e47\u003c/span\u003e]. I explore this idea further by examining how researchers can ethically apply the new GAI tools.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003ch3\u003eEthical and safe use of generative AI in research\u003c/h3\u003e\n\u003cp\u003eThere is nothing inherently wrong with using generative AI during data analysis as long as the researcher does so responsibly [\u003cspan class=\"CitationRef\"\u003e48\u003c/span\u003e]. For the reader's purposes, I recommend using generally established ethical guidelines for generative AI use in research [\u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e50\u003c/span\u003e]:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003ebeing transparent in your use of generative AI during your research processes\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003ebeing responsible for the inputs and how you use the outputs\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eprotecting the data privacy of your participants\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eusing AI for specific purposes in the research process and reporting the use as one would with any software [\u003cspan class=\"CitationRef\"\u003e51\u003c/span\u003e]\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003econsidering and limiting potential bias from the AI platform\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eemphasis on the humanisation and centrality of participants in research\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAs a researcher, you need to understand that Chatbots are tools. The user still needs to have the required foundational knowledge to use the instrument for the intended purpose in the same way you need to know why you select statistical tests when using GUI software.\u003c/p\u003e\n\u003cp\u003eDesigners of Chatbots add guardrails to prevent offensive, inflammatory or dangerous outputs in attempts to protect the users [\u003cspan class=\"CitationRef\"\u003e52\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e53\u003c/span\u003e]. However, jailbreaking still occurs; some individuals can find ways around the guardrails [\u003cspan class=\"CitationRef\"\u003e54\u003c/span\u003e]. Jailbreaking is a complex issue and can be part of the innovative strategies of users. Guardrails have downsides, such as over-correction and replacing one type of bias with another \u0026ndash; users need to be aware of this when employing Chatbots [\u003cspan class=\"CitationRef\"\u003e55\u003c/span\u003e]. While the designers of generative AI try to protect users with guardrails, the researcher needs to understand settings to ensure safer use. To enhance the ethical use of ChatGPT and your data, you can go to privacy settings, choose \u0026ldquo;Settings\u0026rdquo; \u0026ndash; \u0026ldquo;Data control\u0026rdquo; \u0026ndash; \u0026ldquo;Chat History and Training\u0026rdquo;, then switch off the history and training [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. In Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, I show the guidelines for the ethical use of AI and Chatbots in MMR and data analysis.\u003c/p\u003e\n\u003cp\u003eI advise that you refrain from uploading any sensitive data. Ensure you remove any identifying information from data sets before using generative AI to analyse the data. Use resources such as ChatGPT responsibly by rephrasing, repurposing, and reintegrating responses, just as you would information from other sources [\u003cspan class=\"CitationRef\"\u003e56\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e57\u003c/span\u003e]. Critically evaluate all the information you receive from AI platforms and apply your interpretation. Be transparent about how and when you used artificial intelligence in your writing, analysis, and presentation. Cite the Large-Language Model (LLM) you used. Chatbots are sophisticated but only sometimes reliable, explainable and never accountable for what they produce [\u003cspan class=\"CitationRef\"\u003e58\u003c/span\u003e]. Therefore, the researcher is responsible for fair and ethical use.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResearch questions\u003c/strong\u003e\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cem\u003eHow can generative Artificial Intelligence assist with data analysis of Mixed Methods Research (MMR)?\u003c/em\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cem\u003eTo what extent does generative Artificial Intelligence yield reliable and valid findings for MMR data analysis?\u003c/em\u003e\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003eThe data used as a demonstration in the current article comes from a project that focused on enhancing first-year engineering education in South Africa. The data were collected as part of a concurrent triangulation research design, and the findings were reported in various publications (Author \u0026amp; Author, 2020; Author et al., 2022). We wanted to help first year engineering students by understanding student motivation, expectations, and perspectives and how these align with lecturer insights and curriculum design.\u003c/p\u003e\n\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n \u003ch2\u003eInstruments\u003c/h2\u003e\n \u003cp\u003eThe quantitative data were collected using the \u003cem\u003eAcademic Pathways of People Learning Engineering Survey\u003c/em\u003e (APPLES), which contains sections on learning expectations, beneficial and enjoyable educational experiences, skills and abilities required to become an engineer, motivation for studying engineering and post-graduate expectations [\u003cspan class=\"CitationRef\"\u003e59\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e60\u003c/span\u003e]. The APPLES survey was adapted to have forced-choice options, a decision for which evidence is presented in Author (2024). The qualitative data were collected using in-depth interviews for which the researchers designed a guide. The survey contained two open-ended questions where additional qualitative data were collected. In the open-ended questions, students were asked to explain why they chose engineering and provided any other information they deemed appropriate.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\n \u003ch2\u003eSample\u003c/h2\u003e\n \u003cp\u003eThe quantitative data collection yielded a sample of 969 first-year engineering students, who were primarily men (71%), Caucasian (58%) or Black African (26%) and came from middle-class or affluent homes (81%). Students were, on average, 19 years old. The students who participated in the in-depth interviews (14) were all Black African by the study\u0026apos;s design, predominantly male (9 out of 14 participants) and came from varied socio-economic backgrounds.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\n \u003ch2\u003eData Analysis\u003c/h2\u003e\n \u003cp\u003eIn the original analysis, I used the Rasch dichotomous model via Winsteps\u0026copy; 5.4.0.0 [\u003cspan class=\"CitationRef\"\u003e61\u003c/span\u003e] and SPSS (Statistical Package for the Social Sciences) to generate descriptive and inferential statistics [\u003cspan class=\"CitationRef\"\u003e62\u003c/span\u003e]. The qualitative interview data were transcribed and then analysed by two research psychologists using reflexive thematic analysis (TA) (Author et al., 2022). After that, the specialist inspection was conducted by two engineering educators, and the final themes and write-up were the result of a consensus being reached among the team of four researchers [\u003cspan class=\"CitationRef\"\u003e63\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e65\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eFor the current article and demonstration, I reran the analysis through ChatGPT\u0026rsquo;s \u003cem\u003eAdvanced Data Analysis bot\u003c/em\u003e, and my custom settings are shown Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. Custom ChatGPTs are available, and instructions can be customised [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. I recommend that the user try out custom GPTs for themselves; the current tutorial utilises the \u003cem\u003eAdvanced Data Analyst\u003c/em\u003e, one of the original customs ChatGPTs [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e]. Setting up custom instructions by clicking on your profile is recommended, and many good resources are available on the internet for custom instructions [\u003cspan class=\"CitationRef\"\u003e66\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eTo use the custom instruction shown in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, click on your \u003cstrong\u003eprofile icon\u003c/strong\u003e in ChatGPT (bottom left-hand corner), choose the \u003cem\u003eCustom instructions\u003c/em\u003e option, and paste in the text. The text provided is a generic guideline, and can be adjusted to suit specific individual preferences and should reflect the type of research you are conducting [\u003cspan class=\"CitationRef\"\u003e67\u003c/span\u003e].\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eCustom instructions for mixed methods analysis in ChatGPT\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eQuestion\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHeading\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSpecification\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"13\" align=\"left\"\u003e\n \u003cp\u003eWhat would you like ChatGPT to know about you to provide better responses?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\" align=\"left\"\u003e\n \u003cp\u003eKey Responsibilities\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIntegration of qualitative and quantitative data analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCoding, thematic analysis, and statistical analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSynthesis of findings from both data types to generate comprehensive insights\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" align=\"left\"\u003e\n \u003cp\u003eKnowledge or expertise\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eExpertise in qualitative coding thematic analysis, and quantitative statistical methods\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFamiliarity with integrating data from different sources to form cohesive conclusions.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eProficiency in software tools for both qualitative (e.g., NVivo, ATLAS.ti) and quantitative analysis (e.g., SPSS, R)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCurrent Projects\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[Add your projects which reflect mixed methods]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eJargon or Terminology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQualitative coding, Reflexive Thematic Analysis, mixed methods integration, statistical analysis, multimodal data analysis, Rasch Analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"2\" align=\"left\"\u003e\n \u003cp\u003eGoals and Objectives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTo create rich, multidimensional interpretations from diverse data sets, consider context, and apply interdisciplinary knowledge to enrich interpretations.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCreate meaningful interpretations from textual data, consider context, apply interdisciplinary knowledge to interpretations, limit bias\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" align=\"left\"\u003e\n \u003cp\u003eInteractions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEngage in both social science and statistical data analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNarrative and analytical writing for diverse audiences, including academic, policy, and public sectors\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEducational and practical application of mixed methods research findings\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"9\" align=\"left\"\u003e\n \u003cp\u003eHow would you like ChatGPT to respond?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTone\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eProfessional, empathetic, understanding, and clear\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLevel of detail\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eProvide comprehensive narratives that integrate findings from both qualitative and quantitative data, supported by quotes and statistical evidence\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePreferred References\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eScientific papers, psychological and educational frameworks, academic journals, primary sources across both qualitative and quantitative fields\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eExamples or analogies\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUse examples from journal articles, research reports, or thesis chapters that effectively demonstrate the integration of mixed methods\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAvoidance of Ambiguity\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePresent clear, analytical findings that seamlessly blend insights from both qualitative and quantitative research, grounded in theory and empirical evidence.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eResources links\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAccess to interdisciplinary data science libraries, mixed methods research papers, and online courses focusing on integrated data analysis techniques\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFollow up questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eResearch impact, future studies\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eProblem-solving method\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eApply a systematic approach to mixed methods data analysis, detailing the process of qualitative coding, thematic generation, and quantitative statistical analysis, including the integration phase, to draw overarching conclusions\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eConfidence of answers\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eInclude a confidence level with each piece of advice or information provided, especially regarding integrating data and interpreting mixed methods analysis results.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\n \u003ch2\u003eEthical considerations\u003c/h2\u003e\n \u003cp\u003eThe present study was approved by the ethics committee of the Faculty of Engineering, Built Environment \u0026amp; IT (EBIT) at the University of Pretoria (EBIT/46/2020). The study was performed in accordance with the ethical guidelines of the University of Pretoria. All participants in this work have provided freely given, informed consent to participants in the study. The author affirms that the participants also provided informed consent for their data to be used for publication purposes.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Tutorial and Results – Application","content":"\u003cp\u003eHere, I present the roles that generative AI can play in the various types of data analysis. I give prompt examples, and I evaluate the reliability and validity of using ChatGPT for MMR. When asking ChatGPT to conduct analysis, I recommend that you give it frameworks, evaluation criteria or any other standards that it could use to improve outputs [\u003cspan class=\"CitationRef\"\u003e68\u003c/span\u003e]. Ask ChatGPT to be brutally honest, to look for problems and lack of cohesion, to evaluate your writing critically, and to find ways to overcome positivity bias. ChatGPT can act as a research assistant who is available, teachable, and inexpensive. Meet your artificial assistant, ChatGPT, also referred to here as Chad (the only good Chad you will ever meet). The Chad is a na\u0026iuml;ve but well-meaning intern working for $20 monthly. Chad needs much supervision and makes silly mistakes but can add value when managed well. Chad is a well-mannered Chatbot (thanks to the guardrails) with much potential. Unfortunately, he is a people-pleaser, which can sometimes be annoying.\u003c/p\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n \u003ch2\u003eQualitative data analysis with ChatGPT\u003c/h2\u003e\n \u003cp\u003eChatGPT and other similar generative AI platforms have immense capacity to analyse multimodal data [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e]. ChatGPT can play several roles in qualitative analysis, including coding, where you ask ChatGPT to generate codes for textual data based on frameworks or theories. In addition to textual analysis, ChatGPT can detect sentiment, such as the emotional tone of the text [\u003cspan class=\"CitationRef\"\u003e69\u003c/span\u003e]. ChatGPT can also generate themes and evaluate qualitative analysis for consistency by acting as a specialist inspector. The possibility of generating visualisations and tables with ChatGPT-4 is also helpful in quantifying or summarising aspects of the qualitative data. You can paste a limited amount of qualitative text into ChatGPT-3.5. In ChatGPT-4, the \u003cem\u003eAdvanced Data Analysis\u003c/em\u003e module makes it possible to upload data, ask for Excel codes per line, do more checks and comparisons, and create figures. Consider ChatGPT as an additional coder or a research assistant in this process. However, crucially, do your analysis and not rely solely on the Chatbot as it may miss rich aspects of your data or misinterpret the text.\u003c/p\u003e\n \u003cp\u003eIn my experience, inductive coding does not work well in generative Artificial Intelligence settings, and I advise you to avoid asking the Chatbot to cold code. Instead, ChatGPT should be given both an analytical framework and a theoretical or conceptual framework to guide the coding and theme generation, as shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e (see also the Prompt Library in the appendix). Examples of analytical frameworks to consider using include thematic, narrative, content, and discourse analysis. Another suggestion is to code the first few lines or paragraphs to show ChatGPT as examples.\u003c/p\u003e\n \u003cp\u003eChatGPT assigns generic codes such as \u0026quot;other\u0026quot; when unsure or struggling to interpret the text. Such codes and themes require additional investigation as complex and valuable information is often submerged into these commonly named themes. When facing token restrictions, analyse a few interviews at a time; after that, ask ChatGPT to adjust themes based on additional interviews. As can be seen in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, Chad provides well-written themes and quotes the participants as requested. The theoretical framework has also been woven into his answers.\u003c/p\u003e\n \u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e demonstrates the prompt used to ask ChatGPT to code open-ended survey items. I uploaded the Excel file with the responses and labelled the columns where I wanted the bot to add the codes per line.\u003c/p\u003e\n \u003cp\u003eIn addition to the downloadable Excel file Chad produced, reasonable themes were also written, as seen in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e. Checking the Excel file to see how every line was coded was very useful, and many vague statements had been given the code \u0026quot;other\u0026quot;, and I could compare this to the human coding, which was more nuanced and consequential.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\n \u003ch2\u003eQuantitative data analysis with ChatGPT\u003c/h2\u003e\n \u003cp\u003eGenerative Artificial Intelligence (GAI) fulfils additional roles that GUI software did to a limited extent or not at all. ChatGPT can assist with cleaning data, for example, finding out-of-range values in data files. The \u003cem\u003eAdvanced Data Analysis\u003c/em\u003e custom ChatGPT can also scan your data file and recommend analysis. Here, I suggest sharing a codebook with the Chatbot with variables, labels, categories, and any required explanations, such as the Level of measurement. Use the suggestions from the Chatbot with care, as ChatGPT might suggest analysis that cannot or should not be done with the type of data presented. Chad can quickly generate descriptive statistics, as I will demonstrate here, as well as a paragraph with initial interpretation for the researcher to consider using. Commonly used inferential statistics can also be conducted in ChatGPT, with the bonus of interpretation. Currently, generative AI is fine for run-of-the-mill statistical analysis. Using generative AI to run more sophisticated statistical models requires further exploration. For example, I tried to conduct some structural equation modelling (SEM) with ChatGPT, but Chatbot has not yet been able to handle such complex analysis.\u003c/p\u003e\n \u003cp\u003eIn Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, I show the prompt asking for Chad\u0026apos;s recommended data analysis.\u003c/p\u003e\n \u003cp\u003eAs can be seen in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, Chad recommends many different types of analysis for my specific data set. However, not all of them are feasible or applicable. Nonetheless, this is a good starting point and ChatGPT may suggest types of analysis you had yet to consider.\u003c/p\u003e\n \u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e is where I asked ChatGPT to produce descriptive statistics in a specified table format. I also asked for an interpretation and write-up of the table. While I did not fully agree with its interpretation, this again provides a good departure point to help the researcher start the writing process.\u003c/p\u003e\n \u003cp\u003eIn Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e, I illustrate the prompt for asking for inferential statistics from the Chad. Instead of asking for specific analysis, I asked ChatGPT to explore various relationships and to report statistical significance in an interpretative paragraph.\u003c/p\u003e\n \u003cp\u003eAs can be seen from the response, the GAI accurately interprets the \u003cem\u003ep\u003c/em\u003e-values, though it fails to report the effect sizes. I continued prompting beyond this point to obtain all the information I needed. Remember that just like a real assistant, you should keep speaking with the bot until you receive everything you need. See the appendix for the other prompts used.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\n \u003ch2\u003eMixed methods data analysis with ChatGPT\u003c/h2\u003e\n \u003cp\u003eA wide range of MMR analyses can be aided by generative AI, including concurrent triangulation, sequential explanatory and exploratory analysis, transformative designs (here, you must supply the paradigm) and multi-stage evaluations. ChatGPT-4 can also produce more complex figures and tables based on its own or your analysis, for example, side-by-side joint displays, statistics displayed by theme or vice versa and interview questions joint displays [\u003cspan class=\"CitationRef\"\u003e70\u003c/span\u003e]. I have also created some infographics with ChatGPT-4, but these still require much input from the researcher, and it may be easier just to create your own in a different software environment. In ChatGPT-3.5, you can paste your findings from both the quantitative and qualitative analysis and work step by step with the bot to integrate analysis and output. To successfully use ChatGPT for mixed methods analysis, I recommend that you separately analyse the quantitative and qualitative data and combine the findings into a single document before asking the bot to assist with MMR analysis. In Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e, I prompt ChatGPT to use my combined quantitative and qualitative findings document and conduct a sequential explanatory analysis using this study\u0026apos;s theoretical framework. I specified both the analytical and the theoretical (or conceptual) frameworks as this yields the best results. Researchers should use comprehensive frameworks when conducting and analysing MMR data, as shown by Corrigan and Onwuegbuzie [\u003cspan class=\"CitationRef\"\u003e71\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eThe output received in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e was sparse, but further prompting into aspects reported by Chad led to more valuable outputs.\u003c/p\u003e\n \u003cp\u003eBased on the findings, I asked ChatGPT to create a joint table (see Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e8\u003c/span\u003e). While I would not use the table in its current format in a publication, I would use this summary to guide my writing and inform the creation of other graphics.\u003c/p\u003e\n \u003cp\u003eI recommend using generative AI to create initial tables, figures and text based on the quantitative and qualitative data. The final integration should be based on the aims of the study. When asking ChatGPT to create side by side type of tables, check this against your own version to make sure everything relevant is included. Write your interpretations based on your research aims and objectives; do not rely solely on the bot.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\n \u003ch2\u003eReliability and validity of generative AI application\u003c/h2\u003e\n \u003cp\u003eIn Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, I compare the original analysis done by the research team and the outputs received from ChatGPT-4\u0026apos;s \u003cem\u003eAdvanced Data Analysis\u003c/em\u003e Chatbot.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eComparison of original quantitative and qualitative analysis with ChatGPT\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eType\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eOriginal (Researchers)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eChatGPT version\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQualitative interviews\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: The academic program\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Academic and Personal Growth\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQualitative interviews\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Student support\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Transition Challenges\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQualitative interviews\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: The role of peers in student success\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Importance of Social Connections\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQualitative interviews\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Behavioral and attitudinal factors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQuantitative data analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDemographic profile (Table \u0026amp; Text)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDemographic profile (Table \u0026amp; Text)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQuantitative data analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRecommended non-parametric analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRecommended non-parametric but also other not applicable analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eQuantitative data analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNon-parametric analysis and interpretation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNon-parametric analysis done but less detail was provided\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOpen-ended survey questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Intrinsic Psychological Motivation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Personal Interest and Passion\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOpen-ended survey questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Engineering as Social Good\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOpen-ended survey questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Focused on career opportunities\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Career aspirations\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOpen-ended survey questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Intrinsic behavioural motivation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Influence of Technology and Innovation\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOpen-ended survey questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTheme: Curiosity and Problem-Solving\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eComparing the original qualitative analysis done by the team of researchers to the ChatGPT themes, most were similar. The content of the themes is also consistent, but the most significant difference was related to the extent of coverage. ChatGPT found most of the same themes but gave shorter, less rich write-ups. ChatGPT also missed a theme we identified (behavioural and attitudinal factors).\u003c/p\u003e\n \u003cp\u003eIn terms of the quantitative analysis, the Chatbot suggested creating demographic profiles (for example, gender, age, and ethnicity), mean distributions (for continuous variables) and summaries for the categorical variables. Other suggestions included cross-tabulations, non-parametric analysis, factor analysis (for constructs in the survey) and regression analysis to estimate predictors of the continuous variables. All the former suggestions I considered myself and are reasonable. Further suggestions, which I had yet to consider, included cluster analysis to group the students (this was not feasible with the data set), creating profiles within the engineering student population, and outcomes based on educational preferences. These last suggestions were exciting but irrelevant for my data set, once again pointing to the fact that the scholar needs to understand their data and the statistical possibilities. The demographic tables, figures, and interpretations provided by ChatGPT were of high quality and a good match for our own demographic profiles. When I asked ChatGPT to suggest inferential analysis, it did suggest the same non-parametric tests that I initially used, but it also suggested less useful and sometimes irrelevant analysis.\u003c/p\u003e\n \u003cp\u003eWhile descriptive statistics can quickly and reliably be done using ChatGPT, researchers should use their knowledge and judgment of statistics when deciding which tests to run in generative AI. The outputs are deemed acceptable and accurate and could be used in a research report or journal article, with the caveat that the text interpretation should be rewritten. Look for inaccuracies or exaggerations in the Chatbot\u0026rsquo;s output; for example, in my outputs, I found the claim that the sample is diverse to be an exaggeration.\u003c/p\u003e\n \u003cp\u003eThe open-ended survey questions revealed most of the same themes, but again, ChatGPT\u0026apos;s descriptions and unpacking of the themes were shallow when compared to human analysis. ChatGPT also missed a theme when compared to the human coders. At the same time, Chad had a theme that the humans did not identify. This comparison between human and artificial analysis can create avenues for identifying additional themes. ChatGPT contributes to analysis, but human judgement and insight should also be present throughout the process.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e\n \u003ch2\u003eTroubleshooting\u003c/h2\u003e\n \u003cp\u003eAs is the case with all technology, ChatGPT can experience technical difficulties. Troubleshooting for ChatGPT includes clearing the history, cookies, and cache for the \u0026quot;all-time\u0026quot; option. Next, you can restart the session. Lastly, use different browsers (for example, Firefox works well) or disable all the Chrome extensions.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Discussion and key findings","content":"\u003cp\u003eGenerative AI is here to stay, and we must find ways to use it ethically and responsibly [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e]. This is why the current study demonstrates how to conduct MMR analysis and integration using an artificial intelligence platform, ChatGPT. I also acknowledge that all types of technology have drawbacks, and a lack of transparency could call into question our research's methodological integrity, one of the biggest threats we face with GAI [\u003cspan class=\"CitationRef\"\u003e72\u003c/span\u003e]. In this paper, I provide guidelines on using generative AI for mixed methods data analysis while balancing potential pitfalls. The main findings and recommendations from the current article are shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e9\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eIn the quantitative section, I demonstrate how ChatGPT can quickly provide suggestions for data analysis techniques. Most of the suggestions are valid, and the bot has also improved since I used it last year, as it can now more accurately identify parametric from non-parametric statistical models. Providing the Chatbot with a codebook facilitates the data analysis, like setting up variables in software such as SPSS. Generative AI can provide accurate descriptive and inferential statistical outputs, but it\u0026rsquo;s ability to conduct more sophisticated analysis such as structural equation modelling (SEM) has yet to be tested.\u003c/p\u003e\n\u003cp\u003eIn the qualitative section, the Chatbot should be provided with an analytical and a theoretical model to improve the analysis. ChatGPT does not handle inductive coding well, and I suggest providing the bot with examples of how you want it to code. You could also specify if you want it to act as a splitter or lumper. For more informative text, such as interviews, ChatGPT produces coherent narrative themes and can quote participants. It was less useful for shallow qualitative data, such as answers to open-ended questions. I suspect the latter is due to the inherent vagueness of the responses, which even human coders find challenging.\u003c/p\u003e\n\u003cp\u003eMixed methods research (MMR) analysis works best if the quantitative and qualitative analyses are done separately before asking the bot to help integrate the findings. When the QUAL and QUANT are neatly combined in a document with clear headings, a prompt can be provided to the GAI wherein one asks it to help synthesise the findings. Here, I recommend specifying which MMR analytical approach you want it to use. The side-by-side tables it produces are also helpful, but as always, the researcher must add their own insights.\u003c/p\u003e\n\u003cp\u003eThe reliability and validity of straightforward tasks, such as producing descriptive statistics, are perfect in the generative AI environment. More interpretative outputs, such as narratives derived from qualitative data, are less reliable and valid when using ChatGPT. While it can detect most of the same themes, it sometimes misses them. The outputs provided by the GAI are also less rich and tend to be shallower summaries. Therefore, the researcher can consider the GAI an assistant in coding but should still create their own themes and write-ups to compare to bot outputs. Overall, the reliability and validity of using generative AI in analysing MMR data is acceptable. The researcher needs to have the required foundational knowledge and be discerning, rephrasing, repurposing, and reintegrating the outputs before final publication. One of the main advantages of using generative AI is the quick turnaround time for data analysis. Speedier analysis is most definitely desirable in a world where rapid publication is required in higher education [\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eLastly, we may ask ourselves why we would create new knowledge in a world where artificial intelligence can produce equivalent artefacts. Yet there have always been others doing something similar, and who could do it better than we could. Excellent poets, writers, scholars, artists, and every type of creator have existed for millennia. We continue to create despite others doing the same work because everyone has a unique point of view. Different from another person or Chatbot. We create new knowledge as scholars because we need to contextualise what we do for our settings. We add value through our humanity, and others crave that human connection. Human production of creative work will become even more desirable in the future when machines make so much. Creativity is a natural part of who we are \u0026ndash; we are our most authentic selves when we create. Authentic self-expression through knowledge creation can fulfil an identity need to feel connected to ourselves and our world through adding new insights and intellectual artefacts.\u003c/p\u003e\n\u003cdiv id=\"Sec20\" class=\"Section2\"\u003e\n\u003ch2\u003eSignificance of the findings\u003c/h2\u003e\n\u003cp\u003eThe current paper demonstrates innovative ways to use GAI for quantitative, qualitative, and mixed methods data analysis. I also showcase new ways to integrate findings with the assistance of generative AI to improve the interpretation of mixed methods data. The reader is provided with guidelines on using GAI for efficiency and automation to make handling large quantities of data easier. Generative AI misses themes sometimes but can also reveal themes and patterns humans overlook. Therefore, I state the case for combining human insight with generative artificial intelligence to obtain results beyond the abilities of either; the sum of the parts is indeed greater. My study can be used across many disciplines or for cross-disciplinary work where mixed methods are prominent. I have detailed my steps and provided a prompt library for easy replicability and application. The current paper serves as capacity building for researchers globally to advance their use and understanding of AI tools in research methodology. I explicitly addressed ethical considerations in this paper to clarify privacy, consent, and practical implications for researchers who want to use generative AI. My findings here are meant to strengthen digital literacy in a fast-developing technological landscape.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec21\" class=\"Section2\"\u003e\n\u003ch2\u003eLimitations of the study\u003c/h2\u003e\n\u003cp\u003eThe tutorial and application were limited to an available data set, and the example was drawn from higher education. I only used one platform, ChatGPT, and I used the paid version. Using AI in data analysis has its limitations, as discussed in the paper. The quality of the results depends significantly on the knowledge and skills of the researcher. The platform user should have the requisite knowledge of mixed methods data analysis, synthesis, and application. Chatbots cannot replace human insight.\u003c/p\u003e\n\u003cp\u003eFurthermore, the fact that bias and unfairness are built into artificial generative intelligence is well known [\u003cspan class=\"CitationRef\"\u003e73\u003c/span\u003e]. Therefore, the researcher must actively manage the inputs and how the outputs are used, aware that there may be systemic, computational and human biases embedded in chats [\u003cspan class=\"CitationRef\"\u003e74\u003c/span\u003e]. The landscape of generative AI is also changing rapidly [\u003cspan class=\"CitationRef\"\u003e75\u003c/span\u003e], and the current paper is limited to the time and space in which it was written. ChatGPT is not the only platform available to researchers, and future researchers could look at platforms such as Perplexity (see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.perplexity.ai/\u003c/span\u003e\u003c/span\u003e), Jenni (see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://jenni.ai/\u003c/span\u003e\u003c/span\u003e) and Consensus (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://consensus.app/\u003c/span\u003e\u003c/span\u003e).\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eFunding statement\u003c/h2\u003e\n\u003cp\u003eNo funding was received for the current work.\u003c/p\u003e\n\u003ch2\u003eConflicts of interest\u003c/h2\u003e\n\u003cp\u003eThere is no conflict of interest.\u003c/p\u003e\n\u003ch2\u003eData Availability\u003c/h2\u003e\n\u003cp\u003eThe datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003ch2\u003eCode availability\u003c/h2\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003ch2\u003eConsent statement\u003c/h2\u003e\n\u003cp\u003eAll participants in this work have provided freely-given informed consent to participants in the study. The author affirms that the participants also provided informed consent for their data to be used for publication purposes.\u003c/p\u003e\n\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\n\u003cp\u003eI am the only author of this paper; I confirm the single authorship.\u003c/p\u003e\n\u003ch2\u003eAcknowledgement\u003c/h2\u003e\n\u003cp\u003eThe Living Lab for Innovative Teaching at the University of Pretoria (LLITUP) supported the development of a workshop to test the ideas presented in this paper. I am very grateful to them for their time, inputs and congeniality.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eHoward, J., \u003cem\u003eArtificial intelligence: Implications for the future of work.\u003c/em\u003e American Journal of Industrial Medicine, 2019. \u003cstrong\u003e62\u003c/strong\u003e(11): p. 917-926.\u003c/li\u003e\n\u003cli\u003eDwivedi, Y.K., et al., \u003cem\u003eArtificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy.\u003c/em\u003e International Journal of Information Management, 2021. \u003cstrong\u003e57\u003c/strong\u003e: p. 101994.\u003c/li\u003e\n\u003cli\u003e\u0026Aring;gerfalk, P.J., \u003cem\u003eArtificial intelligence as digital agency.\u003c/em\u003e European Journal of Information Systems, 2020. \u003cstrong\u003e29\u003c/strong\u003e(1): p. 1-8.\u003c/li\u003e\n\u003cli\u003eOoi, K.-B., et al., \u003cem\u003eThe potential of generative artificial intelligence across disciplines: Perspectives and future directions.\u003c/em\u003e Journal of Computer Information Systems, 2023: p. 1-32.\u003c/li\u003e\n\u003cli\u003eDewey, J., M.C. Pautz, and M.K. Diede, \u003cem\u003eHow do we Address Faculty Burnout? Start by Exploring Faculty Motivation.\u003c/em\u003e Innovative Higher Education, 2023.\u003c/li\u003e\n\u003cli\u003eStrzelecki, A., \u003cem\u003eStudents\u0026rsquo; Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology.\u003c/em\u003e Innovative Higher Education, 2023.\u003c/li\u003e\n\u003cli\u003eBahroun, Z., et al., \u003cem\u003eTransforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis.\u003c/em\u003e Sustainability, 2023. \u003cstrong\u003e15\u003c/strong\u003e(17): p. 12983.\u003c/li\u003e\n\u003cli\u003eWikipedia. \u003cem\u003eChatGPT\u003c/em\u003e. 2024 [cited 2024, March 4; Available from: https://en.wikipedia.org/wiki/ChatGPT.\u003c/li\u003e\n\u003cli\u003eRawas, S., \u003cem\u003eChatGPT: Empowering lifelong learning in the digital age of higher education.\u003c/em\u003e Education and Information Technologies, 2023.\u003c/li\u003e\n\u003cli\u003eRuiz-Rojas, L.I., et al., \u003cem\u003eEmpowering education with generative artificial intelligence tools: Approach with an instructional design matrix.\u003c/em\u003e Sustainability, 2023. \u003cstrong\u003e15\u003c/strong\u003e(15): p. 11524.\u003c/li\u003e\n\u003cli\u003eRatten, V. and P. Jones, \u003cem\u003eGenerative artificial intelligence (ChatGPT): Implications for management educators.\u003c/em\u003e The International Journal of Management Education, 2023. \u003cstrong\u003e21\u003c/strong\u003e(3): p. 100857.\u003c/li\u003e\n\u003cli\u003eHmoud, M., et al., \u003cem\u003eHigher Education Students\u0026rsquo; Task Motivation in the Generative Artificial Intelligence Context: The Case of ChatGPT.\u003c/em\u003e Information, 2024. \u003cstrong\u003e15\u003c/strong\u003e(1): p. 33.\u003c/li\u003e\n\u003cli\u003eKoh, E. and S. Doroudi, \u003cem\u003eLearning, teaching, and assessment with generative artificial intelligence: towards a plateau of productivity.\u003c/em\u003e Learning: Research and Practice, 2023. \u003cstrong\u003e9\u003c/strong\u003e(2): p. 109-116.\u003c/li\u003e\n\u003cli\u003eDempere, J., et al., \u003cem\u003eThe impact of ChatGPT on higher education.\u003c/em\u003e Dempere J, Modugu K, Hesham A and Ramasamy LK (2023) The impact of ChatGPT on higher education. Front. Educ, 2023. \u003cstrong\u003e8\u003c/strong\u003e: p. 1206936.\u003c/li\u003e\n\u003cli\u003eIslam, I. and M.N. Islam, \u003cem\u003eExploring the opportunities and challenges of ChatGPT in academia.\u003c/em\u003e Discover Education, 2024. \u003cstrong\u003e3\u003c/strong\u003e(31).\u003c/li\u003e\n\u003cli\u003eDavidson, T., \u003cem\u003eStart generating: Harnessing generative artificial intelligence for sociological research.\u003c/em\u003e Retrieved from osf. io/preprints/socarxiv/u9nft, 2023.\u003c/li\u003e\n\u003cli\u003ePack, A. and J. Maloney, \u003cem\u003eUsing Generative Artificial Intelligence for Language Education Research: Insights from Using \u0026lt;scp\u0026gt;OpenAI\u0026lt;/scp\u0026gt;\u0026apos;s \u0026lt;scp\u0026gt;ChatGPT\u0026lt;/scp\u0026gt;.\u003c/em\u003e TESOL Quarterly, 2023. \u003cstrong\u003e57\u003c/strong\u003e(4): p. 1571-1582.\u003c/li\u003e\n\u003cli\u003eGoodfellow, I., et al., \u003cem\u003eGenerative adversarial networks.\u003c/em\u003e Communications of the ACM, 2020. \u003cstrong\u003e63\u003c/strong\u003e(11): p. 139-144.\u003c/li\u003e\n\u003cli\u003eAdamopoulou, E. and L. Moussiades. \u003cem\u003eAn overview of chatbot technology\u003c/em\u003e. in \u003cem\u003eIFIP international conference on artificial intelligence applications and innovations\u003c/em\u003e. 2020. Springer.\u003c/li\u003e\n\u003cli\u003eHadi, M.U., et al., \u003cem\u003eA survey on large language models: Applications, challenges, limitations, and practical usage.\u003c/em\u003e Authorea Preprints, 2023.\u003c/li\u003e\n\u003cli\u003eKorteling, J., et al., \u003cem\u003eHuman-versus artificial intelligence.\u003c/em\u003e Frontiers in artificial intelligence, 2021. \u003cstrong\u003e4\u003c/strong\u003e: p. 622364.\u003c/li\u003e\n\u003cli\u003eCooms, S. and V. Saunders, \u003cem\u003ePoetic inquiry: a tool for decolonising qualitative research.\u003c/em\u003e Qualitative Research Journal, 2024. \u003cstrong\u003e24\u003c/strong\u003e(1): p. 3-22.\u003c/li\u003e\n\u003cli\u003eOpenAI. \u003cem\u003eChatGPT (September 25 Version) [Large language model].\u003c/em\u003e 2024; Available from: https://chat.openai.com.\u003c/li\u003e\n\u003cli\u003eBarros, A., A. Prasad, and M. Śliwa, \u003cem\u003eGenerative artificial intelligence and academia: Implication for research, teaching and service.\u003c/em\u003e Management Learning, 2023. \u003cstrong\u003e54\u003c/strong\u003e(5): p. 597-604.\u003c/li\u003e\n\u003cli\u003eCostello, G.R., K.R. Davis, and O.S. Crocco, \u003cem\u003eLearning by Doing: Student \u0026amp; Faculty Reflections on a Collaborative Model for Conducting and Publishing Mixed Methods Research in a Graduate Course.\u003c/em\u003e Innovative Higher Education, 2022. \u003cstrong\u003e47\u003c/strong\u003e(6): p. 1067-1084.\u003c/li\u003e\n\u003cli\u003eMalina, M.A., H.S. N\u0026oslash;rreklit, and F.H. Selto, \u003cem\u003eLessons learned: advantages and disadvantages of mixed method research.\u003c/em\u003e Qualitative Research in Accounting \u0026amp; Management, 2011. \u003cstrong\u003e8\u003c/strong\u003e(1): p. 59-71.\u003c/li\u003e\n\u003cli\u003eOnwuegbuzie, A.J. and C. Poth, \u003cem\u003eSpecial Issue: Mixed Methods.\u003c/em\u003e International Journal of Qualitative Methods, 2015. \u003cstrong\u003e14\u003c/strong\u003e(2): p. 122-125.\u003c/li\u003e\n\u003cli\u003eCoolidge, F.L., \u003cem\u003eStatistics: A gentle introduction\u003c/em\u003e. 2020: Sage Publications.\u003c/li\u003e\n\u003cli\u003eChambers, J.M., \u003cem\u003eStatistical Computing: History and Trends.\u003c/em\u003e The American Statistician, 1980. \u003cstrong\u003e283\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eAbatan, S.M. and M. Olayemi, \u003cem\u003eThe Role of Statistical Software in Data Analysis \u003c/em\u003eInternational Journal of Applied Research and Studies (iJARS) 2014. \u003cstrong\u003e3\u003c/strong\u003e(8).\u003c/li\u003e\n\u003cli\u003eKlinke, S. \u003cem\u003eStatistical user interfaces\u003c/em\u003e. Center for Applied Statistics and Economics (CASE) REport, 2004. \u003cstrong\u003e35\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eMartinez, W.L., \u003cem\u003eGraphical user interfaces.\u003c/em\u003e Wiley Interdisciplinary Reviews: Computational Statistics, 2011. \u003cstrong\u003e3\u003c/strong\u003e(2): p. 119-133.\u003c/li\u003e\n\u003cli\u003eField, A.P., \u003cem\u003eDiscovering statistics using IBM SPSS statistics\u003c/em\u003e. 2024: Sage publications limited.\u003c/li\u003e\n\u003cli\u003eVindrola-Padros, C. and G.A. Johnson, \u003cem\u003eRapid Techniques in Qualitative Research: A Critical Review of the Literature.\u003c/em\u003e Qualitative Health Research, 2020. \u003cstrong\u003e30\u003c/strong\u003e(10): p. 1596-1604.\u003c/li\u003e\n\u003cli\u003eKantor, J., \u003cem\u003eBest practices for implementing ChatGPT, large language models, and artificial intelligence in qualitative and survey-based research.\u003c/em\u003e JAAD international, 2024. \u003cstrong\u003e14\u003c/strong\u003e: p. 22-23.\u003c/li\u003e\n\u003cli\u003eCave, S. and K. Dihal, \u003cem\u003eHopes and fears for intelligent machines in fiction and reality.\u003c/em\u003e Nature machine intelligence, 2019. \u003cstrong\u003e1\u003c/strong\u003e(2): p. 74-78.\u003c/li\u003e\n\u003cli\u003eMcClure, P.K., \u003cem\u003e\u0026ldquo;You\u0026rsquo;re fired,\u0026rdquo; says the robot: The rise of automation in the workplace, technophobes, and fears of unemployment.\u003c/em\u003e Social Science Computer Review, 2018. \u003cstrong\u003e36\u003c/strong\u003e(2): p. 139-156.\u003c/li\u003e\n\u003cli\u003eHinks, T., \u003cem\u003eArtificial Intelligence Perceptions and Life Satisfaction.\u003c/em\u003e Journal of Happiness Studies, 2024. \u003cstrong\u003e25\u003c/strong\u003e(1): p. 1-14.\u003c/li\u003e\n\u003cli\u003eSmolansky, A., et al. \u003cem\u003eEducator and student perspectives on the impact of generative AI on assessments in higher education\u003c/em\u003e. in \u003cem\u003eProceedings of the tenth ACM conference on Learning@ Scale\u003c/em\u003e. 2023.\u003c/li\u003e\n\u003cli\u003eCouncil on Higher Education, C. \u003cem\u003eA Closer Look at Plagiarism within the Context of Student Academic Writing in an Era of Generative Artificial Intelligence\u003c/em\u003e. Briefly Speaking, 2024. \u003cstrong\u003e27\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eAtkinson, R., \u003cem\u003eDon\u0026apos;t fear AI (Volume 2)\u003c/em\u003e. 2019: European Investment Bank.\u003c/li\u003e\n\u003cli\u003eBaker, M., \u003cem\u003eThe Western Mathematic and the Ontological Turn: Ethnomathematics and Cosmotechnics for the Pluriverse\u003c/em\u003e, in \u003cem\u003eIndigenous Knowledge and Ethnomathematics\u003c/em\u003e. 2023, Springer. p. 243-276.\u003c/li\u003e\n\u003cli\u003eHui, Y., \u003cem\u003eFor a cosmotechnical event: In honor of Don Ihde and Bernard Stiegler.\u003c/em\u003e Reimagining Philosophy and Technology, Reinventing Ihde, 2020: p. 87-102.\u003c/li\u003e\n\u003cli\u003eIhde, D., \u003cem\u003eTechnology and the lifeworld: From garden to earth.\u003c/em\u003e 1990.\u003c/li\u003e\n\u003cli\u003eFisher, W.P., \u003cem\u003eMeaning and Method in the Social Sciences1.\u003c/em\u003e Human Studies, 2004. \u003cstrong\u003e27\u003c/strong\u003e(4): p. 429-454.\u003c/li\u003e\n\u003cli\u003eRoth, P.A., \u003cem\u003eMeaning and method in the social sciences: A case for methodological pluralism\u003c/em\u003e. 2019: Cornell University Press.\u003c/li\u003e\n\u003cli\u003eFisher Jr, W.P. and S.J. Cano, \u003cem\u003eIdeas and Methods in Person-Centered Outcome Metrology.\u003c/em\u003e Person-Centered Outcome Metrology, 2023: p. 1.\u003c/li\u003e\n\u003cli\u003eSchlagwein, D. and L. Willcocks, \u003cem\u003e\u0026lsquo;ChatGPT et al.\u0026rsquo;: The ethics of using (generative) artificial intelligence in research and science.\u003c/em\u003e Journal of Information Technology, 2023. \u003cstrong\u003e38\u003c/strong\u003e(3): p. 232-238.\u003c/li\u003e\n\u003cli\u003eRothenberger, L., B. Fabian, and E. Arunov. \u003cem\u003eRelevance of Ethical Guidelines for Artificial Intelligence-a Survey and Evaluation\u003c/em\u003e. in \u003cem\u003eECIS\u003c/em\u003e. 2019.\u003c/li\u003e\n\u003cli\u003eSkorburg, J.A., K. O\u0026apos;Doherty, and P. Friesen, \u003cem\u003ePersons or data points? Ethics, artificial intelligence, and the participatory turn in mental health research.\u003c/em\u003e American Psychologist, 2024. \u003cstrong\u003e79\u003c/strong\u003e(1): p. 137.\u003c/li\u003e\n\u003cli\u003eO\u0026rsquo;Kane, P., A. Smith, and M.P. Lerman, \u003cem\u003eBuilding transparency and trustworthiness in inductive research through computer-aided qualitative data analysis software.\u003c/em\u003e Organizational Research Methods, 2021. \u003cstrong\u003e24\u003c/strong\u003e(1): p. 104-139.\u003c/li\u003e\n\u003cli\u003eWang, Y. and L. Singh, \u003cem\u003eAdding guardrails to advanced chatbots.\u003c/em\u003e arXiv preprint arXiv:2306.07500, 2023.\u003c/li\u003e\n\u003cli\u003eAbdelkader, H., et al., \u003cem\u003eML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study.\u003c/em\u003e arXiv preprint arXiv:2401.06513, 2024.\u003c/li\u003e\n\u003cli\u003eBoxleitner, A., \u003cem\u003ePushing Boundaries or Crossing Lines? The Complex Ethics of ChatGPT Jailbreaking.\u003c/em\u003e The Complex Ethics of ChatGPT Jailbreaking (October 17, 2023), 2023.\u003c/li\u003e\n\u003cli\u003eKamalov, F., D. Santandreu Calonge, and I. Gurrib, \u003cem\u003eNew era of artificial intelligence in education: Towards a sustainable multifaceted revolution.\u003c/em\u003e Sustainability, 2023. \u003cstrong\u003e15\u003c/strong\u003e(16): p. 12451.\u003c/li\u003e\n\u003cli\u003eCheng, M.W.T. and I.H.Y. Yim, \u003cem\u003eExamining the use of ChatGPT in public universities in Hong Kong: a case study of restricted access areas.\u003c/em\u003e Discover Education, 2024. \u003cstrong\u003e3\u003c/strong\u003e(1).\u003c/li\u003e\n\u003cli\u003eDaniela, F.-T. \u003cem\u003eAcademic writing and ChatGPT: Students transitioning into college in the shadow of the COVID-19 pandemic\u003c/em\u003e. Discover Education, 2024. \u003cstrong\u003e3\u003c/strong\u003e, 1-10 DOI: 10.1007/s44217-023-00076-5.\u003c/li\u003e\n\u003cli\u003eMari, L., \u003cem\u003eChatbots: facing a cultural revolution and trying to understand it (a non-technical perspective)\u003c/em\u003e, B. Seminar, Editor. 2023: UC Berkeley.\u003c/li\u003e\n\u003cli\u003eDonaldson, K.M., et al., \u003cem\u003eScaling up: Taking the Academic Pathways of People Learning Engineering Survey (APPLES) national\u003c/em\u003e, in \u003cem\u003e2008 IEEE Frontiers in Education Conference\u003c/em\u003e. 2008, IEEE Computer Society. p. F4H-6-F4H-11.\u003c/li\u003e\n\u003cli\u003eSheppard, S.D., et al., \u003cem\u003eExploring the Engineering Student Experience: Findings from the Academic Pathways of People Learning Engineering Survey (APPLES)\u003c/em\u003e. 2010, Center for the Advancement of Engineering Education Technical Report CAEE-TR-10-01.\u003c/li\u003e\n\u003cli\u003eLinacre, J.M., \u003cem\u003eWinsteps\u0026reg; (Version 5.4.0.0) \u003c/em\u003e2023: Portland, Oregon: Winsteps.com.\u003c/li\u003e\n\u003cli\u003eIBM, \u003cem\u003eIBM SPSS Statistics for Windows (Version 28.0)\u003c/em\u003e. 2023, IBM Corp.\u003c/li\u003e\n\u003cli\u003eBraun and Clarke, \u003cem\u003eOne size fits all? What counts as quality practice in (reflexive) thematic analysis?\u003c/em\u003e Qualitative Research in Psychology, 2021. \u003cstrong\u003e18\u003c/strong\u003e(3): p. 328-352.\u003c/li\u003e\n\u003cli\u003eBraun, V. and V. Clarke, \u003cem\u003eThematic analysis : a practical guide\u003c/em\u003e. 2022, London ;: SAGE.\u003c/li\u003e\n\u003cli\u003eBraun, V. and V. Clarke, \u003cem\u003eUsing thematic analysis in psychology.\u003c/em\u003e Qualitative Research in Psychology, 2006. \u003cstrong\u003e3\u003c/strong\u003e(2): p. 77-101.\u003c/li\u003e\n\u003cli\u003eSurach, D., \u003cem\u003eAI foundations: Learn to leverage market moving AI tools\u003c/em\u003e. 2024, YouTube: YouTube.\u003c/li\u003e\n\u003cli\u003eSurach, D., \u003cem\u003e5 ChatGPT Hacks | Take ChatGPT to the NEXT Level!\u003c/em\u003e, in \u003cem\u003eAI Foundations\u003c/em\u003e. 2023: YouTube.\u003c/li\u003e\n\u003cli\u003eStapleton, A., \u003cem\u003eHow To Write An A+ Essay Using AI in 3 Simple Steps\u003c/em\u003e. 2023: YouTube.\u003c/li\u003e\n\u003cli\u003eOwoahene Acheampong, K. and M. Nyaaba, \u003cem\u003eReview of Qualitative Research in the Era of Generative Artificial Intelligence.\u003c/em\u003e Matthew, Review of Qualitative Research in the Era of Generative Artificial Intelligence (January 7, 2024), 2024.\u003c/li\u003e\n\u003cli\u003eGuetterman, T.C., M.D. Fetters, and J.W. Creswell, \u003cem\u003eIntegrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays.\u003c/em\u003e Ann Fam Med, 2015. \u003cstrong\u003e13\u003c/strong\u003e(6): p. 554-61.\u003c/li\u003e\n\u003cli\u003eCorrigan, J.A. and A.J. Onwuegbuzie, \u003cem\u003eToward a meta-framework for conducting mixed methods representation analyses to optimize meta-inferences.\u003c/em\u003e 2020.\u003c/li\u003e\n\u003cli\u003eGrimes, M., et al., \u003cem\u003eFrom scarcity to abundance: Scholars and scholarship in an age of generative artificial intelligence.\u003c/em\u003e Academy of Management Journal, 2023. \u003cstrong\u003e66\u003c/strong\u003e(6): p. 1617-1624.\u003c/li\u003e\n\u003cli\u003eSchwartz, R., et al., \u003cem\u003eTowards a standard for identifying and managing bias in artificial intelligence.\u003c/em\u003e NIST special publication, 2022. \u003cstrong\u003e1270\u003c/strong\u003e(10.6028).\u003c/li\u003e\n\u003cli\u003eHwang, G.-J., et al., \u003cem\u003eVision, challenges, roles and research issues of Artificial Intelligence in Education\u003c/em\u003e. 2020, Elsevier. p. 100001.\u003c/li\u003e\n\u003cli\u003eNishant, R., M. Kennedy, and J. Corbett, \u003cem\u003eArtificial intelligence for sustainability: Challenges, opportunities, and a research agenda.\u003c/em\u003e International Journal of Information Management, 2020. \u003cstrong\u003e53\u003c/strong\u003e.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Appendix A","content":"\u003ch2\u003eAppendix A \u0026ndash;Prompt Library\u003c/h2\u003e\n\u003ch2\u003ePrompts for Qualitative Analysis\u003c/h2\u003e\n\u003cp\u003eSquare brackets indicate aspects you should modify to suit your need.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 1\u003c/strong\u003e \u0026ndash; You are a qualitative researcher. Use [Reflexive thematic analysis], according to Braun and Clark, to analyse the interview and generate themes. Use the [Self-Determination Theory] to interpret the data. The themes should be written as narrative findings in paragraphs, give each theme a name that reflects the main finding of that theme. Use quotes from the interviewee to support your themes. Only analyse the interviewee responses.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 2\u003c/strong\u003e \u0026ndash; Attached are open-ended responses. Code and classify the responses according to [theory] so that you create themes, write up the themes as narratives with quotes from respondents to support your argument. Cite the frequencies of how many times a theme was mentioned. Make the excel sheet available with the codes completed.\u003c/p\u003e\n\u003ch2\u003ePrompts for Quantitative Analysis\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 3\u003c/strong\u003e \u0026ndash; I have cross-sectional survey data, please recommend analyses I could do with my data set. The variables include [demographic questions, such as gender, age, and race]. The constructs measured in the questionnaire include [motivation to attend university and long-term career goals]. [Codebook pasted with prompt].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 4\u003c/strong\u003e \u0026ndash; Summarise the constructs [add list here] in the attached data set to identify trends. Create a table and write a paragraph to illustrate your findings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 5\u003c/strong\u003e \u0026ndash; Please create demographic profiles for my data. Create a table in [APA 7] style as the output with a paragraph which briefly describes the student profile.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 6\u003c/strong\u003e \u0026ndash; Redo the analysis, consider that most of the demographic variables are categorial in nature. Add the categories to the table to make it easier for the reader to understand.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 7 -\u003c/strong\u003e Based on the previous data, run inferential statistical analysis and identify any items that are significant. Include p-values and effect sizes. Show the results in tables with a summary of your interpretation.\u003c/p\u003e\n\u003ch2\u003ePrompts for Mixed Method Analysis\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 8\u003c/strong\u003e \u0026ndash; I have findings from [a questionnaire], and I also conducted [in-depth interviews]. The results from the quantitative and qualitative analyses are attached. Please conduct a [sequential explanatory analysis] and help me to identify patterns, achieve triangulation and integrate the results. Use the [Incentive Theory] to interpret the results. Write two pages based on your sequential explanatory analysis.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 9\u003c/strong\u003e \u0026ndash; Based on the findings from the previous prompt, create a statistics-by-themes joint table display of the findings. Make the table a downloadable Excel.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eOR\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eCreate a side-by-side joint display of the quantitative and qualitative findings. Make the display downloadable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 10\u003c/strong\u003e \u0026ndash; Produce a two-page document of your findings and ensure your report clearly distinguishes between quantitative and qualitative findings and how they integrate. Based on your findings, suggest practical or theoretical implications. Reflect on the effectiveness of the mixed-methods approach for the data collected.\u003c/p\u003e\n\u003ch2\u003ePrompts for obtaining more from Chatbots\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 11\u003c/strong\u003e \u0026ndash; Please give me more information about your analysis and findings\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 12 -\u003c/strong\u003e You are the student\u0026apos;s supervisor. Evaluate the problem statement according to the guidelines. Identify the short-comings and list them. Give the student valuable tips to improve the problem statement. Problem statement guidelines: a) What do we already know about the problem? (Use recent and relevant studies to substantiate, preferably studies no older than 10 years) b) What do we need to know about the problem? (Gap in literature/scholarly field) c) Why does it matter? (importance of the study) d) Conclude with how your study will address the problem.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 13 -\u003c/strong\u003e I want you to evaluate the writing. Act as a critic; be ruthless. Analyse the text and tell me where it can be better.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrompt 14 -\u003c/strong\u003e Please write a review of the attached document and highlight both the strengths and the weaknesses of the submission. Please be as constructive and specific as possible when offering recommendations. Use the following guidelines when examining the document [insert journal guidelines or evaluation criteria here]\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"discover-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"diedu","sideBox":"Learn more about [Discover Education](https://www.springer.com/journal/44217)","snPcode":"44217","submissionUrl":"https://submission.nature.com/new-submission/44217/3","title":"Discover Education","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Generative Artificial Intelligence (AI or GAI), Chat Generative Pre-Trained Transformer (ChatGPT) also known as Chad, Mixed Methods Research (MMR), Large-Language Models (LLMs), Chatbots, data analysis techniques","lastPublishedDoi":"10.21203/rs.3.rs-4176435/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4176435/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eScholars from all disciplines can benefit from learning how to use generative Artificial Intelligence (GAI or AI) for data analysis. The current article used real data to demonstrate the analysis and synthesis of mixed methods research (MMR) data with generative AI. I also explore how reliable and valid data outputs are and how to improve the use of generative AI for research. The demonstration data used emanated from a study done in South Africa, with a quantitative sample size of 969 first-year engineering students and, for the qualitative part, 14 second-year students. In the current article, I compare my original analysis to ChatGPT results. Generative AI is a mind tool that is ideal when utilised with human insight to check the cohesion, consistency, and accuracy of the data analysis. The current content is geared towards enhancing methodological application regardless of field or discipline and includes access to a prompt library and examples of using outputs. For the qualitative analysis, I found that ChatGPT could detect similar themes but missed some, and its write-up was shallower than our human version. The quantitative analysis was accurate for the descriptive statistics, but the researcher had to use best judgment to select the correct inferential analysis. A quantitative and qualitative analysis should be conducted separately in generative AI before asking the bot for help with mixed methods research. I give guidelines and a tutorial on how to use chatbots in an ethically responsible and scientifically sound manner for research in social and human sciences.\u003c/p\u003e","manuscriptTitle":"Enhancing Mixed Methods Research with your generative AI assistant – A Tutorial and Evaluation for Scholars","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-19 16:37:57","doi":"10.21203/rs.3.rs-4176435/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-05-13T05:36:14+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-02T19:16:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"190113181708950905526704203032320706766","date":"2024-05-01T17:13:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-04-30T00:48:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"48092631003330035062846049323706380421","date":"2024-04-28T16:49:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"231528158239942193654110738297375499194","date":"2024-04-26T14:24:17+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-04-19T11:58:35+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-04-19T11:26:49+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-04-15T11:21:23+00:00","index":"","fulltext":""},{"type":"submitted","content":"Discover Education","date":"2024-03-27T13:26:26+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"discover-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"diedu","sideBox":"Learn more about [Discover Education](https://www.springer.com/journal/44217)","snPcode":"44217","submissionUrl":"https://submission.nature.com/new-submission/44217/3","title":"Discover Education","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Discover Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"aaebb94b-2353-4618-be92-a5cecec624cf","owner":[],"postedDate":"April 19th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2024-06-17T06:26:57+00:00","versionOfRecord":[],"versionCreatedAt":"2024-04-19 16:37:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4176435","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4176435","identity":"rs-4176435","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}