Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine.

Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Case Report Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine. Cameron Frederick Atkinson This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4007725/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Systematically gathering grey literature for use in a systematic literature review (SLR) is a challenging task, given the decentralised nature of online resources. Researchers, particularly those in the social sciences, often find themselves navigating government or non-government organisation websites, manually scouring repositories for documents relevant to their SLRs. This important stage is expensive in terms of time and resources required and, in many instances, difficult to repeat. This article introduces an innovative methodology to address these challenges, providing social science researchers with a systematic approach to gather grey literature for inclusion in SLRs. Utilising the computer programming language Python, this articles leverages Google’s API to create a programmable search engine, facilitating a systematic search for grey literature intended for inclusion in a SLR. A case study is presented to demonstrate the efficiency of this method in locating PDF documents, within which two examples are provided. In the first example, documents from Australian government websites ending in “gov.au” related to the topic of (“energy infrastructure” AND resilience) are sought. Secondly, “un.org” is added to illustrate how multiple websites can be searched. Highlighting the effectiveness of this approach, the study successfully locates 100 documents in just 7.5 seconds, automatically saving them into an Excel CSV file for further analysis. To the authors knowledge, this method represents an original approach in the systematic gathering of grey literature for SLRs and highlights the contribution of generative artificial intelligence systems such as ChatGPT 3.5 in assisting to script the necessary code for new SLR tools. Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1. INTRODUCTION The systematic literature review (SLR) is commonly regarded as the pinnacle of evidence gathering and evidence synthesis [ 1 , 2 ]. SLRs seek to gather, appraise, and synthesise evidence pertaining to a phenomenon under investigation [ 3 ]. To accomplish this, SLRs entail comprehensive searches of research databases, aiming to identify all relevant work within the field under review [ 4 , 5 ]. The motivations behind a SLR are multifaceted: (1) to publicise ongoing research and prevent duplications of research; (2) to inform the planning of new research initiatives; (3) to substantiate claims of originality by contrasting existing research against new research [ 3 , 6 ]. This article aims to highlight an innovative approach for researchers, particularly social science researchers engaged in SLRs, to streamline the laborious and time-consuming task of collecting grey literature. Beyond addressing temporal and financial constraints, SLRs are susceptible to becoming outdated due to their prolonged duration required for completion [ 7 – 9 ]. Additionally, the increasing volume of reports, among other grey literatures, presents a challenge for researchers to keep pace with the growing output. Integrating new tools, such as computational methods and generative artificial intelligence (GenAI), can indeed assist in managing and analysing this vast amount of information [ 10 ]. These technologies can enhance the efficiency, transparency, and rigor of information gathering, providing valuable support for social scientists and researchers in dealing with the expanding landscape of grey literature. Consequently, techniques that enhance the speed, efficiency, and rigor of this process prove advantageous for all SLRs. Grey literature represents one element of the broader category known as grey information. Introduced by [ 11 ], grey information serves as a classification encompassing various forms of ‘grey’; including data, literature, and information. In this context, 'grey data' pertains to user-generated web content like tweets or blogs, 'grey literature' encompasses policy documents, standards, and regulations, while 'grey information' involves informal data such as meeting notes or emails [ 12 , 13 ]. This is not the first instance of creating ways to systematically gather grey literature for use in a SLR. For example, [ 14 ] highlights how web-scraping can be employed to gather grey literature from online repositories. In SLRs, grey literature consistently proves to be among the most challenging types of literature to systematically collect [ 15 ]. Bibliographic databases, a mainstay of gathering information pertinent to conducting a SLR, are generally restricted in their ability to locate relevant grey literature [ 16 ]. Therefore, the difficulties associated with the systematic gathering of grey literature can result in its haphazard inclusion or, at times, in its complete exclusion from a review, introducing a research bias into a project [ 17 – 19 ]. In an effort to circumvent these biases, researchers can invest months in the rigorous collection of grey literature, sometimes employing processes that lack repeatability and introduce additional biases [ 11 ]. The issues encountered with the replicability of research contributes to an ongoing methodological challenge, making it difficult, if not impossible, to replicate the findings of numerous scientific projects [ 20 ]. To address the replicability issues surrounding the gathering of grey literature, this article introduces a new methodology to systematically gather grey literature for use in a SLR. Leveraging the power provided by Google’s API to create Programmable Search Engines (PSEs), this article harnesses the robust coding ability of GenAI systems such as ChatGPT 3.5 and employs the computer programming language Python to systematically gather PDF documents from Australian government websites with the (gov.au) and (un.org) URLs. It should be noted here that the method put forward automates what would normally be extensive and manual Google searches. There are no algorithms or machine learning techniques required for this method. However, end-users of this method will need to acquaint themselves with the Pythonic script utilised. Although relatively straightforward, this article does assume some familiarity with coding on the part of the reader. To aid researchers who are unfamiliar, this article sets out a number of steps to be taken to set up the tool. It also provides links to documents that researchers may choose to utilise during this process. Additionally, this article recognises that there are other programming languages and other GenAIs that could have been utilised to create this new method for gathering grey literature. It does not seek to premise one programming language or GenAI over another, but rather seeks to utilise the work of [ 10 ] to highlight the ability of both tools in creating a new method for researchers. As set out in [ 10 ], GenAIs are excellent tools that can assist researchers in devising new ways to conduct research. However, the role of the researcher is firmly embedded in the process of determining what GenAI outputs are functional and true. The upcoming sections of this article will unfold as follows: the ensuing theoretical background section will familiarise readers with the three elements utilised in developing the advocated technique. Firstly, it will introduce the computer programming language Python. Secondly, it will acquaint readers with the GenAI ChatGPT 3.5, which was instrumental in assisting to script and refine the code integral to the technique. Thirdly, the article will shed light on Google’s PSE, serving as the platform for this novel technique. Subsequently, the methods section will present a narrow three step case study to set out the coding devised for this innovative approach, and the following results section will present the outcomes derived from the executed code alongside a guide to utilising the method. The ensuing discussion section will summarise the work conducted, alongside discussing limitations and future research potential. It will then end with concluding remarks. 2. THEORETICAL BACKGROUND 2.1. PYTHON Developed in 1990 by Dutch programmer Guido van Rossum, Python is an interpreted, interactive, object-oriented programming language. Although it has a small kernel, it can be extended by importing external libraries [ 21 ]. In fact, Python maintains a strong position because it contains a vast ecosystem of libraries as well as environments [ 22 ]. Not alone in programming languages (see Ruby [ 23 ], Java [ 24 ], Rust [ 25 ], and C# [ 26 ] for other available languages), Python was chosen as the language used in this article solely due to the authors familiarity with it. The libraries associated with the proposed tool are “OS”, “Requests”, and “CSV”. OS or Operating System is a Python module that allows the user to gain access to operating system information [ 27 ]. Requests is one of the most downloaded Python package with approx. 30 million downloads per week. The package allows users to send HTTP (Hypertext Transfer Protocol) requests easily [ 28 ]. Lastly, the CSV or (Comma Separated Values) format is the most common import and export format for spreadsheets and databases. It provides functionalities to read and write data in CSV format, which is a common and simple way to represent tabular data [ 29 ]. The module allows programmers to work with CSV data without getting into the specifics of the underlying CSV format, making it versatile for dealing with various applications and custom CSV formats [ 29 ]. According to [ 10 ], Python is a programming language that has had great success when paired with GenAIs such as ChatGPT 3.5 for coding assistance. 2.2. GENERATIVE ARTIFICIAL INTELLIGENCE In recent times, the field of artificial intelligence (AI) has experienced a significant surge in research growth, particularly in areas related to chatbots and other conversational agents [ 30 , 31 ]. Significantly, GenAIs are generating diverse arrays of content; text, images, audio, and video [ 32 – 34 ]. At present, GenAI systems are being applied to address common programming challenges such as code summarisation, code review, synthesis, error repairs, and debugging [ 10 , 35 , 36 ]. A popular GenAI, ChatGPT 3.5 by OpenAI, is recognised for its significant potential in examining source code, suggesting changes, and generating code [ 10 , 30 , 35 , 37 ]. This advancement has broad applications for researchers seeking to enhance their research through computational-based methods. Different Generative AI models are currently in use, each exhibiting unique strengths and weaknesses. For a detailed comparative analysis of OpenAI ChatGPT 3.5, Microsoft Bing Chat, and Google Bard, refer to Dao (38). Furthermore, for an extensive list of existing large language models and their comparisons, consult Ray (39). As with the choice for coding with Python, the use of ChatGPT over another GenAI is purely due to the authors familiarity with the tool. As with Python, this article does not seek to place one programming language or GenAI over another. It seeks only to highlight how a new tool for systematically gathering grey literature can be created utilising both instruments. In addition, as with [ 10 ] this article seeks to highlight how GenAI, such as ChatGPT 3.5, can be used as an assistant that contributes to the process of scripting code. Within this process the researcher is still central to developing the code, especially with respect to determining what is ‘functional’ versus what is ‘true’ [ 10 ]. Indeed, [ 10 ] highlights several caveats that researchers should be aware of when utilising GenAIs such as ChatGPT 3.5 for assistance in scripting code for use in research projects. The largest issue to be aware of is the potential ‘bugginess’ of the code that GenAIs are trained on. A secondary issue relates to ethical considerations. As GenAIs are trained on large datasets, for example ChatGPT 3.5 was trained on the GitHub repository, there are ethical concerns when it comes to recognising the original producers of the code used to train the GenAI. [ 10 ] suggests retroactively searching for codes once they are compiled. Fundamental to the tool created for systematically gathering grey literature for a SLR is the PSE provided by Google. 2.3. GOOGLE’S PROGRAMMABLE SEARCH ENGINE Search engines – in particular Google- are the starting point for anyone wanting to locate something on the internet [ 40 ]. Aside from general searches, Google facilitates more sophisticated search features to retrieve information (this research for example uses; site: gov.au and filetype:pdf, to retrieve only PDFs from Australian government websites). The search strategy utilised alongside “site:” and “filetype:” is ("energy infrastructure" AND resilience). A PSE is the tailored version of a generic search engine (Google, Yahoo, Bing etc), in which every feature can be altered to fit a specific purpose(s). There are two primary cases for a PSE: one designed to search the contents of a single website, and the other tailored to concentrate on a particular topic across multiple websites. The full set of instructions to create a google PSE can be found at https://developers.google.com/custom-search/docs/overview . For the work being conducted a Custom search JSON API is also required (to set one up, please see https://developers.google.com/custom-search/v1/overview ). The fundamental advantage of a PSE lies in its ability to empower researchers to create a search engine tailored for searching information on specific topics chosen by the creator. In this instance, to search for Australian government PDFs pertaining to energy infrastructures and resilience. However, should the user wish to search multiple websites, they can add extra URLs to the search query, as will be demonstrated. 3. METHOD 3.1. CASE STUDY This section will lay out the code that ChatGPT 3.5 was prompted to create. It will do this in the form of a narrow three stage case study. Stage one is the full prompt list for creating the code and is included in the supplementary material submitted alongside this article. It is included in the supplementary material due to its size. The script developed in this article is intended for use in another research project. Therefore, ChatGPT 3.5 was prompted with that research project in mind and is not relevant to the method put forward here outside of the prompts used to script the code. Stage two is reviewing and synthesising the newly scripted code. As utilising GenAIs to assist in compiling code is a new methodology in itself, the methodology for this is set out in [ 10 ]. The final stage (three) is executing the newly scripted code. Stage three will also present two examples. The first example will be of a single site search (“gov.au”) for Australian documents. The second example will illustrate how multiple websites can be added, in this instance “gov.au” will be searched alongside “un.org.” The first 100 results for both searches will be provided in the supplementary materials. 3.1.1. STAGE ONE: See supplementary materials. 3.1.2. STAGE TWO: Reviewing and synthesising the scripted code. In this stage, the scripted code from the supplementary materials is put into ChatGPT 3.5. As set out in [ 10 ], this allows the GenAI to provide an overview of the newly scripted code, and also offers an opportunity to use ChatGPT 3.5 to synthesise any parts in order to tighten the coding up further. Table 1 illustrates this process. Table 1 Code scripted with assistance from ChatGPT 3.5. Prompts Responses I am going to give you some python code to review. Can you do this for me? Of course! I'd be happy to help you review your Python code. Please provide the code, and let me know if there's anything specific you would like me to focus on or if you have any questions. #1 import requests import csv import os Certainly! It looks like you've imported the requests , csv , and os modules. These are commonly used modules in Python. Without additional code, it's challenging to provide a comprehensive review, but if you have specific functions or tasks you'd like me to look at, feel free to provide more details or share the rest of the code. Otherwise, if you have any questions about using these modules or if you need assistance with a particular aspect of your code, feel free to ask! #2 def search_AUST_energy_policy_documents(query, api_key, cx, num_results = 10): base_url = " https://www.googleapis.com/customsearch/v1" start_index = 1 all_documents = [] try: while True: params = { 'q': query, 'key': api_key, 'cx': cx, 'fileType': 'pdf', 'num': num_results, 'start': start_index, } response = requests.get(base_url, params = params) if response.status_code = = 200: search_results = response.json() documents = extract_documents(search_results) all_documents.extend(documents) # Update 'start' parameter for the next page start_index + = num_results else: print(f"Error: {response.status_code}") break except Exception as e: print(f"An error occurred: {e}") return all_documents This appears to be a function for searching for Australian energy policy documents using the Google Custom Search JSON API. The function takes a query, an API key, a custom search engine ID (‘cx’), and an optional parameter ‘num_results’ which defaults to 10. The function returns a list of documents in JSON format #3 def extract_documents(search_results): items = search_results.get('items', []) documents = [] for item in items: document = { 'title': item.get('title', ''), 'link': item.get('link', ''), 'snippet': item.get('snippet', ''), } documents.append(document) return documents The ‘extract_documents’ function looks straightforward and serves the purpose of converting search results into a list of dictionaries, where each dictionary represents a document with title, link, and snippet information. #4 def save_to_csv(documents, csv_filename): with open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile: fieldnames = ['Title', 'Link', 'Snippet'] writer = csv.DictWriter(csvfile, fieldnames = fieldnames) writer.writeheader() for doc in documents: writer.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']}) Your ‘save_to_csv’ function looks good and fulfills the purpose of writing the extracted document information to a CSV file. #5 def main(): query = "site:gov.au ('energy infrastructure' AND resilience) filetype:pdf" api_key = "YOUR_API_KEY" # Replace with your actual API key" cx = "YOUR_CUSTOM_SEARCH_ENGINE_ID" # Replace with your actual custom search engine ID csv_filename = “YOUR FILE DESTINATION” num_results = 10 # Adjust as needed search_results = search_un_energy_policy_documents(query, api_key, cx, num_results = num_results) if search_results: save_to_csv(search_results, csv_filename) print(f"{len(search_results)} results saved to {csv_filename}") if __name__ == "__main__": main() Your main function seems to be the entry point for your script, and it orchestrates the execution of the other functions you've defined. I have given you 5 pieces of code. Could you rewrite it to improve the script? Certainly! I'll provide a more polished version of the script, taking into account best practices, improved readability, and additional error handling. Here's a revised version: import os import requests import csv def search_AUST_energy_policy_documents(query, api_key, cx, num_results = 10): base_url = " https://www.googleapis.com/customsearch/v1" start_index = 1 all_documents = [] try: while True: params = { 'q': query, 'key': api_key, 'cx': cx, 'fileType': 'pdf', 'num': num_results, 'start': start_index, } response = requests.get(base_url, params = params) response.raise_for_status() # Raise an HTTPError for bad responses search_results = response.json() documents = extract_documents(search_results) all_documents.extend(documents) # Update 'start' parameter for the next page start_index + = num_results except requests.exceptions.RequestException as e: print(f"An error occurred during the search: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") return all_documents def extract_documents(search_results): result_items = search_results.get('items', []) documents = [] for item in result_items: document = { 'title': item.get('title', 'No Title'), 'link': item.get('link', 'No Link'), 'snippet': item.get('snippet', 'No Snippet'), } documents.append(document) return documents def save_to_csv(documents, csv_filename): try: with open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile: fieldnames = ['Title', 'Link', 'Snippet'] writer = csv.DictWriter(csvfile, fieldnames = fieldnames) writer.writeheader() for doc in documents: writer.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']}) print(f"{len(documents)} results saved to {csv_filename}") except Exception as e: print(f"An error occurred while writing to the CSV file: {e}") def main(): query = 'site:gov.au ("energy infrastructure" AND "resilience") filetype:pdf' api_key = "YOUR_API_KEY" # Replace with your actual API key cx = "YOUR_CUSTOM_SEARCH_ENGINE_ID" # Replace with your actual custom search engine ID csv_filename = os.path.join( “YOUR FILE DESTINATION" ) num_results = 10 # Adjust as needed try: search_results = search_AUST_energy_policy_documents(query, api_key, cx, num_results = num_results) save_to_csv(search_results, csv_filename) if search_results else print("No results found.") except Exception as e: print(f"An unexpected error occurred: {e}") if __name__ == "__main__": main() An error occurred during the search: 400 Client Error: Bad Request for url : https://www.googleapis.com/customsearch [...] The error message indicates a "400 Client Error: Bad Request," which means the request sent to the Google Custom Search API is malformed or incorrect. To troubleshoot and fix this issue, you can consider the following steps: 1. Check API Key and CX : • Ensure that your API key ( api_key ) and custom search engine ID ( cx ) are correct and valid. Double-check that there are no typos or mistakes in these values. 2. Review Query Parameters : • Examine the constructed query ( params ) in your search_AUST_energy_policy_documents function. Ensure that the query parameters are formatted correctly and meet the requirements of the Google Custom Search API. 3. URL Encoding : • Confirm that the query parameters are properly URL-encoded. The error message in your case shows that the URL includes characters like ( , ) , and spaces, which should be properly encoded. 4. Limitations and Quotas : • Check if you are exceeding any limitations or quotas imposed by the Google Custom Search API. For example, there might be restrictions on the number of requests you can make in a given time period. Because I am using a free version, I can only look at 10 pages. Can you change the code to reflect this? Certainly! If you're limited to 10 pages of results, you can modify the search_AUST_energy_policy_documents function to restrict the number of iterations in the while loop. Here's an updated version: def search_AUST_energy_policy_documents(query, api_key, cx, num_results = 10, max_pages = 1): base_url = " https://www.googleapis.com/customsearch/v1" start_index = 1 all_documents = [] try: for _ in range(max_pages): params = { 'q': query, 'key': api_key, 'cx': cx, 'fileType': 'pdf', 'num': num_results, 'start': start_index, } response = requests.get(base_url, params = params) response.raise_for_status() # Raise an HTTPError for bad responses search_results = response.json() if 'error' in search_results: raise Exception(f"Google Custom Search API error: {search_results['error']['message']}") documents = extract_documents(search_results) all_documents.extend(documents) # Update 'start' parameter for the next page start_index + = num_results except requests.exceptions.RequestException as e: print(f"An error occurred during the search: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") return all_documents Thank you, the code is running as expected. You're welcome! I'm glad to hear that the code is running as expected. If you have any more questions, need further assistance, or have other code you'd like me to review, feel free to reach out. Happy coding! Following this, and in line with the code found in [ 14 ], a User-Agent header was included. This was to ensure transparency on the agent end of the code, and to indicate that it was not a ‘bot’ behind the helm. This addition is indicated in Table 2 . Table 2 The addition of a User-Agent Header to the script. Prompt Response Should I include a user-agent header in the code? Yes, including a User-Agent header in your code is a good practice. The User-Agent header provides information about the client making the request, such as the type of browser or application and its version. Some websites and APIs may use this information to identify and differentiate between various clients. Including a User-Agent header helps in the following ways: 1. Identification : It allows the website or API server to identify your request and can be used for statistical or analytical purposes. 2. Avoiding Blocks : Some websites implement access controls or rate limiting based on User-Agent. Providing a User-Agent that looks similar to a typical web browser can help avoid being blocked or flagged as suspicious. 3. Good Etiquette : Including a User-Agent is considered good etiquette. Here's how you can modify your code to include a User-Agent header using the headers parameter in the requests.get method: def search_AUST_energy_policy_documents(query, api_key, cx, num_results = 10, max_pages = 1): base_url = " https://www.googleapis.com/customsearch/v1" start_index = 1 all_documents = [] headers = { 'User-Agent': 'Your-User-Agent-Here', } try: for _ in range(max_pages): params = { 'q': query, 'key': api_key, 'cx': cx, 'fileType': 'pdf', 'num': num_results, 'start': start_index, } response = requests.get(base_url, params = params, headers = headers) response.raise_for_status() # Raise an HTTPError for bad responses search_results = response.json() if 'error' in search_results: raise Exception(f"Google Custom Search API error: {search_results['error']['message']}") documents = extract_documents(search_results) all_documents.extend(documents) # Update 'start' parameter for the next page start_index + = num_results except requests.exceptions.RequestException as e: print(f"An error occurred during the search: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") return all_documents To assist with people utilising the method presented. Table 3 provides a full account of the code. With this new method, researchers will be able to systematically gather grey literature for their SLRs, saving significant time and money in the process. Users will have to ensure that the Python libraries that are utilised are installed on their computers, and that they input the correct information into the areas that are highlighted in all caps, bold, and italics. In the instance of this research project, the author ran the code in a Jupyter notebook in Visual Studio Code. Table 3 Full account of the newly scripted code. import os import requests import csv def search_AUST_energy_policy_documents(query, api_key, cx, num_results = 10, max_pages = 10): base_url = " YOUR COUSTOM SEARCH ENGINE " start_index = 1 all_documents = [] headers = { 'User-Agent' YOUR USER-AGENT INFORMATION , } try: for _ in range(max_pages): params = { 'q': query, 'key': api_key, 'cx': cx, 'fileType': 'pdf', 'num': num_results, 'start': start_index, } response = requests.get(base_url, params = params, headers = headers) response.raise_for_status() # Raise an HTTPError for bad responses search_results = response.json() if 'error' in search_results: raise Exception(f"Google Custom Search API error: {search_results['error']['message']}") documents = extract_documents(search_results) all_documents.extend(documents) # Update 'start' parameter for the next page start_index + = num_results except requests.exceptions.RequestException as e: print(f"An error occurred during the search: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") return all_documents def extract_documents(search_results): result_items = search_results.get('items', []) documents = [] for item in result_items: document = { 'title': item.get('title', 'No Title'), 'link': item.get('link', 'No Link'), 'snippet': item.get('snippet', 'No Snippet'), } documents.append(document) return documents def save_to_csv(documents, csv_filename): try: with open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile: fieldnames = ['Title', 'Link', 'Snippet'] writer = csv.DictWriter(csvfile, fieldnames = fieldnames) writer.writeheader() for doc in documents: writer.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']}) print(f"{len(documents)} results saved to {csv_filename}") except Exception as e: print(f"An error occurred while writing to the CSV file: {e}") def main(): query = 'site:gov.au ("energy infrastructure" AND ("resilience")) filetype:pdf' api_key = " YOUR API KEY " cx = " YOUR CUSTOM SEARCH ENGINE ID " csv_filename = os.path.join( YOUR FILE LOCATION \\. csv" ) num_results = 10 # Adjust as needed try: search_results = search_AUST_energy_policy_documents(query, api_key, cx, num_results = num_results) save_to_csv(search_results, csv_filename) if search_results else print("No results found.") except Exception as e: print(f"An unexpected error occurred: {e}") if __name__ == "__main__": main() 3.1.3. STAGE THREE: execution The executed code provides the output “100 results saved to File location \\ XXX .results.csv”. It accomplished this in 7.5 seconds, and automatically saved a CSV file to the specified location. The next section will provide the results followed by a guide for researchers who wish to utilise this method. 4. FINDINGS The scripted code provided the following results. A CSV file was produced that contained the following information: three columns with the headings ‘Title’, ‘Link’, and ‘Snippet.’ Each column contained the information for 100 PDFs identified in the search. Figures 1 and 2 provides screenshot snippets of the Titles and their Links that were uncovered. Each PDF link, when copied and pasted into a web browser, is functional. The search strategy words “energy infrastructure” AND “resilience” is contained in either the title or the snippet. The full CSV file is provided in the supplementary materials. Figure 3 provides a screenshot snippet of the Blurb accompanying each result. Within the search strategy the URL “un.org” was added. This was done to illustrate the ease in which multiple websites can be searched. A snippet of the Titles and Links from this search can be viewed in Fig. 4 . These results illustrate the speed, rigor, and transparency of the new tool and underscores its importance. Capturing every tier of Australian government, the above results provide grey literature from Victoria, New South Whales, Queensland, Torris Straight, and the Australian Federal Government. Should users wish to narrow the search to these states, they would simply change the URL to “vic.gov.au” for Victoria, or “qld.gov.au” for Queensland. Should users wish to add other URLs such as “un.org” they need to only add the OR Boolean to the URL component of the strategy; for example, “ 'site:gov.au OR site:un.org ("energy infrastructure" AND ("resilience")) filetype:pdf' ” 4.1. GUIDE FOR RESEARCHERS. For researcher who wish to utilise this new methodology, the following steps are recommended. STEP ONE Download Python from https://www.python.org/downloads/ . STEP TWO Familiarise yourself with how libraries are installed in Python. Pip is the preferred installer program to be used in Python. You can find instructions on how to do so at https://packaging.python.org/en/latest/tutorials/installing-packages/ . There are also a large number of YouTube videos that will not only provide in-depth instructions for this stage but will also provide a deeper introduction to Python. STEP THREE Install the libraries required for the tool. You will need to install “OS”, “Requests”, and “CSV”. STEP FOUR Using the google documents provided in the google portion of the theoretical background section, follow the instructions to create a google PSE and to obtain a JSON API. For the PSE component see https://developers.google.com/custom-search/docs/overview . For obtaining the API key see https://developers.google.com/custom-search/v1/overview ). STEP FIVE Copy and paste the full code provide in this methods paper into a Jupyter notebook (other notebooks will also support this code, in this instance the author utilised a Jupyter notebook). You will also need to ensure that you input your own User-Agent Header to the script. Also, change the search query to reflect the information being sought. Lastly, you will need to designate a location for the csv file to be saved in. STEP SIX Run the code. Should any errors be encountered, [ 10 ] offers a method to help resolve them. Figure 5 provides a visual representation of six step process. 5. DISCUSSION AND LIMITATIONS The above sections have presented a new tool for researchers to systematically gather grey literature for SLRs. It has illustrated that Titles, Links, and Blurbs of PDF files pertaining to “energy infrastructure” AND “resilience” can be identified, saved, and accessed in a remarkably small amount of time. This technique will not only save researchers resources (time and money), but it will also increase the rigor, transparency, and efficiency of this stage in a SLR. As highlighted in [ 10 , 41 ], AI, and in this instance GenAI, will have an impact on the world that is akin to the impact of the industrial revolution, perhaps greater. To this end, this is not the first instance of computational-based methods being applied to stages of SLRs. This article not alone in creating new methods that infuse AI and MLTs within SLRs. [ 42 ] provides a thorough account of how AI and MLTs can be applied to different stages of a SLR. Focusing primarily on how Latent Dirichlet Allocation (LDA) Topic Models can be used to abstract and synthesise data obtained during a review, [ 42 ] also introduces AI and MLTs that can be utilised in the title/abstract screening stage, as well as the data extraction stage. [ 14 ] illustrates how Python can be utilised to create a web-scraper to gather grey literature from a single online data repository. Indeed, incorporating AI and MLTs into the SLR process is the topic of contemporary works investigating how they can be inserted into current guidelines. For an account on how AI and MLTs can be incorporated within the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, please see [ 43 ] on how this could be achieved. The scripted code presented in this article will transform how grey literature is gathered for use in a SLR. Although cheap, extensive use of this tool is not one-hundred percent free. The results presented are limited to the free-use terms of Google’s API key. If researchers require a greater number of searches, and more results, then they will have to purchase this ability from Google. Additionally, to keep the scripted code as simple as possible for the reader, more diverse search strategies alongside data organisation and saving functions have not been included. This will be left for future research. [ 10 ] highlighted that there is a large ethical concern regarding appropriate recognition of the coding that GenAIs are trained on. Retrospective searches of coding was suggested as a possible way to attribute codes. At the time of writing, similar code has not been located. Although it is impossible to be certain where the code originated, looking for it is a step forward in the process of solving the ethical issues of utilising GenAIs to script code. A notable limitation of this article is that researchers may not be acquainted with the tools employed. The intention is that by furnishing external resources, direction, and a guide, the technical gap necessary to employ the proposed method has been somewhat alleviated, making it more accessible for a broader audience. 6. FUTURE RESEARCH The tool that has been introduced in this article is intended to be employed to gather pertinent policy documents, standards, laws, and regulations to be utilised in a connected research project. Future research will also include scripting code to increase the sophistication of the queries that can be entered, alongside different methods to organise and save the data generated through this new process. Applications for areas outside of grey literature are also being investigated. 7. CONCLUSION This article has introduced a new and novel tool to aid in gathering grey literature for use in a SLR. Though the use of the computer programming language Python, coding assistance of ChatGPT 3.5, and the ability to create PSEs from Google, this article has introduced a technique for researchers to cheaply, quickly, rigorously, and transparently gather grey literature. No longer requiring long hours and unreliable search processes, this new tool will help every discipline to gather grey literature for their research. Declarations Funding: Natural Hazards Research Australia. Conflicts of Interests/Competing Interests: The author has neither conflicts of interests nor competing interests to declare that are relevant to the content of this article. Data Availability: All data pertinent to the article has been made available in the supplementary materials. Code Availability: All codes utilised in this research project have either been provided in text or are contained in the supplementary materials. Use of AI: The Generative AI ChatGPT 3.5 was utilised in the methods section of this article. Author Contribution C.A, as sole author, is responsible for all aspects of this manuscript. References Thomé AMT, Scavarda LF, Scavarda AJ. Conducting systematic literature review in operations management. (2016). Production Planning & Control, 27(5). 408–20. https://doi.org/10.1080/09537287.2015.1129464 . Pati D, Lorusso LN. How to write a systematic review of the literature. (2018). HERD: Health Environments Research & Design Journal, 11(1). 15–30. https://doi.org/10.1177/1937586717747384 . Petticrew M, Roberts H. Systematic reviews in the social sciences: A practical guide : John Wiley & Sons; 2008. Xiao Y, Watson M. Guidance on Conducting a Systematic Literature Review. (2019). Journal of Planning Education and Research, 39(1). 93–112. https://doi.org/10.1177/0739456x17723971 . Siddaway AP, Wood AM, Hedges LV. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. (2019). Annual Review of Psychology, 70(1). 747–70. https://doi.org/10.1146/annurev-psych-010418-102803 . Paul J, Lim WM, O’Cass A, Hao AW, Bresciani S. Scientific procedures and rationales for systematic literature reviews (SPAR-4‐SLR). (2021). International Journal of Consumer Studies, 45(4). O1-O16. https://doi.org/10.1111/ijcs.12695 . Sundaram G, Berleant D. Automating Systematic Literature Reviews with Natural Language Processing and Text Mining: a Systematic Literature Review. (2022). ArXiv Preprint https://doi.org/10.48550/arXiv.2211.15397 . Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. (2019). Systematic reviews, 8. 1–10. https://doi.org/10.1186/s13643-019-1074-9 . Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. (2007). Annals of internal medicine, 147(4). 224 – 33. https://doi.org/10.7326/0003-4819-147-4-200708210-00179 . Atkinson CF. ChatGPT and computational-based research: benefits, drawbacks, and machine learning applications. (2023). Discover Artificial Intelligence, 3(1). 42. https://doi.org/10.1007/s44163-023-00091-3 . Adams J, Hillier-Brown FC, Moore HJ, Lake AA, Araujo-Soares V, White M, et al. Searching and synthesising ‘grey literature’ and ‘grey information’ in public health: critical reflections on three case studies. (2016). Systematic Reviews, 5(1). 164. https://doi.org/10.1186/s13643-016-0337-y . Kamei F, Wiese I, Pinto G, Ribeiro M, Soares S, editors. On the use of grey literature: A survey with the brazilian software engineering research community. Proceedings of the XXXIV Brazilian Symposium on Software Engineering; 2020. https://doi.org/10.1145/3422392.3422442 Banks M. Blog posts and tweets: the next frontier for grey literature . Grey literature in library and information studies: De Gruyter; 2009. Atkinson C. Cheap, Rigorous, and Transparent: How Web-scraping with Python can Improve Collecting Grey Literature for Systematic Literature Reviews. (2023). Grey Journal (TGJ), 19(3). Paez A. Gray literature: An important resource in systematic reviews. (2017). Journal of Evidence-Based Medicine, 10(3). 233 – 40. https://doi.org/10.1111/jebm.12266 . Grayson L, Gomersall A. Google – boon, bane or both? (2006). Evid Polciy, 2(4). 538 – 42. https://doi.org/10.1332/174426406778881818 . Haddaway NR, Bethel A, Dicks LV, Koricheva J, Macura B, Petrokofsky G, et al. Eight problems with literature reviews and how to fix them. (2020). Nature Ecology & Evolution, 4(12). 1582–9. https://doi.org/10.1038/s41559-020-01295-x . Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. (2015). Systematic Reviews, 4(1). 138. https://doi.org/10.1186/s13643-015-0125-0 . Winters M, Weir A. Grey matters; on the importance of publication bias in systematic reviews. (2017). 51(6). 488–9. https://doi.org/10.1136/bjsports-2016-096679 . Echtler F, Häußler M, editors. Open source, open science, and the replication crisis in HCI. Extended abstracts of the 2018 CHI conference on human factors in computing systems; 2018. https://doi.org/10.1145/3170427.3188395 Nagpal A, Gabrani G, editors. Python for data analytics, scientific and technical applications. 2019 Amity international conference on artificial intelligence (AICAI); 2019: IEEE. https://doi.org/10.1109/AICAI.2019.8701341 Kumar R. Future for scientific computing using Python. (2015). International Journal of Engineering Technologies and Management Research, 2(1). 30–41. https://doi.org/10.29121/ijetmr.v2.i1.2015.28 . Tilkov S, Vinoski S. Node. js: Using JavaScript to build high-performance network programs. (2010). IEEE Internet Computing, 14(6). 80–3. https://doi.org/10.1109/MIC.2010.145 . Arnold K, Gosling J, Holmes D. The Java programming language : Addison Wesley Professional; 2005. Klabnik S, Nichols C. The Rust programming language : No Starch Press; 2023. Hejlsberg A, Torgersen M, Wiltamuth S, Golde P. The C# programming language : Pearson Education; 2008. Python Software Foundation. OS — Miscellaneous operating system interfaces Online: Python Software Foundation; (2024) [Available from: https://docs.python.org/3/library/os.html . Python Software Foundation. Requests 2.31.0 Online: Python Software Foundation; (2024) [Available from: https://pypi.org/project/requests/ . Python Software Foundation. CSV File Reading and Writing Online: Python Software Foundation; (2024) [Available from: https://docs.python.org/3/library/csv.html . Azaria A, Azoulay R, Reches S. ChatGPT is a Remarkable Tool–For Experts. (2023). ArXiv Preprint https://doi.org/10.48550/arXiv.2306.03102 . Olujimi PA, Ade-Ibijola A. NLP techniques for automating responses to customer queries: a systematic review. (2023). Discover Artificial Intelligence, 3(1). 20. https://doi.org/10.1007/s44163-023-00065-5 . Morris MR. Scientists' Perspectives on the Potential for Generative AI in their Fields. (2023). ArXiv Preprint https://doi.org/10.48550/arXiv.2304.01420 . Crawford K, Paglen T. Excavating AI: the politics of images in machine learning training sets. (2021). AI & SOCIETY, 36(4). 1105–16. https://doi.org/10.1007/s00146-021-01162-8 . Hacker P, Engel A, Mauer M, editors. Regulating ChatGPT and other large generative AI models. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency; 2023. https://doi.org/10.1145/3593013.3594067 Tian H, Lu W, Li TO, Tang X, Cheung S-C, Klein J, et al. Is ChatGPT the Ultimate Programming Assistant–How far is it? (2023). ArXiv Preprint https://doi.org/10.48550/arXiv.2304.11938 . Haque MA, Li S. The Potential Use of ChatGPT for Debugging and Bug Fixing. (2023). EAI Endorsed Transactions on AI and Robotics , 2(1). e4-e. https://doi.org/10.4108/airo.v2i1.3276 . Liu J, Xia CS, Wang Y, Zhang L. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. (2023). ArXiv Preprint https://doi.org/10.48550/arXiv.2305.01210 . Dao X-Q. Performance comparison of large language models on vnhsge english dataset: Openai chatgpt, microsoft bing chat, and google bard. (2023). ArXiv Preprint . https://doi.org/10.48550/arXiv.2307.02288 . Ray PP. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. (2023). Internet of Things and Cyber-Physical Systems , 3. 121 – 54. https://doi.org/10.1016/j.iotcps.2023.04.003 . Dhara A. A personalised discovery service using Google custom search engine. (2016). Annals of Library and information Studies , 63. 298–305. Available from: https://nopr.niscpr.res.in/bitstream/123456789/39764/1/ALIS%2063%284%29%20298-305.pdf . Medaglia R, Gil-Garcia JR, Pardo TA. Artificial Intelligence in Government: Taking Stock and Moving Forward. (2021). Social Science Computer Review, 41(1). 123–40. https://doi.org/10.1177/08944393211034087 . Atkinson CF. Cheap, Quick, and Rigorous: Artificial Intelligence and the Systematic Literature Review. (2023). Social Science Computer Review. https://doi.org/10.1177/08944393231196281 . Susnjak T. Prisma-dfllm: An extension of prisma for systematic literature reviews using domain-specific finetuned large language models. (2023). ArXiv Preprint 1–20. https://doi.org/10.48550/arXiv.2306.14905 . Additional Declarations No competing interests reported. Supplementary Files PROMPT1.doc SEARCH2.csv SearchResultsAU.csv Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4007725","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Case Report","associatedPublications":[],"authors":[{"id":284623777,"identity":"fdad3894-e0aa-4b59-bba5-cf8b91cedb89","order_by":0,"name":"Cameron Frederick Atkinson","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGklEQVRIie3QsUrDQBjA8a8E0uVK1gul9RVSAkEw+CwegWxRoUvGc0k2u7aTr5ASiOvJB3EwDxDBLdCtQxEkhYDmpJtni5vD/Zfj4H53xweg0/3vTBBysQ7bAT8tDsTmxh/Id444Qaz0hTVx/AbWQ8LwtvOv3frmadeCP8mEsaEKQqson1XVBmhpClwl4dyrA2NMIHQzYXoq4kBU2HcJApRDjiOOrKgDGAMgywSoibV93EtyJgnpkOXLwNi38NmT4YeS0KgYSOLIjxGzv5wGJiUgekKUr9B6m9u8QjIrwyscJSFbVo13QZzAXSGZn6smtojWOx7jdIql+046ny1S1ry28eXk/jld178MWkZ+jAXAOHJep9PpdEf7At2yaQRAH/uVAAAAAElFTkSuQmCC","orcid":"","institution":"University of Tasmania","correspondingAuthor":true,"prefix":"","firstName":"Cameron","middleName":"Frederick","lastName":"Atkinson","suffix":""}],"badges":[],"createdAt":"2024-03-03 06:46:31","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4007725/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4007725/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53756179,"identity":"52c22382-eb7a-4714-b21d-20668d1a0a33","added_by":"auto","created_at":"2024-03-29 19:01:37","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":193043,"visible":true,"origin":"","legend":"\u003cp\u003eScreenshot snippet of Titles from the new tool.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/7a460c59168a9bda36b3b7ca.png"},{"id":53757833,"identity":"8f9e4ca4-7b01-4b1d-b4a0-a420783824a4","added_by":"auto","created_at":"2024-03-29 19:09:37","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":186035,"visible":true,"origin":"","legend":"\u003cp\u003eScreenshot snippet of the Links from the new tool.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/06972aa90738ba4716d7a8f5.png"},{"id":53756181,"identity":"c6ef686e-8639-4852-a751-8b67bf67ae89","added_by":"auto","created_at":"2024-03-29 19:01:37","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":223603,"visible":true,"origin":"","legend":"\u003cp\u003eScreenshot snippet of the Blurbs accompanying each result from the new tool.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/082042b949d9362e56a238b0.png"},{"id":53756186,"identity":"69f8bc48-e132-4d8a-b306-46813dacbf23","added_by":"auto","created_at":"2024-03-29 19:01:37","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":213222,"visible":true,"origin":"","legend":"\u003cp\u003eScreenshot snippet from the updated search.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/66d9a3778472e99f2c75f3a4.png"},{"id":53756184,"identity":"752c47a7-eb70-4c0f-be34-f0b32389e102","added_by":"auto","created_at":"2024-03-29 19:01:37","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":227985,"visible":true,"origin":"","legend":"\u003cp\u003eStages for setting up tool.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/acc063becadc03a30790ce13.png"},{"id":64423306,"identity":"4a2155b9-4afb-443a-8b59-ec0c88778679","added_by":"auto","created_at":"2024-09-13 03:21:08","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1731296,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/c30d252e-773f-47f8-91cd-680698deb71f.pdf"},{"id":53757832,"identity":"4adbf6fa-3101-417e-8c28-bcf5dfc20b95","added_by":"auto","created_at":"2024-03-29 19:09:37","extension":"doc","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":65949,"visible":true,"origin":"","legend":"","description":"","filename":"PROMPT1.doc","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/0e255f11c237d793de9137a7.doc"},{"id":53758831,"identity":"09de953c-03cb-426a-8259-2a9962841148","added_by":"auto","created_at":"2024-03-29 19:17:37","extension":"csv","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":32150,"visible":true,"origin":"","legend":"","description":"","filename":"SEARCH2.csv","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/104359dec27a59a43d05e38d.csv"},{"id":53756183,"identity":"65fc3701-2844-4de4-8590-c9a066fecf12","added_by":"auto","created_at":"2024-03-29 19:01:37","extension":"csv","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":32499,"visible":true,"origin":"","legend":"","description":"","filename":"SearchResultsAU.csv","url":"https://assets-eu.researchsquare.com/files/rs-4007725/v1/bc04d6b868a705a07c953a89.csv"}],"financialInterests":"No competing interests reported.","formattedTitle":"Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine.","fulltext":[{"header":"1. INTRODUCTION","content":"\u003cp\u003eThe systematic literature review (SLR) is commonly regarded as the pinnacle of evidence gathering and evidence synthesis [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. SLRs seek to gather, appraise, and synthesise evidence pertaining to a phenomenon under investigation [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. To accomplish this, SLRs entail comprehensive searches of research databases, aiming to identify all relevant work within the field under review [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. The motivations behind a SLR are multifaceted: (1) to publicise ongoing research and prevent duplications of research; (2) to inform the planning of new research initiatives; (3) to substantiate claims of originality by contrasting existing research against new research [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThis article aims to highlight an innovative approach for researchers, particularly social science researchers engaged in SLRs, to streamline the laborious and time-consuming task of collecting grey literature. Beyond addressing temporal and financial constraints, SLRs are susceptible to becoming outdated due to their prolonged duration required for completion [\u003cspan additionalcitationids=\"CR8\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Additionally, the increasing volume of reports, among other grey literatures, presents a challenge for researchers to keep pace with the growing output. Integrating new tools, such as computational methods and generative artificial intelligence (GenAI), can indeed assist in managing and analysing this vast amount of information [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. These technologies can enhance the efficiency, transparency, and rigor of information gathering, providing valuable support for social scientists and researchers in dealing with the expanding landscape of grey literature. Consequently, techniques that enhance the speed, efficiency, and rigor of this process prove advantageous for all SLRs.\u003c/p\u003e \u003cp\u003eGrey literature represents one element of the broader category known as grey information. Introduced by [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e], grey information serves as a classification encompassing various forms of \u0026lsquo;grey\u0026rsquo;; including data, literature, and information. In this context, 'grey data' pertains to user-generated web content like tweets or blogs, 'grey literature' encompasses policy documents, standards, and regulations, while 'grey information' involves informal data such as meeting notes or emails [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. This is not the first instance of creating ways to systematically gather grey literature for use in a SLR. For example, [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] highlights how web-scraping can be employed to gather grey literature from online repositories.\u003c/p\u003e \u003cp\u003eIn SLRs, grey literature consistently proves to be among the most challenging types of literature to systematically collect [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. Bibliographic databases, a mainstay of gathering information pertinent to conducting a SLR, are generally restricted in their ability to locate relevant grey literature [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. Therefore, the difficulties associated with the systematic gathering of grey literature can result in its haphazard inclusion or, at times, in its complete exclusion from a review, introducing a research bias into a project [\u003cspan additionalcitationids=\"CR18\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn an effort to circumvent these biases, researchers can invest months in the rigorous collection of grey literature, sometimes employing processes that lack repeatability and introduce additional biases [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. The issues encountered with the replicability of research contributes to an ongoing methodological challenge, making it difficult, if not impossible, to replicate the findings of numerous scientific projects [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. To address the replicability issues surrounding the gathering of grey literature, this article introduces a new methodology to systematically gather grey literature for use in a SLR. Leveraging the power provided by Google\u0026rsquo;s API to create Programmable Search Engines (PSEs), this article harnesses the robust coding ability of GenAI systems such as ChatGPT 3.5 and employs the computer programming language Python to systematically gather PDF documents from Australian government websites with the (gov.au) and (un.org) URLs.\u003c/p\u003e \u003cp\u003eIt should be noted here that the method put forward automates what would normally be extensive and manual Google searches. There are no algorithms or machine learning techniques required for this method. However, end-users of this method will need to acquaint themselves with the Pythonic script utilised. Although relatively straightforward, this article does assume some familiarity with coding on the part of the reader. To aid researchers who are unfamiliar, this article sets out a number of steps to be taken to set up the tool. It also provides links to documents that researchers may choose to utilise during this process. Additionally, this article recognises that there are other programming languages and other GenAIs that could have been utilised to create this new method for gathering grey literature. It does not seek to premise one programming language or GenAI over another, but rather seeks to utilise the work of [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] to highlight the ability of both tools in creating a new method for researchers. As set out in [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], GenAIs are excellent tools that can assist researchers in devising new ways to conduct research. However, the role of the researcher is firmly embedded in the process of determining what GenAI outputs are functional and true.\u003c/p\u003e \u003cp\u003eThe upcoming sections of this article will unfold as follows: the ensuing \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003etheoretical background\u003c/span\u003e section will familiarise readers with the three elements utilised in developing the advocated technique. Firstly, it will introduce the computer programming language Python. Secondly, it will acquaint readers with the GenAI ChatGPT 3.5, which was instrumental in assisting to script and refine the code integral to the technique. Thirdly, the article will shed light on Google\u0026rsquo;s PSE, serving as the platform for this novel technique. Subsequently, the methods section will present a narrow three step case study to set out the coding devised for this innovative approach, and the following results section will present the outcomes derived from the executed code alongside a guide to utilising the method. The ensuing discussion section will summarise the work conducted, alongside discussing limitations and future research potential. It will then end with concluding remarks.\u003c/p\u003e"},{"header":"2. THEORETICAL BACKGROUND","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1. PYTHON\u003c/h2\u003e \u003cp\u003eDeveloped in 1990 by Dutch programmer Guido van Rossum, Python is an interpreted, interactive, object-oriented programming language. Although it has a small kernel, it can be extended by importing external libraries [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. In fact, Python maintains a strong position because it contains a vast ecosystem of libraries as well as environments [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. Not alone in programming languages (see Ruby [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], Java [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e], Rust [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e], and C# [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] for other available languages), Python was chosen as the language used in this article solely due to the authors familiarity with it. The libraries associated with the proposed tool are \u0026ldquo;OS\u0026rdquo;, \u0026ldquo;Requests\u0026rdquo;, and \u0026ldquo;CSV\u0026rdquo;.\u003c/p\u003e \u003cp\u003eOS or Operating System is a Python module that allows the user to gain access to operating system information [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. Requests is one of the most downloaded Python package with approx. 30\u0026nbsp;million downloads per week. The package allows users to send HTTP (Hypertext Transfer Protocol) requests easily [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. Lastly, the CSV or (Comma Separated Values) format is the most common import and export format for spreadsheets and databases. It provides functionalities to read and write data in CSV format, which is a common and simple way to represent tabular data [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. The module allows programmers to work with CSV data without getting into the specifics of the underlying CSV format, making it versatile for dealing with various applications and custom CSV formats [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. According to [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], Python is a programming language that has had great success when paired with GenAIs such as ChatGPT 3.5 for coding assistance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2. GENERATIVE ARTIFICIAL INTELLIGENCE\u003c/h2\u003e \u003cp\u003eIn recent times, the field of artificial intelligence (AI) has experienced a significant surge in research growth, particularly in areas related to chatbots and other conversational agents [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e]. Significantly, GenAIs are generating diverse arrays of content; text, images, audio, and video [\u003cspan additionalcitationids=\"CR33\" citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]. At present, GenAI systems are being applied to address common programming challenges such as code summarisation, code review, synthesis, error repairs, and debugging [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e]. A popular GenAI, ChatGPT 3.5 by OpenAI, is recognised for its significant potential in examining source code, suggesting changes, and generating code [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e]. This advancement has broad applications for researchers seeking to enhance their research through computational-based methods. Different Generative AI models are currently in use, each exhibiting unique strengths and weaknesses. For a detailed comparative analysis of OpenAI ChatGPT 3.5, Microsoft Bing Chat, and Google Bard, refer to Dao (38). Furthermore, for an extensive list of existing large language models and their comparisons, consult Ray (39).\u003c/p\u003e \u003cp\u003eAs with the choice for coding with Python, the use of ChatGPT over another GenAI is purely due to the authors familiarity with the tool. As with Python, this article does not seek to place one programming language or GenAI over another. It seeks only to highlight how a new tool for systematically gathering grey literature can be created utilising both instruments. In addition, as with [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] this article seeks to highlight how GenAI, such as ChatGPT 3.5, can be used as an assistant that contributes to the process of scripting code. Within this process the researcher is still central to developing the code, especially with respect to determining what is \u0026lsquo;functional\u0026rsquo; versus what is \u0026lsquo;true\u0026rsquo; [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. Indeed, [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] highlights several caveats that researchers should be aware of when utilising GenAIs such as ChatGPT 3.5 for assistance in scripting code for use in research projects. The largest issue to be aware of is the potential \u0026lsquo;bugginess\u0026rsquo; of the code that GenAIs are trained on. A secondary issue relates to ethical considerations. As GenAIs are trained on large datasets, for example ChatGPT 3.5 was trained on the GitHub repository, there are ethical concerns when it comes to recognising the original producers of the code used to train the GenAI. [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] suggests retroactively searching for codes once they are compiled. Fundamental to the tool created for systematically gathering grey literature for a SLR is the PSE provided by Google.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3. GOOGLE\u0026rsquo;S PROGRAMMABLE SEARCH ENGINE\u003c/h2\u003e \u003cp\u003eSearch engines \u0026ndash; in particular Google- are the starting point for anyone wanting to locate something on the internet [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. Aside from general searches, Google facilitates more sophisticated search features to retrieve information (this research for example uses; site: gov.au and filetype:pdf, to retrieve only PDFs from Australian government websites). The search strategy utilised alongside \u0026ldquo;site:\u0026rdquo; and \u0026ldquo;filetype:\u0026rdquo; is (\"energy infrastructure\" AND resilience).\u003c/p\u003e \u003cp\u003eA PSE is the tailored version of a generic search engine (Google, Yahoo, Bing etc), in which every feature can be altered to fit a specific purpose(s). There are two primary cases for a PSE: one designed to search the contents of a single website, and the other tailored to concentrate on a particular topic across multiple websites. The full set of instructions to create a google PSE can be found at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://developers.google.com/custom-search/docs/overview\u003c/span\u003e\u003cspan address=\"https://developers.google.com/custom-search/docs/overview\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. For the work being conducted a Custom search JSON API is also required (to set one up, please see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://developers.google.com/custom-search/v1/overview\u003c/span\u003e\u003cspan address=\"https://developers.google.com/custom-search/v1/overview\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e). The fundamental advantage of a PSE lies in its ability to empower researchers to create a search engine tailored for searching information on specific topics chosen by the creator. In this instance, to search for Australian government PDFs pertaining to energy infrastructures and resilience. However, should the user wish to search multiple websites, they can add extra URLs to the search query, as will be demonstrated.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. METHOD","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.1. CASE STUDY\u003c/h2\u003e \u003cp\u003eThis section will lay out the code that ChatGPT 3.5 was prompted to create. It will do this in the form of a narrow three stage case study. Stage one is the full prompt list for creating the code and is included in the supplementary material submitted alongside this article. It is included in the supplementary material due to its size. The script developed in this article is intended for use in another research project. Therefore, ChatGPT 3.5 was prompted with that research project in mind and is not relevant to the method put forward here outside of the prompts used to script the code. Stage two is reviewing and synthesising the newly scripted code. As utilising GenAIs to assist in compiling code is a new methodology in itself, the methodology for this is set out in [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. The final stage (three) is executing the newly scripted code. Stage three will also present two examples. The first example will be of a single site search (\u0026ldquo;gov.au\u0026rdquo;) for Australian documents. The second example will illustrate how multiple websites can be added, in this instance \u0026ldquo;gov.au\u0026rdquo; will be searched alongside \u0026ldquo;un.org.\u0026rdquo; The first 100 results for both searches will be provided in the supplementary materials.\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section3\"\u003e \u003ch2\u003e3.1.1. STAGE ONE: See supplementary materials.\u003c/h2\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section3\"\u003e \u003ch2\u003e3.1.2. STAGE TWO: Reviewing and synthesising the scripted code.\u003c/h2\u003e \u003cp\u003eIn this stage, the scripted code from the supplementary materials is put into ChatGPT 3.5. As set out in [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], this allows the GenAI to provide an overview of the newly scripted code, and also offers an opportunity to use ChatGPT 3.5 to synthesise any parts in order to tighten the coding up further. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates this process.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eCode scripted with assistance from ChatGPT 3.5.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrompts\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResponses\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eI am going to give you some python code to review. Can you do this for me?\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOf course! I'd be happy to help you review your Python code. Please provide the code, and let me know if there's anything specific you would like me to focus on or if you have any questions.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e#1\u003c/b\u003e\u003c/p\u003e \u003cp\u003eimport requests\u003c/p\u003e \u003cp\u003eimport csv\u003c/p\u003e \u003cp\u003eimport os\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCertainly! It looks like you've imported the \u003cb\u003erequests\u003c/b\u003e, \u003cb\u003ecsv\u003c/b\u003e, and \u003cb\u003eos\u003c/b\u003e modules. These are commonly used modules in Python. Without additional code, it's challenging to provide a comprehensive review, but if you have specific functions or tasks you'd like me to look at, feel free to provide more details or share the rest of the code. Otherwise, if you have any questions about using these modules or if you need assistance with a particular aspect of your code, feel free to ask!\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e#2\u003c/b\u003e\u003c/p\u003e \u003cp\u003edef search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;10):\u003c/p\u003e \u003cp\u003ebase_url = \"\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.googleapis.com/customsearch/v1\"\u003c/span\u003e\u003cspan address=\"https://www.googleapis.com/customsearch/v1\u0026quot;\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003cp\u003eall_documents = []\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003ewhile True:\u003c/p\u003e \u003cp\u003eparams = {\u003c/p\u003e \u003cp\u003e'q': query,\u003c/p\u003e \u003cp\u003e'key': api_key,\u003c/p\u003e \u003cp\u003e'cx': cx,\u003c/p\u003e \u003cp\u003e'fileType': 'pdf',\u003c/p\u003e \u003cp\u003e'num': num_results,\u003c/p\u003e \u003cp\u003e'start': start_index,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003eresponse\u0026thinsp;=\u0026thinsp;requests.get(base_url, params\u0026thinsp;=\u0026thinsp;params)\u003c/p\u003e \u003cp\u003eif response.status_code\u0026thinsp;=\u0026thinsp;=\u0026thinsp;200:\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;response.json()\u003c/p\u003e \u003cp\u003edocuments\u0026thinsp;=\u0026thinsp;extract_documents(search_results)\u003c/p\u003e \u003cp\u003eall_documents.extend(documents)\u003c/p\u003e \u003cp\u003e# Update 'start' parameter for the next page\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;+\u0026thinsp;=\u0026thinsp;num_results\u003c/p\u003e \u003cp\u003eelse:\u003c/p\u003e \u003cp\u003eprint(f\"Error: {response.status_code}\")\u003c/p\u003e \u003cp\u003ebreak\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred: {e}\")\u003c/p\u003e \u003cp\u003ereturn all_documents\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eThis appears to be a function for searching for Australian energy policy documents using the Google Custom Search JSON API. The function takes a query, an API key, a custom search engine ID (\u0026lsquo;cx\u0026rsquo;), and an optional parameter \u0026lsquo;num_results\u0026rsquo; which defaults to 10. The function returns a list of documents in JSON format\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e#3\u003c/b\u003e\u003c/p\u003e \u003cp\u003edef extract_documents(search_results):\u003c/p\u003e \u003cp\u003eitems\u0026thinsp;=\u0026thinsp;search_results.get('items', [])\u003c/p\u003e \u003cp\u003edocuments = []\u003c/p\u003e \u003cp\u003efor item in items:\u003c/p\u003e \u003cp\u003edocument = {\u003c/p\u003e \u003cp\u003e'title': item.get('title', ''),\u003c/p\u003e \u003cp\u003e'link': item.get('link', ''),\u003c/p\u003e \u003cp\u003e'snippet': item.get('snippet', ''),\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003edocuments.append(document)\u003c/p\u003e \u003cp\u003ereturn documents\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eThe \u0026lsquo;extract_documents\u0026rsquo; function looks straightforward and serves the purpose of converting search results into a list of dictionaries, where each dictionary represents a document with title, link, and snippet information.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e#4\u003c/b\u003e\u003c/p\u003e \u003cp\u003edef save_to_csv(documents, csv_filename):\u003c/p\u003e \u003cp\u003ewith open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile:\u003c/p\u003e \u003cp\u003efieldnames = ['Title', 'Link', 'Snippet']\u003c/p\u003e \u003cp\u003ewriter\u0026thinsp;=\u0026thinsp;csv.DictWriter(csvfile, fieldnames\u0026thinsp;=\u0026thinsp;fieldnames)\u003c/p\u003e \u003cp\u003ewriter.writeheader()\u003c/p\u003e \u003cp\u003efor doc in documents:\u003c/p\u003e \u003cp\u003ewriter.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']})\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYour \u0026lsquo;save_to_csv\u0026rsquo; function looks good and fulfills the purpose of writing the extracted document information to a CSV file.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003e#5\u003c/b\u003e\u003c/p\u003e \u003cp\u003edef main():\u003c/p\u003e \u003cp\u003equery = \"site:gov.au ('energy infrastructure' AND resilience) filetype:pdf\"\u003c/p\u003e \u003cp\u003eapi_key = \"YOUR_API_KEY\" # Replace with your actual API key\"\u003c/p\u003e \u003cp\u003ecx = \"YOUR_CUSTOM_SEARCH_ENGINE_ID\" # Replace with your actual custom search engine ID\u003c/p\u003e \u003cp\u003ecsv_filename = \u0026ldquo;YOUR FILE DESTINATION\u0026rdquo;\u003c/p\u003e \u003cp\u003enum_results\u0026thinsp;=\u0026thinsp;10 \u0026nbsp;# Adjust as needed\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;search_un_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;num_results)\u003c/p\u003e \u003cp\u003eif search_results:\u003c/p\u003e \u003cp\u003esave_to_csv(search_results, csv_filename)\u003c/p\u003e \u003cp\u003eprint(f\"{len(search_results)} results saved to {csv_filename}\")\u003c/p\u003e \u003cp\u003eif __name__ == \"__main__\":\u003c/p\u003e \u003cp\u003emain()\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYour \u003cb\u003emain\u003c/b\u003e function seems to be the entry point for your script, and it orchestrates the execution of the other functions you've defined.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eI have given you 5 pieces of code. Could you rewrite it to improve the script?\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCertainly! I'll provide a more polished version of the script, taking into account best practices, improved readability, and additional error handling. Here's a revised version:\u003c/p\u003e \u003cp\u003eimport os\u003c/p\u003e \u003cp\u003eimport requests\u003c/p\u003e \u003cp\u003eimport csv\u003c/p\u003e \u003cp\u003edef search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;10):\u003c/p\u003e \u003cp\u003ebase_url = \"\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.googleapis.com/customsearch/v1\"\u003c/span\u003e\u003cspan address=\"https://www.googleapis.com/customsearch/v1\u0026quot;\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003cp\u003eall_documents = []\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003ewhile True:\u003c/p\u003e \u003cp\u003eparams = {\u003c/p\u003e \u003cp\u003e'q': query,\u003c/p\u003e \u003cp\u003e'key': api_key,\u003c/p\u003e \u003cp\u003e'cx': cx,\u003c/p\u003e \u003cp\u003e'fileType': 'pdf',\u003c/p\u003e \u003cp\u003e'num': num_results,\u003c/p\u003e \u003cp\u003e'start': start_index,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003eresponse\u0026thinsp;=\u0026thinsp;requests.get(base_url, params\u0026thinsp;=\u0026thinsp;params)\u003c/p\u003e \u003cp\u003eresponse.raise_for_status() # Raise an HTTPError for bad responses\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;response.json()\u003c/p\u003e \u003cp\u003edocuments\u0026thinsp;=\u0026thinsp;extract_documents(search_results)\u003c/p\u003e \u003cp\u003eall_documents.extend(documents)\u003c/p\u003e \u003cp\u003e# Update 'start' parameter for the next page\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;+\u0026thinsp;=\u0026thinsp;num_results\u003c/p\u003e \u003cp\u003eexcept requests.exceptions.RequestException as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred during the search: {e}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003ereturn all_documents\u003c/p\u003e \u003cp\u003edef extract_documents(search_results):\u003c/p\u003e \u003cp\u003eresult_items\u0026thinsp;=\u0026thinsp;search_results.get('items', [])\u003c/p\u003e \u003cp\u003edocuments = []\u003c/p\u003e \u003cp\u003efor item in result_items:\u003c/p\u003e \u003cp\u003edocument = {\u003c/p\u003e \u003cp\u003e'title': item.get('title', 'No Title'),\u003c/p\u003e \u003cp\u003e'link': item.get('link', 'No Link'),\u003c/p\u003e \u003cp\u003e'snippet': item.get('snippet', 'No Snippet'),\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003edocuments.append(document)\u003c/p\u003e \u003cp\u003ereturn documents\u003c/p\u003e \u003cp\u003edef save_to_csv(documents, csv_filename):\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003ewith open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile:\u003c/p\u003e \u003cp\u003efieldnames = ['Title', 'Link', 'Snippet']\u003c/p\u003e \u003cp\u003ewriter\u0026thinsp;=\u0026thinsp;csv.DictWriter(csvfile, fieldnames\u0026thinsp;=\u0026thinsp;fieldnames)\u003c/p\u003e \u003cp\u003ewriter.writeheader()\u003c/p\u003e \u003cp\u003efor doc in documents:\u003c/p\u003e \u003cp\u003ewriter.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']})\u003c/p\u003e \u003cp\u003eprint(f\"{len(documents)} results saved to {csv_filename}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred while writing to the CSV file: {e}\")\u003c/p\u003e \u003cp\u003edef main():\u003c/p\u003e \u003cp\u003equery = 'site:gov.au (\"energy infrastructure\" AND \"resilience\") filetype:pdf'\u003c/p\u003e \u003cp\u003eapi_key = \"YOUR_API_KEY\" # Replace with your actual API key\u003c/p\u003e \u003cp\u003ecx = \"YOUR_CUSTOM_SEARCH_ENGINE_ID\" # Replace with your actual custom search engine ID\u003c/p\u003e \u003cp\u003ecsv_filename\u0026thinsp;=\u0026thinsp;os.path.join(\u003c/p\u003e \u003cp\u003e\u0026ldquo;YOUR FILE DESTINATION\"\u003c/p\u003e \u003cp\u003e)\u003c/p\u003e \u003cp\u003enum_results\u0026thinsp;=\u0026thinsp;10 # Adjust as needed\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;num_results)\u003c/p\u003e \u003cp\u003esave_to_csv(search_results, csv_filename) if search_results else print(\"No results found.\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003eif __name__ == \"__main__\":\u003c/p\u003e \u003cp\u003emain()\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eAn error occurred during the search: 400 Client Error: Bad Request for url\u003c/b\u003e: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.googleapis.com/customsearch\u003c/span\u003e\u003cspan address=\"https://www.googleapis.com/customsearch\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e \u003cb\u003e[...]\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eThe error message indicates a \"400 Client Error: Bad Request,\" which means the request sent to the Google Custom Search API is malformed or incorrect. To troubleshoot and fix this issue, you can consider the following steps:\u003c/p\u003e \u003cp\u003e1. \u003cb\u003eCheck API Key and CX\u003c/b\u003e:\u003c/p\u003e \u003cp\u003e\u0026bull; Ensure that your API key (\u003cb\u003eapi_key\u003c/b\u003e) and custom search engine ID (\u003cb\u003ecx\u003c/b\u003e) are correct and valid. Double-check that there are no typos or mistakes in these values.\u003c/p\u003e \u003cp\u003e2. \u003cb\u003eReview Query Parameters\u003c/b\u003e:\u003c/p\u003e \u003cp\u003e\u0026bull; Examine the constructed query (\u003cb\u003eparams\u003c/b\u003e) in your \u003cb\u003esearch_AUST_energy_policy_documents\u003c/b\u003e function. Ensure that the query parameters are formatted correctly and meet the requirements of the Google Custom Search API.\u003c/p\u003e \u003cp\u003e3. \u003cb\u003eURL Encoding\u003c/b\u003e:\u003c/p\u003e \u003cp\u003e\u0026bull; Confirm that the query parameters are properly URL-encoded. The error message in your case shows that the URL includes characters like \u003cb\u003e(\u003c/b\u003e, \u003cb\u003e)\u003c/b\u003e, and spaces, which should be properly encoded.\u003c/p\u003e \u003cp\u003e4. \u003cb\u003eLimitations and Quotas\u003c/b\u003e:\u003c/p\u003e \u003cp\u003e\u0026bull; Check if you are exceeding any limitations or quotas imposed by the Google Custom Search API. For example, there might be restrictions on the number of requests you can make in a given time period.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eBecause I am using a free version, I can only look at 10 pages. Can you change the code to reflect this?\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCertainly! If you're limited to 10 pages of results, you can modify the \u003cb\u003esearch_AUST_energy_policy_documents\u003c/b\u003e function to restrict the number of iterations in the while loop. Here's an updated version:\u003c/p\u003e \u003cp\u003edef search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;10, max_pages\u0026thinsp;=\u0026thinsp;1):\u003c/p\u003e \u003cp\u003ebase_url = \"\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.googleapis.com/customsearch/v1\"\u003c/span\u003e\u003cspan address=\"https://www.googleapis.com/customsearch/v1\u0026quot;\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003cp\u003eall_documents = []\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003efor _ in range(max_pages):\u003c/p\u003e \u003cp\u003eparams = {\u003c/p\u003e \u003cp\u003e'q': query,\u003c/p\u003e \u003cp\u003e'key': api_key,\u003c/p\u003e \u003cp\u003e'cx': cx,\u003c/p\u003e \u003cp\u003e'fileType': 'pdf',\u003c/p\u003e \u003cp\u003e'num': num_results,\u003c/p\u003e \u003cp\u003e'start': start_index,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003eresponse\u0026thinsp;=\u0026thinsp;requests.get(base_url, params\u0026thinsp;=\u0026thinsp;params)\u003c/p\u003e \u003cp\u003eresponse.raise_for_status() # Raise an HTTPError for bad responses\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;response.json()\u003c/p\u003e \u003cp\u003eif 'error' in search_results:\u003c/p\u003e \u003cp\u003eraise Exception(f\"Google Custom Search API error: {search_results['error']['message']}\")\u003c/p\u003e \u003cp\u003edocuments\u0026thinsp;=\u0026thinsp;extract_documents(search_results)\u003c/p\u003e \u003cp\u003eall_documents.extend(documents)\u003c/p\u003e \u003cp\u003e# Update 'start' parameter for the next page\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;+\u0026thinsp;=\u0026thinsp;num_results\u003c/p\u003e \u003cp\u003eexcept requests.exceptions.RequestException as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred during the search: {e}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003ereturn all_documents\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eThank you, the code is running as expected.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYou're welcome! I'm glad to hear that the code is running as expected. If you have any more questions, need further assistance, or have other code you'd like me to review, feel free to reach out. Happy coding!\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFollowing this, and in line with the code found in [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], a User-Agent header was included. This was to ensure transparency on the agent end of the code, and to indicate that it was not a \u0026lsquo;bot\u0026rsquo; behind the helm. This addition is indicated in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eThe addition of a User-Agent Header to the script.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrompt\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eResponse\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eShould I include a user-agent header in the code?\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eYes, including a User-Agent header in your code is a good practice. The User-Agent header provides information about the client making the request, such as the type of browser or application and its version. Some websites and APIs may use this information to identify and differentiate between various clients.\u003c/p\u003e \u003cp\u003eIncluding a User-Agent header helps in the following ways:\u003c/p\u003e \u003cp\u003e1. \u003cb\u003eIdentification\u003c/b\u003e: It allows the website or API server to identify your request and can be used for statistical or analytical purposes.\u003c/p\u003e \u003cp\u003e2. \u003cb\u003eAvoiding Blocks\u003c/b\u003e: Some websites implement access controls or rate limiting based on User-Agent. Providing a User-Agent that looks similar to a typical web browser can help avoid being blocked or flagged as suspicious.\u003c/p\u003e \u003cp\u003e3. \u003cb\u003eGood Etiquette\u003c/b\u003e: Including a User-Agent is considered good etiquette.\u003c/p\u003e \u003cp\u003eHere's how you can modify your code to include a User-Agent header using the \u003cb\u003eheaders\u003c/b\u003e parameter in the \u003cb\u003erequests.get\u003c/b\u003e method:\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003edef search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;10, max_pages\u0026thinsp;=\u0026thinsp;1):\u003c/p\u003e \u003cp\u003ebase_url = \"\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.googleapis.com/customsearch/v1\"\u003c/span\u003e\u003cspan address=\"https://www.googleapis.com/customsearch/v1\u0026quot;\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003cp\u003eall_documents = []\u003c/p\u003e \u003cp\u003eheaders = {\u003c/p\u003e \u003cp\u003e'User-Agent': 'Your-User-Agent-Here',\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003efor _ in range(max_pages):\u003c/p\u003e \u003cp\u003eparams = {\u003c/p\u003e \u003cp\u003e'q': query,\u003c/p\u003e \u003cp\u003e'key': api_key,\u003c/p\u003e \u003cp\u003e'cx': cx,\u003c/p\u003e \u003cp\u003e'fileType': 'pdf',\u003c/p\u003e \u003cp\u003e'num': num_results,\u003c/p\u003e \u003cp\u003e'start': start_index,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003eresponse\u0026thinsp;=\u0026thinsp;requests.get(base_url, params\u0026thinsp;=\u0026thinsp;params, headers\u0026thinsp;=\u0026thinsp;headers)\u003c/p\u003e \u003cp\u003eresponse.raise_for_status() # Raise an HTTPError for bad responses\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;response.json()\u003c/p\u003e \u003cp\u003eif 'error' in search_results:\u003c/p\u003e \u003cp\u003eraise Exception(f\"Google Custom Search API error: {search_results['error']['message']}\")\u003c/p\u003e \u003cp\u003edocuments\u0026thinsp;=\u0026thinsp;extract_documents(search_results)\u003c/p\u003e \u003cp\u003eall_documents.extend(documents)\u003c/p\u003e \u003cp\u003e# Update 'start' parameter for the next page\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;+\u0026thinsp;=\u0026thinsp;num_results\u003c/p\u003e \u003cp\u003eexcept requests.exceptions.RequestException as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred during the search: {e}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003ereturn all_documents\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTo assist with people utilising the method presented. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e provides a full account of the code. With this new method, researchers will be able to systematically gather grey literature for their SLRs, saving significant time and money in the process. Users will have to ensure that the Python libraries that are utilised are installed on their computers, and that they input the correct information into the areas that are highlighted in all caps, bold, and italics. In the instance of this research project, the author ran the code in a Jupyter notebook in Visual Studio Code.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eFull account of the newly scripted code.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"1\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eimport os\u003c/p\u003e \u003cp\u003eimport requests\u003c/p\u003e \u003cp\u003eimport csv\u003c/p\u003e \u003cp\u003edef search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;10, max_pages\u0026thinsp;=\u0026thinsp;10):\u003c/p\u003e \u003cp\u003ebase_url = \"\u003cb\u003eYOUR COUSTOM SEARCH ENGINE\u003c/b\u003e\"\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;=\u0026thinsp;1\u003c/p\u003e \u003cp\u003eall_documents = []\u003c/p\u003e \u003cp\u003eheaders = {\u003c/p\u003e \u003cp\u003e'User-Agent' \u003cb\u003eYOUR USER-AGENT INFORMATION\u003c/b\u003e,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003efor _ in range(max_pages):\u003c/p\u003e \u003cp\u003eparams = {\u003c/p\u003e \u003cp\u003e'q': query,\u003c/p\u003e \u003cp\u003e'key': api_key,\u003c/p\u003e \u003cp\u003e'cx': cx,\u003c/p\u003e \u003cp\u003e'fileType': 'pdf',\u003c/p\u003e \u003cp\u003e'num': num_results,\u003c/p\u003e \u003cp\u003e'start': start_index,\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003eresponse\u0026thinsp;=\u0026thinsp;requests.get(base_url, params\u0026thinsp;=\u0026thinsp;params, headers\u0026thinsp;=\u0026thinsp;headers)\u003c/p\u003e \u003cp\u003eresponse.raise_for_status() # Raise an HTTPError for bad responses\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;response.json()\u003c/p\u003e \u003cp\u003eif 'error' in search_results:\u003c/p\u003e \u003cp\u003eraise Exception(f\"Google Custom Search API error: {search_results['error']['message']}\")\u003c/p\u003e \u003cp\u003edocuments\u0026thinsp;=\u0026thinsp;extract_documents(search_results)\u003c/p\u003e \u003cp\u003eall_documents.extend(documents)\u003c/p\u003e \u003cp\u003e# Update 'start' parameter for the next page\u003c/p\u003e \u003cp\u003estart_index\u0026thinsp;+\u0026thinsp;=\u0026thinsp;num_results\u003c/p\u003e \u003cp\u003eexcept requests.exceptions.RequestException as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred during the search: {e}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003ereturn all_documents\u003c/p\u003e \u003cp\u003edef extract_documents(search_results):\u003c/p\u003e \u003cp\u003eresult_items\u0026thinsp;=\u0026thinsp;search_results.get('items', [])\u003c/p\u003e \u003cp\u003edocuments = []\u003c/p\u003e \u003cp\u003efor item in result_items:\u003c/p\u003e \u003cp\u003edocument = {\u003c/p\u003e \u003cp\u003e'title': item.get('title', 'No Title'),\u003c/p\u003e \u003cp\u003e'link': item.get('link', 'No Link'),\u003c/p\u003e \u003cp\u003e'snippet': item.get('snippet', 'No Snippet'),\u003c/p\u003e \u003cp\u003e}\u003c/p\u003e \u003cp\u003edocuments.append(document)\u003c/p\u003e \u003cp\u003ereturn documents\u003c/p\u003e \u003cp\u003edef save_to_csv(documents, csv_filename):\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003ewith open(csv_filename, 'w', newline='', encoding='utf-8') as csvfile:\u003c/p\u003e \u003cp\u003efieldnames = ['Title', 'Link', 'Snippet']\u003c/p\u003e \u003cp\u003ewriter\u0026thinsp;=\u0026thinsp;csv.DictWriter(csvfile, fieldnames\u0026thinsp;=\u0026thinsp;fieldnames)\u003c/p\u003e \u003cp\u003ewriter.writeheader()\u003c/p\u003e \u003cp\u003efor doc in documents:\u003c/p\u003e \u003cp\u003ewriter.writerow({'Title': doc['title'], 'Link': doc['link'], 'Snippet': doc['snippet']})\u003c/p\u003e \u003cp\u003eprint(f\"{len(documents)} results saved to {csv_filename}\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An error occurred while writing to the CSV file: {e}\")\u003c/p\u003e \u003cp\u003edef main():\u003c/p\u003e \u003cp\u003equery = 'site:gov.au (\"energy infrastructure\" AND (\"resilience\")) filetype:pdf'\u003c/p\u003e \u003cp\u003eapi_key = \"\u003cb\u003eYOUR API KEY\u003c/b\u003e\"\u003c/p\u003e \u003cp\u003ecx = \"\u003cb\u003eYOUR CUSTOM SEARCH ENGINE ID\u003c/b\u003e\"\u003c/p\u003e \u003cp\u003ecsv_filename\u0026thinsp;=\u0026thinsp;os.path.join(\u003c/p\u003e \u003cp\u003e\u003cb\u003eYOUR FILE LOCATION\u003c/b\u003e\\\\. csv\"\u003c/p\u003e \u003cp\u003e)\u003c/p\u003e \u003cp\u003enum_results\u0026thinsp;=\u0026thinsp;10 # Adjust as needed\u003c/p\u003e \u003cp\u003etry:\u003c/p\u003e \u003cp\u003esearch_results\u0026thinsp;=\u0026thinsp;search_AUST_energy_policy_documents(query, api_key, cx, num_results\u0026thinsp;=\u0026thinsp;num_results)\u003c/p\u003e \u003cp\u003esave_to_csv(search_results, csv_filename) if search_results else print(\"No results found.\")\u003c/p\u003e \u003cp\u003eexcept Exception as e:\u003c/p\u003e \u003cp\u003eprint(f\"An unexpected error occurred: {e}\")\u003c/p\u003e \u003cp\u003eif __name__ == \"__main__\":\u003c/p\u003e \u003cp\u003emain()\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section3\"\u003e \u003ch2\u003e3.1.3. STAGE THREE: execution\u003c/h2\u003e \u003cp\u003eThe executed code provides the output \u0026ldquo;100 results saved to \u003cb\u003eFile location\u003c/b\u003e\\\\ \u003cb\u003eXXX\u003c/b\u003e.results.csv\u0026rdquo;. It accomplished this in 7.5 seconds, and automatically saved a CSV file to the specified location. The next section will provide the results followed by a guide for researchers who wish to utilise this method.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"4. FINDINGS","content":"\u003cp\u003eThe scripted code provided the following results. A CSV file was produced that contained the following information: three columns with the headings \u0026lsquo;Title\u0026rsquo;, \u0026lsquo;Link\u0026rsquo;, and \u0026lsquo;Snippet.\u0026rsquo; Each column contained the information for 100 PDFs identified in the search. Figures\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e provides screenshot snippets of the Titles and their Links that were uncovered. Each PDF link, when copied and pasted into a web browser, is functional. The search strategy words \u0026ldquo;energy infrastructure\u0026rdquo; AND \u0026ldquo;resilience\u0026rdquo; is contained in either the title or the snippet. The full CSV file is provided in the supplementary materials. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e provides a screenshot snippet of the Blurb accompanying each result.\u003c/p\u003e \u003cp\u003eWithin the search strategy the URL \u0026ldquo;un.org\u0026rdquo; was added. This was done to illustrate the ease in which multiple websites can be searched. A snippet of the Titles and Links from this search can be viewed in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThese results illustrate the speed, rigor, and transparency of the new tool and underscores its importance. Capturing every tier of Australian government, the above results provide grey literature from Victoria, New South Whales, Queensland, Torris Straight, and the Australian Federal Government. Should users wish to narrow the search to these states, they would simply change the URL to \u0026ldquo;vic.gov.au\u0026rdquo; for Victoria, or \u0026ldquo;qld.gov.au\u0026rdquo; for Queensland. Should users wish to add other URLs such as \u0026ldquo;un.org\u0026rdquo; they need to only add the OR Boolean to the URL component of the strategy; for example, \u0026ldquo; 'site:gov.au OR site:un.org (\"energy infrastructure\" AND (\"resilience\")) filetype:pdf' \u0026rdquo;\u003c/p\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e4.1. GUIDE FOR RESEARCHERS.\u003c/h2\u003e \u003cp\u003eFor researcher who wish to utilise this new methodology, the following steps are recommended.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP ONE\u003c/b\u003e \u003c/p\u003e \u003cp\u003eDownload Python from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.python.org/downloads/\u003c/span\u003e\u003cspan address=\"https://www.python.org/downloads/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP TWO\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFamiliarise yourself with how libraries are installed in Python. Pip is the preferred installer program to be used in Python. You can find instructions on how to do so at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://packaging.python.org/en/latest/tutorials/installing-packages/\u003c/span\u003e\u003cspan address=\"https://packaging.python.org/en/latest/tutorials/installing-packages/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. There are also a large number of YouTube videos that will not only provide in-depth instructions for this stage but will also provide a deeper introduction to Python.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP THREE\u003c/b\u003e \u003c/p\u003e \u003cp\u003eInstall the libraries required for the tool. You will need to install \u0026ldquo;OS\u0026rdquo;, \u0026ldquo;Requests\u0026rdquo;, and \u0026ldquo;CSV\u0026rdquo;.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP FOUR\u003c/b\u003e \u003c/p\u003e \u003cp\u003eUsing the google documents provided in the google portion of the \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003etheoretical background\u003c/span\u003e section, follow the instructions to create a google PSE and to obtain a JSON API. For the PSE component see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://developers.google.com/custom-search/docs/overview\u003c/span\u003e\u003cspan address=\"https://developers.google.com/custom-search/docs/overview\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. For obtaining the API key see \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://developers.google.com/custom-search/v1/overview\u003c/span\u003e\u003cspan address=\"https://developers.google.com/custom-search/v1/overview\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP FIVE\u003c/b\u003e \u003c/p\u003e \u003cp\u003eCopy and paste the full code provide in this methods paper into a Jupyter notebook (other notebooks will also support this code, in this instance the author utilised a Jupyter notebook). You will also need to ensure that you input your own User-Agent Header to the script. Also, change the search query to reflect the information being sought. Lastly, you will need to designate a location for the csv file to be saved in.\u003c/p\u003e \u003cp\u003e \u003cb\u003eSTEP SIX\u003c/b\u003e \u003c/p\u003e \u003cp\u003eRun the code. Should any errors be encountered, [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] offers a method to help resolve them. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e provides a visual representation of six step process.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"5. DISCUSSION AND LIMITATIONS","content":"\u003cp\u003eThe above sections have presented a new tool for researchers to systematically gather grey literature for SLRs. It has illustrated that Titles, Links, and Blurbs of PDF files pertaining to \u0026ldquo;energy infrastructure\u0026rdquo; AND \u0026ldquo;resilience\u0026rdquo; can be identified, saved, and accessed in a remarkably small amount of time. This technique will not only save researchers resources (time and money), but it will also increase the rigor, transparency, and efficiency of this stage in a SLR.\u003c/p\u003e \u003cp\u003eAs highlighted in [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e], AI, and in this instance GenAI, will have an impact on the world that is akin to the impact of the industrial revolution, perhaps greater. To this end, this is not the first instance of computational-based methods being applied to stages of SLRs. This article not alone in creating new methods that infuse AI and MLTs within SLRs. [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e] provides a thorough account of how AI and MLTs can be applied to different stages of a SLR. Focusing primarily on how Latent Dirichlet Allocation (LDA) Topic Models can be used to abstract and synthesise data obtained during a review, [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e] also introduces AI and MLTs that can be utilised in the title/abstract screening stage, as well as the data extraction stage. [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] illustrates how Python can be utilised to create a web-scraper to gather grey literature from a single online data repository. Indeed, incorporating AI and MLTs into the SLR process is the topic of contemporary works investigating how they can be inserted into current guidelines. For an account on how AI and MLTs can be incorporated within the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, please see [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e] on how this could be achieved.\u003c/p\u003e \u003cp\u003eThe scripted code presented in this article will transform how grey literature is gathered for use in a SLR. Although cheap, extensive use of this tool is not one-hundred percent free. The results presented are limited to the free-use terms of Google\u0026rsquo;s API key. If researchers require a greater number of searches, and more results, then they will have to purchase this ability from Google. Additionally, to keep the scripted code as simple as possible for the reader, more diverse search strategies alongside data organisation and saving functions have not been included. This will be left for future research. [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] highlighted that there is a large ethical concern regarding appropriate recognition of the coding that GenAIs are trained on. Retrospective searches of coding was suggested as a possible way to attribute codes. At the time of writing, similar code has not been located. Although it is impossible to be certain where the code originated, looking for it is a step forward in the process of solving the ethical issues of utilising GenAIs to script code.\u003c/p\u003e \u003cp\u003eA notable limitation of this article is that researchers may not be acquainted with the tools employed. The intention is that by furnishing external resources, direction, and a guide, the technical gap necessary to employ the proposed method has been somewhat alleviated, making it more accessible for a broader audience.\u003c/p\u003e"},{"header":"6. FUTURE RESEARCH","content":"\u003cp\u003eThe tool that has been introduced in this article is intended to be employed to gather pertinent policy documents, standards, laws, and regulations to be utilised in a connected research project. Future research will also include scripting code to increase the sophistication of the queries that can be entered, alongside different methods to organise and save the data generated through this new process. Applications for areas outside of grey literature are also being investigated.\u003c/p\u003e"},{"header":"7. CONCLUSION","content":"\u003cp\u003eThis article has introduced a new and novel tool to aid in gathering grey literature for use in a SLR. Though the use of the computer programming language Python, coding assistance of ChatGPT 3.5, and the ability to create PSEs from Google, this article has introduced a technique for researchers to cheaply, quickly, rigorously, and transparently gather grey literature. No longer requiring long hours and unreliable search processes, this new tool will help every discipline to gather grey literature for their research.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNatural Hazards Research Australia.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of Interests/Competing Interests:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe author has neither conflicts of interests nor competing interests to declare that are relevant to the content of this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data pertinent to the article has been made available in the supplementary materials.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode Availability:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll codes utilised in this research project have either been provided in text or are contained in the supplementary materials.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eUse of AI:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Generative AI ChatGPT 3.5 was utilised in the methods section of this article.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cb\u003eAuthor Contribution\u003c/b\u003e\u003c/p\u003e\u003cp\u003eC.A, as sole author, is responsible for all aspects of this manuscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eThom\u0026eacute; AMT, Scavarda LF, Scavarda AJ. Conducting systematic literature review in operations management. (2016). Production Planning \u0026amp; Control, 27(5). 408\u0026ndash;20. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/09537287.2015.1129464\u003c/span\u003e\u003cspan address=\"10.1080/09537287.2015.1129464\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePati D, Lorusso LN. How to write a systematic review of the literature. (2018). HERD: Health Environments Research \u0026amp; Design Journal, 11(1). 15\u0026ndash;30. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/1937586717747384\u003c/span\u003e\u003cspan address=\"10.1177/1937586717747384\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePetticrew M, Roberts H. \u003cem\u003eSystematic reviews in the social sciences: A practical guide\u003c/em\u003e: John Wiley \u0026amp; Sons; 2008.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXiao Y, Watson M. Guidance on Conducting a Systematic Literature Review. (2019). Journal of Planning Education and Research, 39(1). 93\u0026ndash;112. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/0739456x17723971\u003c/span\u003e\u003cspan address=\"10.1177/0739456x17723971\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSiddaway AP, Wood AM, Hedges LV. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. (2019). Annual Review of Psychology, 70(1). 747\u0026ndash;70. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1146/annurev-psych-010418-102803\u003c/span\u003e\u003cspan address=\"10.1146/annurev-psych-010418-102803\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePaul J, Lim WM, O\u0026rsquo;Cass A, Hao AW, Bresciani S. Scientific procedures and rationales for systematic literature reviews (SPAR-4‐SLR). (2021). International Journal of Consumer Studies, 45(4). O1-O16. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/ijcs.12695\u003c/span\u003e\u003cspan address=\"10.1111/ijcs.12695\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSundaram G, Berleant D. Automating Systematic Literature Reviews with Natural Language Processing and Text Mining: a Systematic Literature Review. (2022). ArXiv Preprint \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2211.15397\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2211.15397\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. (2019). Systematic reviews, 8. 1\u0026ndash;10. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s13643-019-1074-9\u003c/span\u003e\u003cspan address=\"10.1186/s13643-019-1074-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. (2007). Annals of internal medicine, 147(4). 224\u0026thinsp;\u0026ndash;\u0026thinsp;33. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.7326/0003-4819-147-4-200708210-00179\u003c/span\u003e\u003cspan address=\"10.7326/0003-4819-147-4-200708210-00179\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAtkinson CF. ChatGPT and computational-based research: benefits, drawbacks, and machine learning applications. (2023). Discover Artificial Intelligence, 3(1). 42. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s44163-023-00091-3\u003c/span\u003e\u003cspan address=\"10.1007/s44163-023-00091-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAdams J, Hillier-Brown FC, Moore HJ, Lake AA, Araujo-Soares V, White M, et al. Searching and synthesising \u0026lsquo;grey literature\u0026rsquo; and \u0026lsquo;grey information\u0026rsquo; in public health: critical reflections on three case studies. (2016). Systematic Reviews, 5(1). 164. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s13643-016-0337-y\u003c/span\u003e\u003cspan address=\"10.1186/s13643-016-0337-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKamei F, Wiese I, Pinto G, Ribeiro M, Soares S, editors. On the use of grey literature: A survey with the brazilian software engineering research community. Proceedings of the XXXIV Brazilian Symposium on Software Engineering; 2020. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1145/3422392.3422442\u003c/span\u003e\u003cspan address=\"10.1145/3422392.3422442\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBanks M. \u003cem\u003eBlog posts and tweets: the next frontier for grey literature\u003c/em\u003e. Grey literature in library and information studies: De Gruyter; 2009.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAtkinson C. Cheap, Rigorous, and Transparent: How Web-scraping with Python can Improve Collecting Grey Literature for Systematic Literature Reviews. (2023). Grey Journal (TGJ), 19(3).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePaez A. Gray literature: An important resource in systematic reviews. (2017). Journal of Evidence-Based Medicine, 10(3). 233\u0026thinsp;\u0026ndash;\u0026thinsp;40. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/jebm.12266\u003c/span\u003e\u003cspan address=\"10.1111/jebm.12266\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGrayson L, Gomersall A. Google \u0026ndash; boon, bane or both? (2006). Evid Polciy, 2(4). 538\u0026thinsp;\u0026ndash;\u0026thinsp;42. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1332/174426406778881818\u003c/span\u003e\u003cspan address=\"10.1332/174426406778881818\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaddaway NR, Bethel A, Dicks LV, Koricheva J, Macura B, Petrokofsky G, et al. Eight problems with literature reviews and how to fix them. (2020). Nature Ecology \u0026amp; Evolution, 4(12). 1582\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41559-020-01295-x\u003c/span\u003e\u003cspan address=\"10.1038/s41559-020-01295-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGodin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. (2015). Systematic Reviews, 4(1). 138. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s13643-015-0125-0\u003c/span\u003e\u003cspan address=\"10.1186/s13643-015-0125-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWinters M, Weir A. Grey matters; on the importance of publication bias in systematic reviews. (2017). 51(6). 488\u0026ndash;9. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1136/bjsports-2016-096679\u003c/span\u003e\u003cspan address=\"10.1136/bjsports-2016-096679\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEchtler F, H\u0026auml;u\u0026szlig;ler M, editors. Open source, open science, and the replication crisis in HCI. Extended abstracts of the 2018 CHI conference on human factors in computing systems; 2018. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1145/3170427.3188395\u003c/span\u003e\u003cspan address=\"10.1145/3170427.3188395\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNagpal A, Gabrani G, editors. Python for data analytics, scientific and technical applications. 2019 Amity international conference on artificial intelligence (AICAI); 2019: IEEE. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1109/AICAI.2019.8701341\u003c/span\u003e\u003cspan address=\"10.1109/AICAI.2019.8701341\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKumar R. Future for scientific computing using Python. (2015). International Journal of Engineering Technologies and Management Research, 2(1). 30\u0026ndash;41. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.29121/ijetmr.v2.i1.2015.28\u003c/span\u003e\u003cspan address=\"10.29121/ijetmr.v2.i1.2015.28\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTilkov S, Vinoski S. Node. js: Using JavaScript to build high-performance network programs. (2010). IEEE Internet Computing, 14(6). 80\u0026ndash;3. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1109/MIC.2010.145\u003c/span\u003e\u003cspan address=\"10.1109/MIC.2010.145\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eArnold K, Gosling J, Holmes D. \u003cem\u003eThe Java programming language\u003c/em\u003e: Addison Wesley Professional; 2005.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKlabnik S, Nichols C. \u003cem\u003eThe Rust programming language\u003c/em\u003e: No Starch Press; 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHejlsberg A, Torgersen M, Wiltamuth S, Golde P. \u003cem\u003eThe C# programming language\u003c/em\u003e: Pearson Education; 2008.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePython Software Foundation. OS \u0026mdash; Miscellaneous operating system interfaces Online: Python Software Foundation; (2024) [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://docs.python.org/3/library/os.html\u003c/span\u003e\u003cspan address=\"https://docs.python.org/3/library/os.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePython Software Foundation. Requests 2.31.0 Online: Python Software Foundation; (2024) [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://pypi.org/project/requests/\u003c/span\u003e\u003cspan address=\"https://pypi.org/project/requests/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePython Software Foundation. CSV File Reading and Writing Online: Python Software Foundation; (2024) [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://docs.python.org/3/library/csv.html\u003c/span\u003e\u003cspan address=\"https://docs.python.org/3/library/csv.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAzaria A, Azoulay R, Reches S. ChatGPT is a Remarkable Tool\u0026ndash;For Experts. (2023). ArXiv Preprint \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2306.03102\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2306.03102\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOlujimi PA, Ade-Ibijola A. NLP techniques for automating responses to customer queries: a systematic review. (2023). Discover Artificial Intelligence, 3(1). 20. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s44163-023-00065-5\u003c/span\u003e\u003cspan address=\"10.1007/s44163-023-00065-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMorris MR. Scientists' Perspectives on the Potential for Generative AI in their Fields. (2023). ArXiv Preprint \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2304.01420\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2304.01420\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrawford K, Paglen T. Excavating AI: the politics of images in machine learning training sets. (2021). AI \u0026amp; SOCIETY, 36(4). 1105\u0026ndash;16. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00146-021-01162-8\u003c/span\u003e\u003cspan address=\"10.1007/s00146-021-01162-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHacker P, Engel A, Mauer M, editors. Regulating ChatGPT and other large generative AI models. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency; 2023. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1145/3593013.3594067\u003c/span\u003e\u003cspan address=\"10.1145/3593013.3594067\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTian H, Lu W, Li TO, Tang X, Cheung S-C, Klein J, et al. Is ChatGPT the Ultimate Programming Assistant\u0026ndash;How far is it? (2023). \u003cem\u003eArXiv Preprint\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2304.11938\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2304.11938\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaque MA, Li S. The Potential Use of ChatGPT for Debugging and Bug Fixing. (2023). \u003cem\u003eEAI Endorsed Transactions on AI and Robotics\u003c/em\u003e, 2(1). e4-e. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.4108/airo.v2i1.3276\u003c/span\u003e\u003cspan address=\"10.4108/airo.v2i1.3276\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu J, Xia CS, Wang Y, Zhang L. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. (2023). ArXiv Preprint \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2305.01210\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2305.01210\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDao X-Q. Performance comparison of large language models on vnhsge english dataset: Openai chatgpt, microsoft bing chat, and google bard. (2023). \u003cem\u003eArXiv Preprint\u003c/em\u003e. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2307.02288\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2307.02288\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRay PP. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. (2023). \u003cem\u003eInternet of Things and Cyber-Physical Systems\u003c/em\u003e, 3. 121\u0026thinsp;\u0026ndash;\u0026thinsp;54. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.iotcps.2023.04.003\u003c/span\u003e\u003cspan address=\"10.1016/j.iotcps.2023.04.003\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDhara A. A personalised discovery service using Google custom search engine. (2016). \u003cem\u003eAnnals of Library and information Studies\u003c/em\u003e, 63. 298\u0026ndash;305. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://nopr.niscpr.res.in/bitstream/123456789/39764/1/ALIS%2063%284%29%20298-305.pdf\u003c/span\u003e\u003cspan address=\"https://nopr.niscpr.res.in/bitstream/123456789/39764/1/ALIS%2063%284%29%20298-305.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMedaglia R, Gil-Garcia JR, Pardo TA. Artificial Intelligence in Government: Taking Stock and Moving Forward. (2021). Social Science Computer Review, 41(1). 123\u0026ndash;40. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/08944393211034087\u003c/span\u003e\u003cspan address=\"10.1177/08944393211034087\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAtkinson CF. Cheap, Quick, and Rigorous: Artificial Intelligence and the Systematic Literature Review. (2023). Social Science Computer Review. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/08944393231196281\u003c/span\u003e\u003cspan address=\"10.1177/08944393231196281\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSusnjak T. Prisma-dfllm: An extension of prisma for systematic literature reviews using domain-specific finetuned large language models. (2023). \u003cem\u003eArXiv Preprint\u003c/em\u003e 1\u0026ndash;20. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2306.14905\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2306.14905\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4007725/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4007725/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Systematically gathering grey literature for use in a systematic literature review (SLR) is a challenging task, given the decentralised nature of online resources. Researchers, particularly those in the social sciences, often find themselves navigating government or non-government organisation websites, manually scouring repositories for documents relevant to their SLRs. This important stage is expensive in terms of time and resources required and, in many instances, difficult to repeat. This article introduces an innovative methodology to address these challenges, providing social science researchers with a systematic approach to gather grey literature for inclusion in SLRs. Utilising the computer programming language Python, this articles leverages Google’s API to create a programmable search engine, facilitating a systematic search for grey literature intended for inclusion in a SLR. A case study is presented to demonstrate the efficiency of this method in locating PDF documents, within which two examples are provided. In the first example, documents from Australian government websites ending in “gov.au” related to the topic of (“energy infrastructure” AND resilience) are sought. Secondly, “un.org” is added to illustrate how multiple websites can be searched. Highlighting the effectiveness of this approach, the study successfully locates 100 documents in just 7.5 seconds, automatically saving them into an Excel CSV file for further analysis. To the authors knowledge, this method represents an original approach in the systematic gathering of grey literature for SLRs and highlights the contribution of generative artificial intelligence systems such as ChatGPT 3.5 in assisting to script the necessary code for new SLR tools.","manuscriptTitle":"Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-29 19:01:32","doi":"10.21203/rs.3.rs-4007725/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e1382072-ab1c-413f-a94e-a497f628828b","owner":[],"postedDate":"March 29th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-09-13T03:13:01+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-29 19:01:32","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4007725","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4007725","identity":"rs-4007725","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}