OCTOPUS: Operation Control System for Task Optimization and Job Parallelization via a User-Optimal Scheduler

OCTOPUS: Operation Control System for Task Optimization and Job Parallelization via a User-Optimal Scheduler | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article OCTOPUS: Operation Control System for Task Optimization and Job Parallelization via a User-Optimal Scheduler Sang Soo Han, Hyuk Jun Yoo, Kwan-Young Lee, Donghun Kim This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4254570/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 08 Nov, 2024 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract The materials acceleration platform (MAP), empowered by robotics and artificial intelligence, is a transformative approach for expediting material discovery processes across diverse domains. However, the development of an operating system for MAP faces challenges in simultaneously managing diverse experiments from multiple users. Specifically, when MAP is utilized by multiple users, the overlapping challenges of experimental modules or devices can lead to inefficiencies in both resource utilization and safety hazards. To overcome these challenges, we present an operation control system for MAP, namely, OCTOPUS, which is an acronym for operation control system for task optimization and job parallelization via a user-optimal scheduler. OCTOPUS streamlines experiment scheduling and optimizes resource utilization through integrating its interface node, master node and module nodes. Leveraging process modularization and a network protocol, OCTOPUS ensures the homogeneity, scalability, safety and versatility of MAP. In addition, OCTOPUS embodies a user-optimal scheduler. Job parallelization and task optimization techniques mitigate delays and safety hazards within realistic operational environments, while the closed-packing schedule algorithm efficiently executes multiple jobs with minimal resource waste. This work offers a solution to the challenges encountered within MAP accessed by multiple users, and thereby will facilitate its widespread adoption in material development processes. Physical sciences/Materials science/Techniques and instrumentation/Design, synthesis and processing Physical sciences/Nanoscience and technology/Techniques and instrumentation/Design, synthesis and processing OCTOPUS Materials Acceleration Platform Job parallelization Task optimization Process modularization Network protocol User-optimal scheduler Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Introduction The material acceleration platform (MAP) has revolutionized material discovery through extensive exploration of the chemical space in diverse domains, including organic molecules, perovskites, colloidal quantum dots and nanoparticles. 1–4 Through the integration of robotic automation and AI-based experimental planning, MAP has shown promising potential in enhancing the search efficiency for the target materials 5–8 while ensuring reliability in surveillance-free environments. 9–12 However, the scalability and accessibility of MAP with multiple users have been impeded by limitations in experimental device sharing and resource allocation, necessitating advanced research on operating system (OS). The advent of OS for MAP, exemplified by Chemputer and ChemOS, has provided a promising pathway toward integrated management systems. 13,14 Yet, their accessibility remains restricted, and is primarily constrained by single-researcher execution and limited applicability. Additionally, the inefficiency of device resource allocations due to the lack of experimental device sharing across different applications poses scalability challenges. 15–17 Addressing these constraints requires a paradigm shift toward a multiuser system, which necessitates the development of a central management platform with a master/node relationship capable of managing experimental equipment efficiently. 18 Central to this endeavor, is the modularization of independent experimental processes to facilitate widespread equipment sharing across diverse applications. However, the development of an OS for multiuser systems has posed challenges to standardized methodologies, primarily stemming from equipment heterogeneity, platform delocalization and safety concerns. 19–27 Thus, there is a pressing need for extensive research to develop an advanced OS for MAP accessed by multiple users. When implementing central management systems for processing diverse experiments from multiple users, challenges such as module and device overlap are persistent, leading to resource allocation inefficiencies and safety hazards. 16,17 Module overlap constraints, for instance, result in time wastage and operational inefficiencies, while device overlap issues exacerbate safety concerns from collisions of the activity radius. Addressing these challenges requires a comprehensive scheduling approach that considers resource allocations, safety protocols and operational efficiencies within realistic environments. In response to these challenges, we introduce OCTOPUS, an acronym for an o peration c ontrol system for t ask o ptimization and job p arallelization via a u ser-optimal s cheduler, which is designed to streamline experimental task scheduling, optimize resource utilization and enhance safety protocols. OCTOPUS comprises three core components— an interface node, master node and module nodes— that facilitate client request handling and experimental task scheduling. Leveraging process modularization and a network protocol, OCTOPUS ensures homogeneity, scalability, safety and versatility within a central management platform. OCTOPUS embodies use-optimal schedulers; it integrates job parallelization techniques to mitigate delays by leveraging device standby times, while the task optimization algorithms prevent safety hazards stemming from device collisions or sharing within realistic operational environments. Additionally, the closed-packing schedule (CPS) algorithm enables module resources to be maximally utilized via compact packing. The user-optimal scheduler (US) in OCTOPUS addresses several challenges encountered within MAP accessed by multiple users, and will catalyze its widespread adoption for expedited discovery of novel materials. Results Terminology definition. The intricate nature of the individual components within the MAP necessitates a standardized terminology to eliminate potential confusion. In response, recent research has advocated for a comprehensive summary of terms and definitions pertinent to MAP. 28 Expanding upon this effort, we present a detailed elucidation of the structural framework underlying MAP, which encompasses four primary components, platforms, modules, tasks and actions (Supplementary Figure S1). The platform serves as the foundational framework that interconnects experimental resources, facilitating automated experiments to cater to the diverse needs of multiple users. The concept of platforms has been increasingly emphasized in recent advancements. 29–31 Within a platform, modules represent the fundamental building blocks, each dedicated to executing a specific experimental process. Examples of modules include "BatchSynthesis", "FlowSynthesis", "SprayCoating", "Filtration" and "UV‒Vis". Each module encapsulates the requisite functionalities necessary for its designated experimental process. A task delineates the granular steps comprising a specific experimental process within a module. For instance, tasks within the "BatchSynthesis" module include "PrepareContainer", "AddSolution", "Stir", "Heat", "Mix" and "React", among others. Actions, on the other hand, denote the discrete operations of the experimental devices required for task execution. Experimental devices such as robotic arms, pipettes and stirrers execute various actions to accomplish a task within a module. For example, the "PrepareContainer" task involves sequential actions by a robotic arm of (1) opening vial storage, (2) picking vials and (3) placing vials on a stirrer (Supplementary Figure S1). Through this structured framework, a MAP harmoniously orchestrates all the components, empowering users to execute desired experiments with high precision and efficiency. This hierarchical terminology definition not only enhances clarity and comprehension within the field but also facilitates seamless communication and collaboration among researchers and practitioners working within the automated experimentation realm. Architecture of OCTOPUS. The architectural blueprint of OCTOPUS is depicted in Fig. 1, which highlights and draws inspiration from the adaptive nature of the octopus. OCTOPUS, which comprises three integral components, the interface node, master node and module nodes, orchestrates the seamless execution of the experimental modules within the MAP. The interface node serves as the nexus, bridging multiple users (or clients) in remote environments to the MAP infrastructure. Next, the master node plays a central role in managing a myriad of tasks and actions, employing a sophisticated suite of six components, a job scheduler, a task generator, a task scheduler, an action translator, an action executor and a resource manager. Finally, the module nodes of OCTOPUS are meticulously modularized to accommodate individual experimental processes, as illustrated in Fig. 1. Upon receiving experimental action directives from the master node, the module nodes initiate modularized robotic operations, encompassing a spectrum of actions such as robotic arm maneuvers, solution dispensation and stirrer activation, among others. The hierarchical structure of OCTOPUS (interface node → master node → module nodes) and the harmonious integration of these components elucidate the intricate workflow of the platform. Job submission via the interface node and job scheduler of the master node . The interface node within OCTOPUS functions as a command line interface (CLI), enabling simultaneous job submissions from multiple clients, as illustrated in Fig. 2. Notably, recent advancements in the user interface design for MAP have primarily been tailored to receiving a single job within an OS. 13,14,19–24 However, the ability to accommodate multiple job submissions concurrently is paramount in enhancing the efficiency of a MAP. 25 Thus, the interface node, bolstered by a multithread pool, enables numerous clients to seamlessly request remote experiments across multiple applications, akin to a computing server (Fig. 2). Notably, OCTOPUS operates real chemical experiments rather than computational tasks, distinguishing it from traditional computing server systems 32,33 . Clients are required to generate job scripts based on the JavaScript object notation (JSON) format, and examples of job scripts are provided in Supplementary Figure S2. These job scripts cater to a spectrum of client demands, ranging from manual experiments with predefined input conditions (e.g. a model with the name of "Manual" in Supplementary Figure S2a) to autonomous experiments enabled by AI-decision processes (e.g. model with the names of "BayesianOptimization", "DecisionTree" and "BayesianNeuralNetwork" in Figure S2b). The model names could be changed by the clients depending on the type of AI model built for experimental planning on the platform. Initially, the interface node takes responsibility for managing the client login process, enforcing stringent security policies through hash functions while awaiting client commands (Supplementary Figure S3). Subsequently, clients submit job scripts via CLI, leveraging portable batch system commands, with detailed examples provided in Supplementary Figure S4. 32,33 This transformation of the local platform into a client/server-based ubiquitous platform has profound implications for research continuity, particularly in scenarios constrained by temporal and spatial limitations, such as those imposed by COVID-19. 34,35 The master node of OCTOPUS embodies a central management system capable of orchestrating diverse experiments for multiple clients. 18 Within the master node, the job scheduler, which comprises a job ID generator, job modeling unit, job storage and three distinct queues (waiting, executing and holding), plays a pivotal role in scheduling jobs for the execution of real experiments, as illustrated in Fig. 2. Upon job submission, the job ID generator assigns a unique identifier (ID) to each job, facilitating orderly processing based on submission order. This job ID is subsequently transferred to the job modeling unit, which generates specific job configurations based on the provided job script, comprising process recommendations (manual vs. autonomous), module and task sequences and experimental conditions. Subsequently, the job storage organizes the generated jobs, awaiting the triggering decision. The triggering decision, determined by various factors, including module resource availability and experimental device status, directs the job execution sequence. If the job trigger decides to execute the next job, it transitions the job ID from the waiting queue to the executing queue for real experiment executions (Supplementary Figure S5). Moreover, a holding queue addresses the safety concerns inherent in surveillance-free MAP environments. In the event of severe safety hazards involving, for example, inflammable or hazardous chemicals, 12 the platform automatically places the job on hold, ensuring user safety. Clients retain the autonomy to restart or terminate holding jobs via predefined commands. Several examples of job management (submission, status check, hold, restart and deletion) via CLI are provided in Supplementary Figure S6. We also provide Supplementary Video 1, where the process of logging in, and performing remote and simultaneous job submissions by two users in the interface node of OCTOPUS is recorded for clarity. Job executions in the master node. Following the job submissions in the interface node, we present the job execution workflow in the master node, incorporating the seven components of job scheduler, task generator, task scheduler, action translator, action executor, resource manager and database, which are specifically tailored for closed-loop experiments, as illustrated in Fig. 3. We empower the task generator to retrieve experimental device information, including physical specifications and the setup environment, from the resource manager. For instance, when preparing to execute the “AddSolution” task, the task generator accesses information detailing the device specifications and setup environments of stock solution types and concentrations, among others. Notably, the dynamic updating of experimental device information from the resource manager enhances operational flexibility (Supplementary Figures S7a and S7b). The task generator integrates the job script with actual experimental device information to complete the predefined task template (in JSON format) containing the experimental conditions and task sequences (Fig. 3 and Supplementary Figures S7c-e). Subsequently, the task scheduler supervises resource allocations for task executions by obtaining information on the location indices from the resource manager (Fig. 3 and Supplementary Figure S8). Crucially, the abstraction of tasks, as exemplified in Chemputer 14 and HELAO-async 24 , ensures adaptability across different platforms by omitting specific device operation and location details. 14,24 To concretize abstracted tasks, we implement an action translator, which translates abstracted tasks into concrete actions, incorporating predefined action sequences and device location information. For instance, the "AddSolution" task involves the serial actions of initializing a pump, moving a dispenser and injecting a solution, each executed by different devices (Fig. 3 and Supplementary Figure S9). Next, the action executor transmits device commands for real experiment execution, employing a transmission protocol that follows predefined data types to module nodes. These data types, including the job ID, device name, action type, action data and mode type, are encoded and transmitted to module nodes via the Transmission Control Protocol/Internet Protocol (TCP/IP) (Fig. 3 and Supplementary Fig. S10). The modules execute experimental action and return the resulting data to the database for subsequent experimental planning. More detailed information on the resulting data structure is provided in Supplementary Figure S11. Network protocol -based modularization. To manipulate multiple jobs within OCTOPUS, process modularization is key for efficiently operating experimental processes. We conceptualized four key concepts to explain the benefits of using experimental process modularization and network protocols, which include homogeneity, scalability, safety and versatility, as illustrated in Fig. 4. First, a challenge may arise when different manufacturers utilize various programming languages and operating systems for their own experimental devices, and such heterogeneity may hinder seamless communication between module nodes and devices. Examples of the heterogeneous environments among devices in the "BatchSynthesis" and "UV‒Vis" modules are provided in Supplementary Figure S12a. To achieve a homogeneous environment in OCTOPUS, we opted to connect the experimental devices via the TCP/IP technique (Fig. 4a). Since most programming languages, including C, C++, Python, JAVA and JavaScript, support TCP/IP regardless of the operating system (both Linux and Windows), we independently created device servers to facilitate communications between module nodes and devices via TCP/IP, ensuring platform homogeneity (Supplementary Figure S12b). Second, the TCP/IP network protocol not only ensures homogeneity but also offers significant advantages in terms of scalability. Scalability in MAP refers to the ability to connect experimental modules of multiple laboratories, even across different nations, as illustrated in Fig. 4b. When multiple experimental modules are joined in a TCP/IP-based network, it logically becomes part of the same network, overcoming spatial constraints in MAP and providing infinite possibilities for platform scalability. Notably, synthesized materials cannot be transferred across different regions due to practical issues, including time costs and material degradation. Thus, the absence of material synthesis modules in a specific laboratory could hinder flexible utilization for diverse applications, despite all experimental modules (modules for material synthesis, characterization and property evaluation) being connected within a single network. To address this, we proposed customizing material synthesis modules in each laboratory and integrating all the other local modules into a single platform despite remote distances via the TCP/IP-based network protocol (Fig. 4b). 20,21,36,37 The method for integrating network protocols is detailed in Supplementary Figure S13. Third, to minimize safety concerns arising from physical disconnections, we implemented the user datagram protocol (UDP) with TCP/IP to monitor physical connection issues for platform safety, as illustrated in Fig. 4c. If a miscommunication occurs between the master node, module nodes due to a physical disconnection between module nodes and experimental devices, the UDP-based broadcast executes an emergency stop for the other modules in the internet network to prevent safety issues in surveillance-free systems (Fig. 4c and Supplementary Figure S14a). In addition to the emergency stop, an alert system based on messengers was implemented to provide researchers with notification and monitoring systems, which aids in swiftly recovering the platform from safety issues (Fig. 4c and Supplementary Figure S14b-e). Fourth, a key challenge in MAP is that most platforms have thus far been developed only for a specific purpose; experimental devices are limited to a specific application. 14,38–40 However, with TCP/IP-based process modularization, a static platform can evolve into a versatile platform (Fig. 4d). Researchers can selectively adopt modules tailored to their own applications, allowing customized process design within a single platform and ensuring the versatility of the platform. Job parallelization to address the module overlap challenge . The compositions of the master node and module nodes and their functionalities are investigated with detailed examples. In the following sections, the user-optimal scheduler (US) developed within OCTOPUS is explained, incorporating three techniques, job parallelization, task optimization and CPS. When multiple clients attempt to utilize a MAP with many modules, module overlap issues may occur. Customized scheduling algorithms are necessary to address module overlap issues effectively. One conventional approach is job serialization based on the FCFS (First-Come-First-Serve) concept (Fig. 5a), where the priorities of module usage are assigned based on the job submission order. However, in job serialization, device standby times, where no significant actions are performed in terms of devices, may substantially impair job execution efficiency. An example of device standby times is the chemical reaction period ("React" task) in the "BatchSynthesis" module, where chemical reactions occur in chemical vessels but no actions are performed in terms of devices. To improve module resource utilization, we introduced job parallelization by leveraging device standby times (Fig. 5a and Supplementary Figure S15). During the device standby times of a preceding job, the following job can run in parallel without hindering previously started tasks. As illustrated in Fig. 5b, while job ID 0 progresses during the "React" task in the "BatchSynthesis" module, the "PrepareContainer" task in the next job (job ID 1) can be executed simultaneously by leveraging the bottleneck time (chemical reaction period) of the prior "React" task (Fig. 5b). This specific example of job parallelization is provided in Supplementary Video 2. To evaluate the impact of job parallelization on time efficiency, we conducted virtual experiments on catalysis processes, which typically involve long device standby times for chemical reactions. In this test, we define 10 virtual modules related to catalyst synthesis ("BatchSynthesis", "BallMilling"), preprocessing ("Washing", "Filtration", "Drying", "InkPreparation", "SprayCoating"), characterization ("XRD"), and property evaluation ("HalfCellTest", "FullCellTest"). Each module involves both a device execution time and device standby time, as described in Supplementary Figures S16 and S17. Seven different types of catalysis experiments utilizing parts of these modules were performed (Fig. 5c, Supplementary Figures. S16 and S17). Fig. 5d compares the results between job serialization and parallelization using bar charts. Three performance metrics were implemented, including "job waiting time", "job turnaround time" and "job total time" (Supplementary Figure S18). Job waiting time reflects the difference between the job submission and start time, providing client-centric performance insights. 41 "Job turnaround time" indicates the duration between the start of a job and completion of a job, and it offers administrator-centric performance insights. "Job total time", the sum of "job waiting time" and "job turnaround time", reflects the overall job execution efficiency. Job serialization entails the sequential execution of modules based on a priority order. Analysis of the performance metrics for time efficiency reveals that, in serialized jobs, the lower-priority jobs, determined by job submission order, may experience substantially increased waiting times due to module overlaps. In contrast, job parallelization aims to reduce waiting times by leveraging the device standby times within each module. Consequently, for all seven jobs in these virtual tests, a significant reduction in “job total time” is achieved via job parallelization compared to job serialization (Fig. 5d). For example, for job ID 3, "job total time" decreased significantly with job parallelization (from 10 hours to 6.5 hours) by leveraging the device standby time of modules utilized in the preceding jobs, such as the "BatchSynthesis" module of job ID 1. Similarly, in Job ID 4, "job total time" was nearly halved with job parallelization (10 hours to 5.5 hours), leveraging device standby times in the module, such as the centrifugation of the "Washing" module, oven usage of the "Drying" module and scanning of the "XRD" module, as shown in Supplementary Figure S16. As a result, job parallelization significantly improves the time efficiency, reducing both "job waiting time" and "job turnaround time", ultimately shortening "job total time" (Fig. 5d and Supplementary Table S1). Task optimization with masking table for preventing device overlaps. Up to this point, we operated under the assumption that each module functions independently, with the experimental devices operating without interference. However, as multiple clients submit job requests, the likelihood of multiple tasks within the same module running concurrently increases. In such scenarios, experimental device collisions may occur, raising potential safety concerns (Fig. 6a and Supplementary Figure S19). Moreover, the MAP cost implications are substantial, as each experimental device typically entails significant expenditures to ensure experimental reliability and precision. 15–17 In this regard, implementing device sharing between different modules has become imperative for cost savings within limited budgets, as demonstrated in recent advances. 30,42–44 For instance, a robotic arm may serve for both the "BatchSynthesis" and "UV‒Vis" modules, as illustrated in Fig. 6b and Supplementary Figure S19, enabling cost savings through shared utilization. 8 However, when multiple modules attempt to execute individual tasks simultaneously, conflicts may arise regarding the prioritization of tasks for the robotic arm. Consequently, task optimization is urgently needed to prevent device collisions within the same module, and device overlap challenges across different modules. To overcome these challenges, we introduce a task optimization technique with a masking table. In our approach, each module continuously updates the device status in a tabular format retrieved from the resource manager. Devices currently in use by ongoing tasks are assigned as "True", while unused devices are marked as "False", as shown in the device status table example in Fig. 6c. The resource manager dynamically updates the device status table in real time. Next, it is essential to predefine masking tables for each task, which identify the devices required for a specific task as "True" and those not needed as "False". For instance, when performing the "GetAbsorbance” task in the “UV‒Vis” module, devices such as UV‒Vis spectroscopy, robotic arms and pipettes are involved, and are thus marked as "True", while the other devices are marked as "False" to create task-specific masking tables (Fig. 6c). Prior to performing a specific task, the AND Boolean operation is performed between the device status table and the masking table for the task, resulting in a table for “hold criteria” to determine whether to proceed with the next task (Supplementary Figure S20). If all hold criteria indicate "False", the tasks are executed in parallel, whereas if any of the logical results are found "True", the system waits until the ongoing task is completed, and all the hold criteria indicate “False”. Examples of masking tables are provided in Supplementary Figure S21, and an actual example of a task optimization process with a masking table is provided in Supplementary Video S3. In summary, when experimental devices are shared between modules, task optimization with masking tables enables cost-saving benefits and enhances the efficiency of device utilization while avoiding device collisions and ensuring safety. The closed-packing schedule for optimizing module resources. In computing server systems with a finite number of cores, resource allocation considerations are paramount, and are often documented in job scripts. 32,33 Similarly, in the context of MAP, resource management extends to modules and experimental devices. Here, device resources denote the maximum number of experiments that each module can accommodate. For example, the “BatchSynthesis” module’s resource could be defined by the maximum number of vials simultaneously processable by a magnetic stirrer, while in the “UV‒Vis” module, it could also be determined by the maximum number of vial holders available for UV‒Vis spectrum measurements (Supplementary Figure S22). Thus, given the limited availability of resources in each module, scheduling methods that account for these constraints are essential in practical platforms. To efficiently execute multiple jobs within such constraints, we introduce the closed-packing schedule (CPS) algorithm (Fig. 7 and Supplementary Figure S23). The core concept of CPS lies in splitting tasks in a single job into multiple batches to maximally utilize the remaining resources via compact packing. An example of CPS is illustrated in Fig. 7, where three jobs (job IDs 1, 2 and 3) are sequentially submitted with different batch sizes (the number of required vials) of eight, four and eight, respectively, and a stirrer can accommodate up to 16 vials. With active job parallelization, at the execution of job ID 3, 12 resources out of a total of 16 are already occupied by preceding jobs, leaving only 4 resources unused. Then, the CPS divides the tasks of job ID 3 into two batches to allow the first batch to be executed first by fully occupying the 4 unused resources (compact packing), and to allow the other batch to be executed later as soon as the preceding jobs are completed and resources become available. As demonstrated in the example, the CPS aims to effectively minimize the waste of module resources by splitting the tasks within a job based on the computation of the remaining resources. Performance test of the user-optimal scheduler. In the previous sections, the user-optimal scheduler developed within OCTOPUS was explained, and involved three distinct scheduling methods, job parallelization, task optimization and CPS. To maximize scheduling efficiency, these three scheduling methods were integrated within OCTOPUS, resulting in a united scheduling system named the user-optimal scheduler (US) in this paper. To benchmark the US, we introduced the conventional FCFS to prioritize jobs based on their order of submission (Supplementary Figure S24). 45 FCFS executes jobs sequentially based on a priority order. We chose the FCFS algorithm as the baseline scheduler for comparison due to its fairness in executing jobs based on its priority order, which is crucial in multiclient systems. Figs. 8a and 8b visualize the execution time efficiency in processing multiple jobs across the two different scheduling schemes of FCFS and US. We employed a platform capable of simulating device collision and sharing issues to reflect a realistic environment. 8 In these comparative tests, 11 jobs with different batch sizes are submitted sequentially, all of which require the use of either the "BatchSynthesis" or "UV‒Vis" module (Supplementary Figure S25). It is important to clearly understand that, in the tests, the techniques of job parallelization, task optimization and CPS-based resource allocation, developed before, are active only for the US, whereas they are not active in the FCFS scheduling scheme. Through the observation of the scheduling results, we demonstrate the efficacy of US, particularly in terms of "job waiting time". The benefits of combined job parallelization and CPS are pronounced in many cases. For example, job IDs 1, 8 and 10 significantly reduced "job waiting time" (8.92 hours to 0.63 hours for job ID 1, 10.21 hours to 0.72 hours for job ID 8, and 10.97 hours to 0 hours for job ID 10). The time reductions are attributed to both leveraging device standby times and splitting jobs via CPS. Notably, these jobs are parallelized during the "React" task (device standby time) in the "BatchSynthesis" module of the preceding jobs (Fig. 8b, Supplementary Figure S26 and Table S2). Similarly, job ID 4 also benefits from both job parallelization and CPS, leading to a reduction in "job waiting time", as shown in Fig. 8c (11.72 hours to 0 hours for job ID 4). This job is parallelized with the preceding job ID 3 as the CPS splits the job according to the remaining resources of the "UV‒Vis" modules (Fig. 8b and Supplementary Figure S27). Meanwhile, the benefit of the task optimization technique is well observed in many cases, such as job IDs 2, 3, 4, 6, 7 and 9. Since the “BatchSynthesis” module and “UV‒Vis” module share a robotic arm, the simultaneous execution of these two modules may cause safety hazards from the device sharing environments. However, with active task optimization, concurrent executions of both modules were safely performed, as demonstrated in Supplementary Figure S28, Table S2 and Video S3. By minimizing "job waiting time", the US also proved more efficient in terms of "job total time" compared to the conventional FCFS scheme. However, we observed an overall and slight increase in "job turnaround time" for US, potentially resulting from a duplication of device standby time (Supplementary Figure S29). For Job IDs 1, 8 and 10, our scheduling system duplicates the device standby time associated with the "React" task, as the jobs are split based on the remaining resources. This redundancy contributes to the accumulation of "job turnaround time". Overall, while US significantly reduces "job waiting time", some delays in "job turnaround time" may occur due to device standby time duplications. Overall, by analyzing the performance metrics for time efficiency, US emerged as highly effective, saving time across all jobs in terms of "job waiting time" (Fig. 8c). Although there is a slight increase in "job turnaround time" due to duplications of device standby time by CPS (Fig. 8d and Supplementary Figure S27), the significant improvement in efficiency in terms of "job waiting time" renders US more efficient, leading to a substantial reduction in “job total time", compared to conventional FCFS (Fig. 8e and Supplementary Table S2). Discussion CLI presents a notable departure from conventional experimentation interfaces, potentially posing challenges for traditional experimental experts. While familiar to researchers in computer science fields, its adoption may present difficulties for those accustomed to hands-on experimentation. Recent studies advocate for the development of a user-friendly web-based interface accessible to both experimental researchers and computational experts. 14,19,22,24 Moreover, the integration of extended reality, virtual reality and augmented reality could enhance the accessibility and usability of OCTOPUS in the future. 46–48 In conclusion, OCTOPUS embodies a multifaceted solution that has been engineered to overcome the challenges inherent in MAP accessed by multiple users, OCTOPUS embodies a multifaceted solution. First, its tripartite structure, comprising the interface node, master node and module nodes, orchestrates seamless client request handling and experimental task scheduling. Through the integration of process modularization and network protocol utilization, OCTOPUS establishes a foundation characterized by homogeneity, scalability, safety and versatility within a central management platform. Furthermore, OCTOPUS presents US. The incorporation of job parallelization techniques serves to alleviate delays, while task optimization algorithms prevent safety hazards potentially arising from device collisions and sharing. In addition, the development of the CPS algorithm within OCTOPUS represents a significant stride in efficiently executing multiple jobs with minimal resource wastage. OCTOPUS will facilitate the management of diverse experiments from multiple users and thereby accelerated the widespread adoption of MAP for expedited material development. Methods Virtual experiments for job parallelization leveraging device standby times . In the virtual experiments related to Fig. 5, we defined the duration of both the device execution time and device standby time of each module as follows. For the "BatchSynthesis", "Filtration", "BallMilling", "InkPreparation", "XRD" and "SprayCoating" modules, a duration of 0.5 hours was assigned to both device execution time and device standby time, resulting in 1 hour of total time for each module (Supplementary Figure S16). For the "Washing" module, which is known for its repetitive removal of impurities, each 0.5-hour duration was assigned to one of the device execution times or device standby times, resulting in 2 hours of total time (Supplementary Figure S16). For the "Drying", "HalfCellTest", and "FullCellTest" modules, which are known for their extended durations for bottleneck processes, durations of 0.5 hours and 1.5 hours, respectively, were assigned to each device execution time and device standby time, resulting in 2 hours of total time (Supplementary Figure S16). The time allocations in each module are provided in detail in Supplementary Figure S16. The experimental devices within each module were assumed to function without mutual interference. The combinations and sequences of modules assigned to each job were carefully chosen based on actual experimental processes, as illustrated in Supplementary Figure S17. For example, for job ID 0 in Fig. 5, an experiment of synthesizing and measuring a Cu catalyst for the CO 2 reduction reaction involves the following modules in order: "BatchSynthesis", "Washing", "Filtration", "Drying", "InkPreparation" and "HalfCellTest". The modules used in other jobs are also described in Supplementary Figure S17. Experiments for task optimization with masking table. Prior to conducting experiments related to Fig. 6, we predefine the masking tables for each task. For examples in the “BatchSynthesis” and “UV‒Vis” modules, as illustrated in Supplementary Figure S19, the masking tables for a task represent the usage of experimental devices during the task, including the robotic arm (shared between two modules), vial storage, linear actuator with solution dispenser, syringe pump, pipette, UV‒Vis spectroscopy. For example, the “AddSolution” task in the “BatchSynthesis” module involves the activation of a linear actuator and pump; thus, these two devices are marked as “True” and all the other devices are marked as “False” in the masking table. The masking tables are structured with Boolean values (True or False). Examples of the masking table are provided in Supplementary Figure S21. In this work, the module and device configuration utilized in this task optimization performance test adhered to our prior publication. 8 Experiments for benchmarking US. The module and device configuration employed in the performance benchmarking test in Fig. 8 are consistent with recent advancements documented in our previous work. 8 Scheduling schemes of FCFS and US are compared in realistic experimental environments. All the resource information was stored and periodically updated by a resource manager at each module node by using a location index based on the listed data types (Supplementary Figure S8). Job parallelization, task optimization and CPS-based resource allocation are active only for the US, whereas they are not active in the FCFS scheduling schemes. In these benchmark tests, the 11 job scripts are submitted based on job submission timelines, as illustrated in Supplementary Figure S25. These 11 job scripts contain information on the experiment type (model names of manual vs. AI optimizations), module selection ("BatchSynthesis" or "UV‒Vis"), batch size, number of closed-loop cycles and task configuration (number of task executions and device standby time). Declarations Data Availability Statement Several examples of our result data, the codes, and related explanations are provided in the following GitHub repository (https://github.com/KIST-CSRC/Octopus). All codes are written in Python 3.7 and all environments could be created via requirements.txt file. Acknowledgements This work was supported by the National Research Foundation of Korea funded by the Ministry of Science and ICT [NRF-2022M3H4A7046278]. Nayeon Kim conceived the concept of image and drawed Supplementary Figure S1. Author Contributions S.S.H, D.K., and K.Y.L. conceived the idea and supervised the project. H.J.Y. conceived the idea, designed OCTOPUS architecture, developed user-optimal scheduler, and performed experiments. All authors contributed to result analysis and manuscript writing. Competing Interests The authors declare no competing financial or non-financial interests. References Higgins, K., Valleti, S. M., Ziatdinov, M., Kalinin, S. V. & Ahmadi, M. Chemical robotics enabled exploration of stability in multicomponent lead halide perovskites via machine learning. ACS Energy Lett. 5 , 3426–3436 (2020). Epps, R. W. et al. Artificial chemist: an autonomous quantum dot synthesis bot. Adv. Mater. 32 , 2001626 (2020). Mekki-Berrada, F. et al. Two-step machine learning enables optimized nanoparticle synthesis. npj Comput. Mater. 7 , 55 (2021). Angelone, D. et al. Convergence of multiple synthetic paradigms in a universally programmable chemical synthesis machine. Nat. Chem. 13 , 63–69 (2021). Häse, F., Roch, L. M., Kreisbeck, C. & Aspuru-Guzik, A. Phoenics: A Bayesian optimizer for chemistry. ACS Cent. Sci. 4 , 1134–1145 (2018). Aldeghi, M., Häse, F., Hickman, R. J., Tamblyn, I. & Aspuru-Guzik, A. Golem: An algorithm for robust experiment and process optimization. Chem. Sci. 12 , 14792–14807 (2021). Häse, F., Aldeghi, M., Hickman, R. J., Roch, L. M. & Aspuru-Guzik, A. Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge. Appl. Phys. Rev. 8 , 031406 (2021). Yoo, H. J. et al. Bespoke Metal Nanoparticle Synthesis at Room Temperature and Discovery of Chemical Knowledge on Nanoparticle Growth via Autonomous Experimentations. Adv. Funct. Mater. (2024) doi:10.1002/adfm.202312561. Yoshikawa, N., Darvish, K., Vakili, M. G., Garg, A. & Aspuru-Guzik, A. Digital pipette: open hardware for liquid transfer in self-driving laboratories. Digit. Discov. 2 , 1745–1751 (2023). Yoshikawa, N. et al. Chemistry Lab Automation via Constrained Task and Motion Planning. arXiv (2022) https://doi.org/10.48550/arXiv.2212.09672. Jiang, Y. et al. Autonomous biomimetic solid dispensing using a dual-arm robotic manipulator. Digit. Discov. 2 , 1733–1744 (2023). Tiong, L. C. O. et al. Machine vision-based detections of transparent chemical vessels toward the safe automation of material synthesis. npj Comput. Mater. 10 , 42 (2024). Sim, M., Ghazi Vakili, M., Hao, H., Hickman, R. J. & Pablo-García, S. ChemOS 2.0: an orchestration architecture for chemical self-driving laboratories . ChemRxiv vol. 5 https://doi.org/10.26434/chemrxiv-2023-v2khf (2023). Steiner, S. et al. Organic synthesis in a modular robotic system driven by a chemical programming language. Science (80-. ). 363 , (2019). Abolhasani, M. & Kumacheva, E. The rise of self-driving labs in chemical and materials sciences. Nat. Synth. 2 , 483–492 (2023). Maffettone, P. M. et al. What is missing in autonomous discovery: open challenges for the community. Digit. Discov. 2 , 1644–1659 (2023). Seifrid, M. et al. Autonomous Chemical Experiments: Challenges and Perspectives on Establishing a Self-Driving Lab. Acc. Chem. Res. 55 , 2454–2466 (2022). Sayfan, G. Mastering kubernetes . (Packt Publishing Ltd, 2017). van der Westhuizen, C. J., du Toit, J., Neyt, N., Riley, D. & Panayides, J. L. Use of open-source software platform to develop dashboards for control and automation of flow chemistry equipment. Digit. Discov. 1 , 596–604 (2022). Rahmanian, F. et al. Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration. Adv. Mater. Interfaces 9 , 2101987 (2022). Strieth-Kalthoff, F. et al. Delocalized, Asynchronous, Closed-Loop Discovery of Organic Laser Emitters . ChemRxiv https://doi.org/10.26434/chemrxiv-2023-wqp0d (2023) doi:10.26434/chemrxiv-2023-wqp0d. Hielscher, M. M., Dörr, M., Schneider, J. & Waldvogel, S. R. LABS: Laboratory Automation and Batch Scheduling – A Modular Open Source Python Program for the Control of Automated Electrochemical Synthesis with a Web Interface. Chem. - An Asian J. 18 , (2023). Tamura, R., Tsuda, K. & Matsuda, S. NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science. arXiv Prepr. (2023) doi:https://doi.org/10.48550/arXiv.2304.13927. Guevarra, D. et al. Orchestrating nimble experiments across interconnected labs. Digit. Discov. 2 , 1806–1812 (2023). Kusne, A. G. & McDannald, A. Scalable multi-agent lab framework for lab optimization. Matter 6 , 1880–1893 (2023). Deneault, J. R. et al. Toward autonomous additive manufacturing: Bayesian optimization on a 3D printer. MRS Bull. 46 , 566–575 (2021). Campbell, S. I. et al. Outlook for artificial intelligence and machine learning at the NSLS-II. Mach. Learn. Sci. Technol. 2 , (2021). Leong, C. J. et al. An object-oriented framework to enable workflow evolution across materials acceleration platforms. Matter 5 , 3124–3134 (2022). Du, X. et al. Elucidating the Full Potential of OPV Materials Utilizing a High-Throughput Robot-Based Platform and Machine Learning. Joule 5 , 495–506 (2021). Coley, C. W. et al. A robotic platform for flow synthesis of organic compounds informed by AI planning. Science (80-. ). 365 , (2019). Jiang, Y. et al. An artificial intelligence enabled chemical synthesis robot for exploration and optimization of nanomaterials. Sci. Adv. 8 , 1–12 (2022). Yoo, A. B., Jette, M. A. & Grondona, M. Slurm: Simple linux utility for resource management. in Workshop on job scheduling strategies for parallel processing 44–60 (2003). Nabrzyski, J., Schopf, J. M. & Weglarz, J. Grid resource management: state of the art and future trends . (Springer Science & Business Media, 2012). Kathryn Vasel. The pandemic forced a massive remote-work experiment. Now comes the hard part. CNN (2021). Park, J. et al. Closed-loop optimization of nanoparticle synthesis enabled by robotics and machine learning. Matter 6 , 677–690 (2023). Canty, R. B. & Jensen, K. F. Sharing reproducible synthesis recipes. Nat. Synth. (2024) doi:10.1038/s44160-023-00478-1. Rauschen, R., Guy, M., Hein, J. E. & Cronin, L. Universal chemical programming language for robotic synthesis repeatability. Nat. Synth. (2024) doi:10.1038/s44160-023-00473-6. Granda, J. M., Donina, L., Dragone, V., Long, D. L. & Cronin, L. Controlling an organic synthesis robot with machine learning to search for new reactivity. Nature 559 , 377–381 (2018). Volk, A. A. et al. AlphaFlow: autonomous discovery and opti-mization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning. Nat. Commun. (2023) doi:https://doi.org/10.1038/s41467-023-37139-y. Soldatov, M. A. et al. Self-Driving Laboratories for Development of New Functional Materials and Optimizing Known Reactions. Nanomaterials 11 , 619 (2021). Rubab, S., Hassan, M. F., Mahmood, A. K. & Shah, S. N. M. Adoptability Study of Bin-Packing for Scheduling Jobs on Volunteer Grid Resources. in Procedia Computer Science vol. 69 2–12 (Elsevier B.V., 2015). MacLeod, B. P. et al. A self-driving laboratory advances the Pareto front for material properties. Nat. Commun. 13 , (2022). Burger, B. et al. A mobile robotic chemist. Nature 583 , 237–241 (2020). MacLeod, B. P. et al. Self-driving laboratory for accelerated discovery of thin-film materials. Sci. Adv. 6 , 1–8 (2020). Putera, A. & Siahaan, U. Comparison Analysis of CPU Scheduling : FCFS, SJF and Round Robin. Int. J. Eng. Dev. Res. 4 , (2016). Li, J., Tu, Y., Liu, R., Lu, Y. & Zhu, X. Toward “On‐Demand” Materials Synthesis and Scientific Discovery through Intelligent Robots. Adv. Sci. 7 , 1901957 (2020). Pells, R. Why scientists are delving into the virtual world. Nature (2023) doi:10.1038/d41586-023-02688-1. Wang, G. et al. Development of metaverse for intelligent healthcare. Nat. Mach. Intell. 4 , 922–929 (2022). Additional Declarations There is NO Competing Interest. Supplementary Files SIvideo1.mp4 Supplementary Video 1 SIvideo2.mp4 Supplementary Video 2 SIvideo3.mp4 Supplementary Video 3 SupplementaryInformation.docx Cite Share Download PDF Status: Published Journal Publication published 08 Nov, 2024 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4254570","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":291867606,"identity":"4fccd373-f53f-4f59-b41c-883bd2909096","order_by":0,"name":"Sang Soo Han","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIiWNgGAWjYBADOQlmKEuCWC3GQC2MDSRpSZzBQKwW+fazh1/+qLBJn9nO/vwB4x4bBsnZB/BrMTiTl2bNcyYtdzYzj2EDw7M0Bmm+BAJaGHLMjBnbDufOY+YBOuzAYQY5HkIO639jZviz7XC6HDP7Q6CW/4S1MNzIMX7A23Y4QZqZAeiwAwcYpAlpMbjxxowZ6BfDmc08hjMSDiTzSPYQdFiO8UdgiMlLnD/+4MOHA3ZyEmcIOYyBgQ0REwkMDAR9AgLMH4hRNQpGwSgYBSMYAAC1vjwpaK0nugAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-7925-8105","institution":"Korea Institute of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Sang","middleName":"Soo","lastName":"Han","suffix":""},{"id":291867607,"identity":"040e7838-2c4c-4239-b29e-b9d5a25242e2","order_by":1,"name":"Hyuk Jun Yoo","email":"","orcid":"","institution":"Korea Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Hyuk","middleName":"Jun","lastName":"Yoo","suffix":""},{"id":291867608,"identity":"135a5864-0f7e-4844-802b-98ffe4520b32","order_by":2,"name":"Kwan-Young Lee","email":"","orcid":"","institution":"Korea University","correspondingAuthor":false,"prefix":"","firstName":"Kwan-Young","middleName":"","lastName":"Lee","suffix":""},{"id":291867609,"identity":"436c27a3-e506-4ecd-9d9b-5ec46d99d4ef","order_by":3,"name":"Donghun Kim","email":"","orcid":"","institution":"Korea Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Donghun","middleName":"","lastName":"Kim","suffix":""}],"badges":[],"createdAt":"2024-04-11 23:45:54","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4254570/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4254570/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-024-54067-7","type":"published","date":"2024-11-08T05:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":55898745,"identity":"a1defc9f-6720-476c-85a6-5a36211b6747","added_by":"auto","created_at":"2024-05-06 04:51:58","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":445549,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSchematic design of OCTOPUS. \u003c/strong\u003eThe architecture of OCTOPUS comprises three main components, the interface node, master node and module nodes. Multiple clients submit job scripts via CLI based on the JSON format. The master node manages the submitted jobs, overseeing the job scheduler, task generator, task scheduler, action translator and action executor. The job scheduler prioritizes and schedules jobs, while the task generator creates tasks based on the received script. The task scheduler ensures efficient resource allocations, and the action translator converts task information into device commands. Module nodes operate experimental devices based on instructions from the action executor.\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/17dba8dc08a5d394ff172b68.png"},{"id":55898748,"identity":"08e3fea3-3cd4-47f2-a9bd-7636ff26e03b","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":380790,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eScheme of job submission via the interface node and job scheduler of the master node.\u003c/strong\u003e The interface node enables clients to submit job scripts, with concurrent handling supported by a multithread pool. The job scheduler in the master node consists of six components, a job ID generator, job modeling unit, job storage and three queues (waiting, executing and holding), along with a job trigger. Upon receiving a job script, the job ID generator assigns a unique ID and sends it to the waiting queue and job modeling unit. The job trigger monitors job executions and moves jobs from the waiting queue to the executing queue. Clients can manage the submitted jobs remotely via command lines in the interface node.\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/514ce23c4b317fb97cde08b6.png"},{"id":55898746,"identity":"55615a16-e898-4adc-9c00-d39acf0f67d1","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":389796,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eJob executions at the master node.\u003c/strong\u003e Job executions follow a closed-loop experimentation process involving several components, the job in the executing queue, the resource manager, the task generator, the task scheduler, the action translator, the action executor and the database. When a job is ready for execution, it is matched in the job storage, initiating the experiment. Experiment information is transferred to the task generator, which generates tasks based on experimental conditions and device information. The task scheduler manages task executions, and the action translator converts task information into device commands. After executing the commands, the results are stored in the database, and the job recommends the next experimental conditions in a closed-loop manner.\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/0173e84f48681e08d73f351a.png"},{"id":55898750,"identity":"4b8b1cd1-92d8-4f97-868c-27123bc0b80e","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":581650,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eNetwork protocol-aided process modularization. (a)\u003c/strong\u003e This figure illustrates the use of a device server based on TCP/IP to create a homogeneous experimental environment, even with multiple devices based on different OS or programming languages. \u003cstrong\u003e(b)\u003c/strong\u003e This figure illustrates an example of experimental processes from multiple laboratories across different nations. Modules for material synthesis, characterization and property evaluation can be joined into a single platform by leveraging internal (red line) or external (blue line) TCP/IP-based networks. \u003cstrong\u003e(c)\u003c/strong\u003e The flow chart depicts algorithms for the heartbeat and shutdown function with a UDP-based broadcast, ensuring the stability between the master node, module node and experimental devices. \u003cstrong\u003e(d)\u003c/strong\u003e This figure illustrates an example of a versatile platform, where different sets of modules can be used for diverse applications.\u003c/p\u003e","description":"","filename":"image4.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/1e5b72ea1efde64140c3616f.png"},{"id":55898753,"identity":"0419ebc7-243a-444c-a94d-f3edbf49e266","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":980886,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eJob parallelization to address the module overlap challenge. (a)\u003c/strong\u003e Scheme of job serialization and parallelization, showcasing device execution time (black) and device standby time (shaded black). \u003cstrong\u003e(b)\u003c/strong\u003e Images representing an example of job parallelization. The enlarged image highlights parallelized batch synthesis. \u003cstrong\u003e(c)\u003c/strong\u003eVisualization of module execution results comparing job serialization and parallelization. Shaded red boxes represent the delay time between module executions. \u003cstrong\u003e(d)\u003c/strong\u003e Performance metrics of job serialization and parallelization, including the job waiting time, job turnaround time and job total time.\u003c/p\u003e","description":"","filename":"image5.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/135874b1be38057dffc2c7c4.png"},{"id":55898749,"identity":"c5a0aebd-7b5f-4762-8a6d-28ec61167d58","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":1590182,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTask optimization using a masking table.\u003c/strong\u003e \u003cstrong\u003e(a)\u003c/strong\u003e Example of device overlap challenge - device collision in the same module. \u003cstrong\u003e(b)\u003c/strong\u003eExample of device overlap challenge - device sharing between different modules.\u003cstrong\u003e (c)\u003c/strong\u003e Schematic algorithm of task optimization with a masking table, employing the AND Boolean operation between the device status table and masking table for a specific task.\u003c/p\u003e","description":"","filename":"image6.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/9e2d563a25689d15f959a0dc.png"},{"id":55898752,"identity":"7194f39f-357f-4103-9d37-df4ef6a21b88","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":359412,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eScheme of the closed-packing schedule. \u003c/strong\u003eThe task scheduler with the CPS method aims to utilize the remaining resources of each module efficiently. Job 3, with a batch size of eight, is split into four experiments based on the remaining four stirrer resources in the “BatchSynthesis” module.\u003c/p\u003e","description":"","filename":"image7.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/bc77d3b1980fc042adea15d3.png"},{"id":55898751,"identity":"cf7750b1-2311-4953-9d9e-5a9d535548f4","added_by":"auto","created_at":"2024-05-06 04:51:59","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":387131,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance comparison of scheduling algorithms for multiple jobs. (a-b) \u003c/strong\u003eJob\u003cstrong\u003e \u003c/strong\u003eexecution timelines for 11 jobs for FCFS and US. \u003cstrong\u003e(c-e) \u003c/strong\u003ePerformance metrics of “job waiting time”, “job turnaround time” and “job total time” for FCFS and US.\u003c/p\u003e","description":"","filename":"image8.png","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/3aa0e82da4d27c1bcac272b6.png"},{"id":68606121,"identity":"2cecf202-41aa-4288-acf3-c796abb90e5e","added_by":"auto","created_at":"2024-11-09 08:05:37","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5948719,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/04df2547-3072-4868-8e20-7a017e7c4484.pdf"},{"id":55899042,"identity":"a388bc8e-057f-464d-bfae-8eaf98aebb90","added_by":"auto","created_at":"2024-05-06 04:59:59","extension":"mp4","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":44005964,"visible":true,"origin":"","legend":"Supplementary Video 1","description":"","filename":"SIvideo1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/49d36e24219528e96b7f5af3.mp4"},{"id":55898757,"identity":"c14b0cbd-642b-436d-87e4-efa9b126982b","added_by":"auto","created_at":"2024-05-06 04:52:01","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":137966140,"visible":true,"origin":"","legend":"Supplementary Video 2","description":"","filename":"SIvideo2.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/b4160bb872707a0531dd954e.mp4"},{"id":55898754,"identity":"ec544693-021c-439e-ac08-dd1926ffd11c","added_by":"auto","created_at":"2024-05-06 04:52:00","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":81384810,"visible":true,"origin":"","legend":"Supplementary Video 3","description":"","filename":"SIvideo3.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/9afd1ec20166148e686b0075.mp4"},{"id":55898762,"identity":"f598b54c-af8d-4b83-8b46-659abd9a3bab","added_by":"auto","created_at":"2024-05-06 04:52:05","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":133419953,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformation.docx","url":"https://assets-eu.researchsquare.com/files/rs-4254570/v1/375a67cc23b64ea15709d277.docx"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"OCTOPUS: Operation Control System for Task Optimization and Job Parallelization via a User-Optimal Scheduler","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe material acceleration platform (MAP) has revolutionized\u0026nbsp;material discovery through extensive exploration of the chemical space in diverse domains, including\u0026nbsp;organic molecules, perovskites, colloidal quantum dots and nanoparticles.\u003csup\u003e1\u0026ndash;4\u003c/sup\u003e Through the integration of robotic\u0026nbsp;automation\u0026nbsp;and AI-based\u0026nbsp;experimental\u0026nbsp;planning, MAP has\u0026nbsp;shown\u0026nbsp;promising potential in enhancing\u0026nbsp;the\u0026nbsp;search efficiency for the target materials\u003csup\u003e5\u0026ndash;8\u003c/sup\u003e while ensuring reliability in surveillance-free environments.\u003csup\u003e9\u0026ndash;12\u003c/sup\u003e However, the scalability and accessibility of MAP with multiple users have been impeded by limitations in experimental device sharing and resource allocation, necessitating advanced research\u0026nbsp;on\u0026nbsp;operating system (OS).\u003c/p\u003e\n\u003cp\u003eThe advent of OS for MAP, exemplified by Chemputer and ChemOS,\u0026nbsp;has provided a promising pathway toward integrated management systems.\u003csup\u003e13,14\u003c/sup\u003e Yet, their accessibility remains restricted,\u0026nbsp;and is\u0026nbsp;primarily constrained by single-researcher execution and limited applicability. Additionally, the inefficiency of device resource allocations due to the lack of experimental device sharing across different applications poses scalability challenges.\u003csup\u003e15\u0026ndash;17\u003c/sup\u003e Addressing these constraints requires a paradigm shift toward\u0026nbsp;a multiuser\u0026nbsp;system, which necessitates the development of a central management platform with a master/node relationship capable of managing experimental equipment efficiently.\u003csup\u003e18\u003c/sup\u003e Central to this endeavor, is the modularization of independent experimental processes to facilitate widespread equipment sharing across diverse applications. However, the development of an OS for\u0026nbsp;multiuser\u0026nbsp;systems\u0026nbsp;has\u0026nbsp;posed challenges to standardized methodologies, primarily stemming from equipment heterogeneity, platform delocalization and safety concerns.\u003csup\u003e19\u0026ndash;27\u003c/sup\u003e Thus, there is a pressing need for extensive research to develop an advanced OS for MAP accessed by multiple users.\u003c/p\u003e\n\u003cp\u003eWhen implementing central management systems for processing diverse experiments from multiple users, challenges such as module and device overlap are persistent, leading to resource allocation inefficiencies and safety hazards.\u003csup\u003e16,17\u003c/sup\u003e Module overlap constraints, for instance, result in time wastage and operational inefficiencies, while device overlap issues exacerbate safety concerns from collisions of\u0026nbsp;the\u0026nbsp;activity radius. Addressing these challenges requires a comprehensive scheduling approach that considers resource allocations, safety protocols and operational efficiencies within realistic environments.\u003c/p\u003e\n\u003cp\u003eIn response to these challenges, we introduce OCTOPUS, an acronym for\u0026nbsp;an\u0026nbsp;\u003cstrong\u003eo\u003c/strong\u003eperation \u003cstrong\u003ec\u003c/strong\u003eontrol system for \u003cstrong\u003et\u003c/strong\u003eask \u003cstrong\u003eo\u003c/strong\u003eptimization and job \u003cstrong\u003ep\u003c/strong\u003earallelization via\u0026nbsp;a\u0026nbsp;\u003cstrong\u003eu\u003c/strong\u003eser-optimal \u003cstrong\u003es\u003c/strong\u003echeduler, which is designed to streamline experimental task scheduling, optimize resource utilization and enhance safety protocols. OCTOPUS comprises three core components\u0026mdash; an interface node, master node and module nodes\u0026mdash; that facilitate client request handling and experimental task scheduling. Leveraging process modularization and a network protocol, OCTOPUS ensures homogeneity, scalability, safety and versatility within a central management platform. OCTOPUS embodies use-optimal schedulers; it integrates job parallelization techniques to mitigate delays by leveraging device standby times, while the task optimization algorithms prevent safety hazards stemming from device collisions or sharing within realistic operational environments. Additionally, the closed-packing schedule (CPS) algorithm enables module resources to be maximally utilized via compact packing. The user-optimal scheduler (US) in OCTOPUS addresses several challenges encountered within MAP accessed by multiple users, and will catalyze its widespread adoption for expedited discovery of novel materials.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eTerminology definition.\u003c/strong\u003e The intricate nature of\u0026nbsp;the\u0026nbsp;individual components within\u0026nbsp;the\u0026nbsp;MAP necessitates a standardized terminology to eliminate potential confusion. In response, recent research has advocated for a comprehensive summary of terms and\u0026nbsp;definitions\u0026nbsp;pertinent to MAP.\u003csup\u003e28\u003c/sup\u003e Expanding upon this effort, we present a detailed elucidation of the structural framework underlying MAP, \u0026nbsp;which encompasses four primary components, platforms, modules, tasks and actions (Supplementary Figure S1).\u003c/p\u003e\n\u003cp\u003eThe platform\u0026nbsp;serves as the foundational framework that interconnects experimental resources, facilitating automated experiments to cater to the diverse needs of multiple users. The concept of\u0026nbsp;platforms\u0026nbsp;has been increasingly emphasized in recent\u0026nbsp;advancements.\u003csup\u003e29\u0026ndash;31\u003c/sup\u003e Within a platform, modules represent the fundamental building blocks, each dedicated to executing a specific experimental process. Examples of modules include \u0026quot;BatchSynthesis\u0026quot;, \u0026quot;FlowSynthesis\u0026quot;, \u0026quot;SprayCoating\u0026quot;, \u0026quot;Filtration\u0026quot;\u0026nbsp;and\u0026nbsp;\u0026quot;UV‒Vis\u0026quot;. Each module encapsulates the requisite functionalities necessary for its designated experimental process.\u003c/p\u003e\n\u003cp\u003eA\u0026nbsp;task\u0026nbsp;delineates the granular steps comprising a specific experimental process within a module. For instance, tasks within the \u0026quot;BatchSynthesis\u0026quot; module include \u0026quot;PrepareContainer\u0026quot;, \u0026quot;AddSolution\u0026quot;, \u0026quot;Stir\u0026quot;, \u0026quot;Heat\u0026quot;, \u0026quot;Mix\u0026quot; and \u0026quot;React\u0026quot;, among others. Actions, on the other hand, denote the discrete operations of\u0026nbsp;the\u0026nbsp;experimental devices required for task\u0026nbsp;execution. Experimental devices such as robotic arms, pipettes and stirrers execute various actions to accomplish a task within a module. For example, the \u0026quot;PrepareContainer\u0026quot; task involves sequential actions by a robotic arm of (1) opening vial storage, (2) picking vials and (3) placing vials on a stirrer (Supplementary Figure S1).\u003c/p\u003e\n\u003cp\u003eThrough this structured framework, a MAP harmoniously orchestrates all the components, empowering users to execute desired experiments with high precision and efficiency. This hierarchical terminology definition not only enhances clarity and comprehension within the field but also facilitates seamless communication and collaboration among researchers and practitioners working within the automated experimentation realm.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eArchitecture of\u0026nbsp;OCTOPUS.\u003c/strong\u003e The architectural blueprint of OCTOPUS is depicted in Fig. 1, which highlights and draws inspiration from the adaptive nature of the octopus. OCTOPUS, which comprises three integral components, the interface node, master node and module nodes, orchestrates the seamless execution of the experimental modules within the MAP.\u003c/p\u003e\n\u003cp\u003eThe interface node serves as the nexus, bridging multiple users (or clients) in remote environments to the MAP infrastructure. Next, the master node plays a central role in managing a myriad of tasks and actions, employing a sophisticated suite of six components, a job scheduler, a task generator, a task scheduler, an action translator, an action executor and a resource manager. Finally, the module nodes of OCTOPUS are meticulously modularized to accommodate individual experimental processes, as illustrated in Fig. 1. Upon receiving experimental action directives from the master node, the module nodes initiate modularized robotic operations, encompassing a spectrum of actions such as robotic arm maneuvers, solution dispensation and stirrer activation, among others. The hierarchical structure of OCTOPUS (interface node \u0026rarr; master node \u0026rarr; module nodes) and the harmonious integration of these components elucidate the intricate workflow of the platform.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eJob submission via the\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003einterface node and job scheduler of the master node\u003c/strong\u003e\u003cstrong\u003e.\u003c/strong\u003e The interface node within OCTOPUS functions as a command line interface (CLI), enabling simultaneous job submissions from multiple clients, as illustrated in Fig. 2. Notably, recent advancements in the user interface design for MAP have primarily been tailored to receiving a single job within an OS.\u003csup\u003e13,14,19\u0026ndash;24\u003c/sup\u003e However, the ability to accommodate multiple job submissions concurrently is paramount in enhancing the efficiency of a MAP.\u003csup\u003e25\u003c/sup\u003e Thus, the interface node, bolstered by a multithread pool, enables numerous clients to seamlessly request remote experiments across multiple applications, akin to a computing server (Fig. 2). Notably, OCTOPUS operates real chemical experiments rather than computational tasks, distinguishing it from traditional computing server systems\u003csup\u003e32,33\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eClients are required to generate job scripts based on\u0026nbsp;the\u0026nbsp;JavaScript object notation (JSON) format, and examples of job scripts are provided in\u0026nbsp;Supplementary\u0026nbsp;Figure S2. These job scripts cater to a spectrum of client demands, ranging from manual experiments with predefined input conditions (e.g. a model with the name of\u0026nbsp;\u0026quot;Manual\u0026quot;\u0026nbsp;in\u0026nbsp;Supplementary\u0026nbsp;Figure S2a) to autonomous experiments enabled by AI-decision processes (e.g. model with the names of\u0026nbsp;\u0026quot;BayesianOptimization\u0026quot;,\u0026nbsp;\u0026quot;DecisionTree\u0026quot;\u0026nbsp;and\u0026nbsp;\u0026quot;BayesianNeuralNetwork\u0026quot;\u0026nbsp;in Figure S2b). The\u0026nbsp;model names\u0026nbsp;could be changed by the clients depending on the type of AI model built for\u0026nbsp;experimental\u0026nbsp;planning on the platform.\u0026nbsp;Initially, the interface node takes responsibility for managing the client login process, enforcing stringent security policies through hash functions while awaiting client commands (Supplementary\u0026nbsp;Figure S3).\u0026nbsp;Subsequently, clients submit job scripts via CLI, leveraging\u0026nbsp;portable batch system\u0026nbsp;commands, with detailed examples provided in\u0026nbsp;Supplementary\u0026nbsp;Figure S4.\u003csup\u003e32,33\u003c/sup\u003e This transformation of the local platform into a client/server-based ubiquitous platform\u0026nbsp;has\u0026nbsp;profound implications for research continuity, particularly in scenarios constrained by temporal and spatial limitations, such as those imposed by COVID-19.\u003csup\u003e34,35\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eThe master node of OCTOPUS embodies a central management system capable of orchestrating diverse experiments for multiple clients.\u003csup\u003e18\u003c/sup\u003e Within the master node, the job scheduler, which comprises a job ID generator, job modeling unit, job storage and three distinct queues (waiting, executing and holding), plays a pivotal role in scheduling jobs for the execution of real experiments, as illustrated in Fig. 2. Upon job submission, the job ID generator assigns a unique identifier (ID) to each job, facilitating orderly processing based on submission order. This job ID is subsequently transferred to the job modeling unit, which generates specific job configurations based on the provided job script, comprising process recommendations (manual vs. autonomous), module and task sequences and experimental conditions. Subsequently, the job storage organizes the generated jobs, awaiting the triggering decision.\u003c/p\u003e\n\u003cp\u003eThe triggering decision, determined by various factors,\u0026nbsp;including module resource availability and experimental device status, directs the job execution sequence. If the job trigger decides to execute the next job, it transitions the job ID from the waiting queue to the executing queue for real experiment executions\u0026nbsp;(Supplementary\u0026nbsp;Figure S5). Moreover, a\u0026nbsp;holding queue addresses the safety concerns inherent in surveillance-free\u0026nbsp;MAP environments. In the event of severe safety hazards involving, for example, inflammable or hazardous chemicals,\u003csup\u003e12\u003c/sup\u003e the platform automatically places the job on hold, ensuring user safety. Clients retain the autonomy to restart or terminate holding jobs via predefined commands. Several examples of job management (submission, status check, hold, restart and deletion) via CLI are provided in Supplementary Figure S6. We also provide Supplementary Video 1, where the process of logging in, and performing remote and simultaneous job submissions by two users in the interface node of OCTOPUS is recorded for clarity.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eJob executions in the master node.\u003c/strong\u003e Following the job submissions in the interface node, we present the job execution workflow in the master node, incorporating the seven components of job scheduler, task generator, task scheduler, action translator, action executor, resource manager and database, which are specifically tailored for closed-loop experiments, as illustrated in Fig. 3.\u003c/p\u003e\n\u003cp\u003eWe empower the task generator to retrieve experimental device information, including physical specifications and\u0026nbsp;the\u0026nbsp;setup environment,\u0026nbsp;from the resource manager. For instance, when preparing to execute the \u0026ldquo;AddSolution\u0026rdquo; task, the task generator accesses information detailing the device specifications and setup environments of stock solution types and concentrations, among others.\u0026nbsp;Notably, the dynamic updating of experimental device information from the resource manager enhances operational flexibility (Supplementary Figures S7a and S7b). The task generator integrates the job script with actual experimental device information to complete the predefined task template (in JSON format) containing\u0026nbsp;the\u0026nbsp;experimental conditions and task sequences (Fig. 3 and Supplementary Figures S7c-e).\u0026nbsp;Subsequently, the task scheduler supervises resource allocations for task\u0026nbsp;executions by obtaining\u0026nbsp;information\u0026nbsp;on\u0026nbsp;the location\u0026nbsp;indices\u0026nbsp;from the resource manager (Fig. 3 and Supplementary Figure S8).\u003c/p\u003e\n\u003cp\u003eCrucially, the abstraction of tasks, as exemplified in\u0026nbsp;Chemputer\u003csup\u003e14\u003c/sup\u003e and HELAO-async\u003csup\u003e24\u003c/sup\u003e, ensures adaptability across different platforms by omitting specific device operation and location details.\u003csup\u003e14,24\u003c/sup\u003e To concretize abstracted tasks, we implement an action translator, which translates abstracted tasks into concrete actions, incorporating predefined action sequences and device location information. For instance, the \u0026quot;AddSolution\u0026quot; task involves the serial actions of initializing a pump, moving a dispenser and injecting a solution, each executed by different devices (Fig. 3 and Supplementary Figure S9). Next, the action executor transmits device commands for real experiment execution, employing a transmission protocol that follows predefined data types to module nodes. These data types, including the job ID, device name, action type, action data and mode type, are encoded and transmitted to module nodes via the Transmission Control Protocol/Internet Protocol (TCP/IP) (Fig. 3 and Supplementary Fig. S10). The modules execute experimental action and return the resulting data to the database for subsequent experimental planning. More detailed information on the resulting data structure is provided in Supplementary Figure S11.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNetwork\u0026nbsp;protocol\u003c/strong\u003e\u003cstrong\u003e-based modularization.\u003c/strong\u003e To manipulate multiple jobs within OCTOPUS, process modularization is key for efficiently operating experimental processes. We conceptualized four key concepts to explain the benefits of using experimental process modularization and network protocols, which include homogeneity, scalability, safety and versatility, as illustrated in Fig. 4.\u003c/p\u003e\n\u003cp\u003eFirst, a challenge may arise when different manufacturers\u0026nbsp;utilize\u0026nbsp;various programming languages and operating systems for their own experimental devices, and such heterogeneity may hinder seamless\u0026nbsp;communication\u0026nbsp;between module\u0026nbsp;nodes\u0026nbsp;and devices.\u0026nbsp;Examples\u0026nbsp;of the heterogeneous environments among devices in\u0026nbsp;the\u0026nbsp;\u0026quot;BatchSynthesis\u0026quot;\u0026nbsp;and\u0026nbsp;\u0026quot;UV‒Vis\u0026quot;\u0026nbsp;modules are provided in\u0026nbsp;Supplementary\u0026nbsp;Figure S12a. To achieve a homogeneous environment in OCTOPUS, we opted to connect the experimental devices via the TCP/IP technique (Fig. 4a). Since most programming languages,\u0026nbsp;including C, C++, Python, JAVA and JavaScript,\u0026nbsp;support TCP/IP regardless of the operating system (both Linux and Windows), we independently created device servers to facilitate communications between module\u0026nbsp;nodes\u0026nbsp;and devices via TCP/IP, ensuring platform homogeneity (Supplementary\u0026nbsp;Figure S12b).\u003c/p\u003e\n\u003cp\u003eSecond,\u0026nbsp;the TCP/IP\u0026nbsp;network protocol not only ensures homogeneity but also offers significant advantages in terms of scalability. Scalability in MAP refers to the ability to connect experimental modules of multiple laboratories, even across different nations, as illustrated in Fig. 4b. When\u0026nbsp;multiple\u0026nbsp;experimental modules are joined in a TCP/IP-based network, it logically becomes part of the same network, overcoming spatial constraints in MAP and providing infinite possibilities for platform scalability. Notably, synthesized materials cannot be transferred across different regions due to practical issues,\u0026nbsp;including time costs and material\u0026nbsp;degradation. Thus, the absence of material synthesis modules in a specific laboratory could hinder flexible utilization for diverse applications, despite all experimental modules (modules for material synthesis, characterization and property evaluation) being connected within a single network. To address this, we proposed customizing material synthesis modules in each laboratory and integrating all the other local modules into a single platform despite remote distances via the TCP/IP-based network protocol (Fig. 4b).\u003csup\u003e20,21,36,37\u003c/sup\u003e The method for integrating network protocols is detailed in\u0026nbsp;Supplementary\u0026nbsp;Figure S13.\u003c/p\u003e\n\u003cp\u003eThird, to minimize safety concerns arising from physical disconnections, we implemented\u0026nbsp;the user datagram protocol (UDP) with TCP/IP to monitor physical connection issues for platform safety, as illustrated in Fig. 4c. If a miscommunication occurs between\u0026nbsp;the\u0026nbsp;master node, module nodes due to a physical disconnection between module nodes and experimental devices, the UDP-based broadcast executes an emergency stop for the other modules in the internet network to prevent safety issues in surveillance-free\u0026nbsp;systems\u0026nbsp;(Fig. 4c and\u0026nbsp;Supplementary Figure\u0026nbsp;S14a). In addition to the emergency stop, an alert system based on\u0026nbsp;messengers\u0026nbsp;was implemented to provide researchers with notification and monitoring systems, which aids in swiftly recovering the platform from safety issues (Fig. 4c and\u0026nbsp;Supplementary Figure\u0026nbsp;S14b-e).\u003c/p\u003e\n\u003cp\u003eFourth,\u0026nbsp;a\u0026nbsp;key challenge in MAP is that most platforms have thus far been developed only for a specific purpose;\u0026nbsp;experimental\u0026nbsp;devices are\u0026nbsp;limited\u0026nbsp;to a specific application.\u003csup\u003e14,38\u0026ndash;40\u003c/sup\u003e However, with TCP/IP-based process modularization, a static platform can evolve into a versatile platform (Fig. 4d). Researchers can selectively adopt modules tailored to their own applications, allowing customized process design within a single platform and ensuring the versatility of the platform.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eJob parallelization to address the module overlap challenge\u003c/strong\u003e\u003cstrong\u003e.\u003c/strong\u003e The compositions of the master node and module nodes and their functionalities are investigated with detailed examples. In the following sections, the user-optimal scheduler (US) developed within OCTOPUS is explained, incorporating three techniques, job parallelization, task optimization and CPS.\u003c/p\u003e\n\u003cp\u003eWhen multiple clients attempt to utilize\u0026nbsp;a MAP with many modules, module overlap issues may occur. Customized scheduling algorithms are necessary to address module overlap issues effectively. One conventional approach is job serialization based on the FCFS (First-Come-First-Serve) concept (Fig. 5a), where\u0026nbsp;the\u0026nbsp;priorities of module usage are assigned based on\u0026nbsp;the\u0026nbsp;job submission order. However, in job serialization, device standby times, where no significant actions are performed in terms of devices, may substantially impair job execution efficiency. An example of device standby times is the chemical reaction period (\u0026quot;React\u0026quot;\u0026nbsp;task) in the\u0026nbsp;\u0026quot;BatchSynthesis\u0026quot;\u0026nbsp;module, where chemical\u0026nbsp;reactions occur\u0026nbsp;in chemical vessels but no actions are performed in terms of devices.\u003c/p\u003e\n\u003cp\u003eTo improve module resource utilization, we introduced job parallelization by leveraging device standby times (Fig. 5a and\u0026nbsp;Supplementary Figure\u0026nbsp;S15). During the device standby times of a preceding job, the following job can run in parallel without hindering previously started tasks. As illustrated in Fig. 5b, while job ID 0 progresses during the \u0026quot;React\u0026quot; task in the \u0026quot;BatchSynthesis\u0026quot; module, the\u0026nbsp;\u0026quot;PrepareContainer\u0026quot;\u0026nbsp;task in the next job (job ID 1) can be executed simultaneously by leveraging the bottleneck time (chemical reaction period) of the prior \u0026quot;React\u0026quot; task (Fig. 5b).\u0026nbsp;This specific example of job parallelization is provided in Supplementary Video 2.\u003c/p\u003e\n\u003cp\u003eTo evaluate the impact of job parallelization on time efficiency, we conducted virtual experiments\u0026nbsp;on\u0026nbsp;catalysis processes, which typically involve long device standby times\u0026nbsp;for\u0026nbsp;chemical reactions. In this test, we define 10 virtual modules related to catalyst synthesis (\u0026quot;BatchSynthesis\u0026quot;,\u0026nbsp;\u0026quot;BallMilling\u0026quot;), preprocessing (\u0026quot;Washing\u0026quot;,\u0026nbsp;\u0026quot;Filtration\u0026quot;,\u0026nbsp;\u0026quot;Drying\u0026quot;,\u0026nbsp;\u0026quot;InkPreparation\u0026quot;,\u0026nbsp;\u0026quot;SprayCoating\u0026quot;),\u0026nbsp;characterization\u0026nbsp;(\u0026quot;XRD\u0026quot;), and property\u0026nbsp;evaluation\u0026nbsp;(\u0026quot;HalfCellTest\u0026quot;,\u0026nbsp;\u0026quot;FullCellTest\u0026quot;). Each module involves both a device\u0026nbsp;execution\u0026nbsp;time and device standby time, as described in\u0026nbsp;Supplementary\u0026nbsp;Figures S16 and S17. Seven different types of catalysis experiments utilizing parts of these modules\u0026nbsp;were performed\u0026nbsp;(Fig. 5c,\u0026nbsp;Supplementary\u0026nbsp;Figures.\u0026nbsp;S16 and S17). Fig. 5d compares the results between job serialization and parallelization using bar charts. Three performance metrics were implemented, including\u0026nbsp;\u0026quot;job waiting time\u0026quot;,\u0026nbsp;\u0026quot;job turnaround time\u0026quot;\u0026nbsp;and\u0026nbsp;\u0026quot;job total time\u0026quot;\u0026nbsp;(Supplementary\u0026nbsp;Figure S18). Job waiting time reflects the difference between the job submission and start time, providing client-centric performance insights.\u003csup\u003e41\u003c/sup\u003e \u0026quot;Job turnaround time\u0026quot; indicates the duration between the start of a job and completion of a job, and it offers administrator-centric performance insights. \u0026quot;Job total time\u0026quot;, the sum of \u0026quot;job waiting time\u0026quot; and \u0026quot;job turnaround time\u0026quot;, reflects the overall job execution efficiency.\u003c/p\u003e\n\u003cp\u003eJob serialization entails the sequential execution of modules based on a priority order. Analysis of the performance metrics for time efficiency reveals that, in serialized jobs, the lower-priority jobs, determined by job submission order, may experience substantially increased waiting times due to module overlaps. In contrast, job parallelization aims to reduce waiting times by leveraging the device standby times within each module. Consequently, for all seven jobs in these virtual tests, a significant reduction in \u0026ldquo;job total time\u0026rdquo; is achieved via job parallelization compared to job serialization (Fig. 5d). For example, for job ID 3, \u0026quot;job total time\u0026quot; decreased significantly with job parallelization (from 10 hours to 6.5 hours) by leveraging the device standby time of modules utilized in the preceding jobs, such as the \u0026quot;BatchSynthesis\u0026quot; module of job ID 1. Similarly, in Job ID 4, \u0026quot;job total time\u0026quot; was nearly halved with job parallelization (10 hours to 5.5 hours), leveraging device standby times in the module, such as the centrifugation of the \u0026quot;Washing\u0026quot; module, oven usage of the \u0026quot;Drying\u0026quot; module and scanning of the \u0026quot;XRD\u0026quot; module, as shown in Supplementary Figure S16. As a result, job parallelization significantly improves the time efficiency, reducing both \u0026quot;job waiting time\u0026quot; and \u0026quot;job turnaround time\u0026quot;, ultimately shortening \u0026quot;job total time\u0026quot; (Fig. 5d and Supplementary Table S1).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTask optimization with masking table for preventing device overlaps.\u003c/strong\u003e Up to this point, we operated under the assumption that each module functions independently, with the experimental devices operating without interference. However, as multiple clients submit job requests, the likelihood of multiple tasks within the same module running concurrently increases. In such scenarios, experimental device collisions may occur, raising potential safety concerns (Fig. 6a and Supplementary Figure S19). Moreover, the MAP cost implications are substantial, as each experimental device typically entails significant expenditures to ensure experimental reliability and precision.\u003csup\u003e15\u0026ndash;17\u003c/sup\u003e In this regard, implementing device sharing between different modules has become imperative for cost savings within limited budgets, as demonstrated in recent advances.\u003csup\u003e30,42\u0026ndash;44\u003c/sup\u003e For instance, a robotic arm may serve for both the \u0026quot;BatchSynthesis\u0026quot; and \u0026quot;UV‒Vis\u0026quot; modules, as illustrated in Fig. 6b and Supplementary Figure S19, enabling cost savings through shared utilization.\u003csup\u003e8\u003c/sup\u003e However, when multiple modules attempt to execute individual tasks simultaneously, conflicts may arise regarding the prioritization of tasks for the robotic arm. Consequently, task optimization is urgently needed to prevent device collisions within the same module, and device overlap challenges across different modules.\u003c/p\u003e\n\u003cp\u003eTo overcome these challenges, we introduce a task optimization technique with a masking table. In our approach, each module continuously updates the device status in a tabular format retrieved from the resource manager. Devices currently in use by ongoing tasks are assigned as \u0026quot;True\u0026quot;, while unused devices are marked as \u0026quot;False\u0026quot;, as shown in the device status table example in Fig. 6c. The resource manager dynamically updates the device status table in real time. Next, it is essential to predefine masking tables for each task, which identify the devices required for a specific task as \u0026quot;True\u0026quot; and those not needed as \u0026quot;False\u0026quot;. For instance, when performing the \u0026quot;GetAbsorbance\u0026rdquo; task in the \u0026ldquo;UV‒Vis\u0026rdquo; module, devices such as UV‒Vis spectroscopy, robotic arms and pipettes are involved, and are thus marked as \u0026quot;True\u0026quot;, while the other devices are marked as \u0026quot;False\u0026quot; to create task-specific masking tables (Fig. 6c). Prior to performing a specific task, the AND Boolean operation is performed between the device status table and the masking table for the task, resulting in a table for \u0026ldquo;hold criteria\u0026rdquo; to determine whether to proceed with the next task (Supplementary Figure S20). If all hold criteria indicate \u0026quot;False\u0026quot;, the tasks are executed in parallel, whereas if any of the logical results are found \u0026quot;True\u0026quot;, the system waits until the ongoing task is completed, and all the hold criteria indicate \u0026ldquo;False\u0026rdquo;. Examples of masking tables are provided in Supplementary Figure S21, and an actual example of a task optimization process with a masking table is provided in Supplementary Video S3. In summary, when experimental devices are shared between modules, task optimization with masking tables enables cost-saving benefits and enhances the efficiency of device utilization while avoiding device collisions and ensuring safety.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThe\u0026nbsp;closed-packing schedule for optimizing module resources.\u003c/strong\u003e In computing server systems with a finite number of cores, resource allocation considerations are paramount, and are often documented in job scripts.\u003csup\u003e32,33\u003c/sup\u003e Similarly, in the context of MAP, resource management extends to modules and experimental devices. Here, device resources denote the maximum number of experiments that each module can accommodate. For example, the \u0026ldquo;BatchSynthesis\u0026rdquo; module\u0026rsquo;s resource could be defined by the maximum number of vials simultaneously processable by a magnetic stirrer, while in the \u0026ldquo;UV‒Vis\u0026rdquo; module, it could also be determined by the maximum number of vial holders available for UV‒Vis spectrum measurements (Supplementary Figure S22). Thus, given the limited availability of resources in each module, scheduling methods that account for these constraints are essential in practical platforms.\u003c/p\u003e\n\u003cp\u003eTo efficiently execute multiple jobs within such constraints, we introduce the closed-packing schedule (CPS) algorithm (Fig. 7 and Supplementary Figure S23). The core concept of CPS lies in splitting tasks in a single job into multiple batches to maximally utilize the remaining resources via compact packing. An example of CPS is illustrated in Fig. 7, where three jobs (job IDs 1, 2 and 3) are sequentially submitted with different batch sizes (the number of required vials) of eight, four and eight, respectively, and a stirrer can accommodate up to 16 vials. With active job parallelization, at the execution of job ID 3, 12 resources out of a total of 16 are already occupied by preceding jobs, leaving only 4 resources unused. Then, the CPS divides the tasks of job ID 3 into two batches to allow the first batch to be executed first by fully occupying the 4 unused resources (compact packing), and to allow the other batch to be executed later as soon as the preceding jobs are completed and resources become available. As demonstrated in the example, the CPS aims to effectively minimize the waste of module resources by splitting the tasks within a job based on the computation of the remaining resources.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerformance test of the user-optimal\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;scheduler.\u003c/strong\u003e In the previous sections, the user-optimal scheduler developed within OCTOPUS was explained, and involved three distinct scheduling methods, job parallelization, task optimization and CPS. To maximize scheduling efficiency, these three scheduling methods were integrated within OCTOPUS, resulting in a united scheduling system named the user-optimal scheduler (US) in this paper. To benchmark the US, we introduced the conventional FCFS to prioritize jobs based on their order of submission (Supplementary Figure S24).\u003csup\u003e45\u003c/sup\u003e FCFS executes jobs sequentially based on a priority order. We chose the FCFS algorithm as the baseline scheduler for comparison due to its fairness in executing jobs based on its priority order, which is crucial in multiclient systems.\u003c/p\u003e\n\u003cp\u003eFigs. 8a and 8b visualize the execution time efficiency in processing multiple jobs across the\u0026nbsp;two\u0026nbsp;different scheduling schemes of FCFS and US. We employed a platform capable of simulating device collision and sharing issues to reflect a realistic environment.\u003csup\u003e8\u003c/sup\u003e In these comparative tests, 11 jobs with different batch sizes are submitted sequentially, all of which require the\u0026nbsp;use\u0026nbsp;of\u0026nbsp;either\u0026nbsp;the\u0026nbsp;\u0026quot;BatchSynthesis\u0026quot;\u0026nbsp;or\u0026nbsp;\u0026quot;UV‒Vis\u0026quot;\u0026nbsp;module (Supplementary\u0026nbsp;Figure S25). It is important to clearly understand that, in the tests, the techniques of job parallelization, task optimization and CPS-based resource allocation, developed before, are active only for\u0026nbsp;the\u0026nbsp;US, whereas they are not active in\u0026nbsp;the\u0026nbsp;FCFS scheduling scheme.\u003c/p\u003e\n\u003cp\u003eThrough the observation of the scheduling results, we demonstrate the efficacy of US, particularly in terms of \u0026quot;job waiting time\u0026quot;. The benefits of combined job parallelization and CPS are pronounced in many cases. For example,\u0026nbsp;job IDs 1, 8 and 10\u0026nbsp;significantly reduced \u0026quot;job waiting time\u0026quot; (8.92 hours to 0.63 hours for job ID 1, 10.21 hours to 0.72 hours for job ID 8, and 10.97 hours to 0 hours for job ID 10).\u0026nbsp;The time reductions are attributed to both leveraging device standby times and splitting jobs via CPS. Notably,\u0026nbsp;these jobs are\u0026nbsp;parallelized during the \u0026quot;React\u0026quot; task (device standby time) in the \u0026quot;BatchSynthesis\u0026quot; module of\u0026nbsp;the\u0026nbsp;preceding jobs (Fig. 8b,\u0026nbsp;Supplementary Figure\u0026nbsp;S26 and Table S2). Similarly,\u0026nbsp;job ID 4 also benefits from both job parallelization and CPS,\u0026nbsp;leading to\u0026nbsp;a\u0026nbsp;reduction\u0026nbsp;in\u0026nbsp;\u0026quot;job waiting time\u0026quot;, as shown\u0026nbsp;in Fig. 8c (11.72 hours to 0 hours for job ID 4). This job is parallelized with the preceding job ID 3 as\u0026nbsp;the\u0026nbsp;CPS splits the job\u0026nbsp;according to the remaining resources of\u0026nbsp;the\u0026nbsp;\u0026quot;UV‒Vis\u0026quot;\u0026nbsp;modules (Fig. 8b and\u0026nbsp;Supplementary Figure\u0026nbsp;S27). Meanwhile, the benefit of the task optimization technique is well observed in\u0026nbsp;many cases,\u0026nbsp;such as job IDs 2, 3, 4, 6, 7 and 9. Since\u0026nbsp;the\u0026nbsp;\u0026ldquo;BatchSynthesis\u0026rdquo; module and \u0026ldquo;UV‒Vis\u0026rdquo; module\u0026nbsp;share\u0026nbsp;a robotic arm, the simultaneous\u0026nbsp;execution\u0026nbsp;of these two modules may cause safety hazards from the device sharing environments. However, with\u0026nbsp;active\u0026nbsp;task optimization, concurrent executions of both modules were safely performed, as demonstrated in\u0026nbsp;Supplementary\u0026nbsp;Figure S28, Table S2\u0026nbsp;and Video S3.\u003c/p\u003e\n\u003cp\u003eBy minimizing \u0026quot;job waiting time\u0026quot;,\u0026nbsp;the\u0026nbsp;US\u0026nbsp;also\u0026nbsp;proved more efficient in terms of \u0026quot;job total time\u0026quot; compared to the conventional FCFS scheme. However, we observed an overall and slight increase\u0026nbsp;in\u0026nbsp;\u0026quot;job turnaround time\u0026quot; for US, potentially resulting from a duplication of\u0026nbsp;device standby\u0026nbsp;time (Supplementary\u0026nbsp;Figure S29). For Job IDs 1, 8 and 10, our scheduling system duplicates the\u0026nbsp;device standby\u0026nbsp;time associated with the \u0026quot;React\u0026quot; task,\u0026nbsp;as the jobs are split based on\u0026nbsp;the\u0026nbsp;remaining resources. This redundancy contributes to the accumulation of \u0026quot;job turnaround time\u0026quot;. Overall, while US significantly reduces \u0026quot;job waiting time\u0026quot;,\u0026nbsp;some\u0026nbsp;delays in \u0026quot;job turnaround time\u0026quot; may occur due to\u0026nbsp;device standby\u0026nbsp;time duplications.\u003c/p\u003e\n\u003cp\u003eOverall, by analyzing the performance metrics for time efficiency, US emerged as highly effective, saving time across all jobs in terms of \u0026quot;job waiting time\u0026quot; (Fig. 8c). Although there is a slight increase in \u0026quot;job turnaround time\u0026quot; due to duplications of device standby time by CPS (Fig. 8d and Supplementary Figure S27), the significant improvement in efficiency in terms of \u0026quot;job waiting time\u0026quot; renders US more efficient, leading to a substantial reduction in \u0026ldquo;job total time\u0026quot;, compared to conventional FCFS (Fig. 8e and Supplementary Table S2).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eCLI presents a notable departure from conventional experimentation interfaces, potentially posing challenges for traditional experimental experts. While familiar to researchers in computer science fields, its adoption may present difficulties for those accustomed to hands-on experimentation. Recent studies advocate for the development of a user-friendly web-based interface accessible to both experimental researchers and computational experts.\u003csup\u003e14,19,22,24\u003c/sup\u003e Moreover, the integration of extended reality, virtual reality and augmented reality could enhance the accessibility and usability of OCTOPUS in the future.\u003csup\u003e46\u0026ndash;48\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eIn conclusion, OCTOPUS embodies a multifaceted solution that has been engineered to overcome the challenges inherent in MAP accessed by multiple users, OCTOPUS embodies a multifaceted solution. First, its tripartite structure, comprising the interface node, master node and module nodes, orchestrates seamless client request handling and experimental task scheduling. Through the integration of process modularization and network protocol utilization, OCTOPUS establishes a foundation characterized by homogeneity, scalability, safety and versatility within a central management platform. Furthermore, OCTOPUS presents US. The incorporation of job parallelization techniques serves to alleviate delays, while task optimization algorithms prevent safety hazards potentially arising from device collisions and sharing. In addition, the development of the CPS algorithm within OCTOPUS represents a significant stride in efficiently executing multiple jobs with minimal resource wastage. OCTOPUS will facilitate the management of diverse experiments from multiple users and thereby accelerated the widespread adoption of MAP for expedited material development.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eVirtual experiments for job parallelization leveraging\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003edevice standby\u003c/strong\u003e \u003cstrong\u003etimes\u003c/strong\u003e\u003cstrong\u003e.\u0026nbsp;\u003c/strong\u003eIn\u0026nbsp;the virtual experiments\u0026nbsp;related to Fig. 5, we defined the duration of both the device execution time and device standby\u0026nbsp;time of each module as follows. For the\u0026nbsp;\u0026quot;BatchSynthesis\u0026quot;,\u0026nbsp;\u0026quot;Filtration\u0026quot;,\u0026nbsp;\u0026quot;BallMilling\u0026quot;,\u0026nbsp;\u0026quot;InkPreparation\u0026quot;,\u0026nbsp;\u0026quot;XRD\u0026quot;\u0026nbsp;and\u0026nbsp;\u0026quot;SprayCoating\u0026quot;\u0026nbsp;modules,\u0026nbsp;a duration of 0.5 hours was assigned to both device execution time and device standby\u0026nbsp;time, resulting in 1 hour of total time for each module (Supplementary\u0026nbsp;Figure S16). For the \u0026quot;Washing\u0026quot; module, which is known for its repetitive removal of impurities, each 0.5-hour duration was assigned to one of the device execution times or device standby\u0026nbsp;times, resulting in 2 hours of total time (Supplementary\u0026nbsp;Figure S16). For the\u0026nbsp;\u0026quot;Drying\u0026quot;,\u0026nbsp;\u0026quot;HalfCellTest\u0026quot;, and\u0026nbsp;\u0026quot;FullCellTest\u0026quot;\u0026nbsp;modules, which are known for their extended durations for bottleneck processes, durations of 0.5 hours and 1.5 hours, respectively, were assigned to each device execution time and device standby\u0026nbsp;time, resulting in 2 hours of total time (Supplementary\u0026nbsp;Figure S16). The time allocations in each module are provided in detail in\u0026nbsp;Supplementary\u0026nbsp;Figure S16. The experimental devices within each module were assumed to function without mutual interference. The combinations and sequences of modules assigned to each job were carefully chosen based on actual experimental processes, as illustrated in\u0026nbsp;Supplementary\u0026nbsp;Figure S17. For example, for job ID 0 in Fig. 5, an experiment of synthesizing and measuring a Cu catalyst for the CO\u003csub\u003e2\u003c/sub\u003e reduction reaction involves the following modules in order: \u0026quot;BatchSynthesis\u0026quot;, \u0026quot;Washing\u0026quot;, \u0026quot;Filtration\u0026quot;, \u0026quot;Drying\u0026quot;, \u0026quot;InkPreparation\u0026quot; and \u0026quot;HalfCellTest\u0026quot;. The modules used in other jobs are also described in Supplementary Figure S17.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExperiments for task optimization with masking table.\u0026nbsp;\u003c/strong\u003ePrior to conducting\u0026nbsp;experiments related to Fig. 6, we predefine the masking tables for each task.\u0026nbsp;For examples\u0026nbsp;in the \u0026ldquo;BatchSynthesis\u0026rdquo; and \u0026ldquo;UV‒Vis\u0026rdquo; modules, as illustrated in Supplementary\u0026nbsp;Figure S19, the masking tables for a task represent the usage of experimental devices during the task, including the robotic arm (shared between two modules), vial storage, linear actuator with solution dispenser, syringe pump, pipette, UV‒Vis spectroscopy. For example, the \u0026ldquo;AddSolution\u0026rdquo; task in the \u0026ldquo;BatchSynthesis\u0026rdquo; module involves the activation of a linear actuator and pump; thus, these two devices are marked as \u0026ldquo;True\u0026rdquo; and all the other devices are marked as \u0026ldquo;False\u0026rdquo; in the masking table.\u0026nbsp;The masking tables are structured with Boolean values (True or False).\u0026nbsp;Examples of the masking table are provided in Supplementary\u0026nbsp;Figure S21. In this work, the module and device configuration utilized in this task optimization performance test adhered to our prior publication.\u003csup\u003e8\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExperiments for benchmarking US.\u0026nbsp;\u003c/strong\u003eThe module and device configuration employed in the performance benchmarking test\u0026nbsp;in Fig. 8 are consistent with recent advancements documented in our previous work.\u003csup\u003e8\u003c/sup\u003e Scheduling schemes of FCFS and US are compared in realistic experimental environments. All the resource information was stored and periodically updated by a resource manager at each module node by using a location index based on the listed data types (Supplementary Figure S8). Job parallelization, task optimization and CPS-based resource allocation are active only for the US, whereas they are not active in the FCFS scheduling schemes. In these benchmark tests, the 11 job scripts are submitted based on job submission timelines, as illustrated in Supplementary Figure S25. These 11 job scripts contain information on the experiment type (model names of manual vs. AI optimizations), module selection (\u0026quot;BatchSynthesis\u0026quot; or \u0026quot;UV‒Vis\u0026quot;), batch size, number of closed-loop cycles and task configuration (number of task executions and device standby time).\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eData Availability Statement\u003c/h2\u003e\n\u003cp\u003eSeveral examples of our result data, the codes, and related explanations are provided in the following GitHub repository (https://github.com/KIST-CSRC/Octopus). All codes are written in Python 3.7 and all environments could be created via requirements.txt file.\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003cp\u003eThis work was supported by the National Research Foundation of Korea funded by the Ministry of Science and ICT [NRF-2022M3H4A7046278]. Nayeon Kim conceived the concept of image and drawed Supplementary Figure S1.\u003c/p\u003e\n\u003ch2\u003eAuthor Contributions\u003c/h2\u003e\n\u003cp\u003eS.S.H, D.K., and K.Y.L. conceived the idea and supervised the project. H.J.Y. conceived the idea, designed OCTOPUS architecture, developed user-optimal scheduler, and performed experiments. All authors contributed to result analysis and manuscript writing.\u003c/p\u003e\n\u003ch2\u003eCompeting Interests\u003c/h2\u003e\n\u003cp\u003eThe authors declare no competing financial or non-financial interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eHiggins, K., Valleti, S. M., Ziatdinov, M., Kalinin, S. V. \u0026amp; Ahmadi, M. Chemical robotics enabled exploration of stability in multicomponent lead halide perovskites via machine learning. \u003cem\u003eACS Energy Lett.\u003c/em\u003e \u003cstrong\u003e5\u003c/strong\u003e, 3426\u0026ndash;3436 (2020).\u003c/li\u003e\n\u003cli\u003eEpps, R. W. \u003cem\u003eet al.\u003c/em\u003e Artificial chemist: an autonomous quantum dot synthesis bot. \u003cem\u003eAdv. Mater.\u003c/em\u003e \u003cstrong\u003e32\u003c/strong\u003e, 2001626 (2020).\u003c/li\u003e\n\u003cli\u003eMekki-Berrada, F. \u003cem\u003eet al.\u003c/em\u003e Two-step machine learning enables optimized nanoparticle synthesis. \u003cem\u003enpj Comput. Mater.\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 55 (2021).\u003c/li\u003e\n\u003cli\u003eAngelone, D. \u003cem\u003eet al.\u003c/em\u003e Convergence of multiple synthetic paradigms in a universally programmable chemical synthesis machine. \u003cem\u003eNat. Chem.\u003c/em\u003e \u003cstrong\u003e13\u003c/strong\u003e, 63\u0026ndash;69 (2021).\u003c/li\u003e\n\u003cli\u003eH\u0026auml;se, F., Roch, L. M., Kreisbeck, C. \u0026amp; Aspuru-Guzik, A. Phoenics: A Bayesian optimizer for chemistry. \u003cem\u003eACS Cent. Sci.\u003c/em\u003e \u003cstrong\u003e4\u003c/strong\u003e, 1134\u0026ndash;1145 (2018).\u003c/li\u003e\n\u003cli\u003eAldeghi, M., H\u0026auml;se, F., Hickman, R. J., Tamblyn, I. \u0026amp; Aspuru-Guzik, A. Golem: An algorithm for robust experiment and process optimization. \u003cem\u003eChem. Sci.\u003c/em\u003e \u003cstrong\u003e12\u003c/strong\u003e, 14792\u0026ndash;14807 (2021).\u003c/li\u003e\n\u003cli\u003eH\u0026auml;se, F., Aldeghi, M., Hickman, R. J., Roch, L. M. \u0026amp; Aspuru-Guzik, A. Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge. \u003cem\u003eAppl. Phys. Rev.\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 031406 (2021).\u003c/li\u003e\n\u003cli\u003eYoo, H. J. \u003cem\u003eet al.\u003c/em\u003e Bespoke Metal Nanoparticle Synthesis at Room Temperature and Discovery of Chemical Knowledge on Nanoparticle Growth via Autonomous Experimentations. \u003cem\u003eAdv. Funct. Mater.\u003c/em\u003e (2024) doi:10.1002/adfm.202312561.\u003c/li\u003e\n\u003cli\u003eYoshikawa, N., Darvish, K., Vakili, M. G., Garg, A. \u0026amp; Aspuru-Guzik, A. Digital pipette: open hardware for liquid transfer in self-driving laboratories. \u003cem\u003eDigit. Discov.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 1745\u0026ndash;1751 (2023).\u003c/li\u003e\n\u003cli\u003eYoshikawa, N. \u003cem\u003eet al.\u003c/em\u003e Chemistry Lab Automation via Constrained Task and Motion Planning. \u003cem\u003earXiv\u003c/em\u003e (2022) https://doi.org/10.48550/arXiv.2212.09672.\u003c/li\u003e\n\u003cli\u003eJiang, Y. \u003cem\u003eet al.\u003c/em\u003e Autonomous biomimetic solid dispensing using a dual-arm robotic manipulator. \u003cem\u003eDigit. Discov.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 1733\u0026ndash;1744 (2023).\u003c/li\u003e\n\u003cli\u003eTiong, L. C. O. \u003cem\u003eet al.\u003c/em\u003e Machine vision-based detections of transparent chemical vessels toward the safe automation of material synthesis. \u003cem\u003enpj Comput. Mater.\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 42 (2024).\u003c/li\u003e\n\u003cli\u003eSim, M., Ghazi Vakili, M., Hao, H., Hickman, R. J. \u0026amp; Pablo-Garc\u0026iacute;a, S. \u003cem\u003eChemOS 2.0: an orchestration architecture for chemical self-driving laboratories\u003c/em\u003e. \u003cem\u003eChemRxiv\u003c/em\u003e vol. 5 https://doi.org/10.26434/chemrxiv-2023-v2khf (2023).\u003c/li\u003e\n\u003cli\u003eSteiner, S. \u003cem\u003eet al.\u003c/em\u003e Organic synthesis in a modular robotic system driven by a chemical programming language. \u003cem\u003eScience (80-. ).\u003c/em\u003e \u003cstrong\u003e363\u003c/strong\u003e, (2019).\u003c/li\u003e\n\u003cli\u003eAbolhasani, M. \u0026amp; Kumacheva, E. The rise of self-driving labs in chemical and materials sciences. \u003cem\u003eNat. Synth.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 483\u0026ndash;492 (2023).\u003c/li\u003e\n\u003cli\u003eMaffettone, P. M. \u003cem\u003eet al.\u003c/em\u003e What is missing in autonomous discovery: open challenges for the community. \u003cem\u003eDigit. Discov.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 1644\u0026ndash;1659 (2023).\u003c/li\u003e\n\u003cli\u003eSeifrid, M. \u003cem\u003eet al.\u003c/em\u003e Autonomous Chemical Experiments: Challenges and Perspectives on Establishing a Self-Driving Lab. \u003cem\u003eAcc. Chem. Res.\u003c/em\u003e \u003cstrong\u003e55\u003c/strong\u003e, 2454\u0026ndash;2466 (2022).\u003c/li\u003e\n\u003cli\u003eSayfan, G. \u003cem\u003eMastering kubernetes\u003c/em\u003e. (Packt Publishing Ltd, 2017).\u003c/li\u003e\n\u003cli\u003evan der Westhuizen, C. J., du Toit, J., Neyt, N., Riley, D. \u0026amp; Panayides, J. L. Use of open-source software platform to develop dashboards for control and automation of flow chemistry equipment. \u003cem\u003eDigit. Discov.\u003c/em\u003e \u003cstrong\u003e1\u003c/strong\u003e, 596\u0026ndash;604 (2022).\u003c/li\u003e\n\u003cli\u003eRahmanian, F. \u003cem\u003eet al.\u003c/em\u003e Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration. \u003cem\u003eAdv. Mater. Interfaces\u003c/em\u003e \u003cstrong\u003e9\u003c/strong\u003e, 2101987 (2022).\u003c/li\u003e\n\u003cli\u003eStrieth-Kalthoff, F. \u003cem\u003eet al.\u003c/em\u003e \u003cem\u003eDelocalized, Asynchronous, Closed-Loop Discovery of Organic Laser Emitters\u003c/em\u003e. \u003cem\u003eChemRxiv\u003c/em\u003e https://doi.org/10.26434/chemrxiv-2023-wqp0d (2023) doi:10.26434/chemrxiv-2023-wqp0d.\u003c/li\u003e\n\u003cli\u003eHielscher, M. M., D\u0026ouml;rr, M., Schneider, J. \u0026amp; Waldvogel, S. R. LABS: Laboratory Automation and Batch Scheduling \u0026ndash; A Modular Open Source Python Program for the Control of Automated Electrochemical Synthesis with a Web Interface. \u003cem\u003eChem. - An Asian J.\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, (2023).\u003c/li\u003e\n\u003cli\u003eTamura, R., Tsuda, K. \u0026amp; Matsuda, S. NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science. \u003cem\u003earXiv Prepr.\u003c/em\u003e (2023) doi:https://doi.org/10.48550/arXiv.2304.13927.\u003c/li\u003e\n\u003cli\u003eGuevarra, D. \u003cem\u003eet al.\u003c/em\u003e Orchestrating nimble experiments across interconnected labs. \u003cem\u003eDigit. Discov.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 1806\u0026ndash;1812 (2023).\u003c/li\u003e\n\u003cli\u003eKusne, A. G. \u0026amp; McDannald, A. Scalable multi-agent lab framework for lab optimization. \u003cem\u003eMatter\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 1880\u0026ndash;1893 (2023).\u003c/li\u003e\n\u003cli\u003eDeneault, J. R. \u003cem\u003eet al.\u003c/em\u003e Toward autonomous additive manufacturing: Bayesian optimization on a 3D printer. \u003cem\u003eMRS Bull.\u003c/em\u003e \u003cstrong\u003e46\u003c/strong\u003e, 566\u0026ndash;575 (2021).\u003c/li\u003e\n\u003cli\u003eCampbell, S. I. \u003cem\u003eet al.\u003c/em\u003e Outlook for artificial intelligence and machine learning at the NSLS-II. \u003cem\u003eMach. Learn. Sci. Technol.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, (2021).\u003c/li\u003e\n\u003cli\u003eLeong, C. J. \u003cem\u003eet al.\u003c/em\u003e An object-oriented framework to enable workflow evolution across materials acceleration platforms. \u003cem\u003eMatter\u003c/em\u003e \u003cstrong\u003e5\u003c/strong\u003e, 3124\u0026ndash;3134 (2022).\u003c/li\u003e\n\u003cli\u003eDu, X. \u003cem\u003eet al.\u003c/em\u003e Elucidating the Full Potential of OPV Materials Utilizing a High-Throughput Robot-Based Platform and Machine Learning. \u003cem\u003eJoule\u003c/em\u003e \u003cstrong\u003e5\u003c/strong\u003e, 495\u0026ndash;506 (2021).\u003c/li\u003e\n\u003cli\u003eColey, C. W. \u003cem\u003eet al.\u003c/em\u003e A robotic platform for flow synthesis of organic compounds informed by AI planning. \u003cem\u003eScience (80-. ).\u003c/em\u003e \u003cstrong\u003e365\u003c/strong\u003e, (2019).\u003c/li\u003e\n\u003cli\u003eJiang, Y. \u003cem\u003eet al.\u003c/em\u003e An artificial intelligence enabled chemical synthesis robot for exploration and optimization of nanomaterials. \u003cem\u003eSci. Adv.\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 1\u0026ndash;12 (2022).\u003c/li\u003e\n\u003cli\u003eYoo, A. B., Jette, M. A. \u0026amp; Grondona, M. Slurm: Simple linux utility for resource management. in \u003cem\u003eWorkshop on job scheduling strategies for parallel processing\u003c/em\u003e 44\u0026ndash;60 (2003).\u003c/li\u003e\n\u003cli\u003eNabrzyski, J., Schopf, J. M. \u0026amp; Weglarz, J. \u003cem\u003eGrid resource management: state of the art and future trends\u003c/em\u003e. (Springer Science \u0026amp; Business Media, 2012).\u003c/li\u003e\n\u003cli\u003eKathryn Vasel. The pandemic forced a massive remote-work experiment. Now comes the hard part. \u003cem\u003eCNN\u003c/em\u003e (2021).\u003c/li\u003e\n\u003cli\u003ePark, J. \u003cem\u003eet al.\u003c/em\u003e Closed-loop optimization of nanoparticle synthesis enabled by robotics and machine learning. \u003cem\u003eMatter\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 677\u0026ndash;690 (2023).\u003c/li\u003e\n\u003cli\u003eCanty, R. B. \u0026amp; Jensen, K. F. Sharing reproducible synthesis recipes. \u003cem\u003eNat. Synth.\u003c/em\u003e (2024) doi:10.1038/s44160-023-00478-1.\u003c/li\u003e\n\u003cli\u003eRauschen, R., Guy, M., Hein, J. E. \u0026amp; Cronin, L. Universal chemical programming language for robotic synthesis repeatability. \u003cem\u003eNat. Synth.\u003c/em\u003e (2024) doi:10.1038/s44160-023-00473-6.\u003c/li\u003e\n\u003cli\u003eGranda, J. M., Donina, L., Dragone, V., Long, D. L. \u0026amp; Cronin, L. Controlling an organic synthesis robot with machine learning to search for new reactivity. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e559\u003c/strong\u003e, 377\u0026ndash;381 (2018).\u003c/li\u003e\n\u003cli\u003eVolk, A. A. \u003cem\u003eet al.\u003c/em\u003e AlphaFlow: autonomous discovery and opti-mization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning. \u003cem\u003eNat. Commun.\u003c/em\u003e (2023) doi:https://doi.org/10.1038/s41467-023-37139-y.\u003c/li\u003e\n\u003cli\u003eSoldatov, M. A. \u003cem\u003eet al.\u003c/em\u003e Self-Driving Laboratories for Development of New Functional Materials and Optimizing Known Reactions. \u003cem\u003eNanomaterials\u003c/em\u003e \u003cstrong\u003e11\u003c/strong\u003e, 619 (2021).\u003c/li\u003e\n\u003cli\u003eRubab, S., Hassan, M. F., Mahmood, A. K. \u0026amp; Shah, S. N. M. Adoptability Study of Bin-Packing for Scheduling Jobs on Volunteer Grid Resources. in \u003cem\u003eProcedia Computer Science\u003c/em\u003e vol. 69 2\u0026ndash;12 (Elsevier B.V., 2015).\u003c/li\u003e\n\u003cli\u003eMacLeod, B. P. \u003cem\u003eet al.\u003c/em\u003e A self-driving laboratory advances the Pareto front for material properties. \u003cem\u003eNat. Commun.\u003c/em\u003e \u003cstrong\u003e13\u003c/strong\u003e, (2022).\u003c/li\u003e\n\u003cli\u003eBurger, B. \u003cem\u003eet al.\u003c/em\u003e A mobile robotic chemist. \u003cem\u003eNature\u003c/em\u003e \u003cstrong\u003e583\u003c/strong\u003e, 237\u0026ndash;241 (2020).\u003c/li\u003e\n\u003cli\u003eMacLeod, B. P. \u003cem\u003eet al.\u003c/em\u003e Self-driving laboratory for accelerated discovery of thin-film materials. \u003cem\u003eSci. Adv.\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 1\u0026ndash;8 (2020).\u003c/li\u003e\n\u003cli\u003ePutera, A. \u0026amp; Siahaan, U. Comparison Analysis of CPU Scheduling : FCFS, SJF and Round Robin. \u003cem\u003eInt. J. Eng. Dev. Res.\u003c/em\u003e \u003cstrong\u003e4\u003c/strong\u003e, (2016).\u003c/li\u003e\n\u003cli\u003eLi, J., Tu, Y., Liu, R., Lu, Y. \u0026amp; Zhu, X. Toward \u0026ldquo;On‐Demand\u0026rdquo; Materials Synthesis and Scientific Discovery through Intelligent Robots. \u003cem\u003eAdv. Sci.\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 1901957 (2020).\u003c/li\u003e\n\u003cli\u003ePells, R. Why scientists are delving into the virtual world. \u003cem\u003eNature\u003c/em\u003e (2023) doi:10.1038/d41586-023-02688-1.\u003c/li\u003e\n\u003cli\u003eWang, G. \u003cem\u003eet al.\u003c/em\u003e Development of metaverse for intelligent healthcare. \u003cem\u003eNat. Mach. Intell.\u003c/em\u003e \u003cstrong\u003e4\u003c/strong\u003e, 922\u0026ndash;929 (2022).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"OCTOPUS, Materials Acceleration Platform, Job parallelization, Task optimization, Process modularization, Network protocol, User-optimal scheduler","lastPublishedDoi":"10.21203/rs.3.rs-4254570/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4254570/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The materials acceleration platform (MAP), empowered by robotics and artificial intelligence, is a transformative approach for expediting material discovery processes across diverse domains. However, the development of an operating system for MAP faces challenges in simultaneously managing diverse experiments from multiple users. Specifically, when MAP is utilized by multiple users, the overlapping challenges of experimental modules or devices can lead to inefficiencies in both resource utilization and safety hazards. To overcome these challenges, we present an operation control system for MAP, namely, OCTOPUS, which is an acronym for operation control system for task optimization and job parallelization via a user-optimal scheduler. OCTOPUS streamlines experiment scheduling and optimizes resource utilization through integrating its interface node, master node and module nodes. Leveraging process modularization and a network protocol, OCTOPUS ensures the homogeneity, scalability, safety and versatility of MAP. In addition, OCTOPUS embodies a user-optimal scheduler. Job parallelization and task optimization techniques mitigate delays and safety hazards within realistic operational environments, while the closed-packing schedule algorithm efficiently executes multiple jobs with minimal resource waste. This work offers a solution to the challenges encountered within MAP accessed by multiple users, and thereby will facilitate its widespread adoption in material development processes.","manuscriptTitle":"OCTOPUS: Operation Control System for Task Optimization and Job Parallelization via a User-Optimal Scheduler","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-06 04:51:53","doi":"10.21203/rs.3.rs-4254570/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"3d1ff4a9-2818-45ca-a3d2-4e10d2ca3929","owner":[],"postedDate":"May 6th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":30748105,"name":"Physical sciences/Materials science/Techniques and instrumentation/Design, synthesis and processing"},{"id":30748106,"name":"Physical sciences/Nanoscience and technology/Techniques and instrumentation/Design, synthesis and processing"}],"tags":[],"updatedAt":"2024-11-09T08:05:25+00:00","versionOfRecord":{"articleIdentity":"rs-4254570","link":"https://doi.org/10.1038/s41467-024-54067-7","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2024-11-08 05:00:00","publishedOnDateReadable":"November 8th, 2024"},"versionCreatedAt":"2024-05-06 04:51:53","video":"","vorDoi":"10.1038/s41467-024-54067-7","vorDoiUrl":"https://doi.org/10.1038/s41467-024-54067-7","workflowStages":[]},"version":"v1","identity":"rs-4254570","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4254570","identity":"rs-4254570","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}