General Deep Learning Framework for Emissivity Engineering

General Deep Learning Framework for Emissivity Engineering | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article General Deep Learning Framework for Emissivity Engineering Run Hu, Shilv Yu, Xi Wang, Zihe Chen, Peng Zhou, Yuheng Deng, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3140708/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 05 Dec, 2023 Read the published version in Light: Science & Applications → Version 1 posted 14 You are reading this latest preprint version Abstract Wavelength-selective thermal emitters have been frequently adopted as a typical platform for emissivity engineering to achieve desired target emissivity spectra for broad applications such as thermal camouflage, radiative cooling, and gas sensing, etc. However, previous design methods fail to tackle the simultaneous design of both materials and structures, either fixing materials to design structures or fixing structures to select proper materials, hindering the establishment of a general design framework for emissivity engineering applicable across different applications. Herein, we employ the deep Q-learning network algorithm, a reinforcement learning method based on deep learning framework, to design multilayer wavelength-selective thermal emitters for a diverse range of applications, including thermal camouflage, radiative cooling and gas sensing. With magnetron sputtering, these emitters are fabricated and measured, validating the desired emissivity spectra with the designed ones. The main merits of the deep Q-learning algorithm include that it can 1) autonomously select suitable materials from a self-built material library and 2) autonomously optimize structures, thus realizing simultaneous optimization of materials and structures for various emissivity engineering applications. The present method is demonstrated to be feasible and efficient in designing multilayer wavelength-selective thermal emitters, offering a general framework for emissivity engineering and paving the way for efficient design of nonlinear optimization problems across various physical fields. emissivity engineering structure optimization deep Q-learning network wavelength-selective thermal emitters thermal camouflage radiative cooling gas sensing Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Introduction All objects in nature emit thermal radiation outwardly at anytime and anywhere in a broadband, non-selective, incoherent, diffusive, and reciprocal manner. 1,2 Thanks to the fast development of thermal metamaterials and metasurfaces in recent years, thermal radiation has been demonstrated to be engineered with comprehensive control of spectral, directional, and dynamic characteristics, enabling higher-efficiency regulation of radiative heat transfer from natural objects. Among them, the spectral emissivity engineering of thermal radiation enables more applications, such as energy harvesting, 3 thermal management, 4–5 radiative cooling, 6 thermal camouflage, 7 infrared (IR) sensing, 8 far-/near-field radiation control, 9 thermophotovoltaics, 10–11 thermography, 12 heat-assisted magnetic recording, 13 etc. The emissivity engineering aims to select materials and design nanostructures to achieve specific functionalities with a target emissivity spectrum. The common physics of selective emissivity comes from the excitation of different photon modes, which leads to the local enhancement or suppression of the internal electric field, thus allowing for control over the radiation emission at different wavelengths. Wavelength-selective thermal emitters (WS-EMs), as the main platform for emissivity engineering, can be designed by multilayers, photonic crystals, nano-grating, 14 nano antennas arrays, 15 multiple-quantum-well, 16 Fabry-Perot cavities, 17 etc. As one of the simplest structures of WS-EMs, multilayers are frequently employed which are composed of alternating layers of materials with different refractive indices 18 , and relatively easy for fabrication at a low cost for extensive applications in thermal camouflage (TC), 19–21 radiative cooling (RC), 22–24 gas sensing (GS), 25–27 etc. In general, different applications require distinct emissivity spectra as illustrated in Fig. 1 . For instance, TC necessitates low emissivity within the long-wavelength IR range (8–14 µm), which is call as atmosphere window (AM) due to its high transmittance, to prevent detection by the most IR detectors when the background temperature is low. Additionally, it is advantageous for the emissivity outside the AM to remain as high as possible to facilitate further radiative heat dissipation. 28,29 To achieve TC, Peng et al. designed a silver/germanium (Ag/Ge) multilayered structure, where impedance matching is utilized to manipulate the radiation characteristics. 30 Zhu et al. designed a Ge/ZnS multilayer on a silica aerogel substrate with efficient radiative cooling capability for TC in high ambient temperature. 31 In contrast, RC aims to achieve passive cooling by radiating the heat directly to the outer space at ~ 3 K via the high emissivity within the AM. In addition, a high reflectivity in the solar band is necessarily required to reflect as much solar energy as possible to maximize the cooling power, ultimately achieving net energy outflow and reducing object temperature. 32 Raman et al. adopted needle optimization method to design a seven-layer HfO 2 /SiO 2 emitter. The fabricated multilayer emitter achieved daytime RC under direct solar irradiance for the first time, which reflected 97% of solar irradiance and cooled to 4.9 ℃ below the ambient temperature. 33 Similarly, Ma et al. optimized seven-layer SiO 2 /Si 3 N 4 emitter using evolutionary algorithm, and the emitter was highly reflective towards solar radiation and had a broadband high emissivity within the AM. 34 Different from the broadband emissivity spectra for RC and TC, the emissivity spectrum for GS needs narrow-band peaks which match the absorption peaks of the detected gas. Sakurai et al. 35 and Xi et al. 26 both utilized machine-learning to design and optimize multilayered structures and achieved ultra-narrowband emission peaks at multiple wavelengths. In particular, Xi et al. 26 obtained the whole database of multilayered Ge/SiO 2 WS-EMs for narrowband emissivity spectra in the wavelength range of 3 to 10 µm, and the highest Q -factor reaches 508 far beyond the Q -factor record in the literature. As far as we are concerned from the literature, although many combinations of materials of multilayer WS-EMs have been proposed for emissivity engineering, both material selection and structural design still rely on physics-inspired methods and past design experience or guidelines. They either fix materials to design structures, or fix structures to select proper materials, which are inefficient and difficult to achieve optimal design. To further improve the performance of multilayer WS-EMs, machine-learning optimization algorithms have shown unique advantages in structure optimization and designing problems. 36–38 However, designers still have to conduct extensive searches of existing work to determine suitable materials and initial structural parameters for the design goals before optimization 26,34,35 . This is attributed to the enormous optimization space that arises when considering both material selection and structure parameters optimization simultaneously, leading to a significant increase in computing resources and inefficiency of machine learning algorithm. Hence, one open question comes that whether the materials selection and structures optimization can be performed synchronously for emissivity engineering across different applications? Recently, in light of the exceptional success in domains related to computer science and engineering, including natural language processing, computer vision, image processing, speech recognition, etc., deep learning has attracted increasing attention, especially for material structure optimization 38 . Through establishing the artificial neural network and the data-driven method, deep learning obtains the mapping relationship between data pairs. Among the multitudinous neural network, deep Q-learning network (DQN), a reinforcement learning (RL) method based on deep learning (DL), can effectively make optimal decisions from millions of choices by learning from the history experiences, thus it is particularly well-adapted to address the large optimization space problem. In addition, as a kind of RL method, DQN is policy based, which means that the dataset used for neural network training does not need to be collected beforehand, but is derived from the accumulation of historical policies and corresponding responses. Through combining the RL and DL, DQN has been proved to be extraordinarily competent for Atari games, and can reach the level of human experts in games such as Go and Chess, because it is able to formulate the best next action based on the current state of the game to achieve the highest game score. 39 In this study, DQN is adopted to achieve a general framework for emissivity engineering with high accuracy and efficiency. Firstly, a framework of DQN is established and the initialization method in DQN is improved to be capable of automatic design and optimization of multilayer WS-EMs. Subsequently, a commonly used material library for WS-EMs is set up for DQN to select materials according to different applications. After that, the multilayer WS-EMs for three applications including TC, RC, and GS, are designed and optimized, which are then experimentally validated. The selection of materials and the design of the structure are independently completed by DQN within the extensive optimization space. The designed multilayer WS-EMs all exhibit extremely exceptional performance in these three applications, validating DQN as a general deep learning framework for emissivity engineering. Methods The roadmap of optimization process of DQN is illustrated in Fig. 2 . The whole optimization process can be described as an interactive progress with the environment. The state of the environment, which consists of the material ID number and the thicknesses of each layer, represents the materials and structural parameters of the current multilayer. Here we set up the multilayer WS-EMs as a 5-layer structure composed of alternating two materials. It is worth mentioning that while increasing the number of layers can meet more rigorous emissivity spectrum requirements, it also significantly expands the optimization space by several orders of magnitude, requiring greater computing power and longer design time. Consequently, the state can be represented by a 1×7 vector containing material and structure information. The two materials are selected from the self-built material library, as shown in Table 1 , which contains 8 commonly used materials for emissivity engineering. Their optical properties (refractive index) are referred to E. Palik’s and Qurry’s books 40,41 and other research work 42,43 (See Supplementary Information S1). Regarding the substrate material, it needs to be selected according to specific design goals, we chose silver for RC, silicon for TC, and tungsten for GS. Each layer thickness is varied within the range of 20 nm–1000 nm with a uniform step size of 20 nm, which results in a total of 50 possible steps for each layer. Considering the 8 materials available, the above structural configuration leads to 8×7×50 5 = 1.75×10 10 potential candidate structures. The sheer volume of candidate structure necessitates enormous computility, making it nearly impossible through manual design and inefficient for conventional machine learning algorithms. After the physical information of the multilayer structure is encoded into digital information, it is inputted into an artificial neural network. The network, called ‘agent’ in DQN, consists of an input layer, three fully connected layers and an output layer. The number of neurons in the three fully connected layers is 24, 48 and 24, respectively. These layers perform computations on the input data, extracting relevant features and learning patterns from the encoded structural information. The output layer of the agent is referred to as the “action” layer. It generates a single value and each value corresponds to a policy that can be applied to update the current state (structure). More details about the actions and their corresponding policies can be found in Table 2 , which provides a mapping between the output values of the action layer and the structural modifications they represent. Then the transfer matrix method (TMM) is adopted to simulate the radiation characteristics of the new state (new structure), and obtain its emissivity according to Kirchhoff’s law. To evaluate the performance of the new structure, a reward R is obtained from the emissivity spectra. The reward serves as feedback for the agent and plays a crucial role in determining the convergence direction of the DQN model. The specific definition of the reward will depend on the desired application or emissivity target, and further details regarding the reward for TC, RC and GS will be provided later. In the DQN, a Q -Function Q ( s , a ) is defined to represent the maximum future reward for each action a at a given state s , and DQN chooses the action with the highest Q at each state. To optimize the Q -Function, a replay buffer is utilized to store historical data consisting of state-action pairs and corresponding rewards. These datasets are used to train the neural network, enabling it to learn and improve the estimation of Q -Function, namely the chosen of actions, so as to achieve higher reward. Meanwhile, Epsilon Greedy Exploration algorithm is employed to balance exploration and exploitation. The epsilon in the algorithm is a variable between 0 and 1 with the initial value of 1, indicating that actions are randomly selected. As training progresses, epsilon gradually decreases over iterations. During action selection, a random number is generated by DQN, and if this number is less than epsilon, a random action is chosen. Otherwise, the action is determined based on the Q -function. In addition to the components mentioned earlier, the dual network structure of the main network and the target network intermittently updates the target network parameters to enhance the stability of training process. More detailed description of the DQN principle can be referred to Ref. 44 . Finally, it is crucial to design an appropriate initialization method to make DQN capable for multilayer optimization. Here we introduce an iteration threshold, the function of which is to judge whether the iteration should continue. In each iteration, DQN continues to accept the state, take the action, simulate the emissivity spectra, feedback and then accept the next state. Once the reward of a new state falls below the iteration threshold, the structure will be reinitialized for the next iteration. It is important to note that the ‘train from buffer’ mechanism results in the number of simulations or the number of calculated structures is not equal to the number of iterations. In essence, the design and optimization process of DQN can be likened to playing a game. The game will continue until the mission fails, at which point it needs to be initialized and restarted. An ingenious initialization method can help achieve higher scores efficiently. Table 1 Material library of multilayer WS-EMs for TC, RC and GS. Material ID Material 1 Ge 2 ZnSe 3 Si 4 SiO 2 5 TiO 2 6 ZnS 7 Si 3 N 4 8 MgF 2 Table 2 Definitions of actions used in DQN. Action No. Action Definition 0 Decrease the material ID of Material Ⅰ by 1.(min 1) 1 Increase the material ID of Material Ⅰ by 1.(max 8) 2 Decrease the material ID of Material Ⅱ by 1.(min 1) 3 Increase the material ID of Material Ⅱ by 1.(max 8) 4 Decrease the Thickness Ⅰ by 20.(min 20) 5 Increase the Thickness Ⅰ by 20.(max 1000) 6 Decrease the Thickness Ⅱ by 20.(min 20) 7 Increase the Thickness Ⅱ by 20.(max 1000) 8 Decrease the Thickness Ⅲ by 20.(min 20) 9 Increase the Thickness Ⅲ by 20.(max 1000) 10 Decrease the Thickness Ⅳ by 20.(min 20) 11 Increase the Thickness Ⅳ by 20.(max 1000) 12 Decrease the Thickness Ⅴ by 20.(min 20) 13 Increase the Thickness Ⅴ by 20.(max 1000) Results and discussion In order to showcase the generality and effectiveness of the DQN algorithm, we design multilayer WS-EMs in the following for three applications in emissivity engineering, including TC, RC and GS, respectively, under the same optimization framework and utilizing a common material library As mentioned earlier, the reward function needs to be meticulously defined to ensure that the optimization progress in the desired direction. So firstly, for TC, since an ideal TC emitter requires low emissivity inside AM (8–14 µm) but high emissivity outside, we therefore define the reward R as the difference between the average emissivity inside and outside the AM, which can be calculated as: $$R=\frac{{\int_{5}^{8} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda +\int_{{14}}^{{20}} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda } } }}{{\int_{5}^{8} {{I_{BB}}(\lambda ,T)d\lambda +\int_{{14}}^{{20}} {{I_{BB}}(\lambda ,T)d\lambda } } }} - \frac{{\int_{8}^{{14}} {\varepsilon (\lambda ){I_{BB}}(\lambda ,T)d\lambda } }}{{\int_{8}^{{14}} {{I_{BB}}(\lambda ,T)d\lambda } }}$$ 1 where \({I_{BB}}=h{c^2}/{\lambda ^5} \cdot {[\exp (hc/\lambda {k_B}T) - 1]^{ - 1}}\) is the spectral radiance of a blackbody at wavelength λ and temperature T . h and k B are the Planck’s constant and Boltzmann constant, respectively and c is the speed of light. \(\varepsilon (\lambda )\) is the emissivity spectrum of the designed TC emitter. The temperature here is set to 350 K, which is slightly higher than the average surface temperature of armored vehicles in the military. 45 The reward R yields a value between 0 and 1 based on Eq. ( 1 ). By pre-trial, the iteration threshold is set as 0.2, that is to say, when the reward R of a state falls below this iteration during the iteration process, the iteration will be stopped, and agent will re-initialize a new state and proceed to the next iteration. In addition, the rewards R less than 0.2 are mandatorily modified to − 0.2, which signals to the agent that the states corresponding to the negative rewards do not meet the design requirements. As for the state initialization method, it is set to be randomly initialized at the beginning, namely, two materials are randomly selected from the material library, and the thickness of each layer is randomly generated within the range described above. When the reward R of a state exceeds the iteration threshold, the state with the highest historical reward is chosen as the initial structure for the next iteration. This initialization method may introduce randomness to the optimization results. To mitigate the impact of randomness, the optimization process is 5 times to obtain the optimal TC emitter structure. Each run consists of 1000 iterations, which is sufficient to reduce epsilon in the Epsilon Greedy algorithm to its minimum value. This ensures that the agent dominates the selection of actions. Once the optimization is completed, the optimal structure is experimentally fabricated using magnetron sputtering to demonstrate the feasibility of the structural optimization. The schematic of resulting optimal structure and corresponding scanning electron microscopy (SEM) image of fabricated multilayer are shown in Fig. 3 a. It can be seen that DQN finally choose ZnS and Ge as the materials for the TC emitter. The thicknesses of each layer are also presented in in Fig. 3 a, including the value of designed and the ones obtained from the SEM image of the fabricated sample. It is evident that the layer thicknesses in the optimal TC emitter are irregular and aperiodic, which is difficult to design accurately for manual optimization. However, due to the manufacturing precision, there are certain deviations between the thicknesses of fabricated sample and its designed values, resulting in the discrepancy of their corresponding emissivity spectra as depicted in Fig. 3 b. In addition, the differences between the optical properties of the sputtered materials used for fabrication and the input parameters used in the numerical simulation also make a certain impact. Nevertheless, both the designed and fabricated structures exhibit low emissivity within the AM and high emissivity outside the window. The calculated average normal emissivity in AM of simulation is 0.19, while 0.80 is obtained outside the AM, resulting in the reward value of 0.61. The excellent camouflage effect is attributed to low thermal emission in the AM (IR camera detected band) and high emission outside AM for further radiative cooling. For further verification, the normalized electric field intensities of the optimal structure at 6.65 µm and 8.93 µm are plotted in Fig. 3 c. The intensity of the electric field at 8.93 µm is degraded heavily, which means a forbidden band is formed in AM resulting in low absorption (and therefore low emissivity) in this band. While the intensity outside AM remains relatively unchanged, resulting in high emissivity for the structure with the lossy SiO 2 substrate. The emissivity of the optimal structure as a function of incident angle and wavelength is shown in Fig. 3 d, indicating the angular independence of the excellent performance. In order to demonstrate the efficiency of the optimization under the framework of DQN algorithm, we quantitatively analyze the reward R as a function of the percentage of the number of calculated structures. As shown in Fig. 4 a, DQN only calculated less than 0.2% of the all the calculated structures to obtain 70% and 90% of the maximum reward and calculated only 4.428% of the structures to find the optimal structure for TC. It can be obviously seen that, with the progress of optimization iterations, the emissivity within the AM decreases continuously, while the emissivity outside the window gradually increases, aiming to achieve a better camouflage effect. In addition, the material combinations of structures with 70% and 90% of the maximum reward are the same as optimal structure as shown in Fig. S2, which indicates that DQN is capable to select appropriate materials at a rapid pace and then performs subsequent structural optimization. The parametric distribution curves of each layer thickness are presented in Fig. 4 b, which indicated that the optimal layer thicknesses are derived from the peak of the curves. To further validate the correctness of the optimal structure, Bayesian optimization (BO) is adopted to optimize the emitter for TC under the specified material combination, namely ZnS and Ge. The histories of the rewards are shown in Fig. 4 c, which reveals that the maximum reward and the corresponding structure configuration obtained by BO are the same as those achieved by DQN. The more detailed information about BO for TC is provided in Supplementary Information Note 1. For designing a RC emitter, the objective is to maximize the emissivity within the AM, while minimizing it in the solar band so as to achieve maximum net energy power outflow. The net energy power also called cooling power, which can be denoted by $${P_{cooling}}(T)={P_{rad}}(T) - {P_{atm}}({T_{amb}}) - {P_{sun}}(\theta ) - {P_{cond+conv}}$$ 2 where \({P_{rad}}\) is the output power from the RC emitter, \({P_{atm}}\) is the input power from the atmosphere radiation, \({P_{sun}}\) is the input power from the sun and \({P_{cond+conv}}\) describes the heat exchange between the RC emitter and the environment by conduction and convection. T and T amb are the temperature of RC emitter and ambient, respectively. \(\theta\) is the angle of solar radiation. A more detailed calculation method of each power is provided in the Supplementary Information Note 2. In the following calculation, the conjugate heat transfer coefficient in \({P_{cond+conv}}\) is set as \({h_c}=5W/({m^2} \cdot K)\) and the ambient temperature is kept at to simulate a breeze situation 46 . Obviously, the greater the cooling power, the better the performance of the designed RC emitter. However, it seems not intuitive to use cooling power as reward, and it is difficult to set a suitable iteration threshold. So, the reward R is set as the difference between the steady-state temperature ( T steady ) of the RC emitter and the ambient temperature, namely the temperature drop below the T amb . If the \({P_{cooling}}\) is positive at the initial temperature \({T_{init}}\) ( \({T_{init}}\) = \({T_{amb}}\) ), the RC emitter starts to be cooled down. As the temperature of cooler decreases, the cooling power \({P_{cool}}\) also reduces until \({P_{cool}}({T_{steady}})=0\) . At that time, the RC emitter reaches an equilibrium state and the \({T_{steady}}\) can be obtained from the Eq. ( 2 ). 22 Previous studies have shown that the temperature difference ( \(\Delta T={T_{amb}} - {T_{steady}}\) ) can reach 8 ℃ or even higher 6,33,47 , so the iteration threshold is set as 5 ℃. Similar to the previous design for TC, the rewards R less than 5 will be mandatorily modified to − 5. The structure initialization method is also set to random initialization at beginning until the reward is larger than the iteration threshold, and the optimal structure is selected as the initial structure for subsequent iterations. The optimization is also implemented for 5 times with 1000 iterations each to eliminate the randomness of the optimal structures and materials. The design and optimization results of RC emitter are presented in Fig. 5 a. SiO 2 and TiO 2 are finally chosen as the materials for the optimal RC structure. The layer thickness of the optimal RC emitter also exhibits irregular and aperiodic. The emissivity spectra of the designed and fabricated structures are shown in Fig. 5 b. It can be seen that the designed RC emitter exhibits near zero emissivity in solar spectrum band, allowing it to reflect the input solar radiation energy. In contrast, a high emissivity is obtained within the AM, enabling it radiates heat efficiently to outer space. Due to the differences between the thickness of the fabricated sample and designed values, their emissivity spectra are not completely consistent. The reward R of the optimal RC emitter is 16.99, which means it can maintain 16.99 ℃ below the ambient temperature at thermal equilibrium in theory. The cooling power at the initial temperature is 132.40 W/m 2 . The equilibrium temperature difference and cooling power both exhibit the excellent performance of the designed RC emitter. The normalized electric field intensities of the optimal structure in the visible wavelength band and AM are illustrated in Fig. 5 c, indicating the strong reflection of the Ag substrate and the high emissivity caused by the electric field enhancement, respectively. Furthermore, the angular independence of emissivity spectrum can also be observed within an angle of less than 80 ° as shown in Fig. 5 d. The optimization process is quantitatively shown in Fig. 6 c. In the early stage of optimization, the reward R increases sharply, which means that DQN can quickly identify suitable materials for the RC emitter and performs optimization under this material combination until the optimization process tends to be smooth (as shown in Fig. S3). The material combination of the structure yielding 50% of maximum reward is Si/SiO 2 , which indicates that DQN replaces Si with TiO 2 to achieve better cooling performance as shown in Fig. S3a. During the smooth optimization period, the thickness of each layer is continuously optimized to further enhance the radiative cooling performance. When calculating less than 2% of the candidate structures, the RC emitter could reach a temperature drop of 14.94 ℃ below the ambient temperature at a steady state. After 1000 iterations, only 6.31% of structures need to be calculated to find the structure for RC emitter with the maximum reward. To further exhibit the details of the optimization, the parametric distribution curves of each layer thickness are shown in Fig. 6 b. In addition, except for the material combination of the optimal RC emitter, other material combinations are shown in Fig. 6 c. It can be seen that ZnS and Si 3 N 4 also exhibit potential as the materials of RC emitter, in addition to TiO 2 and SiO 2 . The occurrence of less frequent material combinations can be explained by the random initialization of the DQN and the random selection of the Epsilon Greedy Exploration algorithm in DQN. In the final part of this study, we adopt DQN to tackle a more rigorous task, that is, to achieve peak emissivity at a fixed wavelength for GS. More specifically, the target is to obtain a narrow-band emission peak with a high emissivity at the wavelength of absorption peak of the detected gas, while the emissivity at other wavelength is zero to eliminate the impact of absorption by other gases. Here, we take carbon dioxide (CO 2 ) as the target gas, which has an absorption peak at 4.26 µm. The reward R is defined as the difference between the average emissivity within and outside the narrow band: $$R={\varepsilon _t} \times Q$$ 3 where Q is used to ensure that a narrow-band emission peak can be generated in the GS WS-EMs and \({\varepsilon _t}\) is to ensure a high emissivity at target wavelength, 4.26 µm. Maximize the product of the two terms to optimize the resulting GS emitter with a narrow-band emission peak that matches the carbon dioxide absorption peak. By pre-train, the iteration threshold is set to 2. The optimization was implemented for 5 times with 1000 iterations in each round to eliminate the randomness of materials and structures. As shown in Fig. 7 a, the Si and SiO 2 are chosen as the materials of GS emitter by DQN. The emissivity spectra of the optimized structure are shown in Fig. 7 b. The simulation result shows that a sharp and high emissivity peak can be realized with the optimized structures at 4.26 µm, and the emissivity outside the narrow-band is close to zero. The corresponding emissivity of the peak is 0.9996, and the reward R of the structure is 60.62. The result shows that the designed WS-EM is sufficient to be an excellent CO 2 sensor. Due to the thickness deviation of the fabricated sample, the measured wavelength of the emissivity peak deviates from the target wavelength but still within the CO 2 absorption peak. The emission peak is located at 4.3 µm and the peak value is 0.905. In addition, the sample generates a certain low emission outside the absorption peak. Figure 7 c displays the normalized electric field intensities of the optimal structure at 4.26 µm and 5 µm. Due to the excitation of the localized Tamm plasmon state, the electric field intensity is significantly enhanced at the thickness of 0.3 µm from the top of the substrate, resulting in peak emissivity at 4.26 µm. However, there is no notable enhancement of the intensity of electric field at 5 µm, resulting in near-zero emissivity at this wavelength. The incident angle related emissivity spectrum is displayed in Fig. 7 d. It can be seen that the angle independent only occurs within 30 °, but it does not have any effect on gas sensing since the emitter typically faces the detected gas in the normal direction. The optimization process of the GS emitter is presented in Fig. 8 c. In the early stage of optimization, the emitter has only a small emissivity peak within the research band and the wavelength of emissivity peak deviates from 4.26 µm. As the iteration progresses, the more suitable material combinations can be found so that the emissivity peak becomes more obvious and the wavelength of emissivity gradually approaches the target wavelength of 4.26 µm. Eventually, a near perfect emissivity peak is achieved at 4.26 µm with a Q-factor of 60.64. Further insights into the structure evolution during the optimization process can be obtained from Fig. S5. The distribution of each layer thickness as well as the material combinations are shown in Fig. 8 b and 8 c, respectively. Compared to the emitter for RC, although a greater variety of material combinations are generated, the number is small. This indicates that the combination of Si and SiO2 is undoubtedly the most suitable choice for achieving the target emissivity spectrum for CO 2 sensing. Conclusion In summary, we present a general deep learning framework, i.e. DQN, for emissivity engineering. To demonstrate the powerfulness of the DQN algorithm, three multilayer WS-EMs are designed for typical applications, namely TC, RC and GS, which can autonomously select suitable materials from the same self-built material library for different design targets and optimize to the best structural parameters within a huge optimization space. The three design tasks are based on the same structural framework, so they can share the same material library, and can be easily converted from task to task by setting the corresponding reward function and modifying a few parameters. Therefore, DQN shows extremely remarkable efficiency and significant advantages when facing similar design tasks. Although we only present the design of 1D structures, DQN is also applicable for more complex 2D or 3D structures. Moreover, its applicability is not limited to emissivity engineering and can be further expanded to other fields. Materials and methods Simulation The reflection and transmission of the multilayer WS-EMs were calculated by transfer matrix method based on Fresnel equations. The emissivity was obtained from the corresponding reflection and transmission according to the law of conservation of energy. The code of DQN was written based on the Keras package in TensorFlow by Python. Sample Fabrication The designed multilayer WS-EM samples were all deposited by a magnetron sputter (Kurt J. LesKer-VD75). The deposition rates of SiO 2 , TiO 2 , Ag, Si, W, Ge and ZnS are 2, 3, 6, 9, 3, 5 and 11 nm/min, respectively. Optical Characterization The infrared emissivity of the multilayer WS-EM samples was measured using a Fourier transform infrared spectrometer (Nicolet iN10, Thermo Scientific). Declarations Author Contributions S.Y. and R.H. conceived the study. S.Y., W.X., Z.C. and R.H. developed the code and performed numerical simulations. P.Z., Y.D. and W.L. performed the experiment. S.Y. and R.H. wrote the manuscript and analyzed the results. J.S., W.L. and R.H. revised the manuscript. W.L. and R.H. supervised the study. All the authors provided feedback and contributed to the manuscript. Acknowledgement The authors would like to acknowledge the financial support by National Natural Science Foundation of China (52211540005, 52076087, 52161160332), the Open Project Program of Wuhan National Laboratory for Optoelectronics (2021WNLOKF004), Wuhan City Science and Technology Program (2020010601012197), Knowledge Innovation Shuguang Program. W.L. acknowledges the financial support from Key Research and Development plan of Hubei Province (Grant no. 2021BGE037). J.S. acknowledges the financial support from JSPS Bilateral Joint Research Projects (120227404). Data availability The data that support this research’s findings are available and can be provided based on the request to the corresponding authors. Conflict of interest The authors declare no competing financial interest. References Baranov, D. G. et al. A. Nanophotonic engineering of far-field thermal emitters. Nat. Mater. 18 , 920–930 (2019). Li, W. & Fan, S. Nanophotonic control of thermal radiation for energy applications [Invited]. Opt. Express, OE , 26 , 15995–16021 (2018 ) . Byrnes, S. J., Blanchard, R. & Capasso, F. Harvesting renewable energy from Earth’s mid-infrared emissions. Proceedings of the National Academy of Sciences 111 , 3927–3932 (2014). Hsu, P.-C. et al. Radiative human body cooling by nanoporous polyethylene textile. Science 353 , 1019–1023 (2016). Xu, J., Mandal, J. & Raman, A. P. Broadband directional control of thermal emission. Science 372 , 393–397 (2021). Hossain, M. M., Jia, B. & Gu, M. A metamaterial emitter for highly efficient radiative cooling. Adv. Opt. Mater. 3 , 1047–1051 (2015). Zhu, H. et al. Multispectral camouflage for infrared, visible, lasers and microwave with radiative cooling. Nat. Commun. 12 , 1805 (2021). He, M. et al. Deterministic inverse design of tamm plasmon thermal emitters with multi-resonant control. Nat. Mater. 20 , 1663–1669 (2021). Biehs, S.-A., Tschikin, M. & Ben-Abdallah, P. Hyperbolic metamaterials as an analog of a blackbody in the near field. Phys. Rev. Lett. 109 , 104301 (2012). Lenert, A. et al. Nanophotonic solar thermophotovoltaic device. Nat. Nanotech. 9 , 126–130 (2014). Hu, R. et al. Machine learning-optimized Tamm emitter for high-performance thermophotovoltaic system with detailed balance analysis. Nano Energy 72 , 104687 (2020). Yang, R. & He, Y. Optically and non-optically excited thermography for composites: A review. Infrared Phys. Techn. 75 , 26–50 (2016). Cen, Z. H. et al. Optical property study of FePt-C nanocomposite thin film for heat-assisted magnetic recording. Opt. Express , 21 , 9906 (2013) Greffet, J.-J. et al. Coherent emission of light by thermal sources. Nature 416, 61–64 (2002). Liu, B., Gong, W., Yu, B., Li, P. & Shen, S. Perfect thermal emission by nanoscale transmission line resonators. Nano Lett. 17 , 666–672 (2017). De Zoysa, M. et al. Conversion of broadband to narrowband thermal emission through energy recycling. Nat. Photon. 6 , 535–539 (2012). Ying, Y. et al. Whole LWIR directional thermal emission based on ENZ thin films. Laser & Photonics Rev. , 16 , 2200018 (2022). Yue, Y. & Gong, J. P. Tunable one-dimensional photonic crystals from soft materials. J. Photoch. Photobio. C 23 , 45–67 (2015). Pan, M. et al. Multi-band middle-infrared-compatible camouflage with thermal management via simple photonic structures. Nano Energy 69 , 104449 (2020). Kim, J., Park, C. & Hahn, J. W. Metal–semiconductor–metal metasurface for multiband infrared stealth technology using camouflage color pattern in visible range. Adv. Opt. Mater. 10 , 2101930 (2022). Deng, Z. et al. Nanostructured Ge/ZnS films for multispectral camouflage with low visibility and low thermal emission. ACS Appl. Nano Mater. 5 , 5119–5127 (2022). Sheng, C., An, Y., Du, J. & Li, X. Colored radiative cooler under optical Tamm resonance. ACS Photonics 6 , 2545–2552 (2019). Yao, K. et al. Near-perfect selective photonic crystal emitter with nanoscale layers for daytime radiative cooling. ACS Appl. Nano Mater. 2 , 5512–5519 (2019). Zhu, Y. et al. Color-preserving passive radiative cooling for an actively temperature-regulated enclosure. Light Sci. Appl. 11 , 122 (2022). Xu, H., Wu, P., Zhu, C., Elbaz, A. & Gu Z. Z. Photonic crystal for gas sensing. J. Mater. Chem. C 1 , 6087–6098 (2013). Xi, W., Liu, Y., Song, J., Hu, R. & Luo, X. High-throughput screening of a high-Q mid-infrared Tamm emitter by material informatics. Opt. Lett. 46 , 888 (2021). Yang, Z.-Y. et al. Narrowband wavelength selective thermal emitters by confined Tamm plasmon polaritons. ACS Photonics 4 , 2212–2219 (2017). Kang, Q., Li, D., Guo, K., Gao, J. & Guo, Z. Tunable thermal camouflage based on GST plasmonic metamaterial. Nanomaterials 11 , 260 (2021). Hu, R. et al. Thermal camouflaging metamaterials. Materials Today 45 , 120–141 (2021). Peng, L., Liu, D., Cheng, H., Zhou, S. & Zu, M. A multilayer film based selective thermal emitter for infrared stealth technology. Adv. Opt. Mater. , 6 , 1801006 (2018). Zhu, H. et al. High-temperature infrared camouflage with efficient thermal management. Light Sci. Appl. 9 , 60 (2020). Fan, S. & Li, W. Photonics and thermodynamics concepts in radiative cooling. Nat. Photon. 16 , 189 (2022). Raman, A. P., Anoma, M. A., Zhu, L., Rephaeli, E. & Fan, S. Passive radiative cooling below ambient air temperature under direct sunlight. Nature , 515 , 540–544 (2014). Ma, H. et al. Multilayered SiO 2 /Si 3 N 4 photonic emitter to achieve high-performance all-day radiative cooling. Sol. Energ. Mater. Sol. C. 212 , 110584 (2020). Sakurai, A. et al. Ultranarrow-band wavelength-selective thermal emission with aperiodic multilayered metamaterials designed by Bayesian optimization. ACS Cent. Sci. 5 , 319–326 (2019). Hu, R. et al. Machine-learning-optimized aperiodic superlattice minimizes coherent phonon heat conduction. Phys. Rev. X 10 , 021050 (2020). Molesky, S. et al. Inverse design in nanophotonics. Nat. Photon. 12 , 659–670 (2018). Ma, W. et al. Deep learning for the design of photonic structures. Nat. Photon. 15 , 77–90 (2021). Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518 , 529–533 (2015). Palik, E. D. Handbook of Optical Constants of Solids. Academic Press , (1998). Querry, M. R. Optical constants of minerals and other materials from the millimeter to the ultraviolet. Chemical Research, Development & Engineering Center, U.S. Army Armament Munitions Chemical Command , (1987). Siefke, T. et al. Materials pushing the application limits of wire grid polarizers further into the deep ultraviolet spectral range. Adv. Opt. Mater. 4 , 1780–1786 (2016). Yang, H. U. et al. Optical dielectric function of silver. Phys. Rev. B 91 , 235137 (2015). Hasselt, H. van, Guez, A. & Silver, D. Deep reinforcement learning with double Q-learning. Proceedings of the AAAI Conference on Artificial Intelligence 30 , (2016). Liu, Y. et al. Dynamic thermal camouflage via a liquid-crystal-based radiative metasurface. Nanophotonics 9 , 855–863 (2020). Xi, W., Liu, Y., Zhao, W., Hu, R. & Luo, X. Colored radiative cooling: How to balance color display and radiative cooling performance. Int. J. Therm. Sci. 170 , 107172 (2021). Guo, J., Ju, S., Lee, Y., Gunay, A. A. & Shiomi, J. Photonic design for color compatible radiative cooling accelerated by materials informatics. Int. J. Heat Mass Tran. 195 , 123193 (2022). Additional Declarations (Not answered) Supplementary Files SupplementaryInformation0705.docx Cite Share Download PDF Status: Published Journal Publication published 05 Dec, 2023 Read the published version in Light: Science & Applications → Version 1 posted Editorial decision: revise 21 Aug, 2023 Review # 3 received at journal 16 Aug, 2023 Reviewer # 5 agreed at journal 11 Aug, 2023 Review # 4 received at journal 10 Aug, 2023 Review # 1 received at journal 10 Aug, 2023 Reviewer # 4 agreed at journal 04 Aug, 2023 Reviewer # 3 agreed at journal 03 Aug, 2023 Review # 2 received at journal 01 Aug, 2023 Reviewer # 2 agreed at journal 24 Jul, 2023 Reviewer # 1 agreed at journal 23 Jul, 2023 Reviewers invited by journal 17 Jul, 2023 Submission checks completed at journal 12 Jul, 2023 Editor assigned by journal 04 Jul, 2023 First submitted to journal 04 Jul, 2023 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3140708","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":219264184,"identity":"cd1db663-be9f-438d-a88d-fc9f92f21ee5","order_by":0,"name":"Run Hu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuElEQVRIiWNgGAWjYBACAzBZcQDC4yFeyxmgFjaStDC2kaLFXCL3mXThvDuJ8+c3MD5428Ygb05Ii+WMdDPpmdueJW44xsBsOLeNwXBnAyGH3Uhjk+bddjhxAxsDkNHGkGBwgCgtcw4nzm9jYP9NgpaGw4kNxxjYmInTcuYZszXPscPGG44lNkvOOSdhuIGgluNpjLd5ag7Lzm8+fPDDmzIbeYK2IAHGBiAhQbz6UTAKRsEoGAW4AQCogT2WP8O/zAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0003-0274-9982","institution":"Huazhong University of Science and Technology","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Run","middleName":"","lastName":"Hu","suffix":""},{"id":219264185,"identity":"42bfd315-5a95-492f-9be3-b6398e2bdf4d","order_by":1,"name":"Shilv Yu","email":"","orcid":"","institution":"Huazhong University of Science and Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Shilv","middleName":"","lastName":"Yu","suffix":""},{"id":219264186,"identity":"12ec3869-5e94-4444-9fb8-9c2cf4d65c28","order_by":2,"name":"Xi Wang","email":"","orcid":"","institution":"Huazhong University of Science and Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Xi","middleName":"","lastName":"Wang","suffix":""},{"id":219264187,"identity":"2fbe6597-39af-444b-8526-b1aa600b560a","order_by":3,"name":"Zihe Chen","email":"","orcid":"","institution":"Huazhong University of Science and Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zihe","middleName":"","lastName":"Chen","suffix":""},{"id":219264188,"identity":"a0c7298f-c9a2-4849-a5fb-10a547ec0393","order_by":4,"name":"Peng Zhou","email":"","orcid":"","institution":"Wuhan University of Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Peng","middleName":"","lastName":"Zhou","suffix":""},{"id":219264189,"identity":"65fe24d5-d8b6-4f60-9b6f-afc8d9f8a723","order_by":5,"name":"Yuheng Deng","email":"","orcid":"","institution":"Hubei University of Arts and Science","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yuheng","middleName":"","lastName":"Deng","suffix":""},{"id":219264190,"identity":"0e7d7d85-77ba-4ecc-bdfe-5b79388958de","order_by":6,"name":"Wangnan li","email":"","orcid":"https://orcid.org/0000-0003-3558-6956","institution":"Hubei University of Arts and Science","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Wangnan","middleName":"","lastName":"li","suffix":""},{"id":219264191,"identity":"eaebe276-9b03-421f-a8e6-8a9882a2c273","order_by":7,"name":"Junichiro Shiomi","email":"","orcid":"","institution":"The University of Tokyo","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Junichiro","middleName":"","lastName":"Shiomi","suffix":""}],"badges":[],"createdAt":"2023-07-05 02:30:43","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3140708/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3140708/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41377-023-01341-w","type":"published","date":"2023-12-05T05:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":40341976,"identity":"83562315-97ba-4795-a9ab-b95db61f2b13","added_by":"auto","created_at":"2023-07-20 22:18:20","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":571360,"visible":true,"origin":"","legend":"\u003cp\u003eEmissivity engineering of multilayer WS-EMs designed and optimized by Deep Q-learning network (DQN) for radiative cooling, thermal camouflage, and gas sensing, respectively. The schematic for the emissivity requirement for different applications are included. The basic elements for the DQN network are also illustrated.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/de49dce550a0eea6ac08ef36.png"},{"id":40342438,"identity":"31599f07-383e-4004-b7ab-8581ffd17c9d","added_by":"auto","created_at":"2023-07-20 22:42:20","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":544642,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic for the multilayer structure and DQN model. (a) Five-layer multilayer structure composed of two alternating materials. (b) Schematic of the DQN model. The state consists of two materials and five layers of thickness of the multilayer, then the state parameters are fed into the DQN to generate an Action. Then take the action to update the state. Transfer matrix method (TMM) is adopted to simulate the new state, and reward is obtained to feed back to neural network (agent). The new state is fed into the DQN for next iteration. Each pair of state, action and reward is recorded as dataset to train the neural network, so that it can take the action that increases the reward and finally get the corresponding state with the maximum reward.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/389643a68ed163b9c35252bc.png"},{"id":40341972,"identity":"e2665ee2-411d-40fb-906c-8a3160817622","added_by":"auto","created_at":"2023-07-20 22:18:20","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":718713,"visible":true,"origin":"","legend":"\u003cp\u003eThe result of TC emitter designed by DQN. (a) Schematic and SEM images of the optimal TC structure. ZnS and Ge are chosen as the material and the layer thicknesses of simulation and experiment are presented. (b) Emissivity spectrum of the optimal TC emitter. (c) The normalized electric field intensity for optimal TC emitter at various wavelengths (\u003cem\u003eλ \u003c/em\u003e= 6.65, 8.93 μm). (d) The emissivity as the function of incident angle and wavelength.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/c49ff20daac97d4fe1a07454.png"},{"id":40342020,"identity":"1c48f87a-7d59-4807-8613-1645f7d014f2","added_by":"auto","created_at":"2023-07-20 22:26:20","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":619737,"visible":true,"origin":"","legend":"\u003cp\u003eThe optimization process for TC. (a) Maximum of reward \u003cem\u003eR\u003c/em\u003e as a function of the percentage of calculated structures for TC by DQN. (b) The parametric distribution curves of each layer thickness. (c) Maximum of reward \u003cem\u003eR\u003c/em\u003e as a function of the percentage of calculated structures for TC by BO\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/fe6e32a6e67f9fd5ec543d81.png"},{"id":40341971,"identity":"2333d17d-a1cd-42c8-b37a-2364017a4c50","added_by":"auto","created_at":"2023-07-20 22:18:20","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":691903,"visible":true,"origin":"","legend":"\u003cp\u003eOptimization result of RC performed by DQN. (a) Schematic and the SEM image of the optimal RC structure. TiO\u003csub\u003e2\u003c/sub\u003e and SiO\u003csub\u003e2\u003c/sub\u003e are chosen as the materials and the layer thicknesses of simulation and experiment are presented. (b) Emissivity spectrum of the RC emitter. (c) The normalized electric field intensity for optimal emitter at various wavelengths (\u003cem\u003eλ \u003c/em\u003e= 0.5, 8.5 μm). (d) The emissivity as the function of incident angle and wavelength.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/9c77d4d6350926ddbc202232.png"},{"id":40342021,"identity":"795d7fe2-722c-4690-b15e-d46c89bbb8a8","added_by":"auto","created_at":"2023-07-20 22:26:20","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":533175,"visible":true,"origin":"","legend":"\u003cp\u003eThe optimization process for RC performed by DQN. (a) Maximum of reward \u003cem\u003eR\u003c/em\u003e as a function of the percentage of calculated structures for RC. (b) The parametric distribution curves of each layer thickness. (c) The distribution of material combinations except SiO\u003csub\u003e2\u003c/sub\u003e/TiO\u003csub\u003e2\u003c/sub\u003e.\u0026nbsp;\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/6cad8de54faf8cca75e1de77.png"},{"id":40342186,"identity":"b3237836-e6af-4b5c-8d91-45805261111e","added_by":"auto","created_at":"2023-07-20 22:34:20","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":620576,"visible":true,"origin":"","legend":"\u003cp\u003eOptimization result of GS performed by DQN. (a) Schematic and the SEM image of the optimal GS structure. Si and SiO\u003csub\u003e2\u003c/sub\u003e are chosen as the materials and the layer thicknesses of simulation and experiment are presented. (b) Emissivity spectrum of the GS emitter. (c) The normalized electric field intensity for optimal emitter at various wavelengths (\u003cem\u003eλ \u003c/em\u003e= 4.26, 5 μm). (d) The emissivity as the function of incident angle and wavelength.\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/f2a84b523c245b36753f6a7e.png"},{"id":40341975,"identity":"5afc1769-f679-465f-ad2c-ca0d1c12d56b","added_by":"auto","created_at":"2023-07-20 22:18:20","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":540661,"visible":true,"origin":"","legend":"\u003cp\u003eThe optimization process for GS performed by DQN. (a) Maximum of reward \u003cem\u003eR\u003c/em\u003e as a function of the percentage of calculated structures for GS. (b) The parametric distribution curves of each layer thickness. (c) The distribution of material combinations except Si/SiO\u003csub\u003e2\u003c/sub\u003e.\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/76fcc57bc0ac85819dc8bdc1.png"},{"id":47688946,"identity":"a463f8cb-58f4-4102-8b68-978979250501","added_by":"auto","created_at":"2023-12-06 08:39:49","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1267140,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/206df8e6-fa76-41e6-9a0e-3b2254f79418.pdf"},{"id":40342022,"identity":"8529c71a-9e34-4b76-9c21-2be3f2ee06e5","added_by":"auto","created_at":"2023-07-20 22:26:20","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":507738,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"SupplementaryInformation0705.docx","url":"https://assets-eu.researchsquare.com/files/rs-3140708/v1/0f9ef96d3f1ff1726b0af37e.docx"}],"financialInterests":"(Not answered)","formattedTitle":"General Deep Learning Framework for Emissivity Engineering","fulltext":[{"header":"Introduction","content":"\u003cp\u003eAll objects in nature emit thermal radiation outwardly at anytime and anywhere in a broadband, non-selective, incoherent, diffusive, and reciprocal manner.\u003csup\u003e1,2\u003c/sup\u003e Thanks to the fast development of thermal metamaterials and metasurfaces in recent years, thermal radiation has been demonstrated to be engineered with comprehensive control of spectral, directional, and dynamic characteristics, enabling higher-efficiency regulation of radiative heat transfer from natural objects. Among them, the spectral emissivity engineering of thermal radiation enables more applications, such as energy harvesting,\u003csup\u003e3\u003c/sup\u003e thermal management,\u003csup\u003e4\u0026ndash;5\u003c/sup\u003e radiative cooling,\u003csup\u003e6\u003c/sup\u003e thermal camouflage,\u003csup\u003e7\u003c/sup\u003e infrared (IR) sensing,\u003csup\u003e8\u003c/sup\u003e far-/near-field radiation control,\u003csup\u003e9\u003c/sup\u003e thermophotovoltaics,\u003csup\u003e10\u0026ndash;11\u003c/sup\u003e thermography,\u003csup\u003e12\u003c/sup\u003e heat-assisted magnetic recording,\u003csup\u003e13\u003c/sup\u003e etc. The emissivity engineering aims to select materials and design nanostructures to achieve specific functionalities with a target emissivity spectrum. The common physics of selective emissivity comes from the excitation of different photon modes, which leads to the local enhancement or suppression of the internal electric field, thus allowing for control over the radiation emission at different wavelengths. Wavelength-selective thermal emitters (WS-EMs), as the main platform for emissivity engineering, can be designed by multilayers, photonic crystals, nano-grating,\u003csup\u003e14\u003c/sup\u003e nano antennas arrays,\u003csup\u003e15\u003c/sup\u003e multiple-quantum-well,\u003csup\u003e16\u003c/sup\u003e Fabry-Perot cavities,\u003csup\u003e17\u003c/sup\u003e etc. As one of the simplest structures of WS-EMs, multilayers are frequently employed which are composed of alternating layers of materials with different refractive indices\u003csup\u003e18\u003c/sup\u003e, and relatively easy for fabrication at a low cost for extensive applications in thermal camouflage (TC),\u003csup\u003e19\u0026ndash;21\u003c/sup\u003e radiative cooling (RC),\u003csup\u003e22\u0026ndash;24\u003c/sup\u003e gas sensing (GS),\u003csup\u003e25\u0026ndash;27\u003c/sup\u003e etc.\u003c/p\u003e \u003cp\u003eIn general, different applications require distinct emissivity spectra as illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. For instance, TC necessitates low emissivity within the long-wavelength IR range (8\u0026ndash;14 \u0026micro;m), which is call as atmosphere window (AM) due to its high transmittance, to prevent detection by the most IR detectors when the background temperature is low. Additionally, it is advantageous for the emissivity outside the AM to remain as high as possible to facilitate further radiative heat dissipation.\u003csup\u003e28,29\u003c/sup\u003e To achieve TC, Peng et al. designed a silver/germanium (Ag/Ge) multilayered structure, where impedance matching is utilized to manipulate the radiation characteristics.\u003csup\u003e30\u003c/sup\u003e Zhu et al. designed a Ge/ZnS multilayer on a silica aerogel substrate with efficient radiative cooling capability for TC in high ambient temperature.\u003csup\u003e31\u003c/sup\u003e In contrast, RC aims to achieve passive cooling by radiating the heat directly to the outer space at ~\u0026thinsp;3 K via the high emissivity within the AM. In addition, a high reflectivity in the solar band is necessarily required to reflect as much solar energy as possible to maximize the cooling power, ultimately achieving net energy outflow and reducing object temperature.\u003csup\u003e32\u003c/sup\u003e Raman et al. adopted needle optimization method to design a seven-layer HfO\u003csub\u003e2\u003c/sub\u003e/SiO\u003csub\u003e2\u003c/sub\u003e emitter. The fabricated multilayer emitter achieved daytime RC under direct solar irradiance for the first time, which reflected 97% of solar irradiance and cooled to 4.9 ℃ below the ambient temperature.\u003csup\u003e33\u003c/sup\u003e Similarly, Ma et al. optimized seven-layer SiO\u003csub\u003e2\u003c/sub\u003e/Si\u003csub\u003e3\u003c/sub\u003eN\u003csub\u003e4\u003c/sub\u003e emitter using evolutionary algorithm, and the emitter was highly reflective towards solar radiation and had a broadband high emissivity within the AM.\u003csup\u003e34\u003c/sup\u003e Different from the broadband emissivity spectra for RC and TC, the emissivity spectrum for GS needs narrow-band peaks which match the absorption peaks of the detected gas. Sakurai et al.\u003csup\u003e35\u003c/sup\u003e and Xi et al.\u003csup\u003e26\u003c/sup\u003e both utilized machine-learning to design and optimize multilayered structures and achieved ultra-narrowband emission peaks at multiple wavelengths. In particular, Xi et al.\u003csup\u003e26\u003c/sup\u003e obtained the whole database of multilayered Ge/SiO\u003csub\u003e2\u003c/sub\u003e WS-EMs for narrowband emissivity spectra in the wavelength range of 3 to 10 \u0026micro;m, and the highest \u003cem\u003eQ\u003c/em\u003e-factor reaches 508 far beyond the \u003cem\u003eQ\u003c/em\u003e-factor record in the literature.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAs far as we are concerned from the literature, although many combinations of materials of multilayer WS-EMs have been proposed for emissivity engineering, both material selection and structural design still rely on physics-inspired methods and past design experience or guidelines. They either fix materials to design structures, or fix structures to select proper materials, which are inefficient and difficult to achieve optimal design. To further improve the performance of multilayer WS-EMs, machine-learning optimization algorithms have shown unique advantages in structure optimization and designing problems.\u003csup\u003e36\u0026ndash;38\u003c/sup\u003e However, designers still have to conduct extensive searches of existing work to determine suitable materials and initial structural parameters for the design goals before optimization\u003csup\u003e26,34,35\u003c/sup\u003e. This is attributed to the enormous optimization space that arises when considering both material selection and structure parameters optimization simultaneously, leading to a significant increase in computing resources and inefficiency of machine learning algorithm. Hence, one open question comes that whether the materials selection and structures optimization can be performed synchronously for emissivity engineering across different applications?\u003c/p\u003e \u003cp\u003eRecently, in light of the exceptional success in domains related to computer science and engineering, including natural language processing, computer vision, image processing, speech recognition, etc., deep learning has attracted increasing attention, especially for material structure optimization\u003csup\u003e38\u003c/sup\u003e. Through establishing the artificial neural network and the data-driven method, deep learning obtains the mapping relationship between data pairs. Among the multitudinous neural network, deep Q-learning network (DQN), a reinforcement learning (RL) method based on deep learning (DL), can effectively make optimal decisions from millions of choices by learning from the history experiences, thus it is particularly well-adapted to address the large optimization space problem. In addition, as a kind of RL method, DQN is policy based, which means that the dataset used for neural network training does not need to be collected beforehand, but is derived from the accumulation of historical policies and corresponding responses. Through combining the RL and DL, DQN has been proved to be extraordinarily competent for Atari games, and can reach the level of human experts in games such as Go and Chess, because it is able to formulate the best next action based on the current state of the game to achieve the highest game score.\u003csup\u003e39\u003c/sup\u003e\u003c/p\u003e \u003cp\u003eIn this study, DQN is adopted to achieve a general framework for emissivity engineering with high accuracy and efficiency. Firstly, a framework of DQN is established and the initialization method in DQN is improved to be capable of automatic design and optimization of multilayer WS-EMs. Subsequently, a commonly used material library for WS-EMs is set up for DQN to select materials according to different applications. After that, the multilayer WS-EMs for three applications including TC, RC, and GS, are designed and optimized, which are then experimentally validated. The selection of materials and the design of the structure are independently completed by DQN within the extensive optimization space. The designed multilayer WS-EMs all exhibit extremely exceptional performance in these three applications, validating DQN as a general deep learning framework for emissivity engineering.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e \u003c/p\u003e \u003cp\u003eThe roadmap of optimization process of DQN is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. The whole optimization process can be described as an interactive progress with the environment. The state of the environment, which consists of the material ID number and the thicknesses of each layer, represents the materials and structural parameters of the current multilayer. Here we set up the multilayer WS-EMs as a 5-layer structure composed of alternating two materials. It is worth mentioning that while increasing the number of layers can meet more rigorous emissivity spectrum requirements, it also significantly expands the optimization space by several orders of magnitude, requiring greater computing power and longer design time. Consequently, the state can be represented by a 1\u0026times;7 vector containing material and structure information. The two materials are selected from the self-built material library, as shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, which contains 8 commonly used materials for emissivity engineering. Their optical properties (refractive index) are referred to E. Palik\u0026rsquo;s and Qurry\u0026rsquo;s books\u003csup\u003e40,41\u003c/sup\u003e and other research work\u003csup\u003e42,43\u003c/sup\u003e(See Supplementary Information S1). Regarding the substrate material, it needs to be selected according to specific design goals, we chose silver for RC, silicon for TC, and tungsten for GS. Each layer thickness is varied within the range of 20 nm\u0026ndash;1000 nm with a uniform step size of 20 nm, which results in a total of 50 possible steps for each layer. Considering the 8 materials available, the above structural configuration leads to 8\u0026times;7\u0026times;50\u003csup\u003e5\u003c/sup\u003e= 1.75\u0026times;10\u003csup\u003e10\u003c/sup\u003e potential candidate structures. The sheer volume of candidate structure necessitates enormous computility, making it nearly impossible through manual design and inefficient for conventional machine learning algorithms. After the physical information of the multilayer structure is encoded into digital information, it is inputted into an artificial neural network. The network, called \u0026lsquo;agent\u0026rsquo; in DQN, consists of an input layer, three fully connected layers and an output layer. The number of neurons in the three fully connected layers is 24, 48 and 24, respectively. These layers perform computations on the input data, extracting relevant features and learning patterns from the encoded structural information. The output layer of the agent is referred to as the \u0026ldquo;action\u0026rdquo; layer. It generates a single value and each value corresponds to a policy that can be applied to update the current state (structure). More details about the actions and their corresponding policies can be found in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, which provides a mapping between the output values of the action layer and the structural modifications they represent. Then the transfer matrix method (TMM) is adopted to simulate the radiation characteristics of the new state (new structure), and obtain its emissivity according to Kirchhoff\u0026rsquo;s law. To evaluate the performance of the new structure, a reward \u003cem\u003eR\u003c/em\u003e is obtained from the emissivity spectra. The reward serves as feedback for the agent and plays a crucial role in determining the convergence direction of the DQN model. The specific definition of the reward will depend on the desired application or emissivity target, and further details regarding the reward for TC, RC and GS will be provided later. In the DQN, a \u003cem\u003eQ\u003c/em\u003e-Function \u003cem\u003eQ\u003c/em\u003e (\u003cem\u003es\u003c/em\u003e, \u003cem\u003ea\u003c/em\u003e) is defined to represent the maximum future reward for each action \u003cem\u003ea\u003c/em\u003e at a given state \u003cem\u003es\u003c/em\u003e, and DQN chooses the action with the highest \u003cem\u003eQ\u003c/em\u003e at each state. To optimize the \u003cem\u003eQ\u003c/em\u003e-Function, a replay buffer is utilized to store historical data consisting of state-action pairs and corresponding rewards. These datasets are used to train the neural network, enabling it to learn and improve the estimation of \u003cem\u003eQ\u003c/em\u003e-Function, namely the chosen of actions, so as to achieve higher reward. Meanwhile, Epsilon Greedy Exploration algorithm is employed to balance exploration and exploitation. The epsilon in the algorithm is a variable between 0 and 1 with the initial value of 1, indicating that actions are randomly selected. As training progresses, epsilon gradually decreases over iterations. During action selection, a random number is generated by DQN, and if this number is less than epsilon, a random action is chosen. Otherwise, the action is determined based on the \u003cem\u003eQ\u003c/em\u003e-function. In addition to the components mentioned earlier, the dual network structure of the main network and the target network intermittently updates the target network parameters to enhance the stability of training process. More detailed description of the DQN principle can be referred to Ref.\u003csup\u003e44\u003c/sup\u003e. Finally, it is crucial to design an appropriate initialization method to make DQN capable for multilayer optimization. Here we introduce an iteration threshold, the function of which is to judge whether the iteration should continue. In each iteration, DQN continues to accept the state, take the action, simulate the emissivity spectra, feedback and then accept the next state. Once the reward of a new state falls below the iteration threshold, the structure will be reinitialized for the next iteration. It is important to note that the \u0026lsquo;train from buffer\u0026rsquo; mechanism results in the number of simulations or the number of calculated structures is not equal to the number of iterations. In essence, the design and optimization process of DQN can be likened to playing a game. The game will continue until the mission fails, at which point it needs to be initialized and restarted. An ingenious initialization method can help achieve higher scores efficiently.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eMaterial library of multilayer WS-EMs for TC, RC and GS.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaterial ID\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMaterial\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGe\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eZnSe\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSi\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSiO\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTiO\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eZnS\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSi\u003csub\u003e3\u003c/sub\u003eN\u003csub\u003e4\u003c/sub\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMgF\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDefinitions of actions used in DQN.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAction No.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAction Definition\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the material ID of Material Ⅰ by 1.(min 1)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the material ID of Material Ⅰ by 1.(max 8)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the material ID of Material Ⅱ by 1.(min 1)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the material ID of Material Ⅱ by 1.(max 8)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the Thickness Ⅰ by 20.(min 20)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the Thickness Ⅰ by 20.(max 1000)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the Thickness Ⅱ by 20.(min 20)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the Thickness Ⅱ by 20.(max 1000)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the Thickness Ⅲ by 20.(min 20)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the Thickness Ⅲ by 20.(max 1000)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the Thickness Ⅳ by 20.(min 20)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e11\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the Thickness Ⅳ by 20.(max 1000)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDecrease the Thickness Ⅴ by 20.(min 20)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eIncrease the Thickness Ⅴ by 20.(max 1000)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Results and discussion","content":"\u003cp\u003eIn order to showcase the generality and effectiveness of the DQN algorithm, we design multilayer WS-EMs in the following for three applications in emissivity engineering, including TC, RC and GS, respectively, under the same optimization framework and utilizing a common material library\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAs mentioned earlier, the reward function needs to be meticulously defined to ensure that the optimization progress in the desired direction. So firstly, for TC, since an ideal TC emitter requires low emissivity inside AM (8\u0026ndash;14 \u0026micro;m) but high emissivity outside, we therefore define the reward \u003cem\u003eR\u003c/em\u003e as the difference between the average emissivity inside and outside the AM, which can be calculated as:\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$R=\\frac{{\\int_{5}^{8} {\\varepsilon (\\lambda ){I_{BB}}(\\lambda ,T)d\\lambda +\\int_{{14}}^{{20}} {\\varepsilon (\\lambda ){I_{BB}}(\\lambda ,T)d\\lambda } } }}{{\\int_{5}^{8} {{I_{BB}}(\\lambda ,T)d\\lambda +\\int_{{14}}^{{20}} {{I_{BB}}(\\lambda ,T)d\\lambda } } }} - \\frac{{\\int_{8}^{{14}} {\\varepsilon (\\lambda ){I_{BB}}(\\lambda ,T)d\\lambda } }}{{\\int_{8}^{{14}} {{I_{BB}}(\\lambda ,T)d\\lambda } }}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({I_{BB}}=h{c^2}/{\\lambda ^5} \\cdot {[\\exp (hc/\\lambda {k_B}T) - 1]^{ - 1}}\\)\u003c/span\u003e\u003c/span\u003e is the spectral radiance of a blackbody at wavelength \u003cem\u003eλ\u003c/em\u003e and temperature \u003cem\u003eT\u003c/em\u003e. \u003cem\u003eh\u003c/em\u003e and \u003cem\u003ek\u003c/em\u003e\u003csub\u003e\u003cem\u003eB\u003c/em\u003e\u003c/sub\u003e are the Planck\u0026rsquo;s constant and Boltzmann constant, respectively and \u003cem\u003ec\u003c/em\u003e is the speed of light. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\varepsilon (\\lambda )\\)\u003c/span\u003e\u003c/span\u003e is the emissivity spectrum of the designed TC emitter. The temperature here is set to 350 K, which is slightly higher than the average surface temperature of armored vehicles in the military.\u003csup\u003e45\u003c/sup\u003e The reward \u003cem\u003eR\u003c/em\u003e yields a value between 0 and 1 based on Eq.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). By pre-trial, the iteration threshold is set as 0.2, that is to say, when the reward \u003cem\u003eR\u003c/em\u003e of a state falls below this iteration during the iteration process, the iteration will be stopped, and agent will re-initialize a new state and proceed to the next iteration. In addition, the rewards \u003cem\u003eR\u003c/em\u003e less than 0.2 are mandatorily modified to \u0026minus;\u0026thinsp;0.2, which signals to the agent that the states corresponding to the negative rewards do not meet the design requirements. As for the state initialization method, it is set to be randomly initialized at the beginning, namely, two materials are randomly selected from the material library, and the thickness of each layer is randomly generated within the range described above. When the reward \u003cem\u003eR\u003c/em\u003e of a state exceeds the iteration threshold, the state with the highest historical reward is chosen as the initial structure for the next iteration. This initialization method may introduce randomness to the optimization results. To mitigate the impact of randomness, the optimization process is 5 times to obtain the optimal TC emitter structure. Each run consists of 1000 iterations, which is sufficient to reduce epsilon in the Epsilon Greedy algorithm to its minimum value. This ensures that the agent dominates the selection of actions. Once the optimization is completed, the optimal structure is experimentally fabricated using magnetron sputtering to demonstrate the feasibility of the structural optimization.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe schematic of resulting optimal structure and corresponding scanning electron microscopy (SEM) image of fabricated multilayer are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea. It can be seen that DQN finally choose ZnS and Ge as the materials for the TC emitter. The thicknesses of each layer are also presented in in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea, including the value of designed and the ones obtained from the SEM image of the fabricated sample. It is evident that the layer thicknesses in the optimal TC emitter are irregular and aperiodic, which is difficult to design accurately for manual optimization. However, due to the manufacturing precision, there are certain deviations between the thicknesses of fabricated sample and its designed values, resulting in the discrepancy of their corresponding emissivity spectra as depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb. In addition, the differences between the optical properties of the sputtered materials used for fabrication and the input parameters used in the numerical simulation also make a certain impact. Nevertheless, both the designed and fabricated structures exhibit low emissivity within the AM and high emissivity outside the window. The calculated average normal emissivity in AM of simulation is 0.19, while 0.80 is obtained outside the AM, resulting in the reward value of 0.61. The excellent camouflage effect is attributed to low thermal emission in the AM (IR camera detected band) and high emission outside AM for further radiative cooling. For further verification, the normalized electric field intensities of the optimal structure at 6.65 \u0026micro;m and 8.93 \u0026micro;m are plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec. The intensity of the electric field at 8.93 \u0026micro;m is degraded heavily, which means a forbidden band is formed in AM resulting in low absorption (and therefore low emissivity) in this band. While the intensity outside AM remains relatively unchanged, resulting in high emissivity for the structure with the lossy SiO\u003csub\u003e2\u003c/sub\u003e substrate. The emissivity of the optimal structure as a function of incident angle and wavelength is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed, indicating the angular independence of the excellent performance.\u003c/p\u003e \u003cp\u003eIn order to demonstrate the efficiency of the optimization under the framework of DQN algorithm, we quantitatively analyze the reward \u003cem\u003eR\u003c/em\u003e as a function of the percentage of the number of calculated structures. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea, DQN only calculated less than 0.2% of the all the calculated structures to obtain 70% and 90% of the maximum reward and calculated only 4.428% of the structures to find the optimal structure for TC. It can be obviously seen that, with the progress of optimization iterations, the emissivity within the AM decreases continuously, while the emissivity outside the window gradually increases, aiming to achieve a better camouflage effect. In addition, the material combinations of structures with 70% and 90% of the maximum reward are the same as optimal structure as shown in Fig. S2, which indicates that DQN is capable to select appropriate materials at a rapid pace and then performs subsequent structural optimization. The parametric distribution curves of each layer thickness are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb, which indicated that the optimal layer thicknesses are derived from the peak of the curves. To further validate the correctness of the optimal structure, Bayesian optimization (BO) is adopted to optimize the emitter for TC under the specified material combination, namely ZnS and Ge. The histories of the rewards are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec, which reveals that the maximum reward and the corresponding structure configuration obtained by BO are the same as those achieved by DQN. The more detailed information about BO for TC is provided in Supplementary Information Note 1.\u003c/p\u003e \u003cp\u003eFor designing a RC emitter, the objective is to maximize the emissivity within the AM, while minimizing it in the solar band so as to achieve maximum net energy power outflow. The net energy power also called cooling power, which can be denoted by\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$${P_{cooling}}(T)={P_{rad}}(T) - {P_{atm}}({T_{amb}}) - {P_{sun}}(\\theta ) - {P_{cond+conv}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{rad}}\\)\u003c/span\u003e\u003c/span\u003e is the output power from the RC emitter, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{atm}}\\)\u003c/span\u003e\u003c/span\u003e is the input power from the atmosphere radiation, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{sun}}\\)\u003c/span\u003e\u003c/span\u003e is the input power from the sun and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{cond+conv}}\\)\u003c/span\u003e\u003c/span\u003edescribes the heat exchange between the RC emitter and the environment by conduction and convection. \u003cem\u003eT\u003c/em\u003e and \u003cem\u003eT\u003c/em\u003e\u003csub\u003eamb\u003c/sub\u003e are the temperature of RC emitter and ambient, respectively. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\theta\\)\u003c/span\u003e\u003c/span\u003e is the angle of solar radiation. A more detailed calculation method of each power is provided in the Supplementary Information Note 2. In the following calculation, the conjugate heat transfer coefficient in \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{cond+conv}}\\)\u003c/span\u003e\u003c/span\u003e is set as \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({h_c}=5W/({m^2} \\cdot K)\\)\u003c/span\u003e\u003c/span\u003e and the ambient temperature is kept at \u003cimg src=\"https://myfiles.space/user_files/122228_c8a1650c59388082/122228_custom_files/img1689885791.png\"\u003eto simulate a breeze situation\u003csup\u003e46\u003c/sup\u003e. Obviously, the greater the cooling power, the better the performance of the designed RC emitter. However, it seems not intuitive to use cooling power as reward, and it is difficult to set a suitable iteration threshold. So, the reward \u003cem\u003eR\u003c/em\u003e is set as the difference between the steady-state temperature (\u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003esteady\u003c/em\u003e\u003c/sub\u003e) of the RC emitter and the ambient temperature, namely the temperature drop below the \u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003eamb\u003c/em\u003e\u003c/sub\u003e. If the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{cooling}}\\)\u003c/span\u003e\u003c/span\u003e is positive at the initial temperature \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T_{init}}\\)\u003c/span\u003e\u003c/span\u003e (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T_{init}}\\)\u003c/span\u003e\u003c/span\u003e=\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T_{amb}}\\)\u003c/span\u003e\u003c/span\u003e), the RC emitter starts to be cooled down. As the temperature of cooler decreases, the cooling power \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{cool}}\\)\u003c/span\u003e\u003c/span\u003e also reduces until \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({P_{cool}}({T_{steady}})=0\\)\u003c/span\u003e\u003c/span\u003e. At that time, the RC emitter reaches an equilibrium state and the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({T_{steady}}\\)\u003c/span\u003e\u003c/span\u003e can be obtained from the Eq.\u0026nbsp;(\u003cspan refid=\"Equ2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003csup\u003e22\u003c/sup\u003e Previous studies have shown that the temperature difference (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\Delta T={T_{amb}} - {T_{steady}}\\)\u003c/span\u003e\u003c/span\u003e) can reach 8 ℃ or even higher\u003csup\u003e6,33,47\u003c/sup\u003e, so the iteration threshold is set as 5 ℃. Similar to the previous design for TC, the rewards \u003cem\u003eR\u003c/em\u003e less than 5 will be mandatorily modified to \u0026minus;\u0026thinsp;5. The structure initialization method is also set to random initialization at beginning until the reward is larger than the iteration threshold, and the optimal structure is selected as the initial structure for subsequent iterations. The optimization is also implemented for 5 times with 1000 iterations each to eliminate the randomness of the optimal structures and materials.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe design and optimization results of RC emitter are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea. SiO\u003csub\u003e2\u003c/sub\u003e and TiO\u003csub\u003e2\u003c/sub\u003e are finally chosen as the materials for the optimal RC structure. The layer thickness of the optimal RC emitter also exhibits irregular and aperiodic. The emissivity spectra of the designed and fabricated structures are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb. It can be seen that the designed RC emitter exhibits near zero emissivity in solar spectrum band, allowing it to reflect the input solar radiation energy. In contrast, a high emissivity is obtained within the AM, enabling it radiates heat efficiently to outer space. Due to the differences between the thickness of the fabricated sample and designed values, their emissivity spectra are not completely consistent. The reward \u003cem\u003eR\u003c/em\u003e of the optimal RC emitter is 16.99, which means it can maintain 16.99 ℃ below the ambient temperature at thermal equilibrium in theory. The cooling power at the initial temperature is 132.40 W/m\u003csup\u003e2\u003c/sup\u003e. The equilibrium temperature difference and cooling power both exhibit the excellent performance of the designed RC emitter. The normalized electric field intensities of the optimal structure in the visible wavelength band and AM are illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ec, indicating the strong reflection of the Ag substrate and the high emissivity caused by the electric field enhancement, respectively. Furthermore, the angular independence of emissivity spectrum can also be observed within an angle of less than 80 \u0026deg; as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ed.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe optimization process is quantitatively shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec. In the early stage of optimization, the reward \u003cem\u003eR\u003c/em\u003e increases sharply, which means that DQN can quickly identify suitable materials for the RC emitter and performs optimization under this material combination until the optimization process tends to be smooth (as shown in Fig. S3). The material combination of the structure yielding 50% of maximum reward is Si/SiO\u003csub\u003e2\u003c/sub\u003e, which indicates that DQN replaces Si with TiO\u003csub\u003e2\u003c/sub\u003e to achieve better cooling performance as shown in Fig. S3a. During the smooth optimization period, the thickness of each layer is continuously optimized to further enhance the radiative cooling performance. When calculating less than 2% of the candidate structures, the RC emitter could reach a temperature drop of 14.94 ℃ below the ambient temperature at a steady state. After 1000 iterations, only 6.31% of structures need to be calculated to find the structure for RC emitter with the maximum reward. To further exhibit the details of the optimization, the parametric distribution curves of each layer thickness are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003eb. In addition, except for the material combination of the optimal RC emitter, other material combinations are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec. It can be seen that ZnS and Si\u003csub\u003e3\u003c/sub\u003eN\u003csub\u003e4\u003c/sub\u003e also exhibit potential as the materials of RC emitter, in addition to TiO\u003csub\u003e2\u003c/sub\u003e and SiO\u003csub\u003e2\u003c/sub\u003e. The occurrence of less frequent material combinations can be explained by the random initialization of the DQN and the random selection of the Epsilon Greedy Exploration algorithm in DQN.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eIn the final part of this study, we adopt DQN to tackle a more rigorous task, that is, to achieve peak emissivity at a fixed wavelength for GS. More specifically, the target is to obtain a narrow-band emission peak with a high emissivity at the wavelength of absorption peak of the detected gas, while the emissivity at other wavelength is zero to eliminate the impact of absorption by other gases. Here, we take carbon dioxide (CO\u003csub\u003e2\u003c/sub\u003e) as the target gas, which has an absorption peak at 4.26 \u0026micro;m. The reward R is defined as the difference between the average emissivity within and outside the narrow band:\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$R={\\varepsilon _t} \\times Q$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cem\u003eQ\u003c/em\u003e is used to ensure that a narrow-band emission peak can be generated in the GS WS-EMs and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varepsilon _t}\\)\u003c/span\u003e\u003c/span\u003eis to ensure a high emissivity at target wavelength, 4.26 \u0026micro;m. Maximize the product of the two terms to optimize the resulting GS emitter with a narrow-band emission peak that matches the carbon dioxide absorption peak. By pre-train, the iteration threshold is set to 2. The optimization was implemented for 5 times with 1000 iterations in each round to eliminate the randomness of materials and structures.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003ea, the Si and SiO\u003csub\u003e2\u003c/sub\u003e are chosen as the materials of GS emitter by DQN. The emissivity spectra of the optimized structure are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003eb. The simulation result shows that a sharp and high emissivity peak can be realized with the optimized structures at 4.26 \u0026micro;m, and the emissivity outside the narrow-band is close to zero. The corresponding emissivity of the peak is 0.9996, and the reward \u003cem\u003eR\u003c/em\u003e of the structure is 60.62. The result shows that the designed WS-EM is sufficient to be an excellent CO\u003csub\u003e2\u003c/sub\u003e sensor. Due to the thickness deviation of the fabricated sample, the measured wavelength of the emissivity peak deviates from the target wavelength but still within the CO\u003csub\u003e2\u003c/sub\u003e absorption peak. The emission peak is located at 4.3 \u0026micro;m and the peak value is 0.905. In addition, the sample generates a certain low emission outside the absorption peak. Figure\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003ec displays the normalized electric field intensities of the optimal structure at 4.26 \u0026micro;m and 5 \u0026micro;m. Due to the excitation of the localized Tamm plasmon state, the electric field intensity is significantly enhanced at the thickness of 0.3 \u0026micro;m from the top of the substrate, resulting in peak emissivity at 4.26 \u0026micro;m. However, there is no notable enhancement of the intensity of electric field at 5 \u0026micro;m, resulting in near-zero emissivity at this wavelength. The incident angle related emissivity spectrum is displayed in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003ed. It can be seen that the angle independent only occurs within 30 \u0026deg;, but it does not have any effect on gas sensing since the emitter typically faces the detected gas in the normal direction.\u003c/p\u003e \u003cp\u003eThe optimization process of the GS emitter is presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003ec. In the early stage of optimization, the emitter has only a small emissivity peak within the research band and the wavelength of emissivity peak deviates from 4.26 \u0026micro;m. As the iteration progresses, the more suitable material combinations can be found so that the emissivity peak becomes more obvious and the wavelength of emissivity gradually approaches the target wavelength of 4.26 \u0026micro;m. Eventually, a near perfect emissivity peak is achieved at 4.26 \u0026micro;m with a Q-factor of 60.64. Further insights into the structure evolution during the optimization process can be obtained from Fig. S5. The distribution of each layer thickness as well as the material combinations are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003eb and \u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003ec, respectively. Compared to the emitter for RC, although a greater variety of material combinations are generated, the number is small. This indicates that the combination of Si and SiO2 is undoubtedly the most suitable choice for achieving the target emissivity spectrum for CO\u003csub\u003e2\u003c/sub\u003e sensing.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn summary, we present a general deep learning framework, i.e. DQN, for emissivity engineering. To demonstrate the powerfulness of the DQN algorithm, three multilayer WS-EMs are designed for typical applications, namely TC, RC and GS, which can autonomously select suitable materials from the same self-built material library for different design targets and optimize to the best structural parameters within a huge optimization space. The three design tasks are based on the same structural framework, so they can share the same material library, and can be easily converted from task to task by setting the corresponding reward function and modifying a few parameters. Therefore, DQN shows extremely remarkable efficiency and significant advantages when facing similar design tasks. Although we only present the design of 1D structures, DQN is also applicable for more complex 2D or 3D structures. Moreover, its applicability is not limited to emissivity engineering and can be further expanded to other fields.\u003c/p\u003e"},{"header":"Materials and methods","content":"\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eSimulation\u003c/h2\u003e \u003cp\u003eThe reflection and transmission of the multilayer WS-EMs were calculated by transfer matrix method based on Fresnel equations. The emissivity was obtained from the corresponding reflection and transmission according to the law of conservation of energy. The code of DQN was written based on the Keras package in TensorFlow by Python.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eSample Fabrication\u003c/h2\u003e \u003cp\u003eThe designed multilayer WS-EM samples were all deposited by a magnetron sputter (Kurt J. LesKer-VD75). The deposition rates of SiO\u003csub\u003e2\u003c/sub\u003e, TiO\u003csub\u003e2\u003c/sub\u003e, Ag, Si, W, Ge and ZnS are 2, 3, 6, 9, 3, 5 and 11 nm/min, respectively.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eOptical Characterization\u003c/h2\u003e \u003cp\u003eThe infrared emissivity of the multilayer WS-EM samples was measured using a Fourier transform infrared spectrometer (Nicolet iN10, Thermo Scientific).\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAuthor Contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eS.Y. and R.H. conceived the study. S.Y., W.X., Z.C. and R.H. developed the code and performed numerical simulations. P.Z., Y.D. and W.L. performed the experiment. S.Y. and R.H. wrote the manuscript and analyzed the results. J.S., W.L. and R.H. revised the manuscript. W.L. and R.H. supervised the study. All the authors provided feedback and contributed to the manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eAcknowledgement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors would like to acknowledge the financial support by National Natural Science Foundation of China (52211540005, 52076087, 52161160332), the Open Project Program of Wuhan National Laboratory for Optoelectronics (2021WNLOKF004), Wuhan City Science and Technology Program (2020010601012197), Knowledge Innovation Shuguang Program. W.L. acknowledges the financial support from Key Research and Development plan of Hubei Province (Grant no. 2021BGE037). J.S. acknowledges the financial support from JSPS Bilateral Joint Research Projects (120227404).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data that support this research\u0026rsquo;s findings are available and can be provided based on the request to the corresponding authors.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eConflict of interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing financial interest.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBaranov, D. G. et al. A. Nanophotonic engineering of far-field thermal emitters. \u003cem\u003eNat. Mater.\u003c/em\u003e\u003cstrong\u003e18\u003c/strong\u003e, 920\u0026ndash;930 (2019).\u003c/li\u003e\n\u003cli\u003eLi, W. \u0026amp; Fan, S. Nanophotonic control of thermal radiation for energy applications [Invited]. \u003cem\u003eOpt. Express, OE\u003c/em\u003e, \u003cstrong\u003e26\u003c/strong\u003e, 15995\u0026ndash;16021 (2018\u003cstrong\u003e)\u003c/strong\u003e.\u003c/li\u003e\n\u003cli\u003eByrnes, S. J., Blanchard, R. \u0026amp; Capasso, F. Harvesting renewable energy from Earth\u0026rsquo;s mid-infrared emissions. \u003cem\u003eProceedings of the National Academy of Sciences\u003c/em\u003e\u003cstrong\u003e111\u003c/strong\u003e, 3927\u0026ndash;3932 (2014).\u003c/li\u003e\n\u003cli\u003eHsu, P.-C. et al. Radiative human body cooling by nanoporous polyethylene textile. \u003cem\u003eScience\u003c/em\u003e\u003cstrong\u003e353\u003c/strong\u003e, 1019\u0026ndash;1023 (2016).\u003c/li\u003e\n\u003cli\u003eXu, J., Mandal, J. \u0026amp; Raman, A. P. Broadband directional control of thermal emission. \u003cem\u003eScience\u003c/em\u003e\u003cstrong\u003e372\u003c/strong\u003e, 393\u0026ndash;397 (2021).\u003c/li\u003e\n\u003cli\u003eHossain, M. M., Jia, B. \u0026amp; Gu, M. A metamaterial emitter for highly efficient radiative cooling. \u003cem\u003eAdv. Opt. Mater.\u003c/em\u003e\u003cstrong\u003e3\u003c/strong\u003e, 1047\u0026ndash;1051 (2015).\u003c/li\u003e\n\u003cli\u003eZhu, H. et al. Multispectral camouflage for infrared, visible, lasers and microwave with radiative cooling. \u003cem\u003eNat. Commun. \u003c/em\u003e\u003cstrong\u003e12\u003c/strong\u003e, 1805 (2021).\u003c/li\u003e\n\u003cli\u003eHe, M. et al. Deterministic inverse design of tamm plasmon thermal emitters with multi-resonant control. \u003cem\u003eNat. Mater.\u003c/em\u003e\u003cstrong\u003e20\u003c/strong\u003e, 1663\u0026ndash;1669 (2021).\u003c/li\u003e\n\u003cli\u003eBiehs, S.-A., Tschikin, M. \u0026amp; Ben-Abdallah, P. Hyperbolic metamaterials as an analog of a blackbody in the near field. \u003cem\u003ePhys. Rev. Lett.\u003c/em\u003e\u003cstrong\u003e109\u003c/strong\u003e, 104301 (2012).\u003c/li\u003e\n\u003cli\u003eLenert, A. et al. Nanophotonic solar thermophotovoltaic device. \u003cem\u003eNat. Nanotech.\u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 126\u0026ndash;130 (2014).\u003c/li\u003e\n\u003cli\u003eHu, R. et al. Machine learning-optimized Tamm emitter for high-performance thermophotovoltaic system with detailed balance analysis. \u003cem\u003eNano Energy\u003c/em\u003e\u003cstrong\u003e72\u003c/strong\u003e, 104687 (2020).\u003c/li\u003e\n\u003cli\u003eYang, R. \u0026amp; He, Y. Optically and non-optically excited thermography for composites: A review. \u003cem\u003eInfrared Phys. Techn.\u003c/em\u003e\u003cstrong\u003e75\u003c/strong\u003e, 26\u0026ndash;50 (2016).\u003c/li\u003e\n\u003cli\u003eCen, Z. H. et al. Optical property study of FePt-C nanocomposite thin film for heat-assisted magnetic recording. \u003cem\u003eOpt. Express\u003c/em\u003e, \u003cstrong\u003e21\u003c/strong\u003e, 9906 (2013)\u003c/li\u003e\n\u003cli\u003eGreffet, J.-J. et al. Coherent emission of light by thermal sources. \u003cem\u003eNature\u003c/em\u003e\u003cstrong\u003e416,\u003c/strong\u003e 61\u0026ndash;64 (2002).\u003c/li\u003e\n\u003cli\u003eLiu, B., Gong, W., Yu, B., Li, P. \u0026amp; Shen, S. Perfect thermal emission by nanoscale transmission line resonators. \u003cem\u003eNano Lett.\u003c/em\u003e\u003cstrong\u003e17\u003c/strong\u003e, 666\u0026ndash;672 (2017).\u003c/li\u003e\n\u003cli\u003eDe Zoysa, M. et al. Conversion of broadband to narrowband thermal emission through energy recycling. \u003cem\u003eNat. Photon.\u003c/em\u003e\u003cstrong\u003e6\u003c/strong\u003e, 535\u0026ndash;539 (2012).\u003c/li\u003e\n\u003cli\u003eYing, Y. et al. Whole LWIR directional thermal emission based on ENZ thin films. \u003cem\u003eLaser \u0026amp; Photonics Rev.\u003c/em\u003e, \u003cstrong\u003e16\u003c/strong\u003e, 2200018 (2022).\u003c/li\u003e\n\u003cli\u003eYue, Y. \u0026amp; Gong, J. P. Tunable one-dimensional photonic crystals from soft materials. \u003cem\u003eJ. Photoch. Photobio. C\u003c/em\u003e\u003cstrong\u003e23\u003c/strong\u003e, 45\u0026ndash;67 (2015).\u003c/li\u003e\n\u003cli\u003ePan, M. et al. Multi-band middle-infrared-compatible camouflage with thermal management via simple photonic structures. \u003cem\u003eNano Energy\u003c/em\u003e\u003cstrong\u003e69\u003c/strong\u003e, 104449 (2020).\u003c/li\u003e\n\u003cli\u003eKim, J., Park, C. \u0026amp; Hahn, J. W. Metal\u0026ndash;semiconductor\u0026ndash;metal metasurface for multiband infrared stealth technology using camouflage color pattern in visible range. \u003cem\u003eAdv. Opt. Mater.\u003c/em\u003e\u003cstrong\u003e10\u003c/strong\u003e, 2101930 (2022).\u003c/li\u003e\n\u003cli\u003eDeng, Z. et al. Nanostructured Ge/ZnS films for multispectral camouflage with low visibility and low thermal emission. \u003cem\u003eACS Appl. Nano Mater.\u003c/em\u003e\u003cstrong\u003e5\u003c/strong\u003e, 5119\u0026ndash;5127 (2022).\u003c/li\u003e\n\u003cli\u003eSheng, C., An, Y., Du, J. \u0026amp; Li, X. Colored radiative cooler under optical Tamm resonance. \u003cem\u003eACS Photonics\u003c/em\u003e\u003cstrong\u003e6\u003c/strong\u003e, 2545\u0026ndash;2552 (2019).\u003c/li\u003e\n\u003cli\u003eYao, K. et al. Near-perfect selective photonic crystal emitter with nanoscale layers for daytime radiative cooling. \u003cem\u003eACS Appl. Nano Mater.\u003c/em\u003e\u003cstrong\u003e2\u003c/strong\u003e, 5512\u0026ndash;5519 (2019).\u003c/li\u003e\n\u003cli\u003eZhu, Y. et al. Color-preserving passive radiative cooling for an actively temperature-regulated enclosure. \u003cem\u003eLight Sci. Appl.\u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 122 (2022).\u003c/li\u003e\n\u003cli\u003eXu, H., Wu, P., Zhu, C., Elbaz, A. \u0026amp; Gu Z. Z. Photonic crystal for gas sensing. \u003cem\u003eJ. Mater. Chem. C\u003c/em\u003e\u003cstrong\u003e1\u003c/strong\u003e, 6087\u0026ndash;6098 (2013).\u003c/li\u003e\n\u003cli\u003eXi, W., Liu, Y., Song, J., Hu, R. \u0026amp; Luo, X. High-throughput screening of a high-Q mid-infrared Tamm emitter by material informatics. \u003cem\u003eOpt. Lett.\u003c/em\u003e\u003cstrong\u003e46\u003c/strong\u003e, 888 (2021).\u003c/li\u003e\n\u003cli\u003eYang, Z.-Y. et al. Narrowband wavelength selective thermal emitters by confined Tamm plasmon polaritons. \u003cem\u003eACS Photonics\u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 2212\u0026ndash;2219 (2017).\u003c/li\u003e\n\u003cli\u003eKang, Q., Li, D., Guo, K., Gao, J. \u0026amp; Guo, Z. Tunable thermal camouflage based on GST plasmonic metamaterial. \u003cem\u003eNanomaterials\u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 260 (2021).\u003c/li\u003e\n\u003cli\u003eHu, R. et al. Thermal camouflaging metamaterials. \u003cem\u003eMaterials Today\u003c/em\u003e\u003cstrong\u003e45\u003c/strong\u003e, 120\u0026ndash;141 (2021).\u003c/li\u003e\n\u003cli\u003ePeng, L., Liu, D., Cheng, H., Zhou, S. \u0026amp; Zu, M. A multilayer film based selective thermal emitter for infrared stealth technology. \u003cem\u003eAdv. Opt. Mater.\u003c/em\u003e, \u003cstrong\u003e6\u003c/strong\u003e, 1801006 (2018).\u003c/li\u003e\n\u003cli\u003eZhu, H. et al. High-temperature infrared camouflage with efficient thermal management. \u003cem\u003eLight Sci. Appl.\u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 60 (2020).\u003c/li\u003e\n\u003cli\u003eFan, S. \u0026amp; Li, W. Photonics and thermodynamics concepts in radiative cooling. \u003cem\u003eNat. Photon.\u003c/em\u003e\u003cstrong\u003e16\u003c/strong\u003e, 189 (2022).\u003c/li\u003e\n\u003cli\u003eRaman, A. P., Anoma, M. A., Zhu, L., Rephaeli, E. \u0026amp; Fan, S. Passive radiative cooling below ambient air temperature under direct sunlight. \u003cem\u003eNature\u003c/em\u003e, \u003cstrong\u003e515\u003c/strong\u003e, 540\u0026ndash;544 (2014).\u003c/li\u003e\n\u003cli\u003eMa, H. et al. Multilayered SiO\u003csub\u003e2\u003c/sub\u003e/Si\u003csub\u003e3\u003c/sub\u003eN\u003csub\u003e4\u003c/sub\u003e photonic emitter to achieve high-performance all-day radiative cooling. \u003cem\u003eSol. Energ. Mater. Sol. C.\u003c/em\u003e\u003cstrong\u003e212\u003c/strong\u003e, 110584 (2020).\u003c/li\u003e\n\u003cli\u003eSakurai, A. et al. Ultranarrow-band wavelength-selective thermal emission with aperiodic multilayered metamaterials designed by Bayesian optimization. \u003cem\u003eACS Cent. Sci.\u003c/em\u003e\u003cstrong\u003e5\u003c/strong\u003e, 319\u0026ndash;326 (2019).\u003c/li\u003e\n\u003cli\u003eHu, R. et al. Machine-learning-optimized aperiodic superlattice minimizes coherent phonon heat conduction. \u003cem\u003ePhys. Rev. X\u003c/em\u003e\u003cstrong\u003e10\u003c/strong\u003e, 021050 (2020).\u003c/li\u003e\n\u003cli\u003eMolesky, S. et al. Inverse design in nanophotonics. \u003cem\u003eNat. Photon.\u003c/em\u003e\u003cstrong\u003e12\u003c/strong\u003e, 659\u0026ndash;670 (2018).\u003c/li\u003e\n\u003cli\u003eMa, W. et al. Deep learning for the design of photonic structures. \u003cem\u003eNat. Photon.\u003c/em\u003e\u003cstrong\u003e15\u003c/strong\u003e, 77\u0026ndash;90 (2021).\u003c/li\u003e\n\u003cli\u003eMnih, V. et al. Human-level control through deep reinforcement learning. \u003cem\u003eNature\u003c/em\u003e\u003cstrong\u003e518\u003c/strong\u003e, 529\u0026ndash;533 (2015).\u003c/li\u003e\n\u003cli\u003ePalik, E. D. Handbook of Optical Constants of Solids. \u003cem\u003eAcademic Press\u003c/em\u003e, (1998).\u003c/li\u003e\n\u003cli\u003eQuerry, M. R. Optical constants of minerals and other materials from the millimeter to the ultraviolet. \u003cem\u003eChemical Research, Development \u0026amp; Engineering Center, U.S. Army Armament Munitions Chemical Command\u003c/em\u003e, (1987).\u003c/li\u003e\n\u003cli\u003eSiefke, T. et al. Materials pushing the application limits of wire grid polarizers further into the deep ultraviolet spectral range. \u003cem\u003eAdv. Opt. Mater.\u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 1780\u0026ndash;1786 (2016).\u003c/li\u003e\n\u003cli\u003eYang, H. U. et al. Optical dielectric function of silver. \u003cem\u003ePhys. Rev. B\u003c/em\u003e\u003cstrong\u003e91\u003c/strong\u003e, 235137 (2015).\u003c/li\u003e\n\u003cli\u003eHasselt, H. van, Guez, A. \u0026amp; Silver, D. Deep reinforcement learning with double Q-learning. \u003cem\u003eProceedings of the AAAI Conference on Artificial Intelligence\u003c/em\u003e\u003cstrong\u003e30\u003c/strong\u003e,\u003cstrong\u003e \u003c/strong\u003e(2016).\u003c/li\u003e\n\u003cli\u003eLiu, Y. et al. Dynamic thermal camouflage via a liquid-crystal-based radiative metasurface. \u003cem\u003eNanophotonics\u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 855\u0026ndash;863 (2020).\u003c/li\u003e\n\u003cli\u003eXi, W., Liu, Y., Zhao, W., Hu, R. \u0026amp; Luo, X. Colored radiative cooling: How to balance color display and radiative cooling performance. \u003cem\u003eInt. J. Therm. Sci.\u003c/em\u003e\u003cstrong\u003e170\u003c/strong\u003e, 107172 (2021).\u003c/li\u003e\n\u003cli\u003eGuo, J., Ju, S., Lee, Y., Gunay, A. A. \u0026amp; Shiomi, J. Photonic design for color compatible radiative cooling accelerated by materials informatics. \u003cem\u003eInt. J. Heat Mass Tran.\u003c/em\u003e\u003cstrong\u003e195\u003c/strong\u003e, 123193 (2022).\u003c/li\u003e\n\u003c/ol\u003e "}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"light-science-and-applications","isNatureJournal":false,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"lsa","sideBox":"Learn more about [Light: Science \u0026 Applications](http://www.nature.com/lsa/)","snPcode":"41377","submissionUrl":"https://mts-lsa.nature.com/","title":"Light: Science \u0026 Applications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"emissivity engineering, structure optimization, deep Q-learning network, wavelength-selective thermal emitters, thermal camouflage, radiative cooling, gas sensing","lastPublishedDoi":"10.21203/rs.3.rs-3140708/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3140708/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWavelength-selective thermal emitters have been frequently adopted as a typical platform for emissivity engineering to achieve desired target emissivity spectra for broad applications such as thermal camouflage, radiative cooling, and gas sensing, etc. However, previous design methods fail to tackle the simultaneous design of both materials and structures, either fixing materials to design structures or fixing structures to select proper materials, hindering the establishment of a general design framework for emissivity engineering applicable across different applications. Herein, we employ the deep Q-learning network algorithm, a reinforcement learning method based on deep learning framework, to design multilayer wavelength-selective thermal emitters for a diverse range of applications, including thermal camouflage, radiative cooling and gas sensing. With magnetron sputtering, these emitters are fabricated and measured, validating the desired emissivity spectra with the designed ones. The main merits of the deep Q-learning algorithm include that it can 1) autonomously select suitable materials from a self-built material library and 2) autonomously optimize structures, thus realizing simultaneous optimization of materials and structures for various emissivity engineering applications. The present method is demonstrated to be feasible and efficient in designing multilayer wavelength-selective thermal emitters, offering a general framework for emissivity engineering and paving the way for efficient design of nonlinear optimization problems across various physical fields.\u003c/p\u003e","manuscriptTitle":"General Deep Learning Framework for Emissivity Engineering","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-07-20 22:18:15","doi":"10.21203/rs.3.rs-3140708/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"revise","date":"2023-08-21T07:46:28+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"This content is not available.","date":"2023-08-16T09:48:17+00:00","index":3,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2023-08-11T06:51:23+00:00","index":5,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2023-08-11T03:31:23+00:00","index":4,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2023-08-10T08:03:55+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2023-08-04T05:42:15+00:00","index":4,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2023-08-03T10:14:59+00:00","index":3,"fulltext":"This content is not available."},{"type":"editorInvitedReview","content":"This content is not available.","date":"2023-08-01T08:39:04+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2023-07-24T09:16:05+00:00","index":2,"fulltext":"This content is not available."},{"type":"reviewerAgreed","content":"This content is not available.","date":"2023-07-24T00:58:06+00:00","index":1,"fulltext":"This content is not available."},{"type":"reviewersInvited","content":"","date":"2023-07-17T14:36:59+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2023-07-13T02:43:34+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2023-07-05T02:30:08+00:00","index":"","fulltext":""},{"type":"submitted","content":"Light: Science \u0026 Applications","date":"2023-07-05T02:30:07+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"light-science-and-applications","isNatureJournal":false,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"lsa","sideBox":"Learn more about [Light: Science \u0026 Applications](http://www.nature.com/lsa/)","snPcode":"41377","submissionUrl":"https://mts-lsa.nature.com/","title":"Light: Science \u0026 Applications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7d8d5348-cf81-438a-9c0f-4ae913578632","owner":[],"postedDate":"July 20th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2023-12-06T08:39:43+00:00","versionOfRecord":{"articleIdentity":"rs-3140708","link":"https://doi.org/10.1038/s41377-023-01341-w","journal":{"identity":"light-science-and-applications","isVorOnly":false,"title":"Light: Science \u0026 Applications"},"publishedOn":"2023-12-05 05:00:00","publishedOnDateReadable":"December 5th, 2023"},"versionCreatedAt":"2023-07-20 22:18:15","video":"","vorDoi":"10.1038/s41377-023-01341-w","vorDoiUrl":"https://doi.org/10.1038/s41377-023-01341-w","workflowStages":[]},"version":"v1","identity":"rs-3140708","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3140708","identity":"rs-3140708","version":["v1"]},"buildId":"rHA-KDH7Qsr4HCuvH75dn","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}