Femto-joule threshold reconfigurable all-optical nonlinear activators for picosecond spiking neural networks

Femto-joule threshold reconfigurable all-optical nonlinear activators for picosecond spiking neural networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Femto-joule threshold reconfigurable all-optical nonlinear activators for picosecond spiking neural networks Hongtao Lin, Ruizhe Liu, Zijia Wang, Chuyu Zhong, Yan Chen, Boshu Sun, and 7 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5162168/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Achieving optical computing with thousands of tera-operations per second per watt per square millimeter (TOPs/W/mm 2 ) is the key to surpassing electrical computing. This realization requires a breakthrough in the design of a new optical computing architecture and nonlinear activation functions. In this work, we propose an on-chip picosecond spiking optical neural network architecture, which can be expected to achieve 2.13×10 3 TOPs/mm 2 . By leveraging the Kerr effect of silicon and the saturable absorption of graphene, we designed an all-optical nonlinear activator based on a graphene-silicon integrated photonic crystal cavity. The ultralow threshold, high-speed, compact, and reconfigurable all-optical nonlinear activator could achieve a 4 fJ activation energy threshold, a 1.05 ps response time, and an ultrasmall size of 15 µm×10 µm. This device provides foundation blocks for the picosecond spiking optical neural network chip to achieve 10 6 TOPs/W/mm 2 level optical computing. Physical sciences/Optics and photonics/Optical materials and structures/Graphene/Optical properties and devices Physical sciences/Optics and photonics/Optical materials and structures/Silicon photonics Physical sciences/Optics and photonics/Applied optics/Optoelectronic devices and components Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction Neural networks, inspired by the information processing mechanisms of the biological nervous system, represent powerful machine learning models 1 . However, traditional electronic-based artificial neural networks face bottlenecks in terms of computational speed and energy consumption 2 , which are limited to within 1 TOPs/W/mm 2 . Compared with neural networks in traditional von Neumann architecture electronic computers 3 , optical neural networks (ONNs) leverage the unique advantages of photons 4 , such as wide bandwidth, low power consumption 5 , high parallelism 6 , and high speed, enabling efficient logical calculations 7 and matrix operations 8 . This opens up broad prospects for applications in artificial intelligence 9 , including image recognition 10 , audio classification 11 , and phase transition system analysis 12 with the potential for ultrafast processing speed and lower power consumption 13 . ONNs, such as optical diffraction-based neural networks 10 , 14 or optical interference-based neural networks 5 , 11 , simulate the operations of biological synapses and neurons. It involves two main computational processes: (a) linear weighting operations and (b) nonlinear activation. The power consumption for linear weighting can be minimized to near zero once phase-change materials are introduced 15 , 16 to achieve nonvolatile devices 17 . Nonlinear activation functions (NAFs) play a crucial role in neural networks 18 as they introduce nonlinear transformations into the output of neurons. This mechanism allows for the development of complex representations in the network while also preventing issues such as gradient vanishing or explosion and enables the network to automatically learn key features from the data 4 . The mechanisms of existing NAF devices can be categorized into optoelectronic and all-optical strategies. The optoelectronic NAF devices mostly rely on opto-electro-optic conversion 19 , 20 or external electrical circuits to excite nonlinearity of materials such as indium tin oxide-graphene heterojunctions 21 , MoS 2 optoresistive RAM switches 22 and graphene‒silicon heterostructures 23 . However, these devices often suffer from a low density of integration, high power consumption, or slow response speeds because of electro-optical conversion. Current all-optical nonlinear activators (ANAs) are based on the nonlinear characteristics of materials, such as stimulated Brillouin scattering 24 , 25 , electromagnetically induced transparency 12 , free carrier absorption 26 , 27 , saturable absorption 28 – 32 , second harmonic generation and its inverse process 33 , cross-phase modulation 34 , and self-phase modulation 35 . Without the need for optical-electrical conversion driving circuits, these devices could achieve a higher density of integration but still face the challenge of simultaneously achieving low thresholds, high speeds, and reconfigurability, which is important for high-speed, low-power consumption optical computing. Here, we designed and demonstrated ultrafast reconfigurable ANAs based on a graphene-integrated silicon photonic crystal microcavity with ultralow thresholds and proposed an on-chip picosecond spiking optical neural network architecture for the first time. By introducing the cavity-enhanced Kerr effect, our reconfigurable ANAs can generate multiple types of NAFs, such as linear-like, ReLU-like, and sigmoid-like activation functions for ONNs. Combining the advantages of the ultrafast saturable absorption effect of graphene, our design successfully achieves an input power of 4 fJ and a response time of 1.05 ps, which indicates that the state-of-the-art figure of merit surpasses that of other ANAs by more than four orders of magnitude. The implementation of our ANAs could also notably enhance the precision of optical neural networks in tasks such as data classification, MNIST, and CIFAR-10 recognition. These activators will be the cornerstones for 10 3 TOPs/mm 2 , 10 6 TOPs/W/mm 2 level on-chip picosecond spiking optical neural network optical computing chips. Results On-chip picosecond spiking optical neural network Current on-chip optical computing architectures are based on modulating continuous wave light 36 , 37 , which has the issue of low power density, making it difficult to activate the material's nonlinear properties. By using ultrafast pulsed light, it is possible to increase the instantaneous power density without exceeding the material's thermal damage threshold while effectively stimulating its nonlinear properties. Therefore, pulsed light is highly suitable for realizing all-optical computing architectures. However, there is limited research on the design and operational mechanisms of picosecond-or even femtosecond-level optical computing architectures. Here, as shown in Fig. 1 a, we propose a wavelength division cascaded picosecond pulse optical computing network architecture for the first time and analyze the performance requirements of the devices involved. The entire architecture consists of a spatiotemporal misalignment multiplexed signal loading layer, a fully connected layer with picosecond-response nonlinear activation capability, and an output layer, as shown in Fig. 1 a.ⅰ. The spatiotemporal misalignment multiplexed signal loading layer includes a high repetition rate picosecond light source, a high-speed broadband modulator, and time-division misalignment units. A high repetition rate (100 GHz) short pulse (150 fs) picosecond light source is generated off-chip (fiber femtosecond light source) or on-chip (mode-locked laser light source), coupled into the on-chip system, and then encoded by a balanced broadband high-speed modulator (100 GHz) 38 . After passing through an on-chip broadband filter, the pulses are split into multiple beams with 1 nm intervals through a wavelength division device (an ID-WDM 39 as shown in Fig. 1 a.ⅳ) and then encoded spatiotemporal misalignment through waveguide delay and combined through an ID-WDM into a waveguide. The fully connected layer with picosecond-response nonlinear activation capability comprises a signal distribution layer, regenerative signal neurons (Fig. 1 a.ⅱ) with linear weights (Fig. 1 a.ⅲ) and NAFs (Fig. 1 b.ⅰ), and a signal bundling layer. The pulses encoded by the last layer are passed through multiple layers of the MMI to distribute the encoded pulse signals to different neurons for processing. Each neuron consists of synapses and activations. The pulses are sent to an ID-WDM and split into different wavelengths, weighted differently 40 and combined through an ID-WDM into a waveguide. Next, the pulses pass through an ANA, and a new wavelength pulse is nonlinearly activated to be transmitted to the next layer of the network. By cascading and changing the intervals between splitting and wavelength division multiplexing channels and regenerating wavelengths, the scale of the fully connected layer can be arbitrarily changed, achieving matrix compression, pooling, transformation, and other optical computing functional modules. After completing the fully connected operation, the signals are sent to the output layer, which is the signal-fully connected layer that directly connects to high-speed detectors for signal output. Under the assumption that the picosecond spiking optical neural network has a 100 GHz clock rate and 16 wavelengths, the signal can be operated at a speed of 1.6 TOPs per neuron. A single neuron, as shown in Fig. 1 b, has two inverse design splitters and one ANA, with a size of 30×25 µm 2 . Furthermore, owing to the near-zero power consumption in the linear weighting operation by the nonvolatile phase change materials, the energy consumption almost hinges on ANA. If it has ultralow threshold power at 10 fJ, our device could achieve a high computing power density (approximately 2.13×10 3 TOPs/mm 2 ) and computing power energy efficiency density (approximately 2.13×10 6 TOPs/W/mm 2 ). The performance estimation is shown in Section I in the Supplementary Information. Thus, the bottleneck of the entire system's power consumption lies in the need for an ultralow threshold, reconfigurable all-optical NAF that supports multiwavelength signal input. Consequently, developing a novel ultralow threshold, ultrafast, ultrasmall size, multiwavelength activatable, reconfigurable ANA is crucial for supporting the next generation of ultrafast processing speed, low power consumption, and large-scale optical computing networks. Silicon-based reconfigurable PhC cavity ANA Owing to the relatively small third-order nonlinear coefficient, exciting third-order nonlinearity on conventional silicon waveguides requires high power, resulting in significant power consumption for device operation 41 . To solve this challenge, a resonant line-defect PhC cavity was designed for reconfigurable ANAs. The device was designed and fabricated on the basis of a standard silicon-on-insulator photonics platform with a two-dimensional periodic circular air hole array and a line defect. A scanning electron microscope image of the device is shown in Fig. 2 a. This PhC resonant cavity offers two key advantages: first, by leveraging the slow-light effect 42 , 43 of the PhC cavity, the interaction between light and the device is enhanced (Fig. 2 b), increasing nonlinear effects and allowing a smaller device footprint. Second, through the design of the PhC cavity, light pulses resonate and increase the energy within the device, further enhancing the third-order nonlinear effects 44 . By inducing changes in the effective refractive index of the silicon device through Kerr third-order nonlinearity, which leads to a redshift in the device's transmission spectrum, as shown in Fig. 1 b.ⅱ, multiple types of NAFs can be constructed by selecting different incident light wavelengths on the basis of specific resonant peaks, thereby achieving reconfigurable ANAs (Fig. 1 b.ⅲ). The transmission spectrum of the device was measured via a continuously tunable laser, as shown in Fig. 2 c. More details of the measurement system are provided in Section II in the Supplementary Information. The nonlinear absorption curve was measured via a femtosecond laser, as shown in Fig. 2 d. Owing to the two-photon absorption effect 45 , the relative transmittance of the device tends to decrease as the input light energy increases. The device exhibited several resonant peaks designed for amplifying third-order nonlinearity (specific design details in Section III in the Supplementary Information), with a resonant peak Q factor on the order of hundreds. The femtosecond laser output was coupled into the device through grating coupling, and the output spectra with different input pulse energies are depicted in Fig. 2 e. The output spectrum redshifts with increasing input pulse energy, which is attributed to the third-order nonlinear effect in silicon, leading to an increase in the effective refractive index of the silicon cavity and resulting in a redshift of the device's resonant peaks. This phenomenon could be explained by classical cavity perturbation theory 46 . A simplified formula to calculate the resonant peak shift \(\:\varDelta\:\lambda\:\) caused by third-order nonlinearity in the microcavity is derived in Section Ⅳ of the Supplementary Information: $$\:\begin{array}{c}\varDelta\:\lambda\:=\frac{\varDelta\:{n}_{eff}}{{n}_{g}}\bullet\:{\lambda\:}_{0}={{n}_{2}}_{eff}\bullet\:Q{P}_{peak}\bullet\:{\lambda\:}_{0}\#\left(equ.1\right)\end{array}$$ where \(\:\varDelta\:{n}_{eff}\) denotes the waveguide effective index change due to the change in the material index caused by the Kerr effect, \(\:{n}_{g}\) denotes the model group index, \(\:\:{\lambda\:}_{0}\) represents the probe resonant wavelength, \(\:{{n}_{2}}_{eff}\) represents the effective third-order nonlinear coefficient of the waveguide, \(\:{P}_{peak}\) represents the pump pulse peak power coupled into the cavity, and the PhC resonant cavity has a quality factor \(\:Q\) . Thus, it can be concluded that the shift of the resonant peak is amplified by the quality factor ( \(\:Q\) ) of the resonant cavity. Figure 2 f shows the variation curve of the center wavelength of the resonance peak at 1539–1540 nm with the change in input light power, along with the calculated change in the corresponding effective refractive index \(\:\varDelta\:{n}_{eff}\) . These results clearly indicate that upon coupling a femtosecond pulsed laser into the ANA, as predicted earlier, strong third-order nonlinear effects are induced, causing a shift in the device's resonant peak. The response time is less than 2 ps, as shown in the inset of Fig. 2 f. In addition, the drifts of the resonant peaks make it possible to achieve reconfigurability and programmability of the nonlinear response in the PhC cavity ANA. At different wavelengths of the resonant peaks, the trends of the device transmittance change induced by the resonant peak shift vary. In other words, different NAF curves can be generated by changing the wavelength of the incident light. Through the introduction of a filter with a 1 nm 3 dB spectral bandwidth into the saturation absorption measurement setup configuration (Section II in the Supplementary Information), the device's transmittance for picosecond pulses at different wavelengths varied with the input light power. The device, excited by light pulses of less than 500 fJ at other wavelengths, generates distinct activation function curves, with trends in line with the changes in the transmission spectrum shown in Fig. 2 g. Therefore, ANAs with hundreds of femtojoule level thresholds can be reconfigured by taking advantage of the Kerr effects in silicon-based PhC devices. However, the picosecond spiking optical neural network needs an ANA with a lower threshold for higher-performance optical computing. Femto-joule threshold graphene-silicon PhC ANA To further reduce the threshold of the ANA, the graphene material was integrated into the silicon PhC cavity. As shown in Fig. 3 a, owing to the Pauli blocking effect, the optical absorption of graphene gradually decreases with increasing light intensity, and once the intensity exceeds the threshold power, it saturates, with a femtosecond-level response time 47 – 49 . Therefore, by leveraging the saturable absorption effect of graphene, we designed a graphene-silicon PhC cavity ANA. Graphene was transferred to the PhC device via a standard wet transfer process 46 and patterned through electron beam lithography. Figure 3 b shows the Raman spectrum of graphene transferred to the sample. The fabrication process flow of our devices and the material properties of the graphene are shown in Section Ⅴ in the Supplementary Information. Figure 3 c shows the transmittance spectra of the device before and after graphene transfer. Although the transfer of graphene increases the device's losses, the resonant peaks are preserved. Owing to the slow-light effect, the interaction between the light pulses and graphene was enhanced 47 , significantly reducing the saturation threshold power of graphene and guaranteeing an ultrafast saturation response time. To verify the ultralow threshold power and ultrafast response speed of the device combined with graphene, saturable absorption tests and pump-probe tests were performed on a graphene-silicon PhC cavity ANA. The saturable absorption curves are shown in Fig. 3 d-e. A comparison between a conventional straight waveguide device covered with 15 µm of graphene (Fig. 3 d) and a graphene-silicon PhC cavity ANA (Fig. 3 e) reveals an ultralow threshold power of 4 fJ (50% saturation transmittance) due to slow light and cavity-enhanced effects. Additionally, pump-probe measurements were also conducted on the device, as shown in Fig. 3 f. The device exhibited increased transmittance after the pump light passed through, returning to its original value within 2 ps, with a full width at half maximum response time of 1.05 ps. Here, an optical nonlinear switch device with ultralow threshold power and ultrafast response time was realized by combining the graphene saturable absorption effect with the slow light cavity enhancement effect. We survey the current state-of-the-art ANAs in Table 1 . Our device has achieved at least four orders of magnitude greater figure of merit than other on-chip ANAs. In addition, by modulating the incident wavelength on the basis of the design of the PhC cavity resonant peaks, multiple different types of NAFs can be achieved. Taking advantage of silicon Kerr third-order nonlinearity effects, as discussed in the above section, the nonlinear response of the graphene-silicon PhC cavity ANA can be reconfigured. When the input pulse was selected near the wavelengths of 1541 nm, 1540 nm and 1534 nm, ReLU-type NAF, sigmoid-type NAF and linear-type NAF could be achieved, as shown in Fig. 3 g-i (details of the configuration can be found in Section VI in the Supplementary Information). Overall, a wavelength-modulated reconfigurable high-speed ANA has been achieved. The device can realize various NAFs on the basis of the design of the transmittance spectrum, with response times of less than 4.5 ps for activation functions. Clearly, the reconfigurable ANA can saturate at such low power levels with a picosecond response time, indicating the potential for achieving more energy-efficient all-optical neural networks. Table 1 Comparison of state-of-the-art ANAs. ‘N/A’ indicates that the result is not reported in the literature and cannot be inferred from the data presented. On-chip ANAs Device Activation energy Threshold Footprint(µm 2 ) Response time Reconfigurability Figure of merit (pJ − 1 ps − 1 ) Si-Gra photonic crystal cavity (This work) 4 fJ ~ 15×10 1.05 ps Yes 238.1 PCM on Si 15 ~ 700 pJ ~ 100×100 0.2 µs No 7.14×10 − 9 Ge-Si PD 27 ~ 0.27 pJ ~ 30×8 50 ps No 0.074 Gra modulator 31 ~ 100 fJ ~ 40×10 < 90 ps No 0.11 PCM on Si MRR 50 (free space excitation) 11.9 pJ N/A < 1 ns No 8.4×10 − 5 SA modulator 32 10 pJ N/A 26 ns No 3.85×10 − 6 Stimulated Brillouin scattering in fiber 25 (potential for on-chip integration) 1 W N/A 100 ps Yes N/A Reconfigurable ANAs and Optical Neural Network Training With the reconfigurable graphene-silicon PhC cavity ANA, a picosecond pulse optical fully connected neural network is established for classification tasks. Figure 4 a shows the schematic architecture of the picosecond pulse optical neural network for classification. Details of the architecture can be found in Section Ⅶ in the Supplementary Information. To provide an initial assessment of the classification ability of the picosecond pulse optical neural network proposed above, a fully connected network is built on PyTorch and scikit-learn libraries. The nonlinear responses generated by our ANAs were fitted into an NAF curve through the linear interpolation method and normalization adjustment (see Section Ⅷ in the Supplementary Information). The NAF curves replaced the classical activation functions in the fully connected network accordingly to solve three kinds of binary classification problems. Three binary datasets are generated for statistical analysis: concentric circles, crescent moon shapes, and linearly separable classification, as shown in Fig. 4 . The size of each binary classification dataset is 1000 instances, divided into training, validation, and testing sets at a 6:2:2 ratio. The comparison is between our designed ANA and the identity function (no activation). As illustrated in Fig. 4 b-e, various activation functions have distinct impacts on the decision boundaries in binary classification tasks, resulting in different levels of final model training accuracy. Sigmoid-type NAF (Fig. 4 c) has the best classification accuracy (96%) on concentric circle datasets and the best classification accuracy (94.5%) on crescent moon datasets. ReLU-type NAF (Fig. 4 b) has the best classification accuracy (89%) on linearly separable datasets. Figure 4 f displays the learning curves for the three datasets. The results align with the widely accepted understanding that sigmoid-type activation functions perform well in binary classification tasks. This is primarily because the sigmoid-type function maps any real number to a range between 0 and 1, making their output highly suitable for interpretation as probabilities. However, owing to the shallow depth of our model, the nonlinear transformations introduced by the activation functions have a more direct and visible impact on the final decision boundary shape, resulting in its sharp angular features in the GSNR AF2’s decision boundary. Compared with GSNR AF2, GSNRs AF1 and 3 display smoother decision boundaries, leading to their gradual activation curve characteristics. Overall, GSNR AF2 is the best option for our network, achieving an average classification accuracy of 92.7% while maintaining high energy efficiency with a low threshold of 60 fJ. The on-chip picosecond pulse ONN not only works effectively on simple tasks such as binary classification tasks but also performs well in more complex image classification tasks. To solve these more challenging tasks, the spatiotemporal misalignment multiplexed picosecond spiking optical neural network proposed above was used, as depicted in Fig. 1 a. This architecture could significantly enhance device reusability and efficiency. Two neural networks are constructed via PyTorch for image classification tasks on the MNIST and CIFAR-10 datasets. The network structures are based on convolutional neural networks 51 and residual networks 52 , and the details of the networks are illustrated in Fig. S8 (see Section Ⅸ in the Supplementary Information). The raw input data samples are shown in Fig. 5 a, f. Both datasets consist of ten classes and follow a standard class-balanced split: 40,000 images for training, 10,000 for validation, and 10,000 for testing. Comprehensive visualizations of the trained networks' internal representations are provided in Fig. 5 b and 5 g. These figures offer an in-depth look at the output of each neural network block, with color coding representing activation intensities. This detailed representation allows for a holistic understanding of how information propagates through the network, from input to output, highlighting the transformations at each stage of the model. To monitor the training process, the current model is evaluated on the validation set at each epoch, generating learning curves, as shown in Fig. 5 c, h. The best model is selected on the basis of its performance on a validation set. In both datasets, ReLU-type GSNR AF shows the best performance, with 97.656% classification accuracy in the MNIST dataset and 83.008% classification accuracy in the CIFAR-10 dataset. The confusion matrices for the test dataset images are presented in Fig. 5 d and Fig. 5 i, providing a comprehensive visualization of the models' classification performance and highlighting potential areas of misclassification. Compared with the identity function, GSNR AF1 demonstrated a 0.293% accuracy improvement on the MNIST test set and a 47.754% accuracy improvement on the CIFAR-10 test set. This substantial difference in accuracy improvement between the two datasets can be attributed to their inherent characteristics and complexity levels. The MNIST dataset consists of simple black-and-white handwritten digit images with relatively linear features. Consequently, a simple linear model could also achieve good classification results. In contrast, the CIFAR-10 dataset contains complex color images of objects that exhibit greater intraclass variations and a more intricate feature space, which requires more robust nonlinear feature extraction capabilities. Accordingly, ReLU-type GSNR AF demonstrates a significant advantage on the CIFAR-10 dataset because it effectively captures and represents complex nonlinear relationships in the data, such as the interactions between object shapes, textures, and colors. The heatmaps in Figs. 5 e and 5 j illustrate the networks' activation patterns across different image regions. Models employing ReLU-type activation functions effectively highlight key features extracted by convolutional layers, such as areas potentially corresponding to car wheels, license plates, and the circular contours of digit '0'. In contrast, although a model without an NAF can detect simple features such as the central void in digit '0', it struggles to effectively learn and emphasize more complex features of the car. In conclusion, GSNR AF1 demonstrates remarkable versatility by effectively capturing nonlinear features, thereby significantly enhancing the model's classification accuracy and feature extraction capabilities across diverse datasets. From the above model results, different tasks require distinct optimal NAFs, which emphasizes the need for reconfigurable ANAs (see Section X in the Supplementary Information). Furthermore, to achieve higher computational efficiency, these NAFs should also possess relatively low thresholds. The graphene-silicon ANA can form a ReLU-type NAF at a wavelength of 1539.5 nm, with a minimum energy consumption of up to 30 fJ. On the basis of the ultralow energy threshold ReLU-type activator, the performance of our picosecond spiking optical neural network in achieving recognition of the MNIST dataset is estimated. A single neuron, as shown in Fig. 1 a.ⅱ, has two inverse design splitters and one ANA, with a size of 30×25 µm 2 . Furthermore, the number of multiplication operations required for the entire convolutional neural network process is calculated to estimate the required number and area for the optical neural network, as Section Ⅺ in the Supplementary Information shows. As estimated, to achieve recognition of the MNIST dataset, our ONN architecture requires 5537 neurons, which can be integrated into an area of 4.15 mm 2 . Consequently, our architecture is compared with the latest electronic GPU (NVIDIA) and other optical computing architectures in Table 2 . Our device can achieve a higher computing power density (approximately 2.13×10 3 TOPs/mm 2 ) and computing power energy efficiency density (approximately 0.71×10 6 TOPs/W/mm 2 ). It has achieved up to two orders of magnitude greater figures of merit than other architectures do, offering better promising performance for all-optical neural networks than electronic neural networks do. Table 2 Comparison of different state-of-the-art ONN architectures and electronic GPUs in terms of energy, area efficiency, and power efficiency. Device Energy consumption per operation (fJ) Computing power density (TOPs/mm 2 ) Computing power energy efficiency density (TOPs/W/mm 2 ) This work 1.875 2.13×10 3 0.71×10 6 Nature. 606 , 501–506, 2022 20 3.45×10 2 3.5 6.09×10 2 Nature. 589 , 52–58, 2021 16 8.5 1.2 1.18×10 4 Science 384 , 202–209, 2024 14 6.2 8.8×10 2 1.26×10 4 NVIDIA GB200 53 4.69 3.54×10 2 1.31×10 − 1 NVIDIA H100 54 22.1 38.9 5.56×10 − 2 AMD MI 300X 55 35.9 8.79 1.17×10 − 2 Intel Gaudi 3 56 30.7 17.8 1.97×10 − 2 Discussion In this work, we demonstrated femtojoule threshold reconfigurable graphene-silicon PhC cavity ANAs and proposed an on-chip wavelength division picosecond spiking optical neural network for accurate and energy-efficient classification tasks. By inducing cavity-enhanced Kerr nonlinearity in silicon, multiple types of NAFs have been constructed in a silicon PhC cavity for the first time. The reconfigurable ANAs could obtain different types of nonlinear transmission responses at different specific wavelengths within a resonant peak. Additionally, by leveraging the slow light effect of the PhC, the optical pump efficiency can be increased, allowing for a reduction in the size of the ANA to 15 µm and an energy threshold of 300 fJ. To achieve a lower power threshold and faster response speed, we effectively combined the saturable absorption properties of graphene with the silicon PhC cavity, resulting in a record low threshold power of 4 fJ and an ultrafast response time of 1.05 ps. To expand on this concept, a deep learning neural network tailored for ANA is constructed, incorporating different forms of NAFs into a neural network computing model, and successfully applied to binary and image (MNIST and CIFAR-10) classification tasks via sigmoid-type and ReLU-type functions. Compared with networks without NAFs, this network achieves significantly lower power consumption and higher accuracy. In conclusion, our demonstrated graphene-silicon PhC cavity ANAs simultaneously achieved ultralow thresholds, high speeds, and reconfigurability. They could significantly support the wavelength division picosecond spiking optical neural network, potentially achieving 2.13×10 3 TOPs/mm 2 and 0.71×10 6 TOPs/W/mm 2 in optical computing chips. This advancement offers a promising solution to meet the demand of the future artificial intelligence era for low-power, high-performance computing. Materials and Methods Device fabrication and measurement The fabrication flowchart and measurement are described in detail in Section Ⅴ in the supplementary information. Declarations Data availability All the data supporting this study are available in the paper and Supplementary Information. Additional data related to this paper are available from the corresponding authors upon request. Competing interests The authors declare no competing interests. Author contributions Conceptualization, H.L.; fabrication, R.L. and C. Z; software, Z.W. and R.L.; measurement setup construction, R.L., C.Z., and Y.C.; device testing, Y.C. and R.L.; investigation, H.L. R.L., Y.C., Z.W., and C.Z.; data curation, R.L. and Z.W.; visualization, R.L. and Z.W.; supervision, H.L., X.H., K.L., L.L., J.Y. and D.G.; All authors contributed to the technical discussions and writing of the paper. Acknowledgments This work was supported by the National Natural Science Foundation of China (92150302 received by H.L., 91950204 received by X.H., 61975179 received by H.L., 12104375 received by L.L. and 52025023 received by K.L.), the National Key Research, Development Program of China (2019YFB2203002 received by H.L.), the Zhejiang Provincial Natural Science Foundation of China (LD22F040002 received by L.L.), and the Key Project of Westlake Institute for Optoelectronics (Grand No. 2024GD002 received by H.L.). The authors would like to acknowledge the fabrication support from the ZJU Micro-Nano Fabrication Center at Zhejiang University and Westlake Center for Micro/Nano Fabrication at Westlake University. The authors would also like to thank Xiaobing Lin for his help in band diagram simulation. References LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). Leiserson, C. E. et al. There’s plenty of room at the Top: What will drive computer performance after Moore’s law? Science 368, eaam9744 (2020). Khan, H. N., Hounshell, D. A. & Fuchs, E. R. H. Science and research policy at the end of Moore’s law. Nat. Electron. 1, 14–21 (2018). Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15, 102–114 (2021). Chen, Z. et al. Deep learning with coherent VCSEL neural networks. Nat. Photonics 17, 723–730 (2023). Bai, B. et al. Microcomb-based integrated photonic processing unit. Nat. Commun. 14, 66 (2023). He, T. et al. On-chip optoelectronic logic gates operating in the telecom band. Nat. Photonics 18, 60–67 (2024). Yan, T. et al. All-optical graph representation learning using integrated diffractive photonic computing units. Sci. Adv. 8, eabn7630 (2022). Zhou, H. et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci. Appl. 11, 30 (2022). Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018). Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11, 441–446 (2017). Zuo, Y. et al. All-optical neural network with nonlinear activation functions. Optica 6, 1132 (2019). Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588, 39–47 (2020). Xu, Z. et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384, 202–209 (2024). Feldmann, J., Youngblood, N., Wright, C. D., Bhaskaran, H. & Pernice, W. H. P. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208–214 (2019). Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52–58 (2021). Wei, M. et al. Electrically programmable phase-change photonic memory for optical neural networks with nanoseconds in situ training capability. Adv. Photonics 5, 046004 (2023). Destras, O., Le Beux, S., De Magalhães, F. G. & Nicolescu, G. Survey on Activation Functions for Optical Neural Networks. ACM Comput Surv 56, (2023). Fard, M. M. P. et al. Experimental realization of arbitrary activation functions for optical neural networks. Opt. Express 28, 12138–12148 (2020). Ashtiani, F., Geers, A. J. & Aflatouni, F. An on-chip photonic deep neural network for image classification. Nature 606, 501–506 (2022). Amin, R. et al. An ITO–graphene heterojunction integrated absorption modulator on Si-photonics for neuromorphic nonlinear activation. APL Photonics 6, 120801 (2021). Xu, Z. et al. Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch. Light Sci. Appl. 11, 288 (2022). Zhong, C. et al. Graphene/silicon heterojunction for reconfigurable phase-relevant activation function in coherent optical neural networks. Nat. Commun. 14, 6939 (2023). Becker, S., Englund, D. & Stiller, B. An optoacoustic field-programmable perceptron for recurrent neural networks. Nat. Commun. 15, 3020 (2024). Slinkov, G., Becker, S., Englund, D. & Stiller, B. All-optical nonlinear activation function based on stimulated Brillouin scattering. Preprint at https://doi.org/10.48550/arXiv.2401.05135 (2024). Jha, A., Huang, C. & Prucnal, P. R. Reconfigurable all-optical nonlinear activation functions for neuromorphic photonics. Opt. Lett. 45, 4819–4822 (2020). Shi, Y. et al. Nonlinear germanium-silicon photodiode for activation and monitoring in photonic neuromorphic networks. Nat. Commun. 13, 6048 (2022). Xu, X. et al. Ultrafast growth of single-crystal graphene assisted by a continuous oxygen supply. Nat. Nanotechnol. 11, 930–935 (2016). Bonaccorso, F., Sun, Z., Hasan, T. & Ferrari, A. C. Graphene photonics and optoelectronics. Nat. Photonics 4, 611–622 (2010). Tari, H., Bile, A., Moratti, F. & Fazio, E. Sigmoid Type Neuromorphic Activation Function Based on Saturable Absorption Behavior of Graphene/PMMA Composite for Intensity Modulation of Surface Plasmon Polariton Signals. Plasmonics 17, 1025–1032 (2022). Liao, K. et al. Matrix eigenvalue solver based on reconfigurable photonic neural network. Nanophotonics 11, 4089–4099 (2022). Guo, X., Barrett, T. D., Wang, Z. M. & Lvovsky, A. I. Backpropagation through nonlinear units for the all-optical training of neural networks. Photonics Res. 9, B71–B80 (2021). Li, G. H. Y. et al. All-optical ultrafast ReLU function for energy-efficient nanophotonic deep learning. Nanophotonics 12, 847–855 (2023). Mourgias-Alexandris, G. et al. An all-optical neuron with sigmoid activation function. Opt. Express 27, 9620–9630 (2019). Vandoorne, K., Dambre, J., Verstraeten, D., Schrauwen, B. & Bienstman, P. Parallel Reservoir Computing Using Optical Amplifiers. IEEE Trans. Neural Netw. 22, 1469–1481 (2011). Xu, X. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589, 44–51 (2021). Miscuglio, M. et al. All-optical nonlinear activation function for photonic neural networks [Invited]. Opt. Mater. Express 8, 3851–3863 (2018). Wang, C. et al. Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages. Nature 562, 101–104 (2018). Piggott, A. Y. et al. Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer. Nat. Photonics 9, 374–377 (2015). Wei, M. et al. Monolithic back-end-of-line integration of phase change materials into foundry-manufactured silicon photonics. Nat. Commun. 15, 2786 (2024). Tanabe, T., Notomi, M., Mitsugi, S., Shinya, A. & Kuramochi, E. All-optical switches on a silicon chip realized using photonic crystal nanocavities. Appl. Phys. Lett. 87, 151112 (2005). Baba, T. Slow light in photonic crystals. Nat. Photonics 2, 465–473 (2008). Yan, S. et al. Slow-light-enhanced energy efficiency for graphene microheaters on silicon photonic crystal waveguides. Nat. Commun. 8, 14411 (2017). Li, S. & Cai, X. High-contrast all optical bistable switching in coupled nonlinear photonic crystal microcavities. Appl. Phys. Lett. 96, 131114 (2010). Rumi, M. & Perry, J. W. Two-photon absorption: an overview of measurements and principles. Adv. Opt. Photonics 2, 451–518 (2010). Lin, H. et al. Chalcogenide glass-on-graphene photonics. Nat. Photonics 11, 798–805 (2017). Gu, T. et al. Regenerative oscillation and four-wave mixing in graphene optoelectronics. Nat. Photonics 6, 554–559 (2012). Marini, A., Cox, J. D. & García De Abajo, F. J. Theory of graphene saturable absorption. Phys. Rev. B 95, 125408 (2017). Zhong, C., Li, J. & Lin, H. Graphene-based all-optical modulators. Front. Optoelectron. 13, 114–128 (2020). Teo, T. Y. et al. Programmable chalcogenide-based all-optical deep neural networks. Nanophotonics 11, 4073–4088 (2022). Kim, Y. Convolutional Neural Networks for Sentence Classification. in Conference on Empirical Methods in Natural Language Processing (2014). He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual Learning for Image Recognition. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (2016). doi: 10.1109/CVPR.2016.90 . NVIDIA DGX B200. NVIDIA https://www.nvidia.com/en-us/data-center/dgx-b200/ . NVIDIA H100 Tensor Core GPU Datasheet. NVIDIA https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet . AMD Instinct ™ MI300X Accelerators. AMD https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html . Intel® Gaudi® 3 AI Accelerator White Paper. Intel https://www.intel.com/content/www/us/en/content-details/817486/intel-gaudi-3-ai-accelerator-white-paper.html . Additional Declarations There is no conflict of interest Supplementary Files SupplementaryMaterialFemtojoulethresholdreconfigurableallopticalnonlinearactivatorsforpicosecondspikingneuralnetworks.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5162168","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":387524747,"identity":"ad2dcc5a-382f-41f6-b120-66e4d3a3d82e","order_by":0,"name":"Hongtao Lin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0klEQVRIiWNgGAWjYBACAxiDn4EhAUgxE6nlABBLNpCsxeAAmE+EFnP27sTHHyoO5xmfP/BMgqHCOrGB/ewBvFose85uNjhw5nCx2YEDaRIMZ9ITG3jyEvA77EbuNomDbbcTtx1sSJNgbDuc2CDBY0BIy/YfIC2bmxmAWv4Rp2UbA0jLBjaQlgZitJw5u1nizJn/iTPOMCRbJBxLN27jySGg5Xjvxg8VFWmJ/f1nEm98qLGW7Wc/g18LEuBJAEcmG7HqgYD9AAmKR8EoGAWjYCQBALP/TSdWrW4oAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0001-7432-3644","institution":"Zhejiang University","correspondingAuthor":true,"prefix":"","firstName":"Hongtao","middleName":"","lastName":"Lin","suffix":""},{"id":387524748,"identity":"a61766db-6afd-4d2a-ac50-83c9ccabc87e","order_by":1,"name":"Ruizhe Liu","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Ruizhe","middleName":"","lastName":"Liu","suffix":""},{"id":387524749,"identity":"56b5c5c5-9a40-46b5-99e5-ffce29e396ed","order_by":2,"name":"Zijia Wang","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Zijia","middleName":"","lastName":"Wang","suffix":""},{"id":387524750,"identity":"064112da-805f-4830-9aad-94700167e573","order_by":3,"name":"Chuyu Zhong","email":"","orcid":"https://orcid.org/0000-0002-3586-7873","institution":"Shenzhen Technology University","correspondingAuthor":false,"prefix":"","firstName":"Chuyu","middleName":"","lastName":"Zhong","suffix":""},{"id":387524751,"identity":"91e5fc7e-b7f2-4a81-80e1-819eee637b58","order_by":4,"name":"Yan Chen","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Yan","middleName":"","lastName":"Chen","suffix":""},{"id":387524752,"identity":"6b1cf201-4add-445a-a667-135d274fcf8c","order_by":5,"name":"Boshu Sun","email":"","orcid":"","institution":"Westlake University","correspondingAuthor":false,"prefix":"","firstName":"Boshu","middleName":"","lastName":"Sun","suffix":""},{"id":387524753,"identity":"7361b6d3-2e39-4515-9ff2-b0982bcf85d2","order_by":6,"name":"Jialing Jian","email":"","orcid":"","institution":"Westlake University","correspondingAuthor":false,"prefix":"","firstName":"Jialing","middleName":"","lastName":"Jian","suffix":""},{"id":387524754,"identity":"9d6eaa2b-a695-4c4c-94b0-b4529784a212","order_by":7,"name":"Hui Ma","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Ma","suffix":""},{"id":387524755,"identity":"e6f3e469-3375-4069-a3ae-4c81487cf86a","order_by":8,"name":"Dawei Gao","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Dawei","middleName":"","lastName":"Gao","suffix":""},{"id":387524756,"identity":"645779c2-3050-45c8-8b5c-9a0c62e01b65","order_by":9,"name":"jianyi yang","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"jianyi","middleName":"","lastName":"yang","suffix":""},{"id":387524757,"identity":"7b2cba73-4496-416e-a6f8-4d29288c37c0","order_by":10,"name":"Lan Li","email":"","orcid":"https://orcid.org/0000-0002-9097-9157","institution":"Westlake University","correspondingAuthor":false,"prefix":"","firstName":"Lan","middleName":"","lastName":"Li","suffix":""},{"id":387524758,"identity":"5c2a547a-0b0f-43e1-a983-b0f42afa99a8","order_by":11,"name":"Kaihui Liu","email":"","orcid":"https://orcid.org/0000-0002-8781-2495","institution":"School of Physics, Peking University","correspondingAuthor":false,"prefix":"","firstName":"Kaihui","middleName":"","lastName":"Liu","suffix":""},{"id":387524759,"identity":"55061086-a10b-4050-a1a5-d383e2f3b57c","order_by":12,"name":"Xiaoyong Hu","email":"","orcid":"https://orcid.org/0000-0002-1545-1491","institution":"Peking University","correspondingAuthor":false,"prefix":"","firstName":"Xiaoyong","middleName":"","lastName":"Hu","suffix":""}],"badges":[],"createdAt":"2024-09-27 04:25:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5162168/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5162168/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":73271522,"identity":"6c0f4e36-c2f9-4464-ba7d-22772e173a9f","added_by":"auto","created_at":"2025-01-08 10:58:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":240995,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSignificance, principle and schematic illustration of the ONN architecture and ANA.\u003c/strong\u003e \u003cstrong\u003ea. \u003c/strong\u003eGeneral block diagram of an on-chip picosecond spiking optical neural network. \u003cstrong\u003eⅰ:\u003c/strong\u003e Wavelength division cascaded picosecond pulse optical computing network architecture, which consists of a spatiotemporal misalignment multiplexed signal loading layer, a signal dividing layer, a fully connected layer with picosecond-response nonlinear activation capability, and an output layer. \u003cstrong\u003eⅱ:\u003c/strong\u003e A schematic illustration of a single picosecond spiking optical neuron, which consists of two inverse design wavelength-division multiplexers (ID-WDMs), m phase change material (PCM) nonvolatile weight operation waveguides, and a reconfigurable photonic crystal (PhC) microcavity ANA. \u003cstrong\u003eⅲ:\u003c/strong\u003e A schematic illustration of PCM nonvolatile weight operation waveguides. \u003cstrong\u003eⅳ:\u003c/strong\u003e A schematic illustration of one-input, m-output ID-WDM. \u003cstrong\u003eb.\u003c/strong\u003e Schematic illustration and principle of the reconfigurable PhC microcavity ANA. \u003cstrong\u003eⅰ:\u003c/strong\u003e Three-dimensional schematic of the reconfigurable PhC microcavity ANA. \u003cstrong\u003eⅱ:\u003c/strong\u003e Schematic diagram of the redshift of the resonant peak caused by the Kerr effect. \u003cstrong\u003eⅲ:\u003c/strong\u003e Relative transmission curves for three randomly selected wavelengths λ\u003csub\u003e1\u003c/sub\u003e, λ\u003csub\u003e2\u003c/sub\u003e, and λ\u003csub\u003e3\u003c/sub\u003e. \u003cstrong\u003eⅳ:\u003c/strong\u003e Band structure of the PhC waveguide. The guided mode (red curve) includes a slow-light region. The blue and orange shaded areas indicate the slab modes.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/a98e2a68cc6b5f540be13aa5.png"},{"id":73271831,"identity":"173b8c39-caa3-4704-b7b0-9b4611dddd03","added_by":"auto","created_at":"2025-01-08 11:06:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":166681,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eDesign, simulation and performance of a silicon reconfigurable PhC cavity ANA.\u003c/strong\u003e \u003cstrong\u003ea.\u003c/strong\u003e Scanning electron microscope image of the top view of the silicon PhC ANA. \u003cstrong\u003eb.\u003c/strong\u003e Simulated optical mode profile in the PhC device. \u003cstrong\u003ec.\u003c/strong\u003e Normalized transmission spectrum of the PhC cavity ANA. \u003cstrong\u003ed.\u003c/strong\u003e Nonlinear absorption curve of the PhC cavity device. \u003cstrong\u003ee.\u003c/strong\u003e Transmission spectra of the PhC cavity under different input optical pulse powers. \u003cstrong\u003ef.\u003c/strong\u003e Peak shift at 1539 nm (purple curve) and the change in theeffective refractive index (orange curve) at different input optical powers. \u003cstrong\u003eg.\u003c/strong\u003e Reconfiguration activation functions at different wavelengths (ⅰ-ⅲ: 1540 nm, 1541 nm and 1548 nm, respectively).\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/557aa40e504a739f7a1accc1.png"},{"id":73272853,"identity":"4702e92c-0727-4a7b-aa46-7a8348dba4f6","added_by":"auto","created_at":"2025-01-08 11:14:13","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":125690,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePrinciple, properties and performance of graphene-silicon PhC cavity ANAs.\u003c/strong\u003e \u003cstrong\u003ea.\u003c/strong\u003e Schematic diagram of thesaturable absorption of graphene. \u003cstrong\u003eb.\u003c/strong\u003e Raman spectrum of graphene. \u003cstrong\u003ec.\u003c/strong\u003e Normalized transmission spectra of a PhC cavity ANA without graphene (blue curve) and with graphene (orange curve). \u003cstrong\u003ed.\u003c/strong\u003eSaturable absorption curve of a straight waveguide covered with a length of 15 μm of graphene. \u003cstrong\u003ee.\u003c/strong\u003e Saturable absorption curve of the PhC cavity with graphene, with a threshold power of 4 fJ (50% saturation transmission rate). \u003cstrong\u003ef.\u003c/strong\u003e Change in the transmission of the probe light as a function of its time delay relative to the pump light. The full width at half maximum response time is approximately 1.05 ps. \u003cstrong\u003eg-i.\u003c/strong\u003eReconfiguration activation functions at different wavelengths.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/a42207382be79482d608e3f1.png"},{"id":73273333,"identity":"4f39caa3-bd6b-41fe-9632-f26e160eaec1","added_by":"auto","created_at":"2025-01-08 11:22:13","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":354015,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eGeneral block diagram of a fully connected network and the performance of graphene/silicon heterojunction nonlinear response activation functions (GSNR AFs) on three binary classification datasets.\u003c/strong\u003e \u003cstrong\u003ea.\u003c/strong\u003e General block diagram of a fully connected binary classification optical neural network, which consists of a 4-channel signal loading layer, three 4×4 fully connected layers with picosecond-response nonlinear activation capability, and an output layer. \u003cstrong\u003eb-d.\u003c/strong\u003e ReLU-type, sigmoid-type and linear-type GSNR AFs derived from a graphene-silicon integrated device and their optimal binary classification results on three test sets. \u003cstrong\u003ee.\u003c/strong\u003e Optimal binary classification results on three datasets without any NAFs. \u003cstrong\u003ef.\u003c/strong\u003e Validation classification accuracy results using different activation functions on three datasets.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/6bf3ddc1f32de095a006d739.png"},{"id":73271526,"identity":"7ac1f89e-602f-4690-af57-16db21affcad","added_by":"auto","created_at":"2025-01-08 10:58:13","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":259939,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance comparison of NAFs on the MNIST and CIFAR-10 datasets. a. \u003c/strong\u003eData examples of the MNIST test set.\u003cstrong\u003e b.\u003c/strong\u003e Visualized activation map trained on the MNIST dataset with GSNR AF1. \u003cstrong\u003ec.\u003c/strong\u003e Validation classification accuracy results using different activation functions on the MINST dataset.\u003cstrong\u003e d.\u003c/strong\u003e Confusion matrix using GSNR AF1 on the MNIST dataset.\u003cstrong\u003ee.\u003c/strong\u003e Heatmap comparison on the MNIST dataset between using GSNR AF1 and not using any activation function. \u003cstrong\u003ef.\u003c/strong\u003e Data examples of the CIFAR-10 dataset. \u003cstrong\u003eg.\u003c/strong\u003e Visualized activation map trained on the CIFAR-10 dataset with GSNR AF1. \u003cstrong\u003eh.\u003c/strong\u003e Validation classification accuracy results using different activation functions on the CIFAR-10 dataset.\u003cstrong\u003e i.\u003c/strong\u003e Confusion matrix using GSNR AF1 on the CIFAR-10 dataset. \u003cstrong\u003ej.\u003c/strong\u003e Heatmap comparison on the CIFAR-10dataset between using GSNR AF1 and not using any activation function.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/8a14bc2bc7be2efda6c9e380.png"},{"id":73274341,"identity":"1dcd1010-9b01-471e-a772-0eca62574590","added_by":"auto","created_at":"2025-01-08 11:30:14","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1825402,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/d9c98bd2-8d35-4c5b-9d25-a3ff2373b94f.pdf"},{"id":73271527,"identity":"f4a4a741-3863-4240-9d3b-c2231ae49212","added_by":"auto","created_at":"2025-01-08 10:58:13","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":5783820,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"SupplementaryMaterialFemtojoulethresholdreconfigurableallopticalnonlinearactivatorsforpicosecondspikingneuralnetworks.docx","url":"https://assets-eu.researchsquare.com/files/rs-5162168/v1/aa419b7cf6a4aa17b34344c6.docx"}],"financialInterests":"There is no conflict of interest","formattedTitle":"Femto-joule threshold reconfigurable all-optical nonlinear activators for picosecond spiking neural networks","fulltext":[{"header":"Introduction","content":"\u003cp\u003eNeural networks, inspired by the information processing mechanisms of the biological nervous system, represent powerful machine learning models\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. However, traditional electronic-based artificial neural networks face bottlenecks in terms of computational speed and energy consumption\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e, which are limited to within 1 TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e. Compared with neural networks in traditional von Neumann architecture electronic computers\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e, optical neural networks (ONNs) leverage the unique advantages of photons\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e, such as wide bandwidth, low power consumption\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e, high parallelism\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e, and high speed, enabling efficient logical calculations\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e and matrix operations\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e. This opens up broad prospects for applications in artificial intelligence\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e, including image recognition\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e, audio classification\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e, and phase transition system analysis\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e with the potential for ultrafast processing speed and lower power consumption\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eONNs, such as optical diffraction-based neural networks\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e,\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e or optical interference-based neural networks\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e,\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e, simulate the operations of biological synapses and neurons. It involves two main computational processes: (a) linear weighting operations and (b) nonlinear activation. The power consumption for linear weighting can be minimized to near zero once phase-change materials are introduced\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e,\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e to achieve nonvolatile devices\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e. Nonlinear activation functions (NAFs) play a crucial role in neural networks\u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e as they introduce nonlinear transformations into the output of neurons. This mechanism allows for the development of complex representations in the network while also preventing issues such as gradient vanishing or explosion and enables the network to automatically learn key features from the data\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe mechanisms of existing NAF devices can be categorized into optoelectronic and all-optical strategies. The optoelectronic NAF devices mostly rely on opto-electro-optic conversion\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e,\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e or external electrical circuits to excite nonlinearity of materials such as indium tin oxide-graphene heterojunctions\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e, MoS\u003csub\u003e2\u003c/sub\u003e optoresistive RAM switches\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e and graphene‒silicon heterostructures\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. However, these devices often suffer from a low density of integration, high power consumption, or slow response speeds because of electro-optical conversion. Current all-optical nonlinear activators (ANAs) are based on the nonlinear characteristics of materials, such as stimulated Brillouin scattering\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e,\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e, electromagnetically induced transparency\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e, free carrier absorption\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e,\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e, saturable absorption\u003csup\u003e\u003cspan additionalcitationids=\"CR29 CR30 CR31\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, second harmonic generation and its inverse process\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, cross-phase modulation\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e, and self-phase modulation\u003csup\u003e\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e\u003c/sup\u003e. Without the need for optical-electrical conversion driving circuits, these devices could achieve a higher density of integration but still face the challenge of simultaneously achieving low thresholds, high speeds, and reconfigurability, which is important for high-speed, low-power consumption optical computing.\u003c/p\u003e \u003cp\u003eHere, we designed and demonstrated ultrafast reconfigurable ANAs based on a graphene-integrated silicon photonic crystal microcavity with ultralow thresholds and proposed an on-chip picosecond spiking optical neural network architecture for the first time. By introducing the cavity-enhanced Kerr effect, our reconfigurable ANAs can generate multiple types of NAFs, such as linear-like, ReLU-like, and sigmoid-like activation functions for ONNs. Combining the advantages of the ultrafast saturable absorption effect of graphene, our design successfully achieves an input power of 4 fJ and a response time of 1.05 ps, which indicates that the state-of-the-art figure of merit surpasses that of other ANAs by more than four orders of magnitude. The implementation of our ANAs could also notably enhance the precision of optical neural networks in tasks such as data classification, MNIST, and CIFAR-10 recognition. These activators will be the cornerstones for 10\u003csup\u003e3\u003c/sup\u003e TOPs/mm\u003csup\u003e2\u003c/sup\u003e, 10\u003csup\u003e6\u003c/sup\u003e TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e level on-chip picosecond spiking optical neural network optical computing chips.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eOn-chip picosecond spiking optical neural network\u003c/h2\u003e \u003cp\u003eCurrent on-chip optical computing architectures are based on modulating continuous wave light\u003csup\u003e\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e,\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e, which has the issue of low power density, making it difficult to activate the material's nonlinear properties. By using ultrafast pulsed light, it is possible to increase the instantaneous power density without exceeding the material's thermal damage threshold while effectively stimulating its nonlinear properties. Therefore, pulsed light is highly suitable for realizing all-optical computing architectures. However, there is limited research on the design and operational mechanisms of picosecond-or even femtosecond-level optical computing architectures. Here, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea, we propose a wavelength division cascaded picosecond pulse optical computing network architecture for the first time and analyze the performance requirements of the devices involved.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe entire architecture consists of a spatiotemporal misalignment multiplexed signal loading layer, a fully connected layer with picosecond-response nonlinear activation capability, and an output layer, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea.ⅰ. The spatiotemporal misalignment multiplexed signal loading layer includes a high repetition rate picosecond light source, a high-speed broadband modulator, and time-division misalignment units. A high repetition rate (100 GHz) short pulse (150 fs) picosecond light source is generated off-chip (fiber femtosecond light source) or on-chip (mode-locked laser light source), coupled into the on-chip system, and then encoded by a balanced broadband high-speed modulator (100 GHz)\u003csup\u003e\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e. After passing through an on-chip broadband filter, the pulses are split into multiple beams with 1 nm intervals through a wavelength division device (an ID-WDM\u003csup\u003e\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea.ⅳ) and then encoded spatiotemporal misalignment through waveguide delay and combined through an ID-WDM into a waveguide.\u003c/p\u003e \u003cp\u003eThe fully connected layer with picosecond-response nonlinear activation capability comprises a signal distribution layer, regenerative signal neurons (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea.ⅱ) with linear weights (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea.ⅲ) and NAFs (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb.ⅰ), and a signal bundling layer. The pulses encoded by the last layer are passed through multiple layers of the MMI to distribute the encoded pulse signals to different neurons for processing. Each neuron consists of synapses and activations. The pulses are sent to an ID-WDM and split into different wavelengths, weighted differently\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e and combined through an ID-WDM into a waveguide. Next, the pulses pass through an ANA, and a new wavelength pulse is nonlinearly activated to be transmitted to the next layer of the network. By cascading and changing the intervals between splitting and wavelength division multiplexing channels and regenerating wavelengths, the scale of the fully connected layer can be arbitrarily changed, achieving matrix compression, pooling, transformation, and other optical computing functional modules. After completing the fully connected operation, the signals are sent to the output layer, which is the signal-fully connected layer that directly connects to high-speed detectors for signal output.\u003c/p\u003e \u003cp\u003eUnder the assumption that the picosecond spiking optical neural network has a 100 GHz clock rate and 16 wavelengths, the signal can be operated at a speed of 1.6 TOPs per neuron. A single neuron, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb, has two inverse design splitters and one ANA, with a size of 30\u0026times;25 \u0026micro;m\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Furthermore, owing to the near-zero power consumption in the linear weighting operation by the nonvolatile phase change materials, the energy consumption almost hinges on ANA. If it has ultralow threshold power at 10 fJ, our device could achieve a high computing power density (approximately 2.13\u0026times;10\u003csup\u003e3\u003c/sup\u003e TOPs/mm\u003csup\u003e2\u003c/sup\u003e) and computing power energy efficiency density (approximately 2.13\u0026times;10\u003csup\u003e6\u003c/sup\u003e TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e). The performance estimation is shown in Section I in the Supplementary Information. Thus, the bottleneck of the entire system's power consumption lies in the need for an ultralow threshold, reconfigurable all-optical NAF that supports multiwavelength signal input. Consequently, developing a novel ultralow threshold, ultrafast, ultrasmall size, multiwavelength activatable, reconfigurable ANA is crucial for supporting the next generation of ultrafast processing speed, low power consumption, and large-scale optical computing networks.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eSilicon-based reconfigurable PhC cavity ANA\u003c/h3\u003e\n\u003cp\u003eOwing to the relatively small third-order nonlinear coefficient, exciting third-order nonlinearity on conventional silicon waveguides requires high power, resulting in significant power consumption for device operation\u003csup\u003e\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e. To solve this challenge, a resonant line-defect PhC cavity was designed for reconfigurable ANAs. The device was designed and fabricated on the basis of a standard silicon-on-insulator photonics platform with a two-dimensional periodic circular air hole array and a line defect. A scanning electron microscope image of the device is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ea. This PhC resonant cavity offers two key advantages: first, by leveraging the slow-light effect\u003csup\u003e\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e,\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e\u003c/sup\u003e of the PhC cavity, the interaction between light and the device is enhanced (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eb), increasing nonlinear effects and allowing a smaller device footprint. Second, through the design of the PhC cavity, light pulses resonate and increase the energy within the device, further enhancing the third-order nonlinear effects\u003csup\u003e\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e\u003c/sup\u003e. By inducing changes in the effective refractive index of the silicon device through Kerr third-order nonlinearity, which leads to a redshift in the device's transmission spectrum, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb.ⅱ, multiple types of NAFs can be constructed by selecting different incident light wavelengths on the basis of specific resonant peaks, thereby achieving reconfigurable ANAs (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb.ⅲ).\u003c/p\u003e \u003cp\u003eThe transmission spectrum of the device was measured via a continuously tunable laser, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec. More details of the measurement system are provided in Section II in the Supplementary Information. The nonlinear absorption curve was measured via a femtosecond laser, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ed. Owing to the two-photon absorption effect\u003csup\u003e\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e\u003c/sup\u003e, the relative transmittance of the device tends to decrease as the input light energy increases. The device exhibited several resonant peaks designed for amplifying third-order nonlinearity (specific design details in Section III in the Supplementary Information), with a resonant peak Q factor on the order of hundreds. The femtosecond laser output was coupled into the device through grating coupling, and the output spectra with different input pulse energies are depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ee. The output spectrum redshifts with increasing input pulse energy, which is attributed to the third-order nonlinear effect in silicon, leading to an increase in the effective refractive index of the silicon cavity and resulting in a redshift of the device's resonant peaks. This phenomenon could be explained by classical cavity perturbation theory\u003csup\u003e\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e\u003c/sup\u003e. A simplified formula to calculate the resonant peak shift \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varDelta\\:\\lambda\\:\\)\u003c/span\u003e\u003c/span\u003e caused by third-order nonlinearity in the microcavity is derived in Section Ⅳ of the Supplementary Information:\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\:\\begin{array}{c}\\varDelta\\:\\lambda\\:=\\frac{\\varDelta\\:{n}_{eff}}{{n}_{g}}\\bullet\\:{\\lambda\\:}_{0}={{n}_{2}}_{eff}\\bullet\\:Q{P}_{peak}\\bullet\\:{\\lambda\\:}_{0}\\#\\left(equ.1\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varDelta\\:{n}_{eff}\\)\u003c/span\u003e\u003c/span\u003e denotes the waveguide effective index change due to the change in the material index caused by the Kerr effect, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{n}_{g}\\)\u003c/span\u003e\u003c/span\u003e denotes the model group index,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\:{\\lambda\\:}_{0}\\)\u003c/span\u003e\u003c/span\u003e represents the probe resonant wavelength, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{{n}_{2}}_{eff}\\)\u003c/span\u003e\u003c/span\u003e represents the effective third-order nonlinear coefficient of the waveguide, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{P}_{peak}\\)\u003c/span\u003e\u003c/span\u003e represents the pump pulse peak power coupled into the cavity, and the PhC resonant cavity has a quality factor \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Q\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e \u003cp\u003eThus, it can be concluded that the shift of the resonant peak is amplified by the quality factor (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Q\\)\u003c/span\u003e\u003c/span\u003e) of the resonant cavity. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef shows the variation curve of the center wavelength of the resonance peak at 1539\u0026ndash;1540 nm with the change in input light power, along with the calculated change in the corresponding effective refractive index \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varDelta\\:{n}_{eff}\\)\u003c/span\u003e\u003c/span\u003e. These results clearly indicate that upon coupling a femtosecond pulsed laser into the ANA, as predicted earlier, strong third-order nonlinear effects are induced, causing a shift in the device's resonant peak. The response time is less than 2 ps, as shown in the inset of Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef.\u003c/p\u003e \u003cp\u003eIn addition, the drifts of the resonant peaks make it possible to achieve reconfigurability and programmability of the nonlinear response in the PhC cavity ANA. At different wavelengths of the resonant peaks, the trends of the device transmittance change induced by the resonant peak shift vary. In other words, different NAF curves can be generated by changing the wavelength of the incident light. Through the introduction of a filter with a 1 nm 3 dB spectral bandwidth into the saturation absorption measurement setup configuration (Section II in the Supplementary Information), the device's transmittance for picosecond pulses at different wavelengths varied with the input light power. The device, excited by light pulses of less than 500 fJ at other wavelengths, generates distinct activation function curves, with trends in line with the changes in the transmission spectrum shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eg.\u003c/p\u003e \u003cp\u003eTherefore, ANAs with hundreds of femtojoule level thresholds can be reconfigured by taking advantage of the Kerr effects in silicon-based PhC devices. However, the picosecond spiking optical neural network needs an ANA with a lower threshold for higher-performance optical computing.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\n\u003ch3\u003eFemto-joule threshold graphene-silicon PhC ANA\u003c/h3\u003e\n\u003cp\u003eTo further reduce the threshold of the ANA, the graphene material was integrated into the silicon PhC cavity. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea, owing to the Pauli blocking effect, the optical absorption of graphene gradually decreases with increasing light intensity, and once the intensity exceeds the threshold power, it saturates, with a femtosecond-level response time\u003csup\u003e\u003cspan additionalcitationids=\"CR48\" citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eTherefore, by leveraging the saturable absorption effect of graphene, we designed a graphene-silicon PhC cavity ANA. Graphene was transferred to the PhC device via a standard wet transfer process\u003csup\u003e\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e\u003c/sup\u003e and patterned through electron beam lithography. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb shows the Raman spectrum of graphene transferred to the sample. The fabrication process flow of our devices and the material properties of the graphene are shown in Section Ⅴ in the Supplementary Information. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec shows the transmittance spectra of the device before and after graphene transfer. Although the transfer of graphene increases the device's losses, the resonant peaks are preserved. Owing to the slow-light effect, the interaction between the light pulses and graphene was enhanced\u003csup\u003e\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e\u003c/sup\u003e, significantly reducing the saturation threshold power of graphene and guaranteeing an ultrafast saturation response time.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo verify the ultralow threshold power and ultrafast response speed of the device combined with graphene, saturable absorption tests and pump-probe tests were performed on a graphene-silicon PhC cavity ANA. The saturable absorption curves are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed-e. A comparison between a conventional straight waveguide device covered with 15 \u0026micro;m of graphene (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed) and a graphene-silicon PhC cavity ANA (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ee) reveals an ultralow threshold power of 4 fJ (50% saturation transmittance) due to slow light and cavity-enhanced effects. Additionally, pump-probe measurements were also conducted on the device, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef. The device exhibited increased transmittance after the pump light passed through, returning to its original value within 2 ps, with a full width at half maximum response time of 1.05 ps.\u003c/p\u003e \u003cp\u003eHere, an optical nonlinear switch device with ultralow threshold power and ultrafast response time was realized by combining the graphene saturable absorption effect with the slow light cavity enhancement effect. We survey the current state-of-the-art ANAs in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. Our device has achieved at least four orders of magnitude greater figure of merit than other on-chip ANAs. In addition, by modulating the incident wavelength on the basis of the design of the PhC cavity resonant peaks, multiple different types of NAFs can be achieved.\u003c/p\u003e \u003cp\u003eTaking advantage of silicon Kerr third-order nonlinearity effects, as discussed in the above section, the nonlinear response of the graphene-silicon PhC cavity ANA can be reconfigured. When the input pulse was selected near the wavelengths of 1541 nm, 1540 nm and 1534 nm, ReLU-type NAF, sigmoid-type NAF and linear-type NAF could be achieved, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eg-i (details of the configuration can be found in Section VI in the Supplementary Information). Overall, a wavelength-modulated reconfigurable high-speed ANA has been achieved. The device can realize various NAFs on the basis of the design of the transmittance spectrum, with response times of less than 4.5 ps for activation functions. Clearly, the reconfigurable ANA can saturate at such low power levels with a picosecond response time, indicating the potential for achieving more energy-efficient all-optical neural networks.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of state-of-the-art ANAs. \u0026lsquo;N/A\u0026rsquo; indicates that the result is not reported in the literature and cannot be inferred from the data presented.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"6\" nameend=\"c6\" namest=\"c1\"\u003e \u003cp\u003eOn-chip ANAs\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eActivation energy Threshold\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eFootprint(\u0026micro;m\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eResponse time\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eReconfigurability\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eFigure of merit\u003c/p\u003e \u003cp\u003e(pJ\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003eps\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSi-Gra photonic crystal cavity (This work)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4 fJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e~\u0026thinsp;15\u0026times;10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.05 ps\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e238.1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePCM on Si\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e~\u0026thinsp;700 pJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e~\u0026thinsp;100\u0026times;100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.2 \u0026micro;s\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e7.14\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;9\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGe-Si PD\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e~\u0026thinsp;0.27 pJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e~\u0026thinsp;30\u0026times;8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e50 ps\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.074\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGra modulator\u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e~\u0026thinsp;100 fJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e~\u0026thinsp;40\u0026times;10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;90 ps\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0.11\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePCM on Si MRR\u003csup\u003e\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u003c/sup\u003e(free space excitation)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e11.9 pJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;1 ns\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e8.4\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;5\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSA modulator\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e10 pJ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e26 ns\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eNo\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3.85\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;6\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStimulated Brillouin scattering in fiber\u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e (potential for on-chip integration)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1 W\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100 ps\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eYes\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e\n\u003ch3\u003eReconfigurable ANAs and Optical Neural Network Training\u003c/h3\u003e\n\u003cp\u003eWith the reconfigurable graphene-silicon PhC cavity ANA, a picosecond pulse optical fully connected neural network is established for classification tasks. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea shows the schematic architecture of the picosecond pulse optical neural network for classification. Details of the architecture can be found in Section Ⅶ in the Supplementary Information. To provide an initial assessment of the classification ability of the picosecond pulse optical neural network proposed above, a fully connected network is built on PyTorch and scikit-learn libraries. The nonlinear responses generated by our ANAs were fitted into an NAF curve through the linear interpolation method and normalization adjustment (see Section Ⅷ in the Supplementary Information). The NAF curves replaced the classical activation functions in the fully connected network accordingly to solve three kinds of binary classification problems.\u003c/p\u003e \u003cp\u003eThree binary datasets are generated for statistical analysis: concentric circles, crescent moon shapes, and linearly separable classification, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e. The size of each binary classification dataset is 1000 instances, divided into training, validation, and testing sets at a 6:2:2 ratio. The comparison is between our designed ANA and the identity function (no activation). As illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb-e, various activation functions have distinct impacts on the decision boundaries in binary classification tasks, resulting in different levels of final model training accuracy. Sigmoid-type NAF (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec) has the best classification accuracy (96%) on concentric circle datasets and the best classification accuracy (94.5%) on crescent moon datasets. ReLU-type NAF (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb) has the best classification accuracy (89%) on linearly separable datasets. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ef displays the learning curves for the three datasets. The results align with the widely accepted understanding that sigmoid-type activation functions perform well in binary classification tasks. This is primarily because the sigmoid-type function maps any real number to a range between 0 and 1, making their output highly suitable for interpretation as probabilities. However, owing to the shallow depth of our model, the nonlinear transformations introduced by the activation functions have a more direct and visible impact on the final decision boundary shape, resulting in its sharp angular features in the GSNR AF2\u0026rsquo;s decision boundary. Compared with GSNR AF2, GSNRs AF1 and 3 display smoother decision boundaries, leading to their gradual activation curve characteristics. Overall, GSNR AF2 is the best option for our network, achieving an average classification accuracy of 92.7% while maintaining high energy efficiency with a low threshold of 60 fJ.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe on-chip picosecond pulse ONN not only works effectively on simple tasks such as binary classification tasks but also performs well in more complex image classification tasks. To solve these more challenging tasks, the spatiotemporal misalignment multiplexed picosecond spiking optical neural network proposed above was used, as depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea. This architecture could significantly enhance device reusability and efficiency. Two neural networks are constructed via PyTorch for image classification tasks on the MNIST and CIFAR-10 datasets. The network structures are based on convolutional neural networks\u003csup\u003e\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e and residual networks\u003csup\u003e\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e\u003c/sup\u003e, and the details of the networks are illustrated in Fig. S8 (see Section Ⅸ in the Supplementary Information). The raw input data samples are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea, f. Both datasets consist of ten classes and follow a standard class-balanced split: 40,000 images for training, 10,000 for validation, and 10,000 for testing. Comprehensive visualizations of the trained networks' internal representations are provided in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb and \u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eg. These figures offer an in-depth look at the output of each neural network block, with color coding representing activation intensities. This detailed representation allows for a holistic understanding of how information propagates through the network, from input to output, highlighting the transformations at each stage of the model.\u003c/p\u003e \u003cp\u003eTo monitor the training process, the current model is evaluated on the validation set at each epoch, generating learning curves, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ec, h. The best model is selected on the basis of its performance on a validation set. In both datasets, ReLU-type GSNR AF shows the best performance, with 97.656% classification accuracy in the MNIST dataset and 83.008% classification accuracy in the CIFAR-10 dataset. The confusion matrices for the test dataset images are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ed and Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ei, providing a comprehensive visualization of the models' classification performance and highlighting potential areas of misclassification. Compared with the identity function, GSNR AF1 demonstrated a 0.293% accuracy improvement on the MNIST test set and a 47.754% accuracy improvement on the CIFAR-10 test set. This substantial difference in accuracy improvement between the two datasets can be attributed to their inherent characteristics and complexity levels. The MNIST dataset consists of simple black-and-white handwritten digit images with relatively linear features. Consequently, a simple linear model could also achieve good classification results. In contrast, the CIFAR-10 dataset contains complex color images of objects that exhibit greater intraclass variations and a more intricate feature space, which requires more robust nonlinear feature extraction capabilities. Accordingly, ReLU-type GSNR AF demonstrates a significant advantage on the CIFAR-10 dataset because it effectively captures and represents complex nonlinear relationships in the data, such as the interactions between object shapes, textures, and colors. The heatmaps in Figs.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ee and \u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ej illustrate the networks' activation patterns across different image regions. Models employing ReLU-type activation functions effectively highlight key features extracted by convolutional layers, such as areas potentially corresponding to car wheels, license plates, and the circular contours of digit '0'. In contrast, although a model without an NAF can detect simple features such as the central void in digit '0', it struggles to effectively learn and emphasize more complex features of the car. In conclusion, GSNR AF1 demonstrates remarkable versatility by effectively capturing nonlinear features, thereby significantly enhancing the model's classification accuracy and feature extraction capabilities across diverse datasets.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFrom the above model results, different tasks require distinct optimal NAFs, which emphasizes the need for reconfigurable ANAs (see Section X in the Supplementary Information). Furthermore, to achieve higher computational efficiency, these NAFs should also possess relatively low thresholds. The graphene-silicon ANA can form a ReLU-type NAF at a wavelength of 1539.5 nm, with a minimum energy consumption of up to 30 fJ. On the basis of the ultralow energy threshold ReLU-type activator, the performance of our picosecond spiking optical neural network in achieving recognition of the MNIST dataset is estimated. A single neuron, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea.ⅱ, has two inverse design splitters and one ANA, with a size of 30\u0026times;25 \u0026micro;m\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Furthermore, the number of multiplication operations required for the entire convolutional neural network process is calculated to estimate the required number and area for the optical neural network, as Section Ⅺ in the Supplementary Information shows. As estimated, to achieve recognition of the MNIST dataset, our ONN architecture requires 5537 neurons, which can be integrated into an area of 4.15 mm\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Consequently, our architecture is compared with the latest electronic GPU (NVIDIA) and other optical computing architectures in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. Our device can achieve a higher computing power density (approximately 2.13\u0026times;10\u003csup\u003e3\u003c/sup\u003e TOPs/mm\u003csup\u003e2\u003c/sup\u003e) and computing power energy efficiency density (approximately 0.71\u0026times;10\u003csup\u003e6\u003c/sup\u003e TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e). It has achieved up to two orders of magnitude greater figures of merit than other architectures do, offering better promising performance for all-optical neural networks than electronic neural networks do.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison of different state-of-the-art ONN architectures and electronic GPUs in terms of energy, area efficiency, and power efficiency.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\"\u0026times;\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevice\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEnergy consumption per operation (fJ)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eComputing power density (TOPs/mm\u003csup\u003e2\u003c/sup\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eComputing power energy efficiency density (TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eThis work\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1.875\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.13\u0026times;10\u003csup\u003e3\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e0.71\u0026times;10\u003csup\u003e6\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eNature.\u003c/em\u003e \u003cb\u003e606\u003c/b\u003e, 501\u0026ndash;506, 2022\u003csup\u003e20\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.45\u0026times;10\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e6.09\u0026times;10\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eNature.\u003c/em\u003e \u003cb\u003e589\u003c/b\u003e, 52\u0026ndash;58, 2021\u003csup\u003e16\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e8.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e1.18\u0026times;10\u003csup\u003e4\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cem\u003eScience\u003c/em\u003e \u003cb\u003e384\u003c/b\u003e, 202\u0026ndash;209, 2024\u003csup\u003e14\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e6.2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8.8\u0026times;10\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e1.26\u0026times;10\u003csup\u003e4\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNVIDIA GB200\u003csup\u003e53\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.54\u0026times;10\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e1.31\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNVIDIA H100\u003csup\u003e54\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e22.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e38.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e5.56\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAMD MI 300X\u003csup\u003e\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e35.9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e8.79\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e1.17\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIntel Gaudi 3\u003csup\u003e56\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e30.7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e17.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\"\u0026times;\" colname=\"c4\"\u003e \u003cp\u003e1.97\u0026times;10\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this work, we demonstrated femtojoule threshold reconfigurable graphene-silicon PhC cavity ANAs and proposed an on-chip wavelength division picosecond spiking optical neural network for accurate and energy-efficient classification tasks. By inducing cavity-enhanced Kerr nonlinearity in silicon, multiple types of NAFs have been constructed in a silicon PhC cavity for the first time. The reconfigurable ANAs could obtain different types of nonlinear transmission responses at different specific wavelengths within a resonant peak. Additionally, by leveraging the slow light effect of the PhC, the optical pump efficiency can be increased, allowing for a reduction in the size of the ANA to 15 \u0026micro;m and an energy threshold of 300 fJ. To achieve a lower power threshold and faster response speed, we effectively combined the saturable absorption properties of graphene with the silicon PhC cavity, resulting in a record low threshold power of 4 fJ and an ultrafast response time of 1.05 ps. To expand on this concept, a deep learning neural network tailored for ANA is constructed, incorporating different forms of NAFs into a neural network computing model, and successfully applied to binary and image (MNIST and CIFAR-10) classification tasks via sigmoid-type and ReLU-type functions. Compared with networks without NAFs, this network achieves significantly lower power consumption and higher accuracy.\u003c/p\u003e \u003cp\u003eIn conclusion, our demonstrated graphene-silicon PhC cavity ANAs simultaneously achieved ultralow thresholds, high speeds, and reconfigurability. They could significantly support the wavelength division picosecond spiking optical neural network, potentially achieving 2.13\u0026times;10\u003csup\u003e3\u003c/sup\u003e TOPs/mm\u003csup\u003e2\u003c/sup\u003e and 0.71\u0026times;10\u003csup\u003e6\u003c/sup\u003e TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e in optical computing chips. This advancement offers a promising solution to meet the demand of the future artificial intelligence era for low-power, high-performance computing.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003eDevice fabrication and measurement\u003c/p\u003e \u003cp\u003eThe fabrication flowchart and measurement are described in detail in Section Ⅴ in the supplementary information.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eData availability\u003c/h2\u003e\n\u003cp\u003eAll the data supporting this study are available in the paper and Supplementary Information. Additional data related to this paper are available from the corresponding authors upon request.\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u003c/h2\u003e \u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e \u003ch2\u003eAuthor contributions\u003c/h2\u003e \u003cp\u003eConceptualization, H.L.; fabrication, R.L. and C. Z; software, Z.W. and R.L.; measurement setup construction, R.L., C.Z., and Y.C.; device testing, Y.C. and R.L.; investigation, H.L. R.L., Y.C., Z.W., and C.Z.; data curation, R.L. and Z.W.; visualization, R.L. and Z.W.; supervision, H.L., X.H., K.L., L.L., J.Y. and D.G.; All authors contributed to the technical discussions and writing of the paper.\u003c/p\u003e\u003ch2\u003eAcknowledgments\u003c/h2\u003e \u003cp\u003eThis work was supported by the National Natural Science Foundation of China (92150302 received by H.L., 91950204 received by X.H., 61975179 received by H.L., 12104375 received by L.L. and 52025023 received by K.L.), the National Key Research, Development Program of China (2019YFB2203002 received by H.L.), the Zhejiang Provincial Natural Science Foundation of China (LD22F040002 received by L.L.), and the Key Project of Westlake Institute for Optoelectronics (Grand No. 2024GD002 received by H.L.). The authors would like to acknowledge the fabrication support from the ZJU Micro-Nano Fabrication Center at Zhejiang University and Westlake Center for Micro/Nano Fabrication at Westlake University. The authors would also like to thank Xiaobing Lin for his help in band diagram simulation.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eLeCun, Y., Bengio, Y. \u0026amp; Hinton, G. Deep learning. Nature 521, 436\u0026ndash;444 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLeiserson, C. E. \u003cem\u003eet al.\u003c/em\u003e There\u0026rsquo;s plenty of room at the Top: What will drive computer performance after Moore\u0026rsquo;s law? Science 368, eaam9744 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKhan, H. N., Hounshell, D. A. \u0026amp; Fuchs, E. R. H. Science and research policy at the end of Moore\u0026rsquo;s law. Nat. Electron. 1, 14\u0026ndash;21 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShastri, B. J. \u003cem\u003eet al.\u003c/em\u003e Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15, 102\u0026ndash;114 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, Z. \u003cem\u003eet al.\u003c/em\u003e Deep learning with coherent VCSEL neural networks. Nat. Photonics 17, 723\u0026ndash;730 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBai, B. \u003cem\u003eet al.\u003c/em\u003e Microcomb-based integrated photonic processing unit. Nat. Commun. 14, 66 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe, T. \u003cem\u003eet al.\u003c/em\u003e On-chip optoelectronic logic gates operating in the telecom band. Nat. Photonics 18, 60\u0026ndash;67 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYan, T. \u003cem\u003eet al.\u003c/em\u003e All-optical graph representation learning using integrated diffractive photonic computing units. Sci. Adv. 8, eabn7630 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou, H. \u003cem\u003eet al.\u003c/em\u003e Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci. Appl. 11, 30 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin, X. \u003cem\u003eet al.\u003c/em\u003e All-optical machine learning using diffractive deep neural networks. Science 361, 1004\u0026ndash;1008 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShen, Y. \u003cem\u003eet al.\u003c/em\u003e Deep learning with coherent nanophotonic circuits. Nat. Photonics 11, 441\u0026ndash;446 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZuo, Y. \u003cem\u003eet al.\u003c/em\u003e All-optical neural network with nonlinear activation functions. Optica 6, 1132 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWetzstein, G. \u003cem\u003eet al.\u003c/em\u003e Inference in artificial intelligence with deep optics and photonics. Nature 588, 39\u0026ndash;47 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, Z. \u003cem\u003eet al.\u003c/em\u003e Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384, 202\u0026ndash;209 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeldmann, J., Youngblood, N., Wright, C. D., Bhaskaran, H. \u0026amp; Pernice, W. H. P. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208\u0026ndash;214 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeldmann, J. \u003cem\u003eet al.\u003c/em\u003e Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52\u0026ndash;58 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWei, M. \u003cem\u003eet al.\u003c/em\u003e Electrically programmable phase-change photonic memory for optical neural networks with nanoseconds in situ training capability. Adv. Photonics 5, 046004 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDestras, O., Le Beux, S., De Magalh\u0026atilde;es, F. G. \u0026amp; Nicolescu, G. Survey on Activation Functions for Optical Neural Networks. ACM Comput Surv 56, (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFard, M. M. P. \u003cem\u003eet al.\u003c/em\u003e Experimental realization of arbitrary activation functions for optical neural networks. Opt. Express 28, 12138\u0026ndash;12148 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAshtiani, F., Geers, A. J. \u0026amp; Aflatouni, F. An on-chip photonic deep neural network for image classification. Nature 606, 501\u0026ndash;506 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAmin, R. \u003cem\u003eet al.\u003c/em\u003e An ITO\u0026ndash;graphene heterojunction integrated absorption modulator on Si-photonics for neuromorphic nonlinear activation. APL Photonics 6, 120801 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, Z. \u003cem\u003eet al.\u003c/em\u003e Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch. Light Sci. Appl. 11, 288 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhong, C. \u003cem\u003eet al.\u003c/em\u003e Graphene/silicon heterojunction for reconfigurable phase-relevant activation function in coherent optical neural networks. Nat. Commun. 14, 6939 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBecker, S., Englund, D. \u0026amp; Stiller, B. An optoacoustic field-programmable perceptron for recurrent neural networks. Nat. Commun. 15, 3020 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSlinkov, G., Becker, S., Englund, D. \u0026amp; Stiller, B. All-optical nonlinear activation function based on stimulated Brillouin scattering. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.48550/arXiv.2401.05135\u003c/span\u003e\u003cspan address=\"10.48550/arXiv.2401.05135\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJha, A., Huang, C. \u0026amp; Prucnal, P. R. Reconfigurable all-optical nonlinear activation functions for neuromorphic photonics. Opt. Lett. 45, 4819\u0026ndash;4822 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShi, Y. \u003cem\u003eet al.\u003c/em\u003e Nonlinear germanium-silicon photodiode for activation and monitoring in photonic neuromorphic networks. Nat. Commun. 13, 6048 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, X. \u003cem\u003eet al.\u003c/em\u003e Ultrafast growth of single-crystal graphene assisted by a continuous oxygen supply. Nat. Nanotechnol. 11, 930\u0026ndash;935 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBonaccorso, F., Sun, Z., Hasan, T. \u0026amp; Ferrari, A. C. Graphene photonics and optoelectronics. Nat. Photonics 4, 611\u0026ndash;622 (2010).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTari, H., Bile, A., Moratti, F. \u0026amp; Fazio, E. Sigmoid Type Neuromorphic Activation Function Based on Saturable Absorption Behavior of Graphene/PMMA Composite for Intensity Modulation of Surface Plasmon Polariton Signals. Plasmonics 17, 1025\u0026ndash;1032 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiao, K. \u003cem\u003eet al.\u003c/em\u003e Matrix eigenvalue solver based on reconfigurable photonic neural network. Nanophotonics 11, 4089\u0026ndash;4099 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo, X., Barrett, T. D., Wang, Z. M. \u0026amp; Lvovsky, A. I. Backpropagation through nonlinear units for the all-optical training of neural networks. Photonics Res. 9, B71\u0026ndash;B80 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, G. H. Y. \u003cem\u003eet al.\u003c/em\u003e All-optical ultrafast ReLU function for energy-efficient nanophotonic deep learning. Nanophotonics 12, 847\u0026ndash;855 (2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMourgias-Alexandris, G. \u003cem\u003eet al.\u003c/em\u003e An all-optical neuron with sigmoid activation function. Opt. Express 27, 9620\u0026ndash;9630 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVandoorne, K., Dambre, J., Verstraeten, D., Schrauwen, B. \u0026amp; Bienstman, P. Parallel Reservoir Computing Using Optical Amplifiers. IEEE Trans. Neural Netw. 22, 1469\u0026ndash;1481 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, X. \u003cem\u003eet al.\u003c/em\u003e 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589, 44\u0026ndash;51 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiscuglio, M. \u003cem\u003eet al.\u003c/em\u003e All-optical nonlinear activation function for photonic neural networks [Invited]. Opt. Mater. Express 8, 3851\u0026ndash;3863 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang, C. \u003cem\u003eet al.\u003c/em\u003e Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages. Nature 562, 101\u0026ndash;104 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePiggott, A. Y. \u003cem\u003eet al.\u003c/em\u003e Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer. Nat. Photonics 9, 374\u0026ndash;377 (2015).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWei, M. \u003cem\u003eet al.\u003c/em\u003e Monolithic back-end-of-line integration of phase change materials into foundry-manufactured silicon photonics. Nat. Commun. 15, 2786 (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTanabe, T., Notomi, M., Mitsugi, S., Shinya, A. \u0026amp; Kuramochi, E. All-optical switches on a silicon chip realized using photonic crystal nanocavities. Appl. Phys. Lett. 87, 151112 (2005).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBaba, T. Slow light in photonic crystals. Nat. Photonics 2, 465\u0026ndash;473 (2008).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYan, S. \u003cem\u003eet al.\u003c/em\u003e Slow-light-enhanced energy efficiency for graphene microheaters on silicon photonic crystal waveguides. Nat. Commun. 8, 14411 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, S. \u0026amp; Cai, X. High-contrast all optical bistable switching in coupled nonlinear photonic crystal microcavities. Appl. Phys. Lett. 96, 131114 (2010).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRumi, M. \u0026amp; Perry, J. W. Two-photon absorption: an overview of measurements and principles. Adv. Opt. Photonics 2, 451\u0026ndash;518 (2010).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin, H. \u003cem\u003eet al.\u003c/em\u003e Chalcogenide glass-on-graphene photonics. Nat. Photonics 11, 798\u0026ndash;805 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGu, T. \u003cem\u003eet al.\u003c/em\u003e Regenerative oscillation and four-wave mixing in graphene optoelectronics. Nat. Photonics 6, 554\u0026ndash;559 (2012).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarini, A., Cox, J. D. \u0026amp; Garc\u0026iacute;a De Abajo, F. J. Theory of graphene saturable absorption. Phys. Rev. B 95, 125408 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhong, C., Li, J. \u0026amp; Lin, H. Graphene-based all-optical modulators. Front. Optoelectron. 13, 114\u0026ndash;128 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTeo, T. Y. \u003cem\u003eet al.\u003c/em\u003e Programmable chalcogenide-based all-optical deep neural networks. Nanophotonics 11, 4073\u0026ndash;4088 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKim, Y. Convolutional Neural Networks for Sentence Classification. in \u003cem\u003eConference on Empirical Methods in Natural Language Processing\u003c/em\u003e (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe, K., Zhang, X., Ren, S. \u0026amp; Sun, J. Deep Residual Learning for Image Recognition. in 2016 \u003cem\u003eIEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e 770\u0026ndash;778 (2016). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2016.90\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2016.90\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNVIDIA DGX B200. \u003cem\u003eNVIDIA\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.nvidia.com/en-us/data-center/dgx-b200/\u003c/span\u003e\u003cspan address=\"https://www.nvidia.com/en-us/data-center/dgx-b200/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNVIDIA H100 Tensor Core GPU Datasheet. \u003cem\u003eNVIDIA\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet\u003c/span\u003e\u003cspan address=\"https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAMD Instinct\u003csup\u003e\u0026trade;\u003c/sup\u003e MI300X Accelerators. \u003cem\u003eAMD\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html\u003c/span\u003e\u003cspan address=\"https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIntel\u0026reg; Gaudi\u0026reg; 3 AI Accelerator White Paper. \u003cem\u003eIntel\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.intel.com/content/www/us/en/content-details/817486/intel-gaudi-3-ai-accelerator-white-paper.html\u003c/span\u003e\u003cspan address=\"https://www.intel.com/content/www/us/en/content-details/817486/intel-gaudi-3-ai-accelerator-white-paper.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5162168/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5162168/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAchieving optical computing with thousands of tera-operations per second per watt per square millimeter (TOPs/W/mm \u003csup\u003e2 \u003c/sup\u003e) is the key to surpassing electrical computing. This realization requires a breakthrough in the design of a new optical computing architecture and nonlinear activation functions. In this work, we propose an on-chip picosecond spiking optical neural network architecture, which can be expected to achieve 2.13×10 \u003csup\u003e3 \u003c/sup\u003eTOPs/mm\u003csup\u003e2\u003c/sup\u003e. By leveraging the Kerr effect of silicon and the saturable absorption of graphene, we designed an all-optical nonlinear activator based on a graphene-silicon integrated photonic crystal cavity. The ultralow threshold, high-speed, compact, and reconfigurable all-optical nonlinear activator could achieve a 4 fJ activation energy threshold, a 1.05 ps response time, and an ultrasmall size of 15 µm×10 µm. This device provides foundation blocks for the picosecond spiking optical neural network chip to achieve 10\u003csup\u003e6\u003c/sup\u003e TOPs/W/mm\u003csup\u003e2\u003c/sup\u003e level optical computing.\u003c/p\u003e","manuscriptTitle":"Femto-joule threshold reconfigurable all-optical nonlinear activators for picosecond spiking neural networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-08 10:58:08","doi":"10.21203/rs.3.rs-5162168/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2e677c6c-eda6-4822-8bfe-6699dab95fc5","owner":[],"postedDate":"January 8th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":41297941,"name":"Physical sciences/Optics and photonics/Optical materials and structures/Graphene/Optical properties and devices"},{"id":41297942,"name":"Physical sciences/Optics and photonics/Optical materials and structures/Silicon photonics"},{"id":41297943,"name":"Physical sciences/Optics and photonics/Applied optics/Optoelectronic devices and components"}],"tags":[],"updatedAt":"2025-03-24T15:47:14+00:00","versionOfRecord":[],"versionCreatedAt":"2025-01-08 10:58:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5162168","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5162168","identity":"rs-5162168","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}