Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision

Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision Wei Wang, Yi Sun, Peiwen Tong, Jiangrong Shen, Hui Xu, Rongrong Cao, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6505999/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 19 Mar, 2026 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract Neuromorphic vision systems based on memristors offer an energy-efficient approach to artificial vision, yet traditional pixel(s)-to-one-memristor architectures remain inefficient in dynamic image processing due to limited temporary storage. Here, inspired by human visual working memory, we propose a novel one-pixel-multiple-memristor (1PnR) architecture with a rolling exposure strategy for fast sequential image acquisition. Furthermore, a data-in-situ computing network for efficient image processing is developed. With network weights mapped to voltage vectors and applied to the image storage memristor array, direct computation is enabled where the image is stored, and the energy-intensive data transmission is eliminated. A hardware prototype of the 1PnR architecture achieved 95.7% recognition accuracy on the Weizmann human action flow dataset. Compared to CMOS-based systems, this architecture is estimated to have a 2000× reduction in latency for image sensing and storage, and a 160× reduction in energy consumption image processing, demonstrating significant potential for future neuromorphic visual systems. Physical sciences/Nanoscience and technology/Nanoscale devices/Electronic devices Physical sciences/Engineering/Electrical and electronic engineering Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Introduction The human visual system is distinguished by its remarkably energy efficiency 1 . Studying and emulating the human visual system offers a promising approach to developing high-performance and low-power artificial vision systems 2 , 3 . Figure 1 illustrates the working process of the human visual system. In essence, the human visual system can be broadly divided into two key components: the retina and the brain. The retina cells convert light signals into neural spiking signals, which are then transmitted to the brain via the optic nerve. In the brain, synaptic cells process these spiking signals, enabling humans to visually perceive the external environment 4 . It’s been reported that there is a visual working memory mechanism in the brain, which stores temporary visual information and performs pre-process for the following neural classification 5 – 7 . This working mode can greatly reduce the visual transmission in the brain nerve and further improve the process efficiency. Recently, various artificial vision systems intimating the biological visual processing mechanism have been proposed 8 – 16 . Among them, memristors are promising candidates for visual information storage and in-memory neuromorphic processing 17 – 19 . The memristor based neuromorphic architecture can avoid the constant data conversion and transmission between physical separated image sensors and processing units in von Neumann architectures, thus provides high power and time efficiency 20 – 23 . In the current implemented systems, images are captured by sensors or optoelectronic devices and transmitted to memristor arrays that store weights for network computation 24 – 27 . While this analog-domain architecture effectively improves processing efficiency, it still requires the transmission of image data from the sensing-storage array to the computing array, resulting in additional latency and power consumption. In addition, the sensor and memristor device is usually integrated as one-sensor-one-memristor 28 – 30 , or multi-sensor-one-memristor structure 31 , 32 . In sequential image processing conditions, read-out and erasing operation is required for previously stored image data before sensing new pixel data, which limits the image sensing speed and energy efficiency. In this work, inspired by the high-efficiency working mode of the visual nerves in the human system, we propose a neuromorphic visual architecture, namely, one-pixel- multiple-memristor (1PnR) computing architecture for sequential image sensing, storing and processing, as shown in Fig. 1 . The light sensors and memristor array, similar to biological retina, are connected by analog circuits as optic nerves for conversion and transmission of the sensory data. The array is divided into 2 parts, one part serves as visual working memory, which stores image data and performs data-in-situ for pre-process. The other part serves as neuromorphic computing core mimicking visual cortex, which stores network weights and carries network computing for classification. Furthermore, the sensors and memristor array are integrated by a gate multiplexing architecture, where a sensor is connected to a gate line of the 1T1R array, offering the ability of storing a single channel of optic input to multiple memristor cells simultaneously, functioning as retinal divergent connectivity for visual information processing. Based on the structure, a rolling exposure strategy for fast sequential image acquisition is then proposed. With the pixel image sensed and stored column by column to the memristor array in analogue domain, both time and energy efficiency can be achieved compared to traditional CMOS system. Moreover, a data-in-situ computing network is proposed for fast image processing. Voltage vectors carrying network weights are applied to the image storage array to implement data-in-situ computing, which avoids transmission of the image data in traditional memristive neuromorphic systems. Finally, a 1PnR hardware system is established for verification of sequential visual sensing and processing. The experimental results reveal 95.7% recognition accuracy for Weizmann human action flow dataset, demonstrating the great classification capacity of the data-in-situ network. Compared to typical CMOS-based systems, the proposed architecture is estimated to have ~ 2000 times reduction of time latency for image sensing and storage, and ~ 160 times reduction of energy consumption for image processing. These results demonstrate great potential for the proposed 1PnR novel architecture to construct neuromorphic visual systems. Results Device characterization and One-Pulse Modulation method An 1T1R memristor array with 8k scale (64 rows × 128 columns) is deployed in this work. Each cell contains a TiN/TaO x /HfO x /TiN memristor and a transistor connected in series, as shown in Fig. 2 a. And the schematic diagram of the 1T1R structure as well as the transistor’s output characteristic are presented in Supplementary Fig. 1 . Figure 2 b is the typical quasi-DC curve of the 1T1R structure. With a 1.6 V gate voltage in SET voltage sweep (0 to 1.0 V) and 3.5 V gate voltage in RESET sweep (0 to -1.15 V), a Resistive Switching (RS) window, defines as the ratio between the High Resistance State (HRS) and the Low Resistance State (LRS), can reach to ~ 10. Electric pulses are then applied to the 1T1R cell, pulse trains with gradually rising amplitudes (0.8 V to 1.3V for SET and 1.3 V to 2.4 V for RESET, width time 80 ns) are used as the modulation signal to achieve more resistance states and larger switching window, and the gate voltage is set to 3 V to turn on the transistor. The result in Fig. 2 c shows the continuous change of the device state under SET and RESET operations, with 100 pulses each. After each pulse, a small pulse of 0.1 V follows behind to read the device conductance. Indeed, the pulse parameters (amplitude and width) are optimized through multiple experiments to achieve better resistive-switching window and linearity. The results reveal that the 1T1R cell has analogue switching behavior under pulse modulation, demonstrating its great potential in multi-value image storage and neuromorphic computing applications. The pixel sensor is achieved by ITO/ZnS/TiN photo-resistive structure. The top electrode is deposited with transparent ITO. ZnS is deposited as a functional film due to its excellent photoactivity. An UV LED with a wavelength of 365 nm is used to investigate optoelectronic characteristics. The experimental results plotted in Supplementary Fig. 2 show that the device has great cycle-to-cycle (C2C) and (device-to-device) D2D uniformity, with the ability to sense the light pulse with various intensities, which can effectively support the implementation of the proposed 1PnR system. To achieve high-frame processing of sequential visual information, an open-loop fast modulation method is then proposed for the 1T1R cell, aiming to significantly improve the writing speed while partially sacrificing precision. This method applies only one parameter-fixed pulse to the 1T1R cell, which can be called as One-Pulse Modulation (OPM). As shown in Fig. 2 d, the pulse is applied to the RL, and the SLs are all grounded. Based on the inter-connection of 1T1R array, all the devices in the same RL can be modulated at the same time, and different gate voltages can be applied to control the modulation states in different SLs. The total pulse width is 1 µs, corresponding to a theoretical modulation frequency of 1 MHz. In our experiments, 8 gate voltage values are used as representatives to evaluate OPM capability and writing fluctuation. The corresponding current response of 1T1R cell is shown in Fig. 2 e, 8 devices with different gate voltages are repeatedly tested 50 times. Before each modulation, the devices are initialized to the same state (30 µS). As the gate voltage gradually drops from 2.0 V to 1.21 V, the response current of the device also gradually decreases. The current fluctuation is relatively large at the initial stage of the pulse and becomes concentrated at the pulse falling edge, indicating that the writing state tends to be uniform under pulse amplitude. The 8 gate voltages ranging from 1.21 V to 2.0 V leading the corresponding conductance averagely distributes in 40 µS∼320 µS, with an interval of 40 µS. Figure 2 f shows the distribution of the 8 conductance states in the form of box plot. The rectangular box represents the 25%∼75% distribution range, and the upper and lower edges represent 1.5 times of the interquartile range. All the device conductance fluctuates near the target value, except for some outliers. And the overall fluctuation range is within 20 µS (half of the interval). It should be noted that the pulse waveform is designed with two voltage segments (1.2 V and 1.6 V) to prevent impact responses caused by voltage jumps from affecting the modulation stability. The experiments of 1 segment pulse are also performed and plotted in Supplementary Fig. 3 , and modulation error of device conductance is compared in Fig. 2 g. The results reveal that 2-seg pulse shows narrow distribution in modulation error, which has better stability. Furthermore, retention tests of the 8 states are performed, the result plotted in Supplementary Fig. 4 shows that the device states can last up to 50000 s, with only small fluctuation. These results demonstrate the great compatibility of the 1T1R cell with OPM method. Rolling exposure on One-Pixel-Multiple-Memristor structure In this work, an architecture integrating the pixel sensors and memristor array is proposed for neuromorphic near-sensor computing system, which we call One-Pixel-Multiple-Memristor (1PnR) Structure. The schematic diagram of the 1PnR structure is shown in Fig. 3 a, where pixel sensors are connected to the gate lines (WL) of the 1T1R array one by one through analog conversion circuits. The circuit has a very simple structure, containing a voltage division part and a differential amplifier, which can convert the light-induced resistance change of the pixel sensor to volage values. To sufficiently verify the 1PnR structure, 30 channels of image sensors and conversion circuits are experimentally demonstrated, as shown in Fig. 3 b. The circuit parameter is adjusted to fit the gate voltage range of the 1T1R array, and the transmission relationship of the circuit from light density to gate voltage, as well as the transmission speed is plotted in Supplementary Fig. 5. The 8k memristor chip is deployed on a transfer board, where all the pins can be controlled by the 1PnR system to apply OPM pulse. Then pixel stimuli with 8 different intensities is applied to 5 randomly picked channels to verify the sensing-storage ability of the 1PnR board. The light intensities are experimentally picked to achieve the same gate voltages in Fig. 2 f for comparison, and 50 repeated tests are performed under each light intensity. Figure 3 c shows the distribution of memristor conductance after the experiments. The statistical results are presented in box plots, with different devices distinguished by colors. As can be seen from the figure, the modulated conductance fluctuates around the target value, and the distribution maintains high consistency between channels. Compared with the results in Fig. 2 f, the 1T1R cells in 1PnR achieved the same 8-state modulation under the same parameters, and the distribution range was also roughly the same. The results indicate that the conversion circuit makes considerable compatibility with the pixel sensors and 1T1R cell, which can support the 1PnR structure to achieve fast storage of pixel images. Based on 1PnR architecture and OPM method, a rolling exposure strategy (RES) is then proposed for fast image acquisition. The workflow of RES is shown in Fig. 2 d, taking the Weizmann human action dataset as example, where the image is binarized and cropped to 24×40. On the left side is a column of 24-pixel sensors grown on a silicon wafer, which are connected to the gate lines of a 24×40 1T1R array through conversion circuits, establishing 24 channels of 1PnR structure. Before image exposure, the memristors array are initialized to same HRS, and the SLs are all grounded. When the first column of the pixel image is under exposure, the pixel information can be converted to gate voltages by the conversion circuits at the same time. Then, the OPM is applied to the first column (RL 0 ) to store the pixel information in parallel. Next, the same exposure method is used to store the image to memristor array column by column, until the entire image is exposed and stored. Utilizing the fabricated pixel sensors and transmission circuit, a 1PnR hardware verification system is implemented, as shown in Supplementary Fig. 6 . UV LED light sources are employed, corresponding one-to-one with the fabricated pixel sensors. The image is converted into UV light signals based on pixel values and directly irradiated to the optical sensors, which simulates the optical exposure process. And the detailed process of the exposure experiment based on the hardware system is illustrated in Supplementary Fig. 7. Besides, to achieve RES verification with larger image size based on the prepared image acquisition hardware, the target column of image (up to 30 pixels) is reshaped to a 5×6 array for UV irradiation. The light signals received by the optical sensors are then reshaped back to one column through the 30-channel conversion circuits and then written to target column of the connected memristor array, which is shown in Supplementary Fig. 8. RES experimental verification is firstly conducted using 3 sample images picked in the Weizmann dataset, containing actions of run, jump and walk. Each sample is processed into a 24×40-pixel image, formed by concatenating 4 consecutive 24×10-pixel frames of human motion, as illustrated in Supplementary Fig. 12 . During the image acquisition experiment of each sample image, 24 sensing channels are used according to the column pixels, and a total of 40 rolling exposure steps are required for the whole sample image. After RES experiment, the target image and restored image from the memristor array is plotted in Fig. 3 e. It can be seen from the results that the action features in restored images are still clear for classification with negligible noise. To further verify the RES strategy, hardware experiment utilizing gray-scaled image and whole 8k memristor array is performed. A portrait image is chosen as an example, which is gray scaled and cropped to 128×64 scale to match the memristor array. According to the design of RES, if the 128 gate lines of the 8k array are fully utilized, a 128×64 image can be perceived in just 64 exposure operations. However, to match the 30-channel circuit prepared in the experiment, the image is divided into 5 groups with 30 pixels, as shown in Supplementary Fig. 9 . The rolling exposure of 30×64 is completed within each group first, and then the next group of images is exposed in sequence. Figure 3 f shows target image and restored image from the memristor array after the RES experiment. Although the restored image has pixel noise caused by writing errors, the original facial features are still clearly distinguishable, which is sufficient to support applications such as face recognition. Figure 3 g shows the distribution of the writing error of the portrait image, which is consistent with the OPM experiment with normal distribution. Indeed, the standard deviation becomes slightly larger due to the conversion of gray pixel values. Figure 3 h shows the performance comparison between the 1PnR architecture with typical CMOS system consisting of image sensors and DDR5 memory. The comparison details are introduced in in Supplementary Note 1 . The results reveal that the proposed 1PnR system has ~ 2000 times reduction of time latency and 1.8 times reduction of energy consumption, which further demonstrates the great potential in fast image acquisition applications. In fact, the proposed 1PnR architecture exhibits remarkable flexibility and scalability for image acquisition. For future optical system applications, when the sensor array scale becomes sufficiently large, it can function as an area sensor for global exposure. Following the operational principle of line-scan CCD cameras, the entire image can be first exposed globally, and then its pixels can be transferred column by column to the memristor array for storage. If multiple memristor arrays are employed, each column of sensors can be connected to a different memristor array. After exposure, writing pulses can be applied simultaneously to different arrays, enabling the entire image to be written into separate memristor arrays at the same time, offering very fast image exposure, as illustrated in Supplementary Fig. 10 . Moreover, if the scale of the memristor array is sufficiently large, all sensor units in the area array can be connected to the same array, allowing an entire image to be written into a single column of memristors. Data-in-situ computing network for sequential vision In traditional memristor-based neuromorphic system, the network weights are mapped to the conductance values of the memristor array, and the samples are mapped to voltage signals by the artificial neurons. The memristor array can achieve the MAC operation efficiently through Ohm's law and Kirchhoff's current law in analog domain. However, in near-sensor integrated computing system, the image data has already been stored in the memristor array. Performing calculations in traditional way requires the image data to be transferred from storage array to computing array, resulting in a large amount of transmitted data, which reduces the overall efficiency of the computing system. Here, we utilize the MAC feature of the data storage array and propose a data-in-situ computing network for memristor-based neuromorphic systems. As shown in Fig. 4 a, the first part of network weights is mapped to voltage weight vector (VWV), which are applied to the data-storage memristor for data-in-situ computing. Then the computing results are fed into a perception for classification, which is implemented on weight-storage memristor array. The data-in-situ computing weights are updated through backpropagation calculation, along with the perception weights in training process. Utilizing data-in-situ network, the image is processed where it is stored, only the data-in-situ computing results with greatly decreased data amount are required to transmission for classification, which is consistent with the high-efficiency mechanism of the visual working memory in the human brain. To evaluate data-in-situ computing network for sequential image processing based on the 1PnR system, the Weizmann Dataset is used for evaluation, which has 10 actions, containing bend, jack, jump, pjump (jump in place), run, side, skip, walk, wave1 (in one hand) and wave2 (in two hands). Each action is performed by 9 people. The dataset is already binarized and aligned from video clips. In our simulation, the dataset is cropped to 24×10. To make sure that every sample includes a whole period of periodic actions (such as walk, run and wave), 4 frames are averagely picked from 12 consecutive frame sequences in the video clip to construct one sample, as shown in Supplementary Fig. 12 . Furthermore, a leave-one-out strategy is used in the sample preparation, which is, 8 people are chosen for training (800 samples for 10 actions) and the rest 1 person is used for testing (100 samples for 10 actions). The network structure is illustrated in Fig. 4 b, where one sample is stored to the 24×40 memristor array by RES. For Weizmann dataset classification, two VWVs (40×1×2) are applied from the column of the array for data-in-situ computing successively, and the results are joined together (48×1) and then fed into 48×8×10 two-layer perception for image classification. The data-in-situ computing network is firstly verified by simulation, 97.82% accuracy can be obtained for the dataset after 100 training epochs, showing great efficiency of the network structure. Furthermore, the impact of VWV direction for data-in-situ computing is analyzed. As shown in Fig. 4 c, when 2 VWVs are both applied from the column of the image, the classification accuracy reaches best to 97.82%, if 2 VWVs are both applied from rows of the array, the accuracy reduces to 92.75%. And if 1 VWV from column and 1 from row, the accuracy is 92.02%. It can be conducted that the sample can keep more features after data-in-situ computing in column direction and achieve better classification performance. Finally, the impact of image noise and weight noise on the network is simulated, which are plotted in Fig. 4 d and Fig. 4 e, respectively. The noise percentage is applied in randomly generated normal distribution, with mean value of 1. Each standard deviation is simulated for 100 cycles. The results show that as the noise deviation rises, the network accuracy decreases accordingly. But the noise on image data shows less impact on the performance, indicating better robustness in the image sensory error. Finally, Experimental verification of data-in-situ computing network system is performed on the 1PnR hardware system. Figure 5 a shows the optical photo of the memristor modulation board. With the support of core controller (FPGA) and array read-write circuit (ADC&DAC), memristor modulation and the MAC computation can be achieved by the system. The mapping layout of the data-in-situ computing network in the memristor array is enlarged. columns 0∼71 is assigned for sample image storage and data-in-situ computing, which can store 3 samples, as shown in the purple box. The hidden layer (HD, 48×8) of the perception needs 16 columns after differential process and is mapped to columns 72∼87 of the array. Besides, the fully connected layer (FC, 8×10) requires 20 columns, which is mapped to the final 88∼127 columns. The network weights (HD and FC) are mapped to the memristor array in range of 100 µS∼200 µS by a modulation script at the host computer. The script adopts a close-loop modulation strategy, and the maximum conductance error is limited to 6 µS to achieve a balance between weight modulation speed and network accuracy. Figure 5 b shows the results of the HD and FC layer mapped to the memristor array, which presents 100% yield for the network computation. During the experiment, every 3 samples are stored to the assigned area through OPM by experiments shown in Fig. 5 a. Then the data-in-situ computing of the image is performed by the hardware system, and the output currents of the 3 representative samples are plotted in Fig. 5 c. The results are obvious differences between the 3 samples, which indicates that data-in-situ computing can successfully extract the sample features for subsequent classification. Next, the results are fed into the HD and FC layer on the hardware successively. The output current of the FC layer for the 3 representative samples is plotted in Fig. 5 d. The largest current is remarked in red, representing the classification result. It’s clear that the 3 samples are both successfully classified. Then, all the test samples are applied to the system for hardware verification. A typical recognition result of the hardware system is shown in Fig. 5 e, with an accuracy of 95.7%. The 2.1% decay compared to the simulation results may be caused by the mapping error of the network weight. The comparison between the ideal calculation current and the hardware output current of all the test samples in FC layer is plotted in Fig. 5 f, which shows considerate consistency. To further evaluate the system performance and stability, repeated experiments are performed combining simulation and hardware measurements. During simulations, random conductance errors in image acquisition process with normal distribution are applied, which is consistent with practical experiment. The results from both simulation in HD and FC layer, experiments in HD layer and simulation in FC layer, both experiments in FC layer are presented in Fig. 5 g, each combination is repeatedly performed for 10 times. The network accuracy with both simulating in two layers reaches 96.5 ± 0.5 (mean value 96.5 with standard deviation 0.5) and slightly decrease to 96.1 ± 0.5 when using experimental results in HD layer and simulation in FC layer. The accuracy loss in experimental results may be caused by the random electric noise in the hardware system. At last, the result of both hardware experiments in two layers reaches 94.7 ± 0.5, demonstrating the good robustness of the hardware system. Furthermore, the energy consumption of the data-in-situ computing architecture for Weizmann Dataset is estimated, details are introduced in Supplementary Note 2 . A CMOS system consisting of a typical digital accelerator-based (HNPU-based) system 33 with DDR5 memory 34 is estimated for comparison, which are plotted in Fig. 5 f. The results show that the proposed data-in-situ computing architecture has about 160 times energy efficiency than typical CMOS systems. To evaluate the general classification ability of data-in-situ computing network, the benchmark is then performed on the commonly used MNIST and Fashion MNIST dataset with noise analysis. The experimental results on MNIST dataset are presented in Supplementary Fig. 13 and Supplementary Note 3 , which demonstrates a 95.9% (simulation) and 92.4% (experimental) classification accuracy. The simulation results on Fashion MNIST dataset are shown in Supplementary Fig. 14. Given the greater complexity of Fashion MNIST images compared to MNIST, 4 voltage vectors are applied to the image for data-in-situ computing to extract more image features, which achieves an 87.17% recognition accuracy. Finally, the comparison of the 1PnR system and other representative works about integrated sensing-storage-computation system is performed and summarized in Supplementary Note 4 . These results further demonstrate the great performance of the data-in-situ computing network for future neuromorphic visual systems. Discussion In this work, a one-pixel-multiple-memristor (1PnR) computing architecture is proposed for sequential image sensing, storing and processing. The sensors and memristor array are integrated by a gate multiplexing architecture, offering the ability of storing a single channel of optic input to multiple memristor cells simultaneously. Then, a rolling exposure strategy for fast sequential image acquisition is proposed. With the pixel image sensed and stored column by column to the memristor array in analogue domain, both time and energy efficiency can be achieved compared to traditional CMOS system. Moreover, a data-in-situ computing network based on memristor array is proposed for fast image processing. Voltage vectors carrying network weights are applied to the image storage array to implement data-in-situ computing, which avoids transmission of the image data in traditional memristive neuromorphic systems. Finally, a 1PnR hardware system is established for verification of sequential visual sensing and processing. The experimental results reveal 95.7% recognition accuracy for Weizmann human action flow dataset, demonstrating the great classification capacity of the data-in-situ network. Compared to typical CMOS-based systems, the proposed architecture is estimated to have 2000 times reduction of time latency for image sensing and storage, and 160 times reduction of energy consumption for image processing, demonstrating the great potential for constructing neuromorphic visual systems. Methods Sample fabrication The fabrication process of the ITO/ZnS/TiN sensor. The bottom electrode (TiN, ~ 40 nm) is deposited by magnetron sputtering, then it is patterned by lithography and wet etching. After the second lithography process, the functional material zinc Sulfide (ZnS, ~ 30 nm) and transparent top electrode ITO (~ 30 nm) are deposited by magnetron sputtering, respectively. The effective device size is determined by the round area in the bottom electrode, with a diameter of 200 µm. Fabrication of 8k memristor chip. The chip has a 64×128 one-transistor-one-resistor (1T1R) memristor array. The transistors in the array are fabricated using a standard 180 nm Si complementary metal–oxide semiconductor process. The memristor cell has structure of TiN/TaO x /HfO 2 /TiN. 12 nm HfO 2 is deposited by atomic layer deposition as resistive switching material, then 45 nm TaO x is deposited by magnetron sputtering as a thermal enhanced layer. The TiN electrode is deposited by magnetron sputtering. Measurement method The electrical test of single device is performed with the Keithley 4200 semiconductor parameter analyzer. The optical light is provided by UV LEDs with a wavelength of 365 nm, and a signal generator is used to control the UV light pulse. The memristor chip and network calculation is measured by our memristor-evaluation system, which is equipped with core controller (FPGA) and array read-write circuits (ADC&DAC). Network simulation method The network simulation is conducted using Python environment and PyTorch framework. The image samples are from Weizmann Dataset of human action flows, which are preprocessed to 24×40 binary images. During the training process, two sets of weights vectors applied to the samples are treated as updatable parameters and iteratively updated using the backpropagation algorithm. The ReLU activation function is employed, the cross-entropy loss function is used for error computation. Research participant consent The authors affirm that human research participants provided informed consent for publication of the images in Fig. 3 f, Supplementary Fig. 8 and Supplementary Fig. 9. Declarations Data availability The source data generated in this study are provided in the Source Data file. Additional data related to this paper can be requested from the authors. Code availability The code of data-in-situ computing simulation will be available from the corresponding authors upon request. Competing interests The authors declare no competing interests. Author contributions Y.S., P.W.T. and W.W. designed the experiments. Y.S. designed and fabricated the sample devices and conducted device-level electrical experiments. C.L.C. and Y.N.W. designed and fabricated the circuit boards. J.R.S. designed and simulated the task algorithms. Y.S. and P.W.T. designed data-in-situ computing architecture. H.X., R.R.C., C.L., and B.S. assisted with data analysis and interpretation. Y.S. and P.W.T. co-wrote the manuscript. All authors discussed the results and revised the manuscript. W.W., Y.C.Y. and Q.J.L. supervised the research. Acknowledgments This work was supported by the National Key R&D Program of China under Grant Nos. 2024YFA1208800 (Q.L.) and 2023YFB4502200 (Y.Y.), Innovation Research Foundation of National University of Defense Technology under Grant Nos. ZK24-09 (Y.S.) and 25-ZZCX-JDZ-18 (W.W.), National Natural Science Foundation of China under Grant Nos. 62404253 (Y.S.), 62304254 (W.W.) and U23A20322 (Q.L.), Guangdong Provincial Key Laboratory of In-Memory Computing Chips under Grant No. 2024B1212020002 (Y.Y.), Shenzhen Science and Technology Program under Grant No. JCYJ20241202125907011 (Y.Y.), as well as Beijing Natural Science Foundation under Grant Nos. L234026 (Y.Y.) and L257010 (Y.Y.). References Gu L et al (2020) A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature 581:278–282 Zhou F et al (2020) Near-sensor and in-sensor computing. Nat Electron 3:664–671 Long Z et al (2023) A neuromorphic bionic eye with filter-free color vision using hemispherical perovskite nanowire array retina. Nat Commun 14:1972 Huang Y et al (2024) Bioinspired sensing-memory-computing integrated vision systems: biomimetic mechanisms, design principles, and applications. Sci China Inf Sci 67:151401 Liebe S et al (2012) Theta coupling between V4 and prefrontal cortex predicts visual short-term memory performance. Nat Neurosci 15:456–462 Liu J et al (2020) Stable maintenance of multiple representational formats in human visual short-term memory, Proc. Natl. Acad. Sci. 117, 32329–32339 Benigno GB et al (2023) Waves traveling over a map of visual space can ignite short-term predictions of sensory input. Nat Commun 14:3409 Lin H et al (2025) In situ training of an in-sensor artificial neural network based on ferroelectric photosensors. Nat Commun 16:421 Feng G (2023) Retinomorphic hardware for in-sensor computing. InfoMat 5:e12473 Xie P et al (2024) Emerging optoelectronic artificial synapses and memristors based on low-dimensional nanomaterials. Appl Phys 11:011303 Chen J et al (2023) Optoelectronic graded neurons for bioinspired in-sensor motion perception. Nat Nanotechnol 18:882–888 Huang P et al (2023) Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction. Nat Commun 14:6736 Zhou F et al (2019) Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat Nanotechnol 14:776–782 Zhou G et al (2023) Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing. Nat Commun 14:8489 Zhang Z et al (2022) All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat Nanotechnol 17:27–32 Hu L et al (2021) All-optically controlled memristor for optoelectronic neuromorphic computing. Adv Funct Mater 31:2005582 Sebastian A et al (2020) Memory devices and applications for in-memory computing. Nat Nanotechnol 15:529–544 Wang Z et al (2020) Resistive switching materials for information processing. Nat Rev Mater 5:173–195 Yao P et al (2020) Fully hardware-implemented memristor convolutional neural network. Nature 577:641–646 Liu Z et al (2025) A memristor-based adaptive neuromorphic decoder for brain–computer interfaces. Nat Electron 8:362–372 Liang X et al (2024) Physical reservoir computing with emerging electronics. Nat Electron 7:193–206 Wang W et al (2021) Neuromorphic Motion Detection and Orientation Selectivity by Volatile Resistive Switching Memories. Adv Intell Syst 3:2000224 Li C et al (2018) Analogue signal and image processing with large memristor crossbars. Nat Electron 1:52–59 Zhang Z et al (2022) In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array. Nat Commun 13:6590 Lao J et al (2022) Ultralow-power machine vision with self-powered sensor reservoir. Adv Sci 9:2106092 Shan X et al (2022) Plasmonic Optoelectronic Memristor Enabling Fully Light-Modulated Synaptic Plasticity for Neuromorphic Vision. Adv Sci 9:2104632 Wang S et al (2020) Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci Rev 8:nwaa172 Chen S et al (2018) An Artificial Flexible Visual Memory System Based on an UV-Motivated Memristor. Adv Mater 30:1705400 Seo S et al (2018) Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat Commun 9:5106 Dang B et al (2023) One-phototransistor–one-memristor array with high-linearity light-tunable weight for optic neuromorphic computing. Adv Mater 35:2204844 Dang B et al (2024) Reconfigurable in-sensor processing based on a multi-phototransistor–one-memristor array. Nat Electron 7:991–1003 Huang H et al (2025) Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing. Nat Nanotechnol 20:93–103 Ahn H-A et al (2024) A 1.01-V 8.5-Gb/s/pin 16-Gb LPDDR5x SDRAM With Advanced I/O Circuitry for High-Speed and Low-Power Applications. IEEE J Solid-State Circuits 59:3479–3487 Han D et al (2021) HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching. IEEE J Solid-State Circuits 56:2858–2869 Additional Declarations There is NO Competing Interest. Supplementary Files InventaryofSupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.docx Inventary of Supplementary Information SourceData.xlsx Dataset 1 SupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.pdf Supplementary Information_Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision SupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.pdf Supplementary Information_Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision Cite Share Download PDF Status: Published Journal Publication published 19 Mar, 2026 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6505999","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":598821544,"identity":"fa48f2dc-d773-4b97-98a1-d01fd7761aaa","order_by":0,"name":"Wei Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/klEQVRIiWNgGAWjYPACCQj1gYGBsQHBJUIL4wwg7iFSCwQw8xCjxeD42cOveSosGPj5jz/+bLvjsOx+BuaDt3kY7PJwajmTl2bNc0aCQXJGjpl07pnDxj0MbMnWPAzJxbi0mB3IMTPObZNgMLjBw8ac23Y4sYeBx0yah+FAYgMuLeffALX8k2CwPw90mCVYC/83/Fpu5Bg/zm0A2sKQYCDNCLGFDa8W+xtvzJj/HJNgkLiRYybZ25Zu3HOYzdhyjkEyTi2S/TnGH2fU1DHw9x9//OFnm7Vse3vzwxtvKuxwagECNlAs1CMUMIMIA9zqQUo+4JUeBaNgFIyCUQAASORSNx87L/cAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-7180-831X","institution":"National University of Defense Technology","correspondingAuthor":true,"prefix":"","firstName":"Wei","middleName":"","lastName":"Wang","suffix":""},{"id":598821545,"identity":"6c66513c-6daa-4cd9-98a8-de562dc5e685","order_by":1,"name":"Yi Sun","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Yi","middleName":"","lastName":"Sun","suffix":""},{"id":598821546,"identity":"4cb6acce-5e22-4988-980a-00bb894ba20d","order_by":2,"name":"Peiwen Tong","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Peiwen","middleName":"","lastName":"Tong","suffix":""},{"id":598821547,"identity":"3f4de104-2d16-46a5-8b45-ec60059f9daf","order_by":3,"name":"Jiangrong Shen","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Jiangrong","middleName":"","lastName":"Shen","suffix":""},{"id":598821548,"identity":"15df3182-7547-401c-9569-93e7c87220f6","order_by":4,"name":"Hui Xu","email":"","orcid":"","institution":"College of Electronic Science and Technology, National University of Defense Technology; Hunan Key Laboratory of Aerospace Intelligent ASIC Technology","correspondingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Xu","suffix":""},{"id":598821549,"identity":"2d326246-d07f-4114-be57-9d4480d5eef5","order_by":5,"name":"Rongrong Cao","email":"","orcid":"https://orcid.org/0000-0001-8760-8577","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Rongrong","middleName":"","lastName":"Cao","suffix":""},{"id":598821550,"identity":"945171b9-1fae-4d96-bcaa-64d6bfdd5d03","order_by":6,"name":"Chang Liu","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Chang","middleName":"","lastName":"Liu","suffix":""},{"id":598821551,"identity":"ac21b01c-6d3f-4692-a51a-f68e805b3f44","order_by":7,"name":"Changlin Chen","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Changlin","middleName":"","lastName":"Chen","suffix":""},{"id":598821552,"identity":"ad5ce270-0e00-4b8c-935a-f914e70193ef","order_by":8,"name":"Bing Song","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Bing","middleName":"","lastName":"Song","suffix":""},{"id":598821553,"identity":"25997e02-1300-4781-a84a-7cad42e9b382","order_by":9,"name":"Yinan Wang","email":"","orcid":"","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Yinan","middleName":"","lastName":"Wang","suffix":""},{"id":598821554,"identity":"1bfdca55-12c4-4117-9ff4-8f1d053a8e4f","order_by":10,"name":"Yuchao Yang","email":"","orcid":"https://orcid.org/0000-0003-4674-4059","institution":"Peking University","correspondingAuthor":false,"prefix":"","firstName":"Yuchao","middleName":"","lastName":"Yang","suffix":""},{"id":598821555,"identity":"6e01517f-401b-4191-adcb-c18660ac85a1","order_by":11,"name":"Qingjiang Li","email":"","orcid":"https://orcid.org/0000-0001-9779-3198","institution":"National University of Defense Technology","correspondingAuthor":false,"prefix":"","firstName":"Qingjiang","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2025-04-22 16:01:53","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6505999/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6505999/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-026-70860-y","type":"published","date":"2026-03-19T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":104404000,"identity":"407ce815-326b-40c4-9b79-588ddeaef3ef","added_by":"auto","created_at":"2026-03-11 12:19:34","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":3988909,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic of the human visual system and the proposed 1PnR visual system. The human vison system contains eyes, optic neurons and brain, the visual working memory in human brain stores temporary sequential visual information and performs pre-process. The proposed artificial visual system based on 1PnR architecture is composed of ITO/ZnS/TiN pixel sensors for image sensing, analogue circuit for data transmission, and an 8k memristor array based on TiN/HfO\u003csub\u003ex\u003c/sub\u003e/TaO\u003csub\u003ex\u003c/sub\u003e/TiN stack. The array is divided into 2 parts, one part serves as working memory, which stores image data and performs data-in-situ for pre-process. The other part serves as neuromorphic computing core, which stores network weights and carries network computing for classification.\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/3adc684e1441f1b2904e542b.jpg"},{"id":104403983,"identity":"3969345e-ed1e-4c77-93fd-444c49436812","added_by":"auto","created_at":"2026-03-11 12:19:32","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":9457051,"visible":true,"origin":"","legend":"\u003cp\u003eThe 1T1R memristor array and One-Pulse-Modulation method. a, Optical image of the 1T1R memristor chip and the TEM image of the TiN/TaO\u003csub\u003ex\u003c/sub\u003e/HfO\u003csub\u003ex\u003c/sub\u003e/TiN memristor structure. (Scale bar: 50 nm) b, The quasi-DC characteristic of TiN/TaO\u003csub\u003ex\u003c/sub\u003e/HfO\u003csub\u003ex\u003c/sub\u003e/TiN memristor. The device shows reversible resistive behaviors with good uniformity. c, 100 conductance states of the 1T1R cell modulated by pulse train, indicating the great multilevel feature. d, The design of One-Pulse-Modulation. The pulse is designed with 2 levels to increase device stability, 8 devices in same RL can be operated by one pulse simultaneously. e, The waveform of the one-pulse modulation test for 8 devices with various gate voltages for different states, each state is performed for 50 cycles. f, The conductance distribution of OPM. 8 distinguishable levels can be achieved, indicating the potential for fast data storage. g, Comparison of the two-segment pulse and one-segment pulse modulation, where 2 segs pulse shows narrower distribution of the conductance error with better modulation stability.\u003c/p\u003e","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/8689bcfb34ad0c235c9fc1e3.jpg"},{"id":104180193,"identity":"498dd8da-584a-48e1-a7ee-5f913904553f","added_by":"auto","created_at":"2026-03-08 17:11:49","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":8769529,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSchematic diagram of the 1PnR architecture and the rolling exposure strategy for fast image acquisition.\u003c/strong\u003e \u003cstrong\u003ea,\u003c/strong\u003e Each pixel sensor is connected to a gate line of the 1T1R array, constructing one-pixel-multiple-memristor architecture for sequential image acquisition. \u003cstrong\u003eb,\u003c/strong\u003eThe scheme and electric board of the transmission circuit, where the pixel’s photo-resistive signal is converted to gate voltage by a differential amplifier. \u003cstrong\u003ec,\u003c/strong\u003e Distribution of the device conductance on electric board using OPM writing strategy. 5 devices on different channels are successfully modulated to 8 states. \u003cstrong\u003ed,\u003c/strong\u003e The rolling exposure strategy starts from the first column of the image using OPM method, then rolls down one by one, until the last column. \u003cstrong\u003ee, \u003c/strong\u003eExperimental result using rolling exposure strategy for 3 samples of Weizmann dataset, left is the target image while right is the restored image read from the memristor array. \u003cstrong\u003ef, \u003c/strong\u003eExperimental results of the portrait image using rolling exposure strategy, the facial features in restored image are still clear for classification with negligible noise. \u003cstrong\u003eg,\u003c/strong\u003e Distribution of the sensing error of the restored image in \u003cstrong\u003ee\u003c/strong\u003e, which is consistent with the OPM performance with normal distribution. \u003cstrong\u003eh,\u003c/strong\u003e Comparison of delay and power consumption between the proposed 1PnR architecture and a typical digital system composed of CMOS sensors and DDR5 memory.\u003c/p\u003e","description":"","filename":"Figure3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/5e554eea3687e9ea59c19d54.jpg"},{"id":104180197,"identity":"43c45466-7100-44e3-9920-52fbf17e03fa","added_by":"auto","created_at":"2026-03-08 17:11:49","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":12062127,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eData-In-situ computing network for image processing.\u003c/strong\u003e \u003cstrong\u003ea,\u003c/strong\u003e Diagram of the data-in-situ computing network architecture. The voltage vector carrying network weights is applied to the data-storage memristor for data-in-situ computing, then the results are fed into a perception for classification. The data-in-situ weights are updated through backpropagation calculation, along with the perception weights in training process. \u003cstrong\u003eb \u003c/strong\u003ethe schematic diagram of the data-in-situ computing network for Weizmann human action dataset based on memristor array. On the left are examples of the dataset. Two voltage vectors carrying the network weights are applied to the array for data-in-situ computing, the results are then fed to a two-layer perception realized on memristor array for classification. \u003cstrong\u003ec-e\u003c/strong\u003e recognition results for the Weizmann dataset of the network simulation, the input voltage directions, image noises and input weights noises are analyzed, respectively.\u003c/p\u003e","description":"","filename":"Figure4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/b682fa5101ae64412d1084b3.jpg"},{"id":104404060,"identity":"85fde3ee-8a7b-4920-8b4e-9c90f34ce47d","added_by":"auto","created_at":"2026-03-11 12:19:40","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":14236655,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eExperimental demonstration of the 1PnR hardware system.\u003c/strong\u003e \u003cstrong\u003ea,\u003c/strong\u003e Optical image of the memristive hardware system, where the memristor array is enlarged. The color blocks in the array reveal the partitions to implement the data-in-situ computing network for Weizmann classification. \u003cstrong\u003eb,\u003c/strong\u003e Weight-transfer errors of the HD (48×8) and FC (8×10) layers to the memristor array by differential group. The transfer errors are limited within 6 μS by an array modulation script. \u003cstrong\u003ec-d,\u003c/strong\u003e Data-in-situ computing current and perception computing current from the hardware system for the 3 test samples. \u003cstrong\u003ee,\u003c/strong\u003e Recognition results for the Weizmann dataset of the hardware system, with an accuracy of 95.7%. The accuracy decay (2.1%) compared to the ideal values may be caused by the mapping error of network weight. \u003cstrong\u003ef, \u003c/strong\u003eComparison between the ideal calculation current and the hardware output current of all the test samples. \u003cstrong\u003eg, \u003c/strong\u003eClassification results of both simulation in HD and FC layer of perception, experiments in HD layer and simulation in FC layer, both experiments in FC layer. The error bars represent mean value and standard deviation of network accuracy obtained from 10 repeated experiments with random noise. \u003cstrong\u003eh, \u003c/strong\u003eEstimated energy consumption of the data-in-situ computing architecture and a typical CMOS system for Weizmann classification, a ~160 times reduction can be achieved.\u003c/p\u003e","description":"","filename":"Figure5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/a42c09047949dd3d8d72b8a9.jpg"},{"id":109158080,"identity":"5739617d-0feb-46f7-815d-d4d6d9356cd5","added_by":"auto","created_at":"2026-05-13 07:06:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":48755679,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/048d47d8-6e4d-4fc8-91b4-f30d467408d4.pdf"},{"id":104180190,"identity":"aa8d213e-8312-4be9-896f-7ff455f6efd0","added_by":"auto","created_at":"2026-03-08 17:11:49","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":55550,"visible":true,"origin":"","legend":"Inventary of Supplementary Information","description":"","filename":"InventaryofSupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.docx","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/719afc205554f33b3c883450.docx"},{"id":104180191,"identity":"30d66cee-970c-47cb-9a1f-d19624861d10","added_by":"auto","created_at":"2026-03-08 17:11:49","extension":"xlsx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":2194536,"visible":true,"origin":"","legend":"Dataset 1","description":"","filename":"SourceData.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/6d2a9d5998441bc703901ecd.xlsx"},{"id":104404940,"identity":"5b2445d7-53dd-4747-821d-55dd635dbaba","added_by":"auto","created_at":"2026-03-11 12:21:25","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":13159666,"visible":true,"origin":"","legend":"Supplementary Information_Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision","description":"","filename":"SupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/db7732bf0d1bec0945199b96.pdf"},{"id":104180194,"identity":"e5032513-67ed-43a0-99cf-12b8666dd9bb","added_by":"auto","created_at":"2026-03-08 17:11:49","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":11362908,"visible":true,"origin":"","legend":"Supplementary Information_Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision","description":"","filename":"SupplementaryInformationDataInsituComputingwithOnePixelMultipleMemristorArchitectureforNeuromorphicSequentialVision.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6505999/v1/13f4efd1dae0291906245515.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision","fulltext":[{"header":"Introduction","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe human visual system is distinguished by its remarkably energy efficiency\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. Studying and emulating the human visual system offers a promising approach to developing high-performance and low-power artificial vision systems\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e,\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates the working process of the human visual system. In essence, the human visual system can be broadly divided into two key components: the retina and the brain. The retina cells convert light signals into neural spiking signals, which are then transmitted to the brain via the optic nerve. In the brain, synaptic cells process these spiking signals, enabling humans to visually perceive the external environment\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e. It\u0026rsquo;s been reported that there is a visual working memory mechanism in the brain, which stores temporary visual information and performs pre-process for the following neural classification\u003csup\u003e\u003cspan additionalcitationids=\"CR6\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e. This working mode can greatly reduce the visual transmission in the brain nerve and further improve the process efficiency.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eRecently, various artificial vision systems intimating the biological visual processing mechanism have been proposed\u003csup\u003e\u003cspan additionalcitationids=\"CR9 CR10 CR11 CR12 CR13 CR14 CR15\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e. Among them, memristors are promising candidates for visual information storage and in-memory neuromorphic processing\u003csup\u003e\u003cspan additionalcitationids=\"CR18\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e. The memristor based neuromorphic architecture can avoid the constant data conversion and transmission between physical separated image sensors and processing units in von Neumann architectures, thus provides high power and time efficiency\u003csup\u003e\u003cspan additionalcitationids=\"CR21 CR22\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. In the current implemented systems, images are captured by sensors or optoelectronic devices and transmitted to memristor arrays that store weights for network computation\u003csup\u003e\u003cspan additionalcitationids=\"CR25 CR26\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e. While this analog-domain architecture effectively improves processing efficiency, it still requires the transmission of image data from the sensing-storage array to the computing array, resulting in additional latency and power consumption. In addition, the sensor and memristor device is usually integrated as one-sensor-one-memristor\u003csup\u003e\u003cspan additionalcitationids=\"CR29\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e, or multi-sensor-one-memristor structure\u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e,\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e. In sequential image processing conditions, read-out and erasing operation is required for previously stored image data before sensing new pixel data, which limits the image sensing speed and energy efficiency.\u003c/p\u003e \u003cp\u003eIn this work, inspired by the high-efficiency working mode of the visual nerves in the human system, we propose a neuromorphic visual architecture, namely, one-pixel- multiple-memristor (1PnR) computing architecture for sequential image sensing, storing and processing, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The light sensors and memristor array, similar to biological retina, are connected by analog circuits as optic nerves for conversion and transmission of the sensory data. The array is divided into 2 parts, one part serves as visual working memory, which stores image data and performs data-in-situ for pre-process. The other part serves as neuromorphic computing core mimicking visual cortex, which stores network weights and carries network computing for classification. Furthermore, the sensors and memristor array are integrated by a gate multiplexing architecture, where a sensor is connected to a gate line of the 1T1R array, offering the ability of storing a single channel of optic input to multiple memristor cells simultaneously, functioning as retinal divergent connectivity for visual information processing. Based on the structure, a rolling exposure strategy for fast sequential image acquisition is then proposed. With the pixel image sensed and stored column by column to the memristor array in analogue domain, both time and energy efficiency can be achieved compared to traditional CMOS system. Moreover, a data-in-situ computing network is proposed for fast image processing. Voltage vectors carrying network weights are applied to the image storage array to implement data-in-situ computing, which avoids transmission of the image data in traditional memristive neuromorphic systems. Finally, a 1PnR hardware system is established for verification of sequential visual sensing and processing. The experimental results reveal 95.7% recognition accuracy for Weizmann human action flow dataset, demonstrating the great classification capacity of the data-in-situ network. Compared to typical CMOS-based systems, the proposed architecture is estimated to have ~\u0026thinsp;2000 times reduction of time latency for image sensing and storage, and ~\u0026thinsp;160 times reduction of energy consumption for image processing. These results demonstrate great potential for the proposed 1PnR novel architecture to construct neuromorphic visual systems.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eDevice characterization and One-Pulse Modulation method\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eAn 1T1R memristor array with 8k scale (64 rows \u0026times; 128 columns) is deployed in this work. Each cell contains a TiN/TaO\u003csub\u003ex\u003c/sub\u003e/HfO\u003csub\u003ex\u003c/sub\u003e/TiN memristor and a transistor connected in series, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ea. And the schematic diagram of the 1T1R structure as well as the transistor\u0026rsquo;s output characteristic are presented in \u003cb\u003eSupplementary Fig.\u0026nbsp;1\u003c/b\u003e. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eb is the typical quasi-DC curve of the 1T1R structure. With a 1.6 V gate voltage in SET voltage sweep (0 to 1.0 V) and 3.5 V gate voltage in RESET sweep (0 to -1.15 V), a Resistive Switching (RS) window, defines as the ratio between the High Resistance State (HRS) and the Low Resistance State (LRS), can reach to ~\u0026thinsp;10. Electric pulses are then applied to the 1T1R cell, pulse trains with gradually rising amplitudes (0.8 V to 1.3V for SET and 1.3 V to 2.4 V for RESET, width time 80 ns) are used as the modulation signal to achieve more resistance states and larger switching window, and the gate voltage is set to 3 V to turn on the transistor. The result in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec shows the continuous change of the device state under SET and RESET operations, with 100 pulses each. After each pulse, a small pulse of 0.1 V follows behind to read the device conductance. Indeed, the pulse parameters (amplitude and width) are optimized through multiple experiments to achieve better resistive-switching window and linearity. The results reveal that the 1T1R cell has analogue switching behavior under pulse modulation, demonstrating its great potential in multi-value image storage and neuromorphic computing applications.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe pixel sensor is achieved by ITO/ZnS/TiN photo-resistive structure. The top electrode is deposited with transparent ITO. ZnS is deposited as a functional film due to its excellent photoactivity. An UV LED with a wavelength of 365 nm is used to investigate optoelectronic characteristics. The experimental results plotted in \u003cb\u003eSupplementary Fig.\u0026nbsp;2\u003c/b\u003e show that the device has great cycle-to-cycle (C2C) and (device-to-device) D2D uniformity, with the ability to sense the light pulse with various intensities, which can effectively support the implementation of the proposed 1PnR system.\u003c/p\u003e \u003cp\u003eTo achieve high-frame processing of sequential visual information, an open-loop fast modulation method is then proposed for the 1T1R cell, aiming to significantly improve the writing speed while partially sacrificing precision. This method applies only one parameter-fixed pulse to the 1T1R cell, which can be called as One-Pulse Modulation (OPM). As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ed, the pulse is applied to the RL, and the SLs are all grounded. Based on the inter-connection of 1T1R array, all the devices in the same RL can be modulated at the same time, and different gate voltages can be applied to control the modulation states in different SLs. The total pulse width is 1 \u0026micro;s, corresponding to a theoretical modulation frequency of 1 MHz. In our experiments, 8 gate voltage values are used as representatives to evaluate OPM capability and writing fluctuation. The corresponding current response of 1T1R cell is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ee, 8 devices with different gate voltages are repeatedly tested 50 times. Before each modulation, the devices are initialized to the same state (30 \u0026micro;S). As the gate voltage gradually drops from 2.0 V to 1.21 V, the response current of the device also gradually decreases. The current fluctuation is relatively large at the initial stage of the pulse and becomes concentrated at the pulse falling edge, indicating that the writing state tends to be uniform under pulse amplitude. The 8 gate voltages ranging from 1.21 V to 2.0 V leading the corresponding conductance averagely distributes in 40 \u0026micro;S\u0026sim;320 \u0026micro;S, with an interval of 40 \u0026micro;S. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef shows the distribution of the 8 conductance states in the form of box plot. The rectangular box represents the 25%\u0026sim;75% distribution range, and the upper and lower edges represent 1.5 times of the interquartile range. All the device conductance fluctuates near the target value, except for some outliers. And the overall fluctuation range is within 20 \u0026micro;S (half of the interval). It should be noted that the pulse waveform is designed with two voltage segments (1.2 V and 1.6 V) to prevent impact responses caused by voltage jumps from affecting the modulation stability. The experiments of 1 segment pulse are also performed and plotted in \u003cb\u003eSupplementary Fig.\u0026nbsp;3\u003c/b\u003e, and modulation error of device conductance is compared in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003eg. The results reveal that 2-seg pulse shows narrow distribution in modulation error, which has better stability. Furthermore, retention tests of the 8 states are performed, the result plotted in \u003cb\u003eSupplementary Fig.\u0026nbsp;4\u003c/b\u003e shows that the device states can last up to 50000 s, with only small fluctuation. These results demonstrate the great compatibility of the 1T1R cell with OPM method.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eRolling exposure on One-Pixel-Multiple-Memristor structure\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn this work, an architecture integrating the pixel sensors and memristor array is proposed for neuromorphic near-sensor computing system, which we call One-Pixel-Multiple-Memristor (1PnR) Structure. The schematic diagram of the 1PnR structure is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea, where pixel sensors are connected to the gate lines (WL) of the 1T1R array one by one through analog conversion circuits. The circuit has a very simple structure, containing a voltage division part and a differential amplifier, which can convert the light-induced resistance change of the pixel sensor to volage values. To sufficiently verify the 1PnR structure, 30 channels of image sensors and conversion circuits are experimentally demonstrated, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb. The circuit parameter is adjusted to fit the gate voltage range of the 1T1R array, and the transmission relationship of the circuit from light density to gate voltage, as well as the transmission speed is plotted in \u003cb\u003eSupplementary Fig.\u0026nbsp;5.\u003c/b\u003e The 8k memristor chip is deployed on a transfer board, where all the pins can be controlled by the 1PnR system to apply OPM pulse. Then pixel stimuli with 8 different intensities is applied to 5 randomly picked channels to verify the sensing-storage ability of the 1PnR board. The light intensities are experimentally picked to achieve the same gate voltages in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef for comparison, and 50 repeated tests are performed under each light intensity. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec shows the distribution of memristor conductance after the experiments. The statistical results are presented in box plots, with different devices distinguished by colors. As can be seen from the figure, the modulated conductance fluctuates around the target value, and the distribution maintains high consistency between channels. Compared with the results in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ef, the 1T1R cells in 1PnR achieved the same 8-state modulation under the same parameters, and the distribution range was also roughly the same. The results indicate that the conversion circuit makes considerable compatibility with the pixel sensors and 1T1R cell, which can support the 1PnR structure to achieve fast storage of pixel images.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eBased on 1PnR architecture and OPM method, a rolling exposure strategy (RES) is then proposed for fast image acquisition. The workflow of RES is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ed, taking the Weizmann human action dataset as example, where the image is binarized and cropped to 24\u0026times;40. On the left side is a column of 24-pixel sensors grown on a silicon wafer, which are connected to the gate lines of a 24\u0026times;40 1T1R array through conversion circuits, establishing 24 channels of 1PnR structure. Before image exposure, the memristors array are initialized to same HRS, and the SLs are all grounded. When the first column of the pixel image is under exposure, the pixel information can be converted to gate voltages by the conversion circuits at the same time. Then, the OPM is applied to the first column (RL\u003csub\u003e0\u003c/sub\u003e) to store the pixel information in parallel. Next, the same exposure method is used to store the image to memristor array column by column, until the entire image is exposed and stored.\u003c/p\u003e \u003cp\u003eUtilizing the fabricated pixel sensors and transmission circuit, a 1PnR hardware verification system is implemented, as shown in \u003cb\u003eSupplementary Fig.\u0026nbsp;6\u003c/b\u003e. UV LED light sources are employed, corresponding one-to-one with the fabricated pixel sensors. The image is converted into UV light signals based on pixel values and directly irradiated to the optical sensors, which simulates the optical exposure process. And the detailed process of the exposure experiment based on the hardware system is illustrated in \u003cb\u003eSupplementary Fig.\u0026nbsp;7.\u003c/b\u003e Besides, to achieve RES verification with larger image size based on the prepared image acquisition hardware, the target column of image (up to 30 pixels) is reshaped to a 5\u0026times;6 array for UV irradiation. The light signals received by the optical sensors are then reshaped back to one column through the 30-channel conversion circuits and then written to target column of the connected memristor array, which is shown in \u003cb\u003eSupplementary Fig.\u0026nbsp;8.\u003c/b\u003e\u003c/p\u003e \u003cp\u003eRES experimental verification is firstly conducted using 3 sample images picked in the Weizmann dataset, containing actions of run, jump and walk. Each sample is processed into a 24\u0026times;40-pixel image, formed by concatenating 4 consecutive 24\u0026times;10-pixel frames of human motion, as illustrated in \u003cb\u003eSupplementary Fig.\u0026nbsp;12\u003c/b\u003e. During the image acquisition experiment of each sample image, 24 sensing channels are used according to the column pixels, and a total of 40 rolling exposure steps are required for the whole sample image. After RES experiment, the target image and restored image from the memristor array is plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ee. It can be seen from the results that the action features in restored images are still clear for classification with negligible noise.\u003c/p\u003e \u003cp\u003eTo further verify the RES strategy, hardware experiment utilizing gray-scaled image and whole 8k memristor array is performed. A portrait image is chosen as an example, which is gray scaled and cropped to 128\u0026times;64 scale to match the memristor array. According to the design of RES, if the 128 gate lines of the 8k array are fully utilized, a 128\u0026times;64 image can be perceived in just 64 exposure operations. However, to match the 30-channel circuit prepared in the experiment, the image is divided into 5 groups with 30 pixels, as shown in \u003cb\u003eSupplementary Fig.\u0026nbsp;9\u003c/b\u003e. The rolling exposure of 30\u0026times;64 is completed within each group first, and then the next group of images is exposed in sequence. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef shows target image and restored image from the memristor array after the RES experiment. Although the restored image has pixel noise caused by writing errors, the original facial features are still clearly distinguishable, which is sufficient to support applications such as face recognition. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eg shows the distribution of the writing error of the portrait image, which is consistent with the OPM experiment with normal distribution. Indeed, the standard deviation becomes slightly larger due to the conversion of gray pixel values. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eh shows the performance comparison between the 1PnR architecture with typical CMOS system consisting of image sensors and DDR5 memory. The comparison details are introduced in in \u003cb\u003eSupplementary Note 1\u003c/b\u003e. The results reveal that the proposed 1PnR system has ~\u0026thinsp;2000 times reduction of time latency and 1.8 times reduction of energy consumption, which further demonstrates the great potential in fast image acquisition applications.\u003c/p\u003e \u003cp\u003eIn fact, the proposed 1PnR architecture exhibits remarkable flexibility and scalability for image acquisition. For future optical system applications, when the sensor array scale becomes sufficiently large, it can function as an area sensor for global exposure. Following the operational principle of line-scan CCD cameras, the entire image can be first exposed globally, and then its pixels can be transferred column by column to the memristor array for storage. If multiple memristor arrays are employed, each column of sensors can be connected to a different memristor array. After exposure, writing pulses can be applied simultaneously to different arrays, enabling the entire image to be written into separate memristor arrays at the same time, offering very fast image exposure, as illustrated in \u003cb\u003eSupplementary Fig.\u0026nbsp;10\u003c/b\u003e. Moreover, if the scale of the memristor array is sufficiently large, all sensor units in the area array can be connected to the same array, allowing an entire image to be written into a single column of memristors.\u003c/p\u003e\n\u003ch3\u003eData-in-situ computing network for sequential vision\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn traditional memristor-based neuromorphic system, the network weights are mapped to the conductance values of the memristor array, and the samples are mapped to voltage signals by the artificial neurons. The memristor array can achieve the MAC operation efficiently through Ohm's law and Kirchhoff's current law in analog domain. However, in near-sensor integrated computing system, the image data has already been stored in the memristor array. Performing calculations in traditional way requires the image data to be transferred from storage array to computing array, resulting in a large amount of transmitted data, which reduces the overall efficiency of the computing system. Here, we utilize the MAC feature of the data storage array and propose a data-in-situ computing network for memristor-based neuromorphic systems. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea, the first part of network weights is mapped to voltage weight vector (VWV), which are applied to the data-storage memristor for data-in-situ computing. Then the computing results are fed into a perception for classification, which is implemented on weight-storage memristor array. The data-in-situ computing weights are updated through backpropagation calculation, along with the perception weights in training process. Utilizing data-in-situ network, the image is processed where it is stored, only the data-in-situ computing results with greatly decreased data amount are required to transmission for classification, which is consistent with the high-efficiency mechanism of the visual working memory in the human brain.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eTo evaluate data-in-situ computing network for sequential image processing based on the 1PnR system, the Weizmann Dataset is used for evaluation, which has 10 actions, containing bend, jack, jump, pjump (jump in place), run, side, skip, walk, wave1 (in one hand) and wave2 (in two hands). Each action is performed by 9 people. The dataset is already binarized and aligned from video clips. In our simulation, the dataset is cropped to 24\u0026times;10. To make sure that every sample includes a whole period of periodic actions (such as walk, run and wave), 4 frames are averagely picked from 12 consecutive frame sequences in the video clip to construct one sample, as shown in \u003cb\u003eSupplementary Fig.\u0026nbsp;12\u003c/b\u003e. Furthermore, a leave-one-out strategy is used in the sample preparation, which is, 8 people are chosen for training (800 samples for 10 actions) and the rest 1 person is used for testing (100 samples for 10 actions).\u003c/p\u003e \u003cp\u003eThe network structure is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb, where one sample is stored to the 24\u0026times;40 memristor array by RES. For Weizmann dataset classification, two VWVs (40\u0026times;1\u0026times;2) are applied from the column of the array for data-in-situ computing successively, and the results are joined together (48\u0026times;1) and then fed into 48\u0026times;8\u0026times;10 two-layer perception for image classification. The data-in-situ computing network is firstly verified by simulation, 97.82% accuracy can be obtained for the dataset after 100 training epochs, showing great efficiency of the network structure. Furthermore, the impact of VWV direction for data-in-situ computing is analyzed. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec, when 2 VWVs are both applied from the column of the image, the classification accuracy reaches best to 97.82%, if 2 VWVs are both applied from rows of the array, the accuracy reduces to 92.75%. And if 1 VWV from column and 1 from row, the accuracy is 92.02%. It can be conducted that the sample can keep more features after data-in-situ computing in column direction and achieve better classification performance. Finally, the impact of image noise and weight noise on the network is simulated, which are plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ed and Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ee, respectively. The noise percentage is applied in randomly generated normal distribution, with mean value of 1. Each standard deviation is simulated for 100 cycles. The results show that as the noise deviation rises, the network accuracy decreases accordingly. But the noise on image data shows less impact on the performance, indicating better robustness in the image sensory error.\u003c/p\u003e \u003cp\u003eFinally, Experimental verification of data-in-situ computing network system is performed on the 1PnR hardware system. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea shows the optical photo of the memristor modulation board. With the support of core controller (FPGA) and array read-write circuit (ADC\u0026amp;DAC), memristor modulation and the MAC computation can be achieved by the system. The mapping layout of the data-in-situ computing network in the memristor array is enlarged. columns 0\u0026sim;71 is assigned for sample image storage and data-in-situ computing, which can store 3 samples, as shown in the purple box. The hidden layer (HD, 48\u0026times;8) of the perception needs 16 columns after differential process and is mapped to columns 72\u0026sim;87 of the array. Besides, the fully connected layer (FC, 8\u0026times;10) requires 20 columns, which is mapped to the final 88\u0026sim;127 columns. The network weights (HD and FC) are mapped to the memristor array in range of 100 \u0026micro;S\u0026sim;200 \u0026micro;S by a modulation script at the host computer. The script adopts a close-loop modulation strategy, and the maximum conductance error is limited to 6 \u0026micro;S to achieve a balance between weight modulation speed and network accuracy. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb shows the results of the HD and FC layer mapped to the memristor array, which presents 100% yield for the network computation.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eDuring the experiment, every 3 samples are stored to the assigned area through OPM by experiments shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea. Then the data-in-situ computing of the image is performed by the hardware system, and the output currents of the 3 representative samples are plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ec. The results are obvious differences between the 3 samples, which indicates that data-in-situ computing can successfully extract the sample features for subsequent classification. Next, the results are fed into the HD and FC layer on the hardware successively. The output current of the FC layer for the 3 representative samples is plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ed. The largest current is remarked in red, representing the classification result. It\u0026rsquo;s clear that the 3 samples are both successfully classified. Then, all the test samples are applied to the system for hardware verification. A typical recognition result of the hardware system is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ee, with an accuracy of 95.7%. The 2.1% decay compared to the simulation results may be caused by the mapping error of the network weight. The comparison between the ideal calculation current and the hardware output current of all the test samples in FC layer is plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ef, which shows considerate consistency.\u003c/p\u003e \u003cp\u003eTo further evaluate the system performance and stability, repeated experiments are performed combining simulation and hardware measurements. During simulations, random conductance errors in image acquisition process with normal distribution are applied, which is consistent with practical experiment. The results from both simulation in HD and FC layer, experiments in HD layer and simulation in FC layer, both experiments in FC layer are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eg, each combination is repeatedly performed for 10 times. The network accuracy with both simulating in two layers reaches 96.5\u0026thinsp;\u0026plusmn;\u0026thinsp;0.5 (mean value 96.5 with standard deviation 0.5) and slightly decrease to 96.1\u0026thinsp;\u0026plusmn;\u0026thinsp;0.5 when using experimental results in HD layer and simulation in FC layer. The accuracy loss in experimental results may be caused by the random electric noise in the hardware system. At last, the result of both hardware experiments in two layers reaches 94.7\u0026thinsp;\u0026plusmn;\u0026thinsp;0.5, demonstrating the good robustness of the hardware system.\u003c/p\u003e \u003cp\u003eFurthermore, the energy consumption of the data-in-situ computing architecture for Weizmann Dataset is estimated, details are introduced in \u003cb\u003eSupplementary Note 2\u003c/b\u003e. A CMOS system consisting of a typical digital accelerator-based (HNPU-based) system\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e with DDR5 memory\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e is estimated for comparison, which are plotted in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ef. The results show that the proposed data-in-situ computing architecture has about 160 times energy efficiency than typical CMOS systems.\u003c/p\u003e \u003cp\u003eTo evaluate the general classification ability of data-in-situ computing network, the benchmark is then performed on the commonly used MNIST and Fashion MNIST dataset with noise analysis. The experimental results on MNIST dataset are presented in \u003cb\u003eSupplementary Fig.\u0026nbsp;13\u003c/b\u003e and \u003cb\u003eSupplementary Note 3\u003c/b\u003e, which demonstrates a 95.9% (simulation) and 92.4% (experimental) classification accuracy. The simulation results on Fashion MNIST dataset are shown in \u003cb\u003eSupplementary Fig.\u0026nbsp;14.\u003c/b\u003e Given the greater complexity of Fashion MNIST images compared to MNIST, 4 voltage vectors are applied to the image for data-in-situ computing to extract more image features, which achieves an 87.17% recognition accuracy. Finally, the comparison of the 1PnR system and other representative works about integrated sensing-storage-computation system is performed and summarized in \u003cb\u003eSupplementary Note 4\u003c/b\u003e. These results further demonstrate the great performance of the data-in-situ computing network for future neuromorphic visual systems.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn this work, a one-pixel-multiple-memristor (1PnR) computing architecture is proposed for sequential image sensing, storing and processing. The sensors and memristor array are integrated by a gate multiplexing architecture, offering the ability of storing a single channel of optic input to multiple memristor cells simultaneously. Then, a rolling exposure strategy for fast sequential image acquisition is proposed. With the pixel image sensed and stored column by column to the memristor array in analogue domain, both time and energy efficiency can be achieved compared to traditional CMOS system. Moreover, a data-in-situ computing network based on memristor array is proposed for fast image processing. Voltage vectors carrying network weights are applied to the image storage array to implement data-in-situ computing, which avoids transmission of the image data in traditional memristive neuromorphic systems. Finally, a 1PnR hardware system is established for verification of sequential visual sensing and processing. The experimental results reveal 95.7% recognition accuracy for Weizmann human action flow dataset, demonstrating the great classification capacity of the data-in-situ network. Compared to typical CMOS-based systems, the proposed architecture is estimated to have 2000 times reduction of time latency for image sensing and storage, and 160 times reduction of energy consumption for image processing, demonstrating the great potential for constructing neuromorphic visual systems.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eSample fabrication\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe fabrication process of the ITO/ZnS/TiN sensor. The bottom electrode (TiN, ~\u0026thinsp;40 nm) is deposited by magnetron sputtering, then it is patterned by lithography and wet etching. After the second lithography process, the functional material zinc Sulfide (ZnS, ~\u0026thinsp;30 nm) and transparent top electrode ITO (~\u0026thinsp;30 nm) are deposited by magnetron sputtering, respectively. The effective device size is determined by the round area in the bottom electrode, with a diameter of 200 \u0026micro;m.\u003c/p\u003e \u003cp\u003eFabrication of 8k memristor chip. The chip has a 64\u0026times;128 one-transistor-one-resistor (1T1R) memristor array. The transistors in the array are fabricated using a standard 180 nm Si complementary metal\u0026ndash;oxide semiconductor process. The memristor cell has structure of TiN/TaO\u003csub\u003ex\u003c/sub\u003e/HfO\u003csub\u003e2\u003c/sub\u003e/TiN. 12 nm HfO\u003csub\u003e2\u003c/sub\u003e is deposited by atomic layer deposition as resistive switching material, then 45 nm TaO\u003csub\u003ex\u003c/sub\u003e is deposited by magnetron sputtering as a thermal enhanced layer. The TiN electrode is deposited by magnetron sputtering.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eMeasurement method\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe electrical test of single device is performed with the Keithley 4200 semiconductor parameter analyzer. The optical light is provided by UV LEDs with a wavelength of 365 nm, and a signal generator is used to control the UV light pulse. The memristor chip and network calculation is measured by our memristor-evaluation system, which is equipped with core controller (FPGA) and array read-write circuits (ADC\u0026amp;DAC).\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e\n\u003ch3\u003eNetwork simulation method\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe network simulation is conducted using Python environment and PyTorch framework. The image samples are from Weizmann Dataset of human action flows, which are preprocessed to 24\u0026times;40 binary images. During the training process, two sets of weights vectors applied to the samples are treated as updatable parameters and iteratively updated using the backpropagation algorithm. The ReLU activation function is employed, the cross-entropy loss function is used for error computation.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eResearch participant consent\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe authors affirm that human research participants provided informed consent for publication of the images in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef, Supplementary Fig.\u0026nbsp;8 and Supplementary Fig.\u0026nbsp;9.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":" \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eData availability\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe source data generated in this study are provided in the Source Data file. Additional data related to this paper can be requested from the authors.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eCode availability\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe code of data-in-situ computing simulation will be available from the corresponding authors upon request.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\u003cp\u003e \u003ch2\u003eCompeting interests\u003c/h2\u003e \u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor contributions\u003c/h2\u003e \u003cp\u003eY.S., P.W.T. and W.W. designed the experiments. Y.S. designed and fabricated the sample devices and conducted device-level electrical experiments. C.L.C. and Y.N.W. designed and fabricated the circuit boards. J.R.S. designed and simulated the task algorithms. Y.S. and P.W.T. designed data-in-situ computing architecture. H.X., R.R.C., C.L., and B.S. assisted with data analysis and interpretation. Y.S. and P.W.T. co-wrote the manuscript. All authors discussed the results and revised the manuscript. W.W., Y.C.Y. and Q.J.L. supervised the research.\u003c/p\u003e\u003ch2\u003eAcknowledgments\u003c/h2\u003e \u003cp\u003eThis work was supported by the National Key R\u0026amp;D Program of China under Grant Nos. 2024YFA1208800 (Q.L.) and 2023YFB4502200 (Y.Y.), Innovation Research Foundation of National University of Defense Technology under Grant Nos. ZK24-09 (Y.S.) and 25-ZZCX-JDZ-18 (W.W.), National Natural Science Foundation of China under Grant Nos. 62404253 (Y.S.), 62304254 (W.W.) and U23A20322 (Q.L.), Guangdong Provincial Key Laboratory of In-Memory Computing Chips under Grant No. 2024B1212020002 (Y.Y.), Shenzhen Science and Technology Program under Grant No. JCYJ20241202125907011 (Y.Y.), as well as Beijing Natural Science Foundation under Grant Nos. L234026 (Y.Y.) and L257010 (Y.Y.).\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eGu L et al (2020) A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature 581:278\u0026ndash;282\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou F et al (2020) Near-sensor and in-sensor computing. Nat Electron 3:664\u0026ndash;671\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLong Z et al (2023) A neuromorphic bionic eye with filter-free color vision using hemispherical perovskite nanowire array retina. Nat Commun 14:1972\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang Y et al (2024) Bioinspired sensing-memory-computing integrated vision systems: biomimetic mechanisms, design principles, and applications. Sci China Inf Sci 67:151401\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiebe S et al (2012) Theta coupling between V4 and prefrontal cortex predicts visual short-term memory performance. Nat Neurosci 15:456\u0026ndash;462\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu J et al (2020) Stable maintenance of multiple representational formats in human visual short-term memory, \u003cem\u003eProc. Natl. Acad. Sci.\u003c/em\u003e 117, 32329\u0026ndash;32339\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBenigno GB et al (2023) Waves traveling over a map of visual space can ignite short-term predictions of sensory input. Nat Commun 14:3409\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin H et al (2025) In situ training of an in-sensor artificial neural network based on ferroelectric photosensors. Nat Commun 16:421\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeng G (2023) Retinomorphic hardware for in-sensor computing. InfoMat 5:e12473\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXie P et al (2024) Emerging optoelectronic artificial synapses and memristors based on low-dimensional nanomaterials. Appl Phys 11:011303\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen J et al (2023) Optoelectronic graded neurons for bioinspired in-sensor motion perception. Nat Nanotechnol 18:882\u0026ndash;888\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang P et al (2023) Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction. Nat Commun 14:6736\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou F et al (2019) Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat Nanotechnol 14:776\u0026ndash;782\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhou G et al (2023) Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing. Nat Commun 14:8489\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang Z et al (2022) All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat Nanotechnol 17:27\u0026ndash;32\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHu L et al (2021) All-optically controlled memristor for optoelectronic neuromorphic computing. Adv Funct Mater 31:2005582\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSebastian A et al (2020) Memory devices and applications for in-memory computing. Nat Nanotechnol 15:529\u0026ndash;544\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang Z et al (2020) Resistive switching materials for information processing. Nat Rev Mater 5:173\u0026ndash;195\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYao P et al (2020) Fully hardware-implemented memristor convolutional neural network. Nature 577:641\u0026ndash;646\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu Z et al (2025) A memristor-based adaptive neuromorphic decoder for brain\u0026ndash;computer interfaces. Nat Electron 8:362\u0026ndash;372\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiang X et al (2024) Physical reservoir computing with emerging electronics. Nat Electron 7:193\u0026ndash;206\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang W et al (2021) Neuromorphic Motion Detection and Orientation Selectivity by Volatile Resistive Switching Memories. Adv Intell Syst 3:2000224\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi C et al (2018) Analogue signal and image processing with large memristor crossbars. Nat Electron 1:52\u0026ndash;59\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang Z et al (2022) In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array. Nat Commun 13:6590\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLao J et al (2022) Ultralow-power machine vision with self-powered sensor reservoir. Adv Sci 9:2106092\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShan X et al (2022) Plasmonic Optoelectronic Memristor Enabling Fully Light-Modulated Synaptic Plasticity for Neuromorphic Vision. Adv Sci 9:2104632\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang S et al (2020) Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception. Natl Sci Rev 8:nwaa172\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen S et al (2018) An Artificial Flexible Visual Memory System Based on an UV-Motivated Memristor. Adv Mater 30:1705400\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeo S et al (2018) Artificial optic-neural synapse for colored and color-mixed pattern recognition. Nat Commun 9:5106\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDang B et al (2023) One-phototransistor\u0026ndash;one-memristor array with high-linearity light-tunable weight for optic neuromorphic computing. Adv Mater 35:2204844\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDang B et al (2024) Reconfigurable in-sensor processing based on a multi-phototransistor\u0026ndash;one-memristor array. Nat Electron 7:991\u0026ndash;1003\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang H et al (2025) Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing. Nat Nanotechnol 20:93\u0026ndash;103\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAhn H-A et al (2024) A 1.01-V 8.5-Gb/s/pin 16-Gb LPDDR5x SDRAM With Advanced I/O Circuitry for High-Speed and Low-Power Applications. IEEE J Solid-State Circuits 59:3479\u0026ndash;3487\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHan D et al (2021) HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching. IEEE J Solid-State Circuits 56:2858\u0026ndash;2869\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6505999/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6505999/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eNeuromorphic vision systems based on memristors offer an energy-efficient approach to artificial vision, yet traditional pixel(s)-to-one-memristor architectures remain inefficient in dynamic image processing due to limited temporary storage. Here, inspired by human visual working memory, we propose a novel one-pixel-multiple-memristor (1PnR) architecture with a rolling exposure strategy for fast sequential image acquisition. Furthermore, a data-in-situ computing network for efficient image processing is developed. With network weights mapped to voltage vectors and applied to the image storage memristor array, direct computation is enabled where the image is stored, and the energy-intensive data transmission is eliminated. A hardware prototype of the 1PnR architecture achieved 95.7% recognition accuracy on the Weizmann human action flow dataset. Compared to CMOS-based systems, this architecture is estimated to have a 2000\u0026times; reduction in latency for image sensing and storage, and a 160\u0026times; reduction in energy consumption image processing, demonstrating significant potential for future neuromorphic visual systems.\u003c/p\u003e","manuscriptTitle":"Data-In-situ Computing with One-Pixel-Multiple-Memristor Architecture for Neuromorphic Sequential Vision","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-08 17:11:44","doi":"10.21203/rs.3.rs-6505999/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"509aded5-48fe-4ad1-995c-d57ebc6ac584","owner":[],"postedDate":"March 8th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":63722178,"name":"Physical sciences/Nanoscience and technology/Nanoscale devices/Electronic devices"},{"id":63722179,"name":"Physical sciences/Engineering/Electrical and electronic engineering"}],"tags":[],"updatedAt":"2026-05-13T07:06:12+00:00","versionOfRecord":{"articleIdentity":"rs-6505999","link":"https://doi.org/10.1038/s41467-026-70860-y","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2026-03-19 04:00:00","publishedOnDateReadable":"March 19th, 2026"},"versionCreatedAt":"2026-03-08 17:11:44","video":"","vorDoi":"10.1038/s41467-026-70860-y","vorDoiUrl":"https://doi.org/10.1038/s41467-026-70860-y","workflowStages":[]},"version":"v1","identity":"rs-6505999","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6505999","identity":"rs-6505999","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}