Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision

Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision weihan li, Qian Ma, Yunfeng Zhang, Xianning Wu, Wei Wang, Shizhao Gao, and 11 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1689931/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 22 Feb, 2023 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract The fifth-generation (5G) wireless communication has an urgent need for target tracking. Digital programmable metasurface (DPM) may offer an intelligent and efficient solution owing to its powerful and flexible controls of electromagnetic waves and advantages of lower cost, less complexity and smaller size than the traditional antenna array. Here, we report an intelligent metasurface system to perform target tracking and wireless communications, in which computer vision integrated with a convolutional neural network (CNN) is used to automatically detect the locations of moving targets, and the dual-polarized DPM integrated with a pre-trained artificial neural network (ANN) serves to realize smart beam tracking and wireless communications. Three groups of experiments are conducted for demonstrating the intelligent system: detection of moving targets, detection of radio-frequency signals, and real-time wireless communications. The proposed method sets the stage for an integrated implementation of target identification, radio environment tracking, and wireless communications. This strategy opens up a new avenue for intelligent wireless networks and self-adaptive systems. Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Introduction In the fifth generation (5G) wireless communication era with large number of devices in use, the demands for Internet of Things (IoT) and intelligence for user-based positioning and tracking services become more urgent. Target tracking is generally based on advanced sensors, such as radar, which can detect and track targets by analyzing and processing the radar echoes of targets. However, the electromagnetic (EM) environment is complicated and changeable, and the detecting systems based on radars tend to be inefficient due to their complexity, high cost and large volume. Therefore, more flexible hardware architecture, higher speed of information processing, and more advanced theory of information and communication are urgently needed for satisfying the substantial demands of user positioning, tracking and extensive applications in the 5G communications. Metamaterials have attracted great interests in the past decades owing to their remarkable EM properties 1 – 6 . They are mainly introduced by their subwavelength unit cells and functional arrangements to manipulate the EM behaviors, yielding many interesting phenomena and devices such as negative refraction 4 and absorber 5 . Metasurfaces are two-dimensional (2D) versions of metamaterials, which are of particular interests because of their planar profile, easy integration and low loss. Digital coding and programmable metasurfaces consist of unique coding elements with discretized reflection phases (e.g., 0° and 180°) represented by digital bits (e.g., ‘0’ and ‘1’), which can be used to manipulate the EM waves in a digitally discrete manner 6 . Digital elements can be tuned when active devices such as positive-intrinsic-negative (PIN) diodes and varactors are integrated, and in this way programmable metasurfaces that possess variety of functions under dynamical controls through different bias sequences have been developed 7 . Furthermore, the design methods for the digital-coding metasurface can be combined with physics, information science and digital signal processing, and therefore lead to the concept of information metamaterials 8 , 9 . For example, information entropy 10 , convolution theorem 11 and addition theorem 12 to describe the information of the coding pattern and the physical pattern have been studied. In practice, the polarization modulations 13 – 17 , amplitude modulations 18 – 21 and transmission-reflection controls 22 – 26 provide additional degrees of freedom to modulate the carrier waves and implement adjustable devices. Reconfigurable intelligent surfaces 27 , 28 (RIS) are boosting smart wireless communications nowadays, and real-time programmable metasurfaces have also been popping up in recent years. A hallmark achievement is the metasurface imager 29 – 33 , which has enabled the control of holographic images in a smart and real-time way. More recently, new wireless communication schemes have been investigated based on space-time-coding digital metasurfaces 34 – 38 to implement both space- and frequency-division multiplexing. However, the vast majority of the above-mentioned tunable, adaptive, and programmable metasurfaces are controlled by human beings. Although several self-adaptive metasurfaces without human interventions were developed to realize invisibility cloaks 39 , intelligent EM sensing 40 and adaptively dynamic reactions 41 , most of the related work is concentrated on the verification of pre-designed functions and performance. Recently, artificial intelligence (AI) and communication systems have developed rapidly, resulting in a wide and important impact on human lives 42 . Nowadays, intelligent wireless communication systems are strongly demanded to capture user locations in complex EM environment, and to establish real-time channels between users. However, it is expensive and complicated to realize real-time and self-adaptive EM responses in the complex environment. Fortunately, with the fast development of computer vision technology, the intuitive, reliable, informative and cost-effective target detection and tracking become possible in many scenarios including ship detection 43 and traffic surveillance system 44 . The major tasks of computer vision include classification, location, detection and segmentation. Among them, the task of visual object detection 45 is to determine whether an image contains the object of interest or not. It is used to find out the objects of a specific category in a given image, and mark the positions of objects in each frame. After that, the process of object tracking 46 – 47 is adopted to continuously estimate the state of the object in subsequent video sequences based on the given position and size of the object in the initial frame. Moreover, AI-enabled computer vision technology 48 – 51 is evolving rapidly, which can solve more complicated problems and serve as an aid to intelligent communications. In this article, we use the advantages of computer vision and flexible controls of the digital programmable metasurface (DPM) to achieve intelligent EM tracking and communications simultaneously. It is an innovative combination of the AI-based intelligent control of EM behaviors and the computer-vision-based accurate classification, detection and tracking. The technology enables real-time and accurate EM responses to meet the challenges in 5G and 6G communications. Here, we propose the concept of an intelligent tracking system using computer vision, and fulfill the design with a dual-polarized DPM. Each element of the metasurface includes two loaded PIN diodes for the dual polarizations. With the aid of a field programmable gate array (FPGA) that processes the coding sequences in real time, the reflection property of each element can be independently controlled and the corresponding EM responses are thus dynamically produced. Embedded with a pre-trained artificial neural network (ANN), DPM can respond to information at a high speed, where information can be collected, fed back, and processed in real time. The information corresponds to the trained voltage sequences, surrounding background, and the moving objects to be tracked, without any human intervention. All bias voltages are automatically calculated by real-time information perception and then instantly supplied to the DPM. The object tracking algorithm based on Yolov4-tiny and the pre-trained ANN are used to imitate the real-time system in intelligent tracking. Experiments were carried out to evaluate the performance of the design, demonstrating that the DPM-based intelligent system exhibits self-adaptability to track moving targets and transmit information to them in real time. The proposed concept will provide new solutions for intelligent meta-systems and new manipulations of EM waves in an unsupervised approach. Results Architecture of the intelligent scheme The schematic of the proposed intelligent system is presented in Fig. 1, which is composed of the dual-polarized DPM and an RGB-D camera. The moving target is represented by a model car running along a certain path in time from t 1 to t 2 . The images of the car are taken by an Intel RealSense Depth Camera D435i (RS-Camera) located on the metasurface at the rate of 40 FPS (frames per second), and each image is selected by the convolutional neural network (CNN) based on Yolov4-tiny. The original image is firstly scaled to [608, 608, 3] when reasoning, and then input to the CSPDarknet53-tiny network for the feature extraction. After passing through five groups of convolution and pooling layers, the feature maps with three different dimensions are obtained, which are fused by the network and Yolo layers. We optimize the network to detect not only the target, but also to collect its position and its elevation and azimuth angles in the coordinates of the RS-Camera (see Supplementary Note 1 for details). The moving object is tracked dynamically and its position information is refreshed in real-time by the RS-Camera, with each refresh followed by a voltage control sequence that feeds the FPGA connected to DPM. The DPM is carefully designed to transmit an adaptive radiating beam towards the moving target based on its changing positions. The coordinate systems of the camera and the DPM (as they are closely located) are unified to ensure the accuracy of the position. As it is extremely time consuming to get all spatial angles through numerical simulation of the coding metasurfaces, an ANN is designed based on the theory of beam-steering coding metasurfaces 52,53 . Through training the neural network, we can promptly obtain the coding sequences of DPM corresponding to all radiation angles that fulfill the moving space of the target. The pre-trained neural network for far-field control and the intelligent tracking algorithm based on Yolov4-tiny have completed the main part of the whole control system. The coding sequence of DPM is obtained through the extraction of position information using the neural network, and the coding sequence refreshed each time is sent to the DPM in the form of voltage through the FPGA. Design of DPM In constructing the DPM, a 1-bit dual-linearly polarized element containing two PIN diodes is proposed, as shown in Fig. 2(a), in which the geometric parameters are designed as follows: a = 25 mm, b = 11.5 mm, c 1 = 6 mm, c 2 = 2.8 mm, and d = 1.5 mm. Two PIN diodes are connected to the central patch through two metal bars along the x - and y -axes to tailor the phases in the orthogonally linear polarizations. The dielectric substrates of the element are made of commercial printed circuit boards (PCBs), in which the upper substrate is F4B with a height h 1 = 3 mm, dielectric constant ε r = 3, and tangent loss tan δ = 0.003, and the lower substrate is F4B with a height h 2 = 1 mm. A metallic sheet is inserted between the two dielectric substrates. In the back layer, the positive side of the PIN diode is conducted with the backside sector structure for the radio-frequency (RF) signal isolation and DC voltage bias. The parameters of the sector are: r = 5 mm and β = 120˚, as shown in Fig. 2(b). The element simulations are performed using commercial software of CST Microwave Studio. We list the schematic diagram of four working states in Fig. 2(c), and the simulated magnitude and phase responses of the reflected waves for different states in Fig. 2(d) and (e), respectively. When the diode along the x -axis is turned ON and the one along the y -axis is OFF, it represents state “10”, and the other three states by parity of reasoning. Given that the structure of the presented element is symmetrical along the x - and y -axes, the phase modulations for the x - and y -polarized incidences are the same. Consequently, we give the results only under the x -polarization. From Fig. 2(d), we observe that the normalized reflected amplitude is almost over 0.8, which guarantees good reflection efficiency at the central frequency of 5.8 GHz (marked in gray). Fig. 2(e) illustrates the phase responses of the element. We remark that the results are for the x -polarized incidence, in which the diode along the y -axis is kept invariant. From Fig. 2(e), it is observed that the OFF-ON states of the x -polarization, e.g., the states of “01” and “11” in the top figure and the states of “00” and “10” in the bottom one, have a phase difference of 180˚ around 5.8 GHz. A total of 324 (18×18) elements constitute the aperture of DPM, as shown in Fig. 3. A dual-polarized rectangular horn antenna is used to illuminate the DPM. Based on the superposition principle 8 , the reflected waves of DPM are the superposition of the reflected waves of all elements. By adequately configuring the digital coding scheme, DPM can steer a single beam or two beams dynamically. Beam steering by DPM We consider the specific situation of target recognition and tracking using DPM, and exam the working mechanism of beam steering (see Supplementary Note 2 for details). The experiment for measuring the far-field patterns of DPM is established in a microwave anechoic chamber, as illustrated in Fig. 3(a). DPM and the feeding horn are both mounted on a turntable. The feeding horn antenna is used to emit the EM wave with a frequency of 5.8 GHz via a signal generator (Keysight E8267D), and a receiving antenna is used to record the scattered powers via a spectrum analyzer (Keysight E4447A). The feeding horn antenna, with a relatively flat wave-front, is placed 0.5 m away from the metasurface. The photographs of the fabricated sample and details of the biasing line are shown in Fig. 3(b) and (c). Considering the cost and complexity of fabrication, the biasing lines of different polarizations on the bottom layer are printed on two sides of the sample, thereby leading to the independent controls of bias voltages. With the aid of the above theoretical analysis, we can achieve precise controls of the scattering patterns at the front space of the metasurface. As the basis of tracking moving targets with beams, we discuss the design of coding scheme for beam steering. Fig. 3(d) plots the measured beams on the E-plane from -40° to 40° with an increment of 10°. The fabricated DPMs present great performance of dynamic beam scanning controlled by the FPGA shown in Fig. 3(c). With the increment of the scanning angle, the gain decreases from 19.43 to 15.54 dB and the beam width becomes wider due to the fact that the effective aperture of the DPMs becomes smaller when the scanning angle increases. For digital coding schemes and simulation results please refer to Supplementary Note 3. The period of the element is relatively large at 5.8 GHz, and the manufacturing technique has some limitations on the overall size of the metasurface, therefore, the metasurface has a relatively small number of 324 elements, leading to the existence of sidelobes in the reflected beam. Nevertheless, the measured results of the y -polarization given in Fig. 3(d) demonstrate that 10-dB sidelobe suppression is still held in measurement, which is sufficient for generating the directive beams. It is noted that the element itself is symmetric and the EM properties under the x -polarization is almost the same. The good performance of designable radiation patterns and spectral power distributions guarantees the feasibility of the proposed intelligent tracking system. Platforms of target detection The RS-Camera is used to collect the target images in real-time at a rate of 40 FPS. The sampled images are then processed by Yolov4-tiny network to obtain the real-time positions and poses (i.e., the elevation and azimuth angles relative to the sampling position) of the target. High-precision detection at a certain detection speed is realized by introducing Mosaic data enhancement, spatial pyramid pool structure (SPPNet), and CSPDarknet53-tiny network with stronger feature extraction ability to the Yolov4-tiny network, as shown in Fig. 4. During the inference of the Yolov4-tiny network, the original image size is firstly scaled to [608, 608, 3] and put into the CSPDarknet53-tiny network for feature extraction. Then the features are fed into two different CNN modules to obtain feature maps with different scales. The feature map with dimensions [38, 38, 256] is obtained after 10 layers of convolution and pooling, and the feature map of [19, 19, 512] is obtained through 4 layers of convolution and pooling. The detection results are obtained by performing 4-layer convolution on the feature maps of [19, 19, 512] to make the number of channels num-anchor×(5+num-class). On the other hand, the 19×19 feature map is doubly up-sampled to a size of 38×38. The feature maps of the same size are superimposed in the backbone network to form a new feature map, which integrates the information of the middle and deep layers for better representation ability, and then performs 2-layer convolution to change the number of channels. After that, the detection results are obtained in a dimension of num_anchor×(5+num_class), where num-class represents the number of categories that can be detected, num-anchor represents the number of anchor boxes, and the five parameters represent the center coordinates, width, height, and confidence of the detection box, respectively (see Supplementary Note 4 for details). To be noted, the RS-Camera is not in the center of the aperture of DPM, and therefore the position information obtained by the RS-Camera has a small deviation in the coordinate system of the DPM. We unified the two coordinate systems in the control system so as to make the position information more accurate. Through debugging, the position information detected by RS-Camera and the beam direction of the DPM are precisely consistent. To characterize the working performance of the networks, we described detailed structure of the Yolov4-tiny network and pre-training ANN (see Supplementary Note 5 for details). To use the pre-trained ANN to decide the corresponding coding sequence, we consider the form of the network as a fully connected neural network. The input is the 2D vector obtained earlier, which is composed of the angles of theta and phi, and the output is an N -dimensional signal sequence composed of “0” and “1”, which is used to control the feeding of metasurface elements. Since the signal sequence is composed of two discrete values, the traditional mean-square error (MSE) as the loss function is not very effective when fitting discrete data. We consider fitting the N -dimensional sequence here as a multi-label and multi-classification problem. There are a total of N tags corresponding to N feeds, and each tag has two states corresponding to “0” and “1” respectively. At this time, we take BCE-loss (Binary Cross Entropy) as the loss function. Because the final output has two discrete values of “0” and “1”, we use the sigmoid function as the activation function of the last layer of the network. Its value ranges from “0” to “1”, and hence can normalize the values calculated by the previous network. After the network training is completed, we set 0.5 as the threshold for judging whether the output is “0” or “1”. We denote that the pre-trained ANN can quickly obtain more dataset to cope with the recognition with smaller resolution. Experimental setup and environment For experimental verification, we made a DPM with a size of 470×500×4mm 3 (18×18 cells). The computer vision detection based on Yolov4-tiny and the pre-trained beam steering ANN facilitate the connection between different parts of the system, so as to complete the intelligent tracking. To validate the above concepts and methods, a target recognition and tracking system is built for experimental demonstration in an indoor scenario. As shown in Fig. 5, the experiment system consists of the transmitting part, the receiving part, and the moving target. During the experimental test, the EM wave centered at 5.8 GHz is emitted from a feeding horn antenna and reflected by the DPM. The thereby generated radiating signal is then received by a patch antenna attached to the moving target (an electronic model car). The receiving antenna (see Supplementary Note 6 for details) is used as the representation of the moving target. The RS-Camera is used to detect the implementation scene, so as to obtain the car’s location in terms of the elevation and azimuth angles. The two angles are then used as the input of the pre-trained ANN, and the coding sequence of DPM is achieved at the output. The coding sequence is powered to the metasurface by FPGA in the form of voltage. A directive beam towards the moving target is generated by the metasurface under the control of the real-time varying coding sequence to keep tracking the target at a sampling speed of 0.2s (see Supplementary Note 7 for details). The speed limit lies in the sampling speed of the RS-Camera. We set the speed of 35 frame rates per second to collect the data, but every three times, the position information is updated and sent to FPGA. In this way, one is able to realize the closed-loop operation of the tracking system, and bridge the gap between visual detection and microwave communications. In order to prove that the system is an adaptive working scene without human intervention, six scenarios are designed in three groups of testing experiments to verify the efficiency and feasibility of the intelligent tracking scheme. Moving target detection and identification We rely on a prototype of 1-bit dual-polarized DPM to carry out the first group of experimental test. We firstly demonstrate the capability of the system to detect the moving model car and track it with directive beams. Fig. 5 shows the constituent part of the metasurface-based transmitter that is mainly composed of a vector network analyzer (VNA) and the 1-bit dual-polarized DPM fed by a linearly polarized horn antenna connected to VNA. A patch antenna is carefully designed at 5.77 GHz to serve as the receiving terminal, which is located at the position of the moving target or fixed at somewhere in the moving path. We design two experiments to verify the tracking scheme through the two-port VNA, whose input and output are connected to the receiving antenna and the feeding horn, respectively. In the first experiment, the receiving antenna is fixed somewhere in the moving path of the car. The reflected beam from DPM is always manipulated towards the car. When the car starts to move, it is far away from the receiving antenna, and the energy received by the antenna (in term of S21 read in VNA) is very low. As the car moves closer to the receiving antenna, the received energy becomes higher. When the car is the closest to the receiving antenna, the received energy is the highest. After that point, the received energy gradually decreases as the car moves away from the receiving antenna. Please refer to Supplementary Video 1 for details. In the second experiment, we demonstrate that the system can identify multiple targets and intelligently switch the target to be tracked. Tolov4-tiny target detection algorithm can classify multiple targets in the field of vision at the same time, and decide the categories to which the targets belong. By judging the category, the position information of the specified target is extracted, and the beam is controlled to point to the specified target. Here, the first model car is located at the start point of the moving path, whilst the second car is located in the middle of the path, close to the fixed receiving antenna. As the first car moves towards the middle of the path, the system recognizes two cars but only returns the position information of the first car. Therefore, the directive beam of DPM tracks the first car in the first half path, and the power received by the receiving antenna gradually increases as the first car approaches. Beyond the midpoint of the moving path, the first car stops and the second car starts to move. Since multi-target recognition can switch between different targets and return the required target information, the beam automatically switches to track the second car in the second half path. Consequently, as the second car moves towards the end of the path, the energy received by the receiving antenna gradually decreases (see Supplementary Video 2 for details). Through the experiments, we initially verify that the designed system based on DPM has the ability to track the moving targets. RF signal detection Next, we build an RF signal detecting system in the experimental scenario to conduct the real-time tracking scheme for obtaining more intuitive detection, as illustrated in Fig. 6. The transmitter primarily consists of a microwave signal generator (Keysight E8267D) and the DPM fed with the linearly polarized horn antenna. Again, the RS-Camera is placed on the top of DPM. A portable RF signal detector, which consists of a receiving patch antenna, a battery, a detector AD8317, and a microcontroller unit (MCU), is attached to the moving car. Detector AD8317 is adopted to accurately measure the RF signal power in 1MHz-10GHz. It is supplied with 3V voltage, and used to convert the RF input signal to the corresponding dB scale with accurate logarithmic consistency. The battery supplies powers to MCU (Arduino), and the DC port on MCU supplies powers to the detector AD8317. The input of the detector AD8317 is connected to the receiving antenna, and the output is connected to MCU for monitoring and processing in real time, as shown in Fig. 6(b). In this way, the portable detector without additional voltage source is realized. We also design two demonstrations in this experiment. Firstly, the detector is placed in the middle of the moving path of the car, together with the receiving antenna. We observe that the received signal increases as the car comes closer and then decreases when the car goes away, as shown in Fig. 6(d). Secondly, the portable RF signal detector is attached to the car to observe the change of RF signals during the movement. We collect the data sets for the detector-loaded car, so that the RS-camera can correctly capture the moving target in the identification process. We monitor the recognition in this experiment, as seen in Supplementary Video 3. Four curves in Fig. 6(d) respectively plot the voltage values obtained by the detector and the corresponding dB calibration values. We observe that when the detector is fixed in the middle of the moving path, the received energy has a highest value when the car is closest to the detector. In contrast, when the detector and the car move together, the received energy is relatively stable with a high value. Through these experiments, we prove that the scheme is real-time and can be quantitatively verified by RF signals. Real-time wireless transmissions Aside from the intelligent tracking, we further demonstrate that the scheme has the power to realize high-speed transmissions of information with the moving target. Here, we present two experiments of real-time video transmissions for instance, as demonstrated in Fig. 7(a) and (b). The video is taken by the camera to capture the change in screen, and sent to the video module which serves as a wireless image transmission module with the frequency ranging from 5.65 to 5.95 GHz. We remark that the working frequency bands of the video module, DPM, and the 5GHz-Wi-Fi all include 5.8 GHz. In the first experiment, the receiver is still placed in the middle of the path, and the video is transmitted only when the car moves close to the receiver (See Supplementary Video 4 for details). As shown in Fig. 7(c), during the experiment, we select five different positions of the car on the moving path for demonstration. The bit error rate of video information transmission can be seen intuitively. When the car is far away from the receiver, the error rate is high and the receiver cannot receive the video information. When the car moves near the receiver, the video can be transmitted clearly. In the second experiment, the receiver is attached to the car, and the video is always transmitted smoothly while the car is moving (see Supplementary Video 5 for details). As shown in Fig. 7(d), we intercepted five different positions in the experimental process when the receiver and the moving car are bound together, and the captured video is kept transmitting at a low bit error rate. To sum up, in these two experiments, the effect of wireless transmission is demonstrated by intercepting five states of the transmitted images, as shown in Fig. 7(c) and (d). When the receiver is fixed in the middle of the moving path, effective wireless transmissions can be realized only when the car is close to the receiver and the beam is manipulated towards them. In contrast, when the receiver is attached to the car, the movement of the car does not affect the transmission of video because the beam is dynamically tracking them. Conclusion For the intelligent system and wireless communications, we proposed a novel scheme of target recognition and tracking system based on DPM and computer vision. In the intelligent system, the RS-Camera combined with Yolov4-tiny is used to detect the position information of the moving target, and process the detected information using a pre-trained ANN to obtain the required coding sequence for the voltage control of DPM. Then intelligently adaptive beams are generated by DPM to track the moving target in real time. This system runs effectively in a closed loop without human intervention. Experimental verifications have been carried out in different scenarios, proving that the proposed scheme has the capabilities of intelligent tracking and information transmissions. This may find promising applications in other intelligent and self-adaptive systems, including intelligent and multi-physical sensing, the internet of things (IoT) technologies. The proposed concept of intelligent tracking metasurface will also open up an avenue for challenges in the 5G and 6G wireless communications, enriching the functions of metasystems. Methods Details on the digital programmable metasurface The 1-bit dual-polarized DPM used in this work operates around the central frequency of 5.8 GHz and contains 18×18 programmable elements. It is designed in the commercial software CST Microwave Studio and fabricated with the printed circuit board technology. Metallic structures on the top and feeding circuits on the bottom are printed on the commercial dielectric substrate F4B with dielectric constant ε r =3 and tangent loss tan δ =0.003. The photographs of fabricated prototypes are shown in Fig. 3. Two PIN diodes (SMP1320 from SKYWORKS) are embedded into each element to control the refection phase. Measurement setups The experimental setup for measuring the far-field patterns was established in a microwave anechoic chamber, as illustrated in Fig. 3. The DPM and the feeding antenna were both mounted on a turntable. A feeding horn antenna was used to emit the monochromatic carrier wave with frequency f = 5.8GHz via a signal generator (Keysight E8267D), and a receiving antenna was used to record the scattered powers via a spectrum analyzer (Keysight E4447A). In the experimental process of moving target detection and identification, a transmitting horn antenna, a receiving patch antenna, a DPM, and a vector network analyzer (VNA, Agilent N5230C) are set up in the anechoic chamber, as shown in Fig. 5. The moving target is represented by one or two model cars, which are captured by the RS-Camera. VNA is used to acquire the response data by measuring the transmission coefficients (S 21 ). To suppress the noise level in measurement, the intermediate bandwidth in VNA is set to 40 MHz. During the experiment of RF signal detection, the detector based on AD8317 and MCU is used to measure the receiving level. The transmitter consists of a microwave signal generator (Keysight E8267D) and the DPM fed with a horn antenna. During the test, voltage values are obtained by the detector, and the output voltage value is connected to the MCU to obtain the corresponding dB scale for real-time monitoring and processing. In the experimental process of real-time wireless transmission, the image transmission module collected a picture of a video played through the notebook, and sent it to the horn antenna after modulation. The video information is transmitted to the DPM through the horn antenna, and then to the receiving module. The receiving module is a receiving antenna, a decoder and a screen connected to the decoder. When the information can be accurately transmitted, the screen will restore the image collected by the image transmission module. The experiment scenarios are shown in Fig. 7, in which the video transmission module and the feeding horn are located under the supporting platform and the horn is about 0.8m away from the DPM. Declarations Acknowledgement. This work was supported by the National Key Research and Development Program of China (2017YFA0700200, 2017YFA0700201, 2017YFA0700202, 2017YFA0700203), the National Natural Science Foundation of China (61971134 and 61631007), the Major Project of Natural Science Foundation of Jiangsu Province (BK20212002), the Fundamental Research Funds for the Central Universities (2242021R41078), and the 111 Project (111-2-05). Author contributions. W. L., W. T. and T. J. C. conceived the idea, conducted the theoretical analysis, and wrote the paper. W. L. and Q. M. proposed the concept of digital programmable metasurfaces and built the proof-of-principle prototype system. W. L., Y. Z., X. W., Jiawei W., S. G., T. Q., T. L., Q. X., T. T. G, Z. Z., F. L. and Jiaxuan W. conducted experiments and data processing. T. J. C., Q. C. and L. L. provided suggestions and comments, and helped to organize and revise the draft. All authors discussed the results and contributed to the manuscript. Code availability. The custom computer codes utilized during the current study are available from the corresponding authors on reasonable request. Data availability. The data that support the finding of this study are available from the corresponding author upon reasonable request. Source data are provided with this paper. References Pendry, J. B. Negative Refraction Makes a Perfect Lens. Phys. Rev. Lett. 85 , 3966–3969 (2000). Pendry, J. B., Schurig, D. & Smith, D. R. Controlling Electromagnetic Fields. Science 312 , 1780–1782 (2006). Zheludev, N. I. & Kivshar, Y. S. From metamaterials to metadevices. Nature Mater 11 , 917–924 (2012). Yu, N. et al. Light Propagation with Phase Discontinuities: Generalized Laws of Reflection and Refraction. Science 334 , 333–337 (2011). Qu, S., Hou, Y. & Sheng, P. Conceptual-based design of an ultrabroadband microwave metamaterial absorber. Proc. Natl. Acad. Sci. U.S.A. 118 , e2110490118 (2021). Tang, W. X., Zhang, H. C., Ma, H. F., Jiang, W. X. & Cui, T. J. Concept, Theory, Design, and Applications of Spoof Surface Plasmon Polaritons at Microwave Frequencies. Advanced Optical Materials 7 , 1800421 (2019). Cui, T. J. et al. digital metamaterials and programmable metamaterials. Light Sci Appl 3 , e218 (2014). Cui, T. J. et al. Information Metamaterial Systems. iScience 23 , 101403 (2020). Li, L., Zhao, H., Liu, C., Li, L. & Cui, T. J. Intelligent metasurfaces: control, communication and computing. eLight 2 , 7 (2022). Cui, T.-J., Liu, S. & Li, L.-L. Information entropy of coding metasurface. Light Sci Appl 5 , e16172–e16172 (2016). Liu, S. et al. Convolution Operations on Coding Metasurface to Reach Flexible and Continuous Controls of Terahertz Beams. Adv. Sci. 3 , 1600156 (2016). Wu, R. Y., Shi, C. B., Liu, S., Wu, W. & Cui, T. J. Addition Theorem for Digital Coding Metamaterials. Advanced Optical Materials 6 , 1701236 (2018). Huang, C. X., Zhang, J., Cheng, Q. & Cui, T. J. Polarization Modulation for Wireless Communications Based on Metasurfaces. Adv. Funct. Mater. 31 , 2103379 (2021). Xu, H.-X. et al. Polarization-insensitive 3D conformal-skin metasurface cloak. Light Sci Appl 10 , 75 (2021). Zhang, X. G. et al. Polarization-Controlled Dual‐Programmable Metasurfaces. Adv. Sci. 7 , 1903382 (2020). Zhang, X. G. et al. Smart Doppler Cloak Operating in Broad Band and Full Polarizations. Adv. Mater. 33 , 2007966 (2021). Ke, J. C. et al. Linear and Nonlinear Polarization Syntheses and Their Programmable Controls based on Anisotropic Time-Domain Digital Coding Metasurface. Small Structures 2 , 2000060 (2021). Qiu, T., Jia, Y., Wang, J., Cheng, Q. & Qu, S. Controllable Reflection-Enhancement Metasurfaces via Amplification Excitation of Transistor Circuit. IEEE Trans. Antennas Propagat. 69 , 1477–1482 (2021). Ma, Q. et al. Controllable and Programmable Nonreciprocity Based on Detachable Digital Coding Metasurface. Adv. Optical Mater. 7 , 1901285 (2019). Chen, L. et al. Dual-polarization programmable metasurface modulator for near-field information encoding and transmission. Photon. Res. 9 , 116 (2021). Xu, H. et al. Radar One-Dimensional Range Profile Dynamic Jamming Based on Programmable Metasurface. Antennas Wirel. Propag. Lett. 20 , 1883–1887 (2021). Li, W. et al. Programmable Coding Metasurface Reflector for Reconfigurable Multibeam Antenna Application. IEEE Trans. Antennas Propagat. 69 , 296–301 (2021). Bao, L. et al. Programmable Reflection–Transmission Shared-Aperture Metasurface for Real‐Time Control of Electromagnetic Waves in Full Space. Adv. Sci. 8 , 2100149 (2021). Wang, H. L., Ma, H. F., Chen, M., Sun, S. & Cui, T. J. A Reconfigurable Multifunctional Metasurface for Full-Space Control of Electromagnetic Waves. Adv. Funct. Mater. 31 , 2100275 (2021). Li, W. et al. Multi-domain functional metasurface with selectivity of polarization in operation frequency and time. J. Phys. D: Appl. Phys. 53 , 495003 (2020). Wu, L. W. et al. Transmission-Reflection Controls and Polarization Controls of Electromagnetic Holograms by a Reconfigurable Anisotropic Digital Coding Metasurface. Adv. Optical Mater. 8 , 2001065 (2020). Dai, L. et al. Reconfigurable Intelligent Surface-Based Wireless Communications: Antenna Design, Prototyping, and Experimental Results. IEEE Access 8 , 45913–45923 (2020). Zhang, Z. & Dai, L. A Joint Precoding Framework for Wideband Reconfigurable Intelligent Surface-Aided Cell-Free Network. IEEE Trans. Signal Process. 69 , 4085–4101 (2021). Liu, C., Yu, W. M., Ma, Q., Li, L. & Cui, T. J. Intelligent coding metasurface holograms by physics-assisted unsupervised generative adversarial network. Photon. Res. 9 , B159 (2021). Li, J. et al. Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv. 7 , eabd7690 (2021). Li, L. et al. Electromagnetic reprogrammable coding-metasurface holograms. Nat Commun 8 , 197 (2017). Li, L. et al. Machine-learning reprogrammable metasurface imager. Nat Commun 10 , 1082 (2019). Li, L. et al. Intelligent metasurface imager and recognizer. Light Sci Appl 8 , 97 (2019). Zhang, L. et al. Space-time-coding digital metasurfaces. Nat Commun 9 , 4334 (2018). Zhao, J. et al. Programmable time-domain digital-coding metasurface for non-linear harmonic manipulation and new wireless communication systems. National Science Review 6 , 231–238 (2019). Zhang, L. et al. A wireless communication scheme based on space- and frequency-division multiplexing using digital metasurfaces. Nat Electron 4 , 218–227 (2021). Chen, M. Z. et al. Accurate and broadband manipulations of harmonic amplitudes and phases to reach 256 QAM millimeter-wave wireless communications by time-domain digital coding metasurface. National Science Review 9 , nwab134 (2022). Dai, J. Y. et al. Wireless Communications through a Simplified Architecture Based on Time-Domain Digital Coding Metasurface. Adv. Mater. Technol. 4 , 1900044 (2019). Qian, C. et al. Deep-learning-enabled self-adaptive microwave cloak without human intervention. Nat. Photonics 14 , 383–390 (2020). Li, H.-Y. et al. Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing. Patterns 1 , 100006 (2020). Ma, Q. et al. Smart metasurface with self-adaptively reprogrammable functions. Light Sci Appl 8 , 98 (2019). Zhao, H. et al. Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals. Nat Commun 11 , 3926 (2020). Shan, Zhao, Pan, Wang, & Zhao. Sea–Sky Line and its Nearby Ships Detection Based on the Motion Attitude of Visible Light Sensors. Sensors 19 , 4004 (2019). Zhang, B. & Zhang, J. A Traffic Surveillance System for Obtaining Comprehensive Information of the Passing Vehicles Based on Instance Segmentation. IEEE Trans. Intell. Transport. Syst. 22 , 7040–7055 (2021). Han, J., Zhang, D., Cheng, G., Liu, N. & Xu, D. Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey. IEEE Signal Process. Mag. 35 , 84–100 (2018). Teng, Z., Xing, J., Wang, Q., Zhang, B. & Fan, J. Deep Spatial and Temporal Network for Robust Visual Object Tracking. IEEE Trans. on Image Process. 29 , 1762–1775 (2020). Yuan, D., Li, X., He, Z., Liu, Q. & Lu, S. Visual object tracking with adaptive structural convolutional network. Knowledge-Based Systems 194 , 105554 (2020). Zhao, Z.-Q., Zheng, P., Xu, S.-T. & Wu, X. Object Detection With Deep Learning: A Review. IEEE Trans. Neural Netw. Learning Syst. 30 , 3212–3232 (2019). Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely Connected Convolutional Networks. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2261–2269 (IEEE, 2017). doi: 10.1109/CVPR.2017.243 . Wu, Y., Qi, Z., Zheng, H., Tao, L. & Gao, W. Deep Image Compression with Latent Optimization and Piece-wise Quantization Approximation. in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 1926–1930 (IEEE, 2021). doi: 10.1109/CVPRW53098 . 2021.00219. Rivadeneira, R. E. et al. Thermal Image Super-Resolution Challenge - PBVS 2021. in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 4354–4362 (IEEE, 2021). doi: 10.1109/CVPRW53098.2021.00492 . Jia, Y. et al. In Situ Customized Illusion Enabled by Global Metasurface Reconstruction. Adv Funct Materials 2109331 (2022) doi: 10.1002/adfm.202109331 . Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat Electron 5 , 113–122 (2022). Additional Declarations There is NO Competing Interest. Supplementary Files SupplementaryInformationIntelligentmetasurfacesystemforautomatictrackingofmovingtargetsandwirelesscommunicationsbasedoncomputervision.docx Supplementary Information SupplementaryVideo1.mp4 Supplementary Video 1 SupplementaryVideo2.mp4 Supplementary Video 2 SupplementaryVideo3.mp4 Supplementary Video 3 SupplementaryVideo4.avi Supplementary Video 4 SupplementaryVideo5.avi Supplementary Video 5 Cite Share Download PDF Status: Published Journal Publication published 22 Feb, 2023 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1689931","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":113572745,"identity":"7b570e43-0d4c-45b7-8c93-43bb34ad126d","order_by":0,"name":"weihan li","email":"","orcid":"https://orcid.org/0000-0001-9837-2563","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"weihan","middleName":"","lastName":"li","suffix":""},{"id":113572746,"identity":"5aa33b79-6470-4f77-b424-48269bdcef0b","order_by":1,"name":"Qian Ma","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Qian","middleName":"","lastName":"Ma","suffix":""},{"id":113572747,"identity":"6abf1199-4164-4c02-a000-16dab7e9aa04","order_by":2,"name":"Yunfeng Zhang","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Yunfeng","middleName":"","lastName":"Zhang","suffix":""},{"id":113572748,"identity":"c19e9f0c-bcd7-4196-9ce4-4d758e46c868","order_by":3,"name":"Xianning Wu","email":"","orcid":"","institution":"Air Force Engineering University","correspondingAuthor":false,"prefix":"","firstName":"Xianning","middleName":"","lastName":"Wu","suffix":""},{"id":113572749,"identity":"4fd50b45-9bc2-4f78-9d8d-164a4f9a7475","order_by":4,"name":"Wei Wang","email":"","orcid":"","institution":"The State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Wang","suffix":""},{"id":113572750,"identity":"677119b6-fdaa-4dee-8e8c-3ea52bf98ef2","order_by":5,"name":"Shizhao Gao","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Shizhao","middleName":"","lastName":"Gao","suffix":""},{"id":113572751,"identity":"4746f6fa-f605-4b26-b651-d796dc33a6cd","order_by":6,"name":"Tianshuo Qiu","email":"","orcid":"","institution":"Air Force Engineering University","correspondingAuthor":false,"prefix":"","firstName":"Tianshuo","middleName":"","lastName":"Qiu","suffix":""},{"id":113572752,"identity":"cac6fbdc-a2ba-4799-bf50-79060d9023d8","order_by":7,"name":"Tonghao Liu","email":"","orcid":"","institution":"Air Force Engineering University","correspondingAuthor":false,"prefix":"","firstName":"Tonghao","middleName":"","lastName":"Liu","suffix":""},{"id":113572753,"identity":"1fddd8f6-aa17-4865-be6d-f2041c5b1126","order_by":8,"name":"Qiang Xiao","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Qiang","middleName":"","lastName":"Xiao","suffix":""},{"id":113572754,"identity":"7590a778-57a5-494d-814a-776f5a2b7c3d","order_by":9,"name":"Jiaxuan Wei","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Jiaxuan","middleName":"","lastName":"Wei","suffix":""},{"id":113572755,"identity":"6aa08f7e-3574-43f1-bfb3-488d37fa59a2","order_by":10,"name":"Ting Ting Gu","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Ting","middleName":"Ting","lastName":"Gu","suffix":""},{"id":113572756,"identity":"6fdc3861-f360-4df2-84c9-87b8f2efb5aa","order_by":11,"name":"Zhize Zhou","email":"","orcid":"","institution":"University of Sheffield","correspondingAuthor":false,"prefix":"","firstName":"Zhize","middleName":"","lastName":"Zhou","suffix":""},{"id":113572757,"identity":"dee0bea6-6649-4d6e-b284-e12750f139ef","order_by":12,"name":"Fashuai Li","email":"","orcid":"","institution":"Zhejiang University","correspondingAuthor":false,"prefix":"","firstName":"Fashuai","middleName":"","lastName":"Li","suffix":""},{"id":113572758,"identity":"7b61d8be-99f3-41d9-8ad7-2bcb2bc0b183","order_by":13,"name":"Qiang Cheng","email":"","orcid":"https://orcid.org/0000-0002-2442-8357","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Qiang","middleName":"","lastName":"Cheng","suffix":""},{"id":113572759,"identity":"8cee4137-924b-44ba-a3c5-ad602e81fc34","order_by":14,"name":"Lianlin Li","email":"","orcid":"https://orcid.org/0000-0001-9394-3638","institution":"peking university","correspondingAuthor":false,"prefix":"","firstName":"Lianlin","middleName":"","lastName":"Li","suffix":""},{"id":113572760,"identity":"a73a9104-98c1-4d8e-a8e4-e7f5cf1b25f2","order_by":15,"name":"Wenxuan Tang","email":"","orcid":"","institution":"Southeast University","correspondingAuthor":false,"prefix":"","firstName":"Wenxuan","middleName":"","lastName":"Tang","suffix":""},{"id":113572761,"identity":"f13ed8e8-7828-4e8c-a75b-89b5326b3a7b","order_by":16,"name":"Tie Jun Cui","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAs0lEQVRIiWNgGAWjYFCCAwwMCRUQpgRRGnjAWs6QpgUIGNtI0WLPePiZxMN5dXIGB5gP3uZhsMsjwpZjZhKJ2w4bGxxgS7bmYUguJkLLGTaglgOJGw7wmEkDfZbYQJyWOXX1Gw7wfyNFSwNzgsEBHjYitRw4ZmyRcOyw4czDbMaWcwySCWthn3H44c0fNXXyfMebH954U2FHWAuDxAEogxlEGBBUDwT8hE0dBaNgFIyCkQ4A+OM4mxohJFoAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-5862-1497","institution":"Southeast University","correspondingAuthor":true,"prefix":"","firstName":"Tie","middleName":"Jun","lastName":"Cui","suffix":""}],"badges":[],"createdAt":"2022-05-24 18:26:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1689931/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1689931/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-023-36645-3","type":"published","date":"2023-02-22T05:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":22652559,"identity":"73c9861e-b188-4073-ad97-4d99fc37a46c","added_by":"auto","created_at":"2022-06-14 18:54:11","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":2139908,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic of the target tracking system.\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/ec28e8e5a0f9a4a2d85df933.jpeg"},{"id":22653102,"identity":"cc304783-3d17-44fb-8c1b-d6504fb0d008","added_by":"auto","created_at":"2022-06-14 19:04:11","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":2829201,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure and performance of the designed DPM. (a) The element structure integrated with two PIN diodes. (b) The bottom view of the coding element. (c) The transmission schematic for the presented metasurface structure, including four switching schemes with two diodes. (d, e) The reflected magnitude and phase responses of the coding element when the PIN diodes are switched on and off.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/d7cbb89dbf00045526f3b7ab.jpeg"},{"id":22652561,"identity":"80d7e364-889a-4c7d-a902-141f639a9798","added_by":"auto","created_at":"2022-06-14 18:54:11","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":3783716,"visible":true,"origin":"","legend":"\u003cp\u003e(a) The far-field experimental setup in an anechoic chamber. (b) Photographs of the fabricated prototype. (c) Zynq-7000 SoC series FPGA for voltage control. (d) The measured far-field patterns when beams on the E-plane vary from -40° to 40° at 5.8 GHz. These experiments verify that the DPMs can shape the far-field patterns in the spatial domain by configuring the digital codes.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/126cd1a322679adb4b6a0978.jpeg"},{"id":22652795,"identity":"fdb67861-4dce-4e1c-a8ff-ecd63e77dcb8","added_by":"auto","created_at":"2022-06-14 18:59:11","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":1368478,"visible":true,"origin":"","legend":"\u003cp\u003eStructure diagram of the Yolov4-tiny network and pre-training ANN.\u003c/p\u003e","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/44aa549bb0deb32283559d33.jpeg"},{"id":22652794,"identity":"1df6a15c-b09b-4846-91f1-691bd22796f1","added_by":"auto","created_at":"2022-06-14 18:59:11","extension":"jpeg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":750266,"visible":true,"origin":"","legend":"\u003cp\u003eExperimental setup and environment. A TE-polarized beam from a feeding horn antenna is incident on the metasurface. The position of the moving car is processed in the Control system, and all bias voltages are instantly calculated and supplied to the metasurface. The reflected waves of the metasurface are detected in real time by the receiving antenna.\u003c/p\u003e","description":"","filename":"floatimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/f621e6cf08d2f8e56278268f.jpeg"},{"id":22652566,"identity":"a83a6ea6-f5f0-4e11-ab7b-ab20e2e9c978","added_by":"auto","created_at":"2022-06-14 18:54:12","extension":"jpeg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":5571934,"visible":true,"origin":"","legend":"\u003cp\u003eExperiments of RF signal detection. (a) A TE-polarized beam from a feeding antenna is incident on the metasurface. The position of model car is fed into the control system, and all bias voltages are instantly calculated and supplied to DPM. The reflected waves are detected in real time by the RF signal detector. (b) The detector AD8317 is connected to the patch antenna from the front. On the back of detector, the battery supplies powers to MCU, and the power port on MCU supplies powers to AD8317. (c) Front view of the experimental scene. (d) RF signal changes when the detector is fixed in the middle of the path or moves with the car. The horizontal ordinate is the moving path of the car from the beginning to the end.\u003c/p\u003e","description":"","filename":"floatimage6.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/67b9c035ee2dd6fe44b202eb.jpeg"},{"id":22652567,"identity":"753c421e-f621-44ea-a1ac-07f24f44f92d","added_by":"auto","created_at":"2022-06-14 18:54:12","extension":"jpeg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":13805923,"visible":true,"origin":"","legend":"\u003cp\u003eExperiments of the wireless communications. (a-b) Experiment scenarios when the receiver is in the middle of the path (a) and when the receiver follows the moving target (b). (c-d) Experimental results of received video frames at five different positions of the car on the moving path in (c) Experiment Scenario 1 and (d) Experiment Scenario 2.\u003c/p\u003e","description":"","filename":"floatimage7.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/a1dbbd9304d7ece4c54b7131.jpeg"},{"id":33329130,"identity":"1272d339-9d72-4beb-bd4a-ad2b60f59541","added_by":"auto","created_at":"2023-02-23 08:10:39","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1516402,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/835fa7cb-36ef-4f35-b864-67806003e23e.pdf"},{"id":22652798,"identity":"0d6448e3-a571-4a2e-9857-620477e3d33a","added_by":"auto","created_at":"2022-06-14 18:59:12","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":14402528,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary Information\u003c/p\u003e","description":"","filename":"SupplementaryInformationIntelligentmetasurfacesystemforautomatictrackingofmovingtargetsandwirelesscommunicationsbasedoncomputervision.docx","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/f75391f448667e5ae3b0108a.docx"},{"id":22652558,"identity":"052ad739-c393-4232-8055-d6f924b8acfb","added_by":"auto","created_at":"2022-06-14 18:54:11","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":8214020,"visible":true,"origin":"","legend":"Supplementary Video 1","description":"","filename":"SupplementaryVideo1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/2289e031a45411786f70160b.mp4"},{"id":22652564,"identity":"6ec17cbc-9ef7-4d49-8f3c-0da7c2324c6f","added_by":"auto","created_at":"2022-06-14 18:54:11","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":7956224,"visible":true,"origin":"","legend":"Supplementary Video 2","description":"","filename":"SupplementaryVideo2.mp4","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/7a66a3a1279c20bae65700a5.mp4"},{"id":22652797,"identity":"b0d26920-f758-4184-93bc-03baefdb7925","added_by":"auto","created_at":"2022-06-14 18:59:11","extension":"mp4","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":10406978,"visible":true,"origin":"","legend":"Supplementary Video 3","description":"","filename":"SupplementaryVideo3.mp4","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/5f11211bec5d161543cd5fab.mp4"},{"id":22652571,"identity":"bafa6b60-2e4b-4702-8f96-48090b6f3a28","added_by":"auto","created_at":"2022-06-14 18:54:13","extension":"avi","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":102424150,"visible":true,"origin":"","legend":"Supplementary Video 4","description":"","filename":"SupplementaryVideo4.avi","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/ad445dfb924a7b300d35ee53.avi"},{"id":22652570,"identity":"6d33cf9e-23e8-4c2d-bcdc-2516544261b3","added_by":"auto","created_at":"2022-06-14 18:54:12","extension":"avi","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":57629642,"visible":true,"origin":"","legend":"Supplementary Video 5","description":"","filename":"SupplementaryVideo5.avi","url":"https://assets-eu.researchsquare.com/files/rs-1689931/v1/ba01eba641a32e214689f6bf.avi"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn the fifth generation (5G) wireless communication era with large number of devices in use, the demands for Internet of Things (IoT) and intelligence for user-based positioning and tracking services become more urgent. Target tracking is generally based on advanced sensors, such as radar, which can detect and track targets by analyzing and processing the radar echoes of targets. However, the electromagnetic (EM) environment is complicated and changeable, and the detecting systems based on radars tend to be inefficient due to their complexity, high cost and large volume. Therefore, more flexible hardware architecture, higher speed of information processing, and more advanced theory of information and communication are urgently needed for satisfying the substantial demands of user positioning, tracking and extensive applications in the 5G communications.\u003c/p\u003e \u003cp\u003eMetamaterials have attracted great interests in the past decades owing to their remarkable EM properties\u003csup\u003e\u003cspan additionalcitationids=\"CR2 CR3 CR4 CR5\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e. They are mainly introduced by their subwavelength unit cells and functional arrangements to manipulate the EM behaviors, yielding many interesting phenomena and devices such as negative refraction\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e and absorber\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. Metasurfaces are two-dimensional (2D) versions of metamaterials, which are of particular interests because of their planar profile, easy integration and low loss. Digital coding and programmable metasurfaces consist of unique coding elements with discretized reflection phases (e.g., 0\u0026deg; and 180\u0026deg;) represented by digital bits (e.g., \u0026lsquo;0\u0026rsquo; and \u0026lsquo;1\u0026rsquo;), which can be used to manipulate the EM waves in a digitally discrete manner\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e. Digital elements can be tuned when active devices such as positive-intrinsic-negative (PIN) diodes and varactors are integrated, and in this way programmable metasurfaces that possess variety of functions under dynamical controls through different bias sequences have been developed\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e. Furthermore, the design methods for the digital-coding metasurface can be combined with physics, information science and digital signal processing, and therefore lead to the concept of information metamaterials\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e. For example, information entropy\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e, convolution theorem\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e and addition theorem\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e to describe the information of the coding pattern and the physical pattern have been studied.\u003c/p\u003e \u003cp\u003eIn practice, the polarization modulations\u003csup\u003e\u003cspan additionalcitationids=\"CR14 CR15 CR16\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e, amplitude modulations\u003csup\u003e\u003cspan additionalcitationids=\"CR19 CR20\" citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e and transmission-reflection controls\u003csup\u003e\u003cspan additionalcitationids=\"CR23 CR24 CR25\" citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e provide additional degrees of freedom to modulate the carrier waves and implement adjustable devices. Reconfigurable intelligent surfaces\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e,\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e (RIS) are boosting smart wireless communications nowadays, and real-time programmable metasurfaces have also been popping up in recent years. A hallmark achievement is the metasurface imager\u003csup\u003e\u003cspan additionalcitationids=\"CR30 CR31 CR32\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, which has enabled the control of holographic images in a smart and real-time way. More recently, new wireless communication schemes have been investigated based on space-time-coding digital metasurfaces\u003csup\u003e\u003cspan additionalcitationids=\"CR35 CR36 CR37\" citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e to implement both space- and frequency-division multiplexing.\u003c/p\u003e \u003cp\u003eHowever, the vast majority of the above-mentioned tunable, adaptive, and programmable metasurfaces are controlled by human beings. Although several self-adaptive metasurfaces without human interventions were developed to realize invisibility cloaks\u003csup\u003e\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e, intelligent EM sensing\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e and adaptively dynamic reactions\u003csup\u003e\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e, most of the related work is concentrated on the verification of pre-designed functions and performance. Recently, artificial intelligence (AI) and communication systems have developed rapidly, resulting in a wide and important impact on human lives\u003csup\u003e\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e. Nowadays, intelligent wireless communication systems are strongly demanded to capture user locations in complex EM environment, and to establish real-time channels between users. However, it is expensive and complicated to realize real-time and self-adaptive EM responses in the complex environment. Fortunately, with the fast development of computer vision technology, the intuitive, reliable, informative and cost-effective target detection and tracking become possible in many scenarios including ship detection\u003csup\u003e\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e\u003c/sup\u003e and traffic surveillance system\u003csup\u003e\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e\u003c/sup\u003e. The major tasks of computer vision include classification, location, detection and segmentation. Among them, the task of visual object detection\u003csup\u003e\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e\u003c/sup\u003e is to determine whether an image contains the object of interest or not. It is used to find out the objects of a specific category in a given image, and mark the positions of objects in each frame. After that, the process of object tracking\u003csup\u003e\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e\u003c/sup\u003e is adopted to continuously estimate the state of the object in subsequent video sequences based on the given position and size of the object in the initial frame. Moreover, AI-enabled computer vision technology\u003csup\u003e\u003cspan additionalcitationids=\"CR49 CR50\" citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e is evolving rapidly, which can solve more complicated problems and serve as an aid to intelligent communications.\u003c/p\u003e \u003cp\u003eIn this article, we use the advantages of computer vision and flexible controls of the digital programmable metasurface (DPM) to achieve intelligent EM tracking and communications simultaneously. It is an innovative combination of the AI-based intelligent control of EM behaviors and the computer-vision-based accurate classification, detection and tracking. The technology enables real-time and accurate EM responses to meet the challenges in 5G and 6G communications. Here, we propose the concept of an intelligent tracking system using computer vision, and fulfill the design with a dual-polarized DPM. Each element of the metasurface includes two loaded PIN diodes for the dual polarizations. With the aid of a field programmable gate array (FPGA) that processes the coding sequences in real time, the reflection property of each element can be independently controlled and the corresponding EM responses are thus dynamically produced. Embedded with a pre-trained artificial neural network (ANN), DPM can respond to information at a high speed, where information can be collected, fed back, and processed in real time. The information corresponds to the trained voltage sequences, surrounding background, and the moving objects to be tracked, without any human intervention. All bias voltages are automatically calculated by real-time information perception and then instantly supplied to the DPM. The object tracking algorithm based on Yolov4-tiny and the pre-trained ANN are used to imitate the real-time system in intelligent tracking. Experiments were carried out to evaluate the performance of the design, demonstrating that the DPM-based intelligent system exhibits self-adaptability to track moving targets and transmit information to them in real time. The proposed concept will provide new solutions for intelligent meta-systems and new manipulations of EM waves in an unsupervised approach.\u003c/p\u003e"},{"header":"Results","content":"\u003ch1\u003eArchitecture of the intelligent scheme\u003c/h1\u003e\n\u003cp\u003eThe schematic of the proposed intelligent system is presented in Fig. 1, which is composed of the dual-polarized DPM and an RGB-D camera. The moving target is represented by a model car running along a certain path in time from \u003cem\u003et\u003csub\u003e1\u003c/sub\u003e\u003c/em\u003e to \u003cem\u003et\u003csub\u003e2\u003c/sub\u003e\u003c/em\u003e. The images of the car are taken by an Intel RealSense Depth Camera D435i (RS-Camera) located on the metasurface at the rate of 40 FPS (frames per second), and each image is selected by the convolutional neural network (CNN) based on Yolov4-tiny. The original image is firstly scaled to [608, 608, 3] when reasoning, and then input to the CSPDarknet53-tiny network for the feature extraction. After passing through five groups of convolution and pooling layers, the feature maps with three different dimensions are obtained, which are fused by the network and Yolo layers. We optimize the network to detect not only the target, but also to collect its position and its elevation and azimuth angles in the coordinates of the RS-Camera (see Supplementary Note 1 for details).\u003c/p\u003e\n\u003cp\u003eThe moving object is tracked dynamically and its position information is refreshed in real-time by the RS-Camera, with each refresh followed by a voltage control sequence that feeds the FPGA connected to DPM. The DPM is carefully designed to transmit an adaptive radiating beam towards the moving target based on its changing positions. The coordinate systems of the camera and the DPM (as they are closely located) are unified to ensure the accuracy of the position. As it is extremely time consuming to get all spatial angles through numerical simulation of the coding metasurfaces, an ANN is designed based on the theory of beam-steering coding metasurfaces\u003csup\u003e52,53\u003c/sup\u003e. Through training the neural network, we can promptly obtain the coding sequences of DPM corresponding to all radiation angles that fulfill the moving space of the target. The pre-trained neural network for far-field control and the intelligent tracking algorithm based on Yolov4-tiny have completed the main part of the whole control system. The coding sequence of DPM is obtained through the extraction of position information using the neural network, and the coding sequence refreshed each time is sent to the DPM in the form of voltage through the FPGA.\u003c/p\u003e\n\u003ch1\u003eDesign of DPM\u003c/h1\u003e\n\u003cp\u003eIn constructing the DPM, a 1-bit dual-linearly polarized element containing two PIN diodes is proposed, as shown in Fig. 2(a), in which the geometric parameters are designed as follows: \u003cem\u003ea\u003c/em\u003e = 25 mm, \u003cem\u003eb\u003c/em\u003e = 11.5 mm, \u003cem\u003ec\u003csub\u003e1\u003c/sub\u003e\u003c/em\u003e = 6 mm, \u003cem\u003ec\u003csub\u003e2\u003c/sub\u003e\u003c/em\u003e = 2.8 mm, and \u003cem\u003ed\u0026nbsp;\u003c/em\u003e= 1.5 mm. Two PIN diodes are connected to the central patch through two metal bars along the \u003cem\u003ex\u003c/em\u003e- and \u003cem\u003ey\u003c/em\u003e-axes to tailor the phases in the orthogonally linear polarizations. The dielectric substrates of the element are made of commercial printed circuit boards (PCBs), in which the upper substrate is F4B with a height \u003cem\u003eh\u003csub\u003e1\u003c/sub\u003e\u003c/em\u003e = 3 mm, dielectric constant \u003cem\u003e\u0026epsilon;\u003csub\u003er\u003c/sub\u003e\u003c/em\u003e = 3, and tangent loss tan\u003cem\u003e\u0026delta;\u003c/em\u003e = 0.003, and the lower substrate is F4B with a height \u003cem\u003eh\u003csub\u003e2\u003c/sub\u003e\u003c/em\u003e = 1 mm. A metallic sheet is inserted between the two dielectric substrates. In the back layer, the positive side of the PIN diode is conducted with the backside sector structure for the radio-frequency (RF) signal isolation and DC voltage bias. The parameters of the sector are: \u003cem\u003er\u003c/em\u003e = 5 mm and \u003cem\u003e\u0026beta;\u003c/em\u003e = 120˚, as shown in Fig. 2(b).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;The element simulations are performed using commercial software of CST Microwave Studio. We list the schematic diagram of four working states in Fig. 2(c), and the simulated magnitude and phase responses of the reflected waves for different states in Fig. 2(d) and (e), respectively. When the diode along the \u003cem\u003ex\u003c/em\u003e-axis is turned ON and the one along the \u003cem\u003ey\u003c/em\u003e-axis is OFF, it represents state \u0026ldquo;10\u0026rdquo;, and the other three states by parity of reasoning. Given that the structure of the presented element is symmetrical along the \u003cem\u003ex\u003c/em\u003e- and \u003cem\u003ey\u003c/em\u003e-axes, the phase modulations for the \u003cem\u003ex\u003c/em\u003e- and \u003cem\u003ey\u003c/em\u003e-polarized incidences are the same. Consequently, we give the results only under the \u003cem\u003ex\u003c/em\u003e-polarization. From Fig. 2(d), we observe that the normalized reflected amplitude is almost over 0.8, which guarantees good reflection efficiency at the central frequency of 5.8 GHz (marked in gray). Fig. 2(e) illustrates the phase responses of the element. We remark that the results are for the \u003cem\u003ex\u003c/em\u003e-polarized incidence, in which the diode along the \u003cem\u003ey\u003c/em\u003e-axis is kept invariant. From Fig. 2(e), it is observed that the OFF-ON states of the \u003cem\u003ex\u003c/em\u003e-polarization, e.g., the states of \u0026ldquo;01\u0026rdquo; and \u0026ldquo;11\u0026rdquo; in the top figure and the states of \u0026ldquo;00\u0026rdquo; and \u0026ldquo;10\u0026rdquo; in the bottom one, have a phase difference of 180˚ around 5.8 GHz.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; A total of 324 (18\u0026times;18) elements constitute the aperture of DPM, as shown in Fig. 3. A dual-polarized rectangular horn antenna is used to illuminate the DPM. Based on the superposition principle\u003csup\u003e8\u003c/sup\u003e, the reflected waves of DPM are the superposition of the reflected waves of all elements. By adequately configuring the digital coding scheme, DPM can steer a single beam or two beams dynamically.\u003c/p\u003e\n\u003ch1\u003eBeam steering by DPM\u003c/h1\u003e\n\u003cp\u003eWe consider the specific situation of target recognition and tracking using DPM, and exam the working mechanism of beam steering (see Supplementary Note 2 for details). The experiment for measuring the far-field patterns of DPM is established in a microwave anechoic chamber, as illustrated in Fig. 3(a). DPM and the feeding horn are both mounted on a turntable. The feeding horn antenna is used to emit the EM wave with a frequency of 5.8 GHz via a signal generator (Keysight E8267D), and a receiving antenna is used to record the scattered powers via a spectrum analyzer (Keysight E4447A). The feeding horn antenna, with a relatively flat wave-front, is placed 0.5 m away from the metasurface. The photographs of the fabricated sample and details of the biasing line are shown in Fig. 3(b) and (c). Considering the cost and complexity of fabrication, the biasing lines of different polarizations on the bottom layer are printed on two sides of the sample, thereby leading to the independent controls of bias voltages. With the aid of the above theoretical analysis, we can achieve precise controls of the scattering patterns at the front space of the metasurface.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAs the basis of tracking moving targets with beams, we discuss the design of coding scheme for beam steering. Fig. 3(d) plots the measured beams on the E-plane from\u0026nbsp;-40\u0026deg; to 40\u0026deg; with an increment of 10\u0026deg;. The fabricated DPMs present great performance of dynamic beam scanning controlled by the FPGA shown in Fig. 3(c). With the increment of the scanning angle, the gain decreases from 19.43 to 15.54 dB and the beam width becomes wider due to the fact that the effective aperture of the DPMs becomes smaller when the scanning angle increases.\u0026nbsp;For digital coding schemes and simulation results please refer to Supplementary Note 3. The period of the element is relatively large at 5.8 GHz, and the manufacturing technique has some limitations on the overall size of the metasurface, therefore, the metasurface has a relatively small number of 324 elements, leading to the existence of sidelobes in the reflected beam. Nevertheless, the measured results of the \u003cem\u003ey\u003c/em\u003e-polarization given in Fig. 3(d) demonstrate that 10-dB sidelobe suppression is still held in measurement, which is sufficient for generating the directive beams. It is noted that the element itself is symmetric and the EM properties under the \u003cem\u003ex\u003c/em\u003e-polarization is almost the same. The good performance of designable radiation patterns and spectral power distributions guarantees the feasibility of the proposed intelligent tracking system.\u0026nbsp;\u003c/p\u003e\n\u003ch1\u003ePlatforms of target detection\u003c/h1\u003e\n\u003cp\u003eThe RS-Camera is used to collect the target images in real-time at a rate of 40 FPS. The sampled images are then processed by Yolov4-tiny network to obtain the real-time positions and poses (i.e., the elevation and azimuth angles relative to the sampling position) of the target. High-precision detection at a certain detection speed is realized by introducing Mosaic data enhancement, spatial pyramid pool structure (SPPNet), and CSPDarknet53-tiny network with stronger feature extraction ability to the Yolov4-tiny network, as shown in Fig. 4.\u003c/p\u003e\n\u003cp\u003eDuring the inference of the Yolov4-tiny network, the original image size is firstly scaled to [608, 608, 3] and put into the CSPDarknet53-tiny network for feature extraction. Then the features are fed into two different CNN modules to obtain feature maps with different scales. The feature map with dimensions [38, 38, 256] is obtained after 10 layers of convolution and pooling, and the feature map of [19, 19, 512] is obtained through 4 layers of convolution and pooling. The detection results are obtained by performing 4-layer convolution on the feature maps of [19, 19, 512] to make the number of channels num-anchor\u0026times;(5+num-class). On the other hand, the 19\u0026times;19 feature map is doubly up-sampled to a size of 38\u0026times;38. The feature maps of the same size are superimposed in the backbone network to form a new feature map, which integrates the information of the middle and deep layers for better representation ability, and then performs 2-layer convolution to change the number of channels. After that, the detection results are obtained in a dimension of num_anchor\u0026times;(5+num_class), where num-class represents the number of categories that can be detected, num-anchor represents the number of anchor boxes, and the five parameters represent the center coordinates, width, height, and confidence of the detection box, respectively (see Supplementary Note 4 for details). To be noted, the RS-Camera is not in the center of the aperture of DPM, and therefore the position information obtained by the RS-Camera has a small deviation in the coordinate system of the DPM. We unified the two coordinate systems in the control system so as to make the position information more accurate. Through debugging, the position information detected by RS-Camera and the beam direction of the DPM are precisely consistent. To characterize the working performance of the networks, we described detailed structure of the Yolov4-tiny network and pre-training ANN (see Supplementary Note 5 for details).\u003c/p\u003e\n\u003cp\u003eTo use the pre-trained ANN to decide the corresponding coding sequence, we consider the form of the network as a fully connected neural network. The input is the 2D vector obtained earlier, which is composed of the angles of theta and phi, and the output is an \u003cem\u003eN\u003c/em\u003e-dimensional signal sequence composed of \u0026ldquo;0\u0026rdquo; and \u0026ldquo;1\u0026rdquo;, which is used to control the feeding of metasurface elements. Since the signal sequence is composed of two discrete values, the traditional mean-square error (MSE) as the loss function is not very effective when fitting discrete data. We consider fitting the \u003cem\u003eN\u003c/em\u003e-dimensional sequence here as a multi-label and multi-classification problem. There are a total of \u003cem\u003eN\u003c/em\u003e tags corresponding to \u003cem\u003eN\u003c/em\u003e feeds, and each tag has two states corresponding to \u0026ldquo;0\u0026rdquo; and \u0026ldquo;1\u0026rdquo; respectively. At this time, we take BCE-loss (Binary Cross Entropy) as the loss function. Because the final output has two discrete values of \u0026ldquo;0\u0026rdquo; and \u0026ldquo;1\u0026rdquo;, we use the sigmoid function as the activation function of the last layer of the network. Its value ranges from \u0026ldquo;0\u0026rdquo; to \u0026ldquo;1\u0026rdquo;, and hence can normalize the values calculated by the previous network. After the network training is completed, we set 0.5 as the threshold for judging whether the output is \u0026ldquo;0\u0026rdquo; or \u0026ldquo;1\u0026rdquo;. We denote that the pre-trained ANN can quickly obtain more dataset to cope with the recognition with smaller resolution.\u0026nbsp;\u003c/p\u003e\n\u003ch1\u003eExperimental setup\u0026nbsp;and environment\u0026nbsp;\u003c/h1\u003e\n\u003cp\u003eFor experimental verification, we made a DPM with a size of 470\u0026times;500\u0026times;4mm\u003csup\u003e3\u003c/sup\u003e (18\u0026times;18 cells). The computer vision detection based on Yolov4-tiny and the pre-trained beam steering ANN facilitate the connection between different parts of the system, so as to complete the intelligent tracking. To validate the above concepts and methods, a target recognition and tracking system is built for experimental demonstration in an indoor scenario.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;As shown in Fig. 5, the experiment system consists of the transmitting part, the receiving part, and the moving target. During the experimental test, the EM wave centered at 5.8 GHz is emitted from a feeding horn antenna and reflected by the DPM. The thereby generated radiating signal is then received by a patch antenna attached to the moving target (an electronic model car). The receiving antenna (see Supplementary Note 6 for details) is used as the representation of the moving target. The RS-Camera is used to detect the implementation scene, so as to obtain the car\u0026rsquo;s location in terms of the elevation and azimuth angles. The two angles are then used as the input of the pre-trained ANN, and the coding sequence of DPM is achieved at the output. The coding sequence is powered to the metasurface by FPGA in the form of voltage. A directive beam towards the moving target is generated by the metasurface under the control of the real-time varying coding sequence to keep tracking the target at a sampling speed of 0.2s (see Supplementary Note 7 for details). The speed limit lies in the sampling speed of the RS-Camera. We set the speed of 35 frame rates per second to collect the data, but every three times, the position information is updated and sent to FPGA. In this way, one is able to realize the closed-loop operation of the tracking system, and bridge the gap between visual detection and microwave communications.\u003c/p\u003e\n\u003cp\u003eIn order to prove that the system is an adaptive working scene without human intervention, six scenarios are designed in three groups of testing experiments to verify the efficiency and feasibility of the intelligent tracking scheme.\u0026nbsp;\u003c/p\u003e\n\u003ch1\u003eMoving target detection and identification\u003c/h1\u003e\n\u003cp\u003eWe rely on a prototype of 1-bit dual-polarized DPM to carry out the first group of experimental test. We firstly demonstrate the capability of the system to detect the moving model car and track it with directive beams. Fig. 5 shows the constituent part of the metasurface-based transmitter that is mainly composed of a vector network analyzer (VNA) and the 1-bit dual-polarized DPM fed by a linearly polarized horn antenna connected to VNA. A patch antenna is carefully designed at 5.77 GHz to serve as the receiving terminal, which is located at the position of the moving target or fixed at somewhere in the moving path. We design two experiments to verify the tracking scheme through the two-port VNA, whose input and output are connected to the receiving antenna and the feeding horn, respectively. In the first experiment, the receiving antenna is fixed somewhere in the moving path of the car. The reflected beam from DPM is always manipulated towards the car. When the car starts to move, it is far away from the receiving antenna, and the energy received by the antenna (in term of S21 read in VNA) is very low. As the car moves closer to the receiving antenna, the received energy becomes higher. When the car is the closest to the receiving antenna, the received energy is the highest. After that point, the received energy gradually decreases as the car moves away from the receiving antenna. Please refer to Supplementary Video 1 for details.\u003c/p\u003e\n\u003cp\u003eIn the second experiment, we demonstrate that the system can identify multiple targets and intelligently switch the target to be tracked. Tolov4-tiny target detection algorithm can classify multiple targets in the field of vision at the same time, and decide the categories to which the targets belong. By judging the category, the position information of the specified target is extracted, and the beam is controlled to point to the specified target. Here, the first model car is located at the start point of the moving path, whilst the second car is located in the middle of the path, close to the fixed receiving antenna. As the first car moves towards the middle of the path, the system recognizes two cars but only returns the position information of the first car. Therefore, the directive beam of DPM tracks the first car in the first half path, and the power received by the receiving antenna gradually increases as the first car approaches. Beyond the midpoint of the moving path, the first car stops and the second car starts to move. Since multi-target recognition can switch between different targets and return the required target information, the beam automatically switches to track the second car in the second half path. Consequently, as the second car moves towards the end of the path, the energy received by the receiving antenna gradually decreases (see Supplementary Video 2 for details). Through the experiments, we initially verify that the designed system based on DPM has the ability to track the moving targets.\u0026nbsp;\u003c/p\u003e\n\u003ch1\u003eRF signal\u0026nbsp;detection\u003c/h1\u003e\n\u003cp\u003eNext, we build an RF signal detecting system in the experimental scenario to conduct the real-time tracking scheme for obtaining more intuitive detection, as illustrated in Fig. 6. The transmitter primarily consists of a microwave signal generator (Keysight E8267D) and the DPM fed with the linearly polarized horn antenna. Again, the RS-Camera is placed on the top of DPM. A portable RF signal detector, which consists of a receiving patch antenna, a battery, a detector AD8317, and a microcontroller unit (MCU), is attached to the moving car. Detector AD8317 is adopted to accurately measure the RF signal power in 1MHz-10GHz. It is supplied with 3V voltage, and used to convert the RF input signal to the corresponding dB scale with accurate logarithmic consistency. The battery supplies powers to MCU (Arduino), and the DC port on MCU supplies powers to the detector AD8317. The input of the detector AD8317 is connected to the receiving antenna, and the output is connected to MCU for monitoring and processing in real time, as shown in Fig. 6(b). In this way, the portable detector without additional voltage source is realized.\u003c/p\u003e\n\u003cp\u003eWe also design two demonstrations in this experiment. Firstly, the detector is placed in the middle of the moving path of the car, together with the receiving antenna. We observe that the received signal increases as the car comes closer and then decreases when the car goes away, as shown in Fig. 6(d). Secondly, the portable RF signal detector is attached to the car to observe the change of RF signals during the movement. We collect the data sets for the detector-loaded car, so that the RS-camera can correctly capture the moving target in the identification process. We monitor the recognition in this experiment, as seen in Supplementary Video 3. Four curves in Fig. 6(d) respectively plot the voltage values obtained by the detector and the corresponding dB calibration values. We observe that when the detector is fixed in the middle of the moving path, the received energy has a highest value when the car is closest to the detector. In contrast, when the detector and the car move together, the received energy is relatively stable with a high value. Through these experiments, we prove that the scheme is real-time and can be quantitatively verified by RF signals.\u003c/p\u003e\n\u003ch1\u003eReal-time wireless transmissions\u003c/h1\u003e\n\u003cp\u003eAside from the intelligent tracking, we further demonstrate that the scheme has the power to realize high-speed transmissions of information with the moving target. Here, we present two experiments of real-time video transmissions for instance, as demonstrated in Fig. 7(a) and (b). The video is taken by the camera to capture the change in screen, and sent to the video module which serves as a wireless image transmission module with the frequency ranging from 5.65 to 5.95 GHz. We remark that the working frequency bands of the video module, DPM, and the 5GHz-Wi-Fi all include 5.8 GHz. In the first experiment, the receiver is still placed in the middle of the path, and the video is transmitted only when the car moves close to the receiver (See Supplementary Video 4 for details).\u003c/p\u003e\n\u003cp\u003eAs shown in Fig. 7(c), during the experiment, we select five different positions of the car on the moving path for demonstration. The bit error rate of video information transmission can be seen intuitively. When the car is far away from the receiver, the error rate is high and the receiver cannot receive the video information. When the car moves near the receiver, the video can be transmitted clearly. In the second experiment, the receiver is attached to the car, and the video is always transmitted smoothly while the car is moving (see Supplementary Video 5 for details). As shown in Fig. 7(d), we intercepted five different positions in the experimental process when the receiver and the moving car are bound together, and the captured video is kept transmitting at a low bit error rate. To sum up, in these two experiments, the effect of wireless transmission is demonstrated by intercepting five states of the transmitted images, as shown in Fig. 7(c) and (d). When the receiver is fixed in the middle of the moving path, effective wireless transmissions can be realized only when the car is close to the receiver and the beam is manipulated towards them. In contrast, when the receiver is attached to the car, the movement of the car does not affect the transmission of video because the beam is dynamically tracking them.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eFor the intelligent system and wireless communications, we proposed a novel scheme of target recognition and tracking system based on DPM and computer vision. In the intelligent system, the RS-Camera combined with Yolov4-tiny is used to detect the position information of the moving target, and process the detected information using a pre-trained ANN to obtain the required coding sequence for the voltage control of DPM. Then intelligently adaptive beams are generated by DPM to track the moving target in real time. This system runs effectively in a closed loop without human intervention. Experimental verifications have been carried out in different scenarios, proving that the proposed scheme has the capabilities of intelligent tracking and information transmissions. This may find promising applications in other intelligent and self-adaptive systems, including intelligent and multi-physical sensing, the internet of things (IoT) technologies. The proposed concept of intelligent tracking metasurface will also open up an avenue for challenges in the 5G and 6G wireless communications, enriching the functions of metasystems.\u003c/p\u003e "},{"header":"Methods","content":"\u003ch1\u003eDetails on the\u0026nbsp;digital programmable metasurface\u003c/h1\u003e\n\u003cp\u003eThe 1-bit dual-polarized DPM used in this work operates around the central frequency of 5.8 GHz and contains 18\u0026times;18 programmable elements. It is designed in the commercial software CST Microwave Studio and fabricated with the printed circuit board technology. Metallic structures on the top and feeding circuits on the bottom are printed on the commercial dielectric substrate F4B with dielectric constant \u003cem\u003e\u0026epsilon;\u003csub\u003er\u003c/sub\u003e\u003c/em\u003e=3 and tangent loss tan\u003cem\u003e\u0026delta;\u003c/em\u003e=0.003. The photographs of fabricated prototypes are shown in Fig. 3. Two PIN diodes (SMP1320 from SKYWORKS) are embedded into each element to control the refection phase.\u003c/p\u003e\n\u003ch1\u003eMeasurement setups\u003c/h1\u003e\n\u003cp\u003eThe experimental setup for measuring the far-field patterns was established in a microwave anechoic chamber, as illustrated in Fig. 3. The DPM and the feeding antenna were both mounted on a turntable. A feeding horn antenna was used to emit the monochromatic carrier wave with frequency \u003cem\u003ef\u003c/em\u003e = 5.8GHz via a signal generator (Keysight E8267D), and a receiving antenna was used to record the scattered powers via a spectrum analyzer (Keysight E4447A).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn the experimental process of moving target detection and identification, a transmitting horn antenna, a receiving patch antenna, a DPM, and a vector network analyzer (VNA, Agilent N5230C) are set up in the anechoic chamber, as shown in Fig. 5. The moving target is represented by one or two model cars, which are captured by the RS-Camera. VNA is used to acquire the response data by measuring the transmission coefficients (S\u003csub\u003e21\u003c/sub\u003e). To suppress the noise level in measurement, the intermediate bandwidth in VNA is set to 40 MHz.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDuring the experiment of RF signal detection, the detector based on AD8317 and MCU is used to measure the receiving level. The transmitter consists of a microwave signal generator (Keysight E8267D) and the DPM fed with a horn antenna. During the test, voltage values are obtained by the detector, and the output voltage value is connected to the MCU to obtain the corresponding dB scale for real-time monitoring and processing.\u003c/p\u003e\n\u003cp\u003eIn the experimental process of real-time wireless transmission, the image transmission module collected a picture of a video played through the notebook, and sent it to the horn antenna after modulation. The video information is transmitted to the DPM through the horn antenna, and then to the receiving module. The receiving module is a receiving antenna, a decoder and a screen connected to the decoder. When the information can be accurately transmitted, the screen will restore the image collected by the image transmission module. The experiment scenarios are shown in Fig. 7, in which the video transmission module and the feeding horn are located under the supporting platform and the horn is about 0.8m away from the DPM.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch1\u003eAcknowledgement.\u003c/h1\u003e\n\u003cp\u003eThis work was supported by the National Key Research and Development Program of China (2017YFA0700200, 2017YFA0700201, 2017YFA0700202, 2017YFA0700203), the National Natural Science Foundation of China (61971134 and 61631007), the Major Project of Natural Science Foundation of Jiangsu Province (BK20212002), the Fundamental Research Funds for the Central Universities (2242021R41078), and the 111 Project (111-2-05).\u003c/p\u003e\n\u003ch1\u003eAuthor contributions.\u003c/h1\u003e\n\u003cp\u003eW. L., W. T. and T. J. C. conceived the idea, conducted the theoretical analysis, and wrote the paper. W. L. and Q. M. proposed the concept of digital programmable metasurfaces and built the proof-of-principle prototype system. W. L., Y. Z., X. W., Jiawei W., S. G., T. Q., T. L., Q. X., T. T. G, Z. Z., F. L. and Jiaxuan W. conducted experiments and data processing. T. J. C., Q. C. and L. L. provided suggestions and comments, and helped to organize and revise the draft. All authors discussed the results and contributed to the manuscript.\u003c/p\u003e\n\u003ch1\u003eCode availability.\u003c/h1\u003e\n\u003cp\u003eThe custom computer codes utilized during the current study are available from the corresponding authors on reasonable request.\u003c/p\u003e\n\u003ch1\u003eData availability.\u003c/h1\u003e\n\u003cp\u003eThe data that support the finding of this study are available from the corresponding author upon reasonable request. Source data are provided with this paper.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003ePendry, J. B. Negative Refraction Makes a Perfect Lens. Phys. Rev. Lett. \u003cb\u003e85\u003c/b\u003e, 3966\u0026ndash;3969 (2000).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePendry, J. B., Schurig, D. \u0026amp; Smith, D. R. Controlling Electromagnetic Fields. Science \u003cb\u003e312\u003c/b\u003e, 1780\u0026ndash;1782 (2006).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZheludev, N. I. \u0026amp; Kivshar, Y. S. From metamaterials to metadevices. Nature Mater \u003cb\u003e11\u003c/b\u003e, 917\u0026ndash;924 (2012).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYu, N. \u003cem\u003eet al.\u003c/em\u003e Light Propagation with Phase Discontinuities: Generalized Laws of Reflection and Refraction. Science \u003cb\u003e334\u003c/b\u003e, 333\u0026ndash;337 (2011).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQu, S., Hou, Y. \u0026amp; Sheng, P. Conceptual-based design of an ultrabroadband microwave metamaterial absorber. \u003cem\u003eProc. Natl. Acad. Sci. U.S.A.\u003c/em\u003e \u003cb\u003e118\u003c/b\u003e, e2110490118 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTang, W. X., Zhang, H. C., Ma, H. F., Jiang, W. X. \u0026amp; Cui, T. J. Concept, Theory, Design, and Applications of Spoof Surface Plasmon Polaritons at Microwave Frequencies. Advanced Optical Materials \u003cb\u003e7\u003c/b\u003e, 1800421 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCui, T. J. \u003cem\u003eet al.\u003c/em\u003e digital metamaterials and programmable metamaterials. Light Sci Appl \u003cb\u003e3\u003c/b\u003e, e218 (2014).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCui, T. J. \u003cem\u003eet al.\u003c/em\u003e Information Metamaterial Systems. iScience \u003cb\u003e23\u003c/b\u003e, 101403 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L., Zhao, H., Liu, C., Li, L. \u0026amp; Cui, T. J. Intelligent metasurfaces: control, communication and computing. eLight \u003cb\u003e2\u003c/b\u003e, 7 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCui, T.-J., Liu, S. \u0026amp; Li, L.-L. Information entropy of coding metasurface. Light Sci Appl \u003cb\u003e5\u003c/b\u003e, e16172\u0026ndash;e16172 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu, S. \u003cem\u003eet al.\u003c/em\u003e Convolution Operations on Coding Metasurface to Reach Flexible and Continuous Controls of Terahertz Beams. Adv. Sci. \u003cb\u003e3\u003c/b\u003e, 1600156 (2016).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, R. Y., Shi, C. B., Liu, S., Wu, W. \u0026amp; Cui, T. J. Addition Theorem for Digital Coding Metamaterials. Advanced Optical Materials \u003cb\u003e6\u003c/b\u003e, 1701236 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang, C. X., Zhang, J., Cheng, Q. \u0026amp; Cui, T. J. Polarization Modulation for Wireless Communications Based on Metasurfaces. Adv. Funct. Mater. \u003cb\u003e31\u003c/b\u003e, 2103379 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, H.-X. \u003cem\u003eet al.\u003c/em\u003e Polarization-insensitive 3D conformal-skin metasurface cloak. Light Sci Appl \u003cb\u003e10\u003c/b\u003e, 75 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, X. G. \u003cem\u003eet al.\u003c/em\u003e Polarization-Controlled Dual‐Programmable Metasurfaces. Adv. Sci. \u003cb\u003e7\u003c/b\u003e, 1903382 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, X. G. \u003cem\u003eet al.\u003c/em\u003e Smart Doppler Cloak Operating in Broad Band and Full Polarizations. Adv. Mater. \u003cb\u003e33\u003c/b\u003e, 2007966 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKe, J. C. \u003cem\u003eet al.\u003c/em\u003e Linear and Nonlinear Polarization Syntheses and Their Programmable Controls based on Anisotropic Time-Domain Digital Coding Metasurface. Small Structures \u003cb\u003e2\u003c/b\u003e, 2000060 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQiu, T., Jia, Y., Wang, J., Cheng, Q. \u0026amp; Qu, S. Controllable Reflection-Enhancement Metasurfaces via Amplification Excitation of Transistor Circuit. IEEE Trans. Antennas Propagat. \u003cb\u003e69\u003c/b\u003e, 1477\u0026ndash;1482 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMa, Q. \u003cem\u003eet al.\u003c/em\u003e Controllable and Programmable Nonreciprocity Based on Detachable Digital Coding Metasurface. Adv. Optical Mater. \u003cb\u003e7\u003c/b\u003e, 1901285 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, L. \u003cem\u003eet al.\u003c/em\u003e Dual-polarization programmable metasurface modulator for near-field information encoding and transmission. Photon. Res. \u003cb\u003e9\u003c/b\u003e, 116 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXu, H. \u003cem\u003eet al.\u003c/em\u003e Radar One-Dimensional Range Profile Dynamic Jamming Based on Programmable Metasurface. Antennas Wirel. Propag. Lett. \u003cb\u003e20\u003c/b\u003e, 1883\u0026ndash;1887 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, W. \u003cem\u003eet al.\u003c/em\u003e Programmable Coding Metasurface Reflector for Reconfigurable Multibeam Antenna Application. IEEE Trans. Antennas Propagat. \u003cb\u003e69\u003c/b\u003e, 296\u0026ndash;301 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBao, L. \u003cem\u003eet al.\u003c/em\u003e Programmable Reflection\u0026ndash;Transmission Shared-Aperture Metasurface for Real‐Time Control of Electromagnetic Waves in Full Space. Adv. Sci. \u003cb\u003e8\u003c/b\u003e, 2100149 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang, H. L., Ma, H. F., Chen, M., Sun, S. \u0026amp; Cui, T. J. A Reconfigurable Multifunctional Metasurface for Full-Space Control of Electromagnetic Waves. Adv. Funct. Mater. \u003cb\u003e31\u003c/b\u003e, 2100275 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, W. \u003cem\u003eet al.\u003c/em\u003e Multi-domain functional metasurface with selectivity of polarization in operation frequency and time. J. Phys. D: Appl. Phys. \u003cb\u003e53\u003c/b\u003e, 495003 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, L. W. \u003cem\u003eet al.\u003c/em\u003e Transmission-Reflection Controls and Polarization Controls of Electromagnetic Holograms by a Reconfigurable Anisotropic Digital Coding Metasurface. Adv. Optical Mater. \u003cb\u003e8\u003c/b\u003e, 2001065 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDai, L. \u003cem\u003eet al.\u003c/em\u003e Reconfigurable Intelligent Surface-Based Wireless Communications: Antenna Design, Prototyping, and Experimental Results. IEEE Access \u003cb\u003e8\u003c/b\u003e, 45913\u0026ndash;45923 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, Z. \u0026amp; Dai, L. A Joint Precoding Framework for Wideband Reconfigurable Intelligent Surface-Aided Cell-Free Network. IEEE Trans. Signal Process. \u003cb\u003e69\u003c/b\u003e, 4085\u0026ndash;4101 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu, C., Yu, W. M., Ma, Q., Li, L. \u0026amp; Cui, T. J. Intelligent coding metasurface holograms by physics-assisted unsupervised generative adversarial network. Photon. Res. \u003cb\u003e9\u003c/b\u003e, B159 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, J. \u003cem\u003eet al.\u003c/em\u003e Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv. \u003cb\u003e7\u003c/b\u003e, eabd7690 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L. \u003cem\u003eet al.\u003c/em\u003e Electromagnetic reprogrammable coding-metasurface holograms. Nat Commun \u003cb\u003e8\u003c/b\u003e, 197 (2017).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L. \u003cem\u003eet al.\u003c/em\u003e Machine-learning reprogrammable metasurface imager. Nat Commun \u003cb\u003e10\u003c/b\u003e, 1082 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, L. \u003cem\u003eet al.\u003c/em\u003e Intelligent metasurface imager and recognizer. Light Sci Appl \u003cb\u003e8\u003c/b\u003e, 97 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, L. \u003cem\u003eet al.\u003c/em\u003e Space-time-coding digital metasurfaces. Nat Commun \u003cb\u003e9\u003c/b\u003e, 4334 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao, J. \u003cem\u003eet al.\u003c/em\u003e Programmable time-domain digital-coding metasurface for non-linear harmonic manipulation and new wireless communication systems. National Science Review \u003cb\u003e6\u003c/b\u003e, 231\u0026ndash;238 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, L. \u003cem\u003eet al.\u003c/em\u003e A wireless communication scheme based on space- and frequency-division multiplexing using digital metasurfaces. Nat Electron \u003cb\u003e4\u003c/b\u003e, 218\u0026ndash;227 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen, M. Z. \u003cem\u003eet al.\u003c/em\u003e Accurate and broadband manipulations of harmonic amplitudes and phases to reach 256 QAM millimeter-wave wireless communications by time-domain digital coding metasurface. National Science Review \u003cb\u003e9\u003c/b\u003e, nwab134 (2022).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDai, J. Y. \u003cem\u003eet al.\u003c/em\u003e Wireless Communications through a Simplified Architecture Based on Time-Domain Digital Coding Metasurface. Adv. Mater. Technol. \u003cb\u003e4\u003c/b\u003e, 1900044 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQian, C. \u003cem\u003eet al.\u003c/em\u003e Deep-learning-enabled self-adaptive microwave cloak without human intervention. Nat. Photonics \u003cb\u003e14\u003c/b\u003e, 383\u0026ndash;390 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, H.-Y. \u003cem\u003eet al.\u003c/em\u003e Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing. Patterns \u003cb\u003e1\u003c/b\u003e, 100006 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMa, Q. \u003cem\u003eet al.\u003c/em\u003e Smart metasurface with self-adaptively reprogrammable functions. Light Sci Appl \u003cb\u003e8\u003c/b\u003e, 98 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao, H. \u003cem\u003eet al.\u003c/em\u003e Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals. Nat Commun \u003cb\u003e11\u003c/b\u003e, 3926 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShan, Zhao, Pan, Wang, \u0026amp; Zhao. Sea\u0026ndash;Sky Line and its Nearby Ships Detection Based on the Motion Attitude of Visible Light Sensors. Sensors \u003cb\u003e19\u003c/b\u003e, 4004 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, B. \u0026amp; Zhang, J. A Traffic Surveillance System for Obtaining Comprehensive Information of the Passing Vehicles Based on Instance Segmentation. IEEE Trans. Intell. Transport. Syst. \u003cb\u003e22\u003c/b\u003e, 7040\u0026ndash;7055 (2021).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHan, J., Zhang, D., Cheng, G., Liu, N. \u0026amp; Xu, D. Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey. IEEE Signal Process. Mag. \u003cb\u003e35\u003c/b\u003e, 84\u0026ndash;100 (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTeng, Z., Xing, J., Wang, Q., Zhang, B. \u0026amp; Fan, J. Deep Spatial and Temporal Network for Robust Visual Object Tracking. IEEE Trans. on Image Process. \u003cb\u003e29\u003c/b\u003e, 1762\u0026ndash;1775 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYuan, D., Li, X., He, Z., Liu, Q. \u0026amp; Lu, S. Visual object tracking with adaptive structural convolutional network. Knowledge-Based Systems \u003cb\u003e194\u003c/b\u003e, 105554 (2020).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao, Z.-Q., Zheng, P., Xu, S.-T. \u0026amp; Wu, X. Object Detection With Deep Learning: A Review. IEEE Trans. Neural Netw. Learning Syst. \u003cb\u003e30\u003c/b\u003e, 3212\u0026ndash;3232 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang, G., Liu, Z., Van Der Maaten, L. \u0026amp; Weinberger, K. Q. Densely Connected Convolutional Networks. in 2017 \u003cem\u003eIEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e 2261\u0026ndash;2269 (IEEE, 2017). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2017.243\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2017.243\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, Y., Qi, Z., Zheng, H., Tao, L. \u0026amp; Gao, W. Deep Image Compression with Latent Optimization and Piece-wise Quantization Approximation. in 2021 \u003cem\u003eIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)\u003c/em\u003e 1926\u0026ndash;1930 (IEEE, 2021). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPRW53098\u003c/span\u003e\u003cspan address=\"10.1109/CVPRW53098\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. 2021.00219.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRivadeneira, R. E. \u003cem\u003eet al.\u003c/em\u003e Thermal Image Super-Resolution Challenge - PBVS 2021. in \u003cem\u003e2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)\u003c/em\u003e 4354\u0026ndash;4362 (IEEE, 2021). doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPRW53098.2021.00492\u003c/span\u003e\u003cspan address=\"10.1109/CVPRW53098.2021.00492\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJia, Y. \u003cem\u003eet al.\u003c/em\u003e In Situ Customized Illusion Enabled by Global Metasurface Reconstruction. Adv Funct Materials 2109331 (2022) doi:\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1002/adfm.202109331\u003c/span\u003e\u003cspan address=\"10.1002/adfm.202109331\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu, C. \u003cem\u003eet al.\u003c/em\u003e A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat Electron \u003cb\u003e5\u003c/b\u003e, 113\u0026ndash;122 (2022).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-1689931/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1689931/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe fifth-generation (5G) wireless communication has an urgent need for target tracking. Digital programmable metasurface (DPM) may offer an intelligent and efficient solution owing to its powerful and flexible controls of electromagnetic waves and advantages of lower cost, less complexity and smaller size than the traditional antenna array. Here, we report an intelligent metasurface system to perform target tracking and wireless communications, in which computer vision integrated with a convolutional neural network (CNN) is used to automatically detect the locations of moving targets, and the dual-polarized DPM integrated with a pre-trained artificial neural network (ANN) serves to realize smart beam tracking and wireless communications. Three groups of experiments are conducted for demonstrating the intelligent system: detection of moving targets, detection of radio-frequency signals, and real-time wireless communications. The proposed method sets the stage for an integrated implementation of target identification, radio environment tracking, and wireless communications. This strategy opens up a new avenue for intelligent wireless networks and self-adaptive systems.\u003c/p\u003e","manuscriptTitle":"Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-06-14 18:54:09","doi":"10.21203/rs.3.rs-1689931/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"c04a23f2-4474-4ed9-a23f-77a04e8a0a8c","owner":[],"postedDate":"June 14th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2023-02-23T08:10:34+00:00","versionOfRecord":{"articleIdentity":"rs-1689931","link":"https://doi.org/10.1038/s41467-023-36645-3","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2023-02-22 05:00:00","publishedOnDateReadable":"February 22nd, 2023"},"versionCreatedAt":"2022-06-14 18:54:09","video":"","vorDoi":"10.1038/s41467-023-36645-3","vorDoiUrl":"https://doi.org/10.1038/s41467-023-36645-3","workflowStages":[]},"version":"v1","identity":"rs-1689931","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1689931","identity":"rs-1689931","version":["v1"]},"buildId":"J0_U0BvcaRcwD8yVFaRlm","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}