Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time

Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time Jiseok Lee, Changil Son, Jinyoung Kim, Dongwon Kang, Seojoung Park, and 13 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4427929/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 12 Sep, 2024 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract Decoupling dynamic touch signals in the optical tactile sensors is highly desired for behavioral tactile applications yet challenging because typical optical sensors mostly measure only static normal force and use imprecise multi-image averaging for dynamic force sensing. Here, we report a highly sensitive upconversion nanocrystals-based behavioral biometric optical tactile sensor that instantaneously and quantitatively decomposes dynamic touch signals into individual components of vertical normal and lateral shear force from a single image in real-time. By mimicking the sensory architecture of human skin, the unique luminescence signal obtained is axisymmetric for static normal forces and non-axisymmetric for dynamic shear forces. Our sensor demonstrates high spatio-temporal screening of small objects and recognizes fingerprints for authentication with high spatial-temporal resolution. Using a dynamic force discrimination machine learning framework, we realized a Braille-to-Speech translation system and a next-generation dynamic biometric recognition system for handwriting. Physical sciences/Engineering/Chemical engineering Physical sciences/Materials science/Materials for devices/Sensors and biosensors Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Full Text Behavioral biometric tactile systems are emerging technologies that identify people by analyzing the way they interact with their devices. For example, by measuring how fast the person writes or moves the pen and how much pressure they exert when interacting with the device, behavioral biometric tactile algorithms can authenticate the person signing a document 1 , 2 . Because personal habits are unique, behavioral biometric data is difficult to replicate, making these technologies better at preventing theft and fraud. Consequently, there is significant interest in using dynamic pressure sensing technology to identify and measure different types of forces that can be used for behavioral recognition 3 – 14 . Optical tactile sensors are an attractive low-cost solution for interactive devices and tactile displays. While typical optical tactile sensors can measure normal force, information on in-plane motion such as velocity can only be obtained by positional analysis of two or more sequential images 15 – 23 . However, the resulting “time-averaged” velocity may not represent the true motion, especially when the directional change of motion is rapid with respect to the available rate of measurement, e.g., frames-per-second (fps) of a camera (Supplementary Fig. 1). For more accurate information on in-plane motion, the optical tactile sensor should ideally also measure “instantaneous” velocity of the in-plane motion. Because instantaneous slip velocity is highly correlated to the shear force applied to the solid-solid interface in contact mechanics, having an optical tactile sensor capable of instantaneously measuring both normal and shear force would be desirable for studying behavioral biometrics. Optical polymers 24 , 25 , photonic crystals 26 , optical light guide 27 , quantum dots 28 and organic dyes 29 , 30 have been used to build optical tactile sensors. However, these systems cannot decompose dynamic elements of forces because they have slow signal response, high limit of detection, and low spatio-temporal resolution. To analyze fast movements such as handwriting in behavioral biometrics, we need optical tactile sensors that can quantitatively decompose applied force into vertical normal and lateral shear forces from a single image. Here, we report a highly sensitive optical tactile sensor system that can instantaneously decouple dynamic touch signals in a single image for behavioral biometric analysis. Our tactile sensor contains a stress concentration layer that resembles the sensory architecture of the human skin and an optical tactile layer that contains an array of upconversion nanocrystals (UCNs). When force is applied on the sensor, the stress concentration layer amplifies and transfers the dynamic force to the tactile layer that interfaces with a total internal reflection (TIR) dove prism. Our UCNs-based optical tactile sensor generates unique axisymmetric or non-axisymmetric upconversion luminescence signals with spatio-temporal patterns that contain information on the motion of the object such as direction, velocity, and magnitude of normal and shear force. Unlike conventional optical tactile sensors, our system can quantitatively decompose the applied force into individual components of vertical normal and lateral shear force from a single image in real-time. We used our sensor system for surface profiling of objects, high-resolution fingerprint recognition and anti-counterfeiting applications. Using a machine learning framework to analyze the luminescence tactile signals, we applied the sensor system in a small-scale Braille-to-Speech (BTS) translation system. Most importantly, our sensor accurately recognized behavioral biometric handwriting patterns. We expect such a simple sensor design to find ample applications in sophisticated behavioral biometric technologies. Our tactile sensor system includes a tactile sensor pad, a TIR dove prism and a charge-coupled device (CCD) camera (Fig. 1a, and Supplementary Fig. 2). The tactile sensor pad is composed of a stress concentration layer at the top, a thin (20 nm) anti-reflective platinum (Pt) layer in between, and a signal generating tactile layer at the bottom (Fig. 1b). The stress concentration layer is designed to mimic the undulating interface of hard epidermis and soft dermis in human skin because such an interface is known to amplify applied forces by increasing interfacial area and localizing the applied force. 32 , 33 Further, mechanical mismatch between the rigid epidermis and soft dermis enhances force transmission from the skin to sensory mechanoreceptors 34 , 35 . To recreate this microarchitecture, we used stiff thermoplastic polyurethane (TPU, 52.5 MPa) and soft polydimethylsiloxane (PDMS, 1.76 MPa) with an undulated two-dimensional (2D) array of micro-hemispheres patterned at the interface (Fig. 1c, and Supplementary Fig. 3). The Pt layer in between serves to suppress diffuse reflection and improve the accuracy of the images (Supplementary Fig. 4). The bottom tactile layer contains an array of PDMS micro-hemispheres embedded with UCNs (Supplementary Fig. 5 and Supplementary Note 1). For our tactile sensor, we synthesized a hexagonal phase yellow luminescent lanthanide-doped UCNs (β-NaYF 4 :Yb 3+ /Er 3+ /Gd 3+ ; 30/2/30 mol%) using a conventional hydrothermal method (Fig. 1d, and 1e, and Supplementary Methods for details). When homogenously embedded into the PDMS micro-hemispheres (Fig. 1f and Supplementary Fig. 6), their luminescence remained unchanged (Fig. 1g, and Methods for details). UCNs are advantageous for our system because they have a high spectral resolution (Full Width at Half Maximum (FWHM) of ~ 20 nm), low optical signal errors and large anti-Stoke shifts. Further, UCNs do not require optical filters for luminescence color purification, have negligible background auto-fluorescence 36 and are optically stable upon environmental perturbations 37 . Moreover, their luminescence colors can be easily tuned 38 by simply adjusting the stoichiometry of lanthanide dopant ions (Supplementary Figs. 7, and 8, and Supplementary Methods for details). UCNs are also amenable to large-scale synthesis 39 . In our system, signal generation depends on the contact between the UCNs-embedded micro-hemispheres (UMs) in the tactile layer and the TIR prism. Irradiating the prism with 980 nm near-infrared (NIR) laser generates evanescent waves in the thin (< 100 nm) boundary layer close to the interface of the prism and UMs layer. When an object moves across our sensor pad, the stress concentration layer amplifies and transfers the applied dynamic forces to the UMs layer. As the contact area between the UM layer and TIR prism increases, the UMs produce a luminescence signal measurable by the CCD camera and luminescence spectrometer (Fig. 1h). This phenomenon can be ascribed to the transformation of the evanescence wave field into propagating far-fields at contact points between the UM and prism, where TIR is frustrated due to the decrease in the refractive index contrast according to the higher refractive index of PDMS (n = 1.4) compared to air (n = 1.0) (Supplementary Figs. 9, and 10 and Supplementary Note 2). At the contact points, the propagating far-field components can now penetrate the UM, effectively illuminating the UCNs within the UM volume. Consequently, locally compressed UMs with a more extensive contact area with the prism generate a stronger luminescence signal, while those without applied force do not yield measurable luminescence. It is noteworthy that the non-linear luminescence property of UCNs suppresses the initial luminescence caused by the evanescence wave field from the minimal contact between the UM layer and the TIR prism in the absence of applied force, consequently reducing background noise 40 . Locally pressed UMs with higher contact area with the prism produce higher luminescence signals while those without applied force do not contact the prism and therefore, do not produce luminescence. When we applied a normal force to the top layer of the tactile sensor by vertically pressing a spherical indenter (5 mm diameter), the resulting luminescence signal was axisymmetric (Fig. 1i). The tactile sensor we fabricated exhibited uniform signal generation at across the entire sensor pad or 10 different sensor pads, and demonstrated highly stable signals under continuous laser excitation (Supplementary Figs. 11–13). However, when we simultaneously applied a lateral shear force in addition to the normal force by moving the pressed indenter along the surface of the top layer, the luminescence signal became non-axisymmetric – the shape of the signal is skewed towards the direction of movement (Fig. 1j). This suggests that detailed information on the surface-parallel motions of the indenter such as the magnitude of normal force, and the velocity, direction and magnitude of lateral shear force can be extracted by analyzing the non-axisymmetric shape of the luminescence signal. These features induced subtle changes in luminescence intensity, and to effectively observe this aspect, we proposed a tactile sensor based on UCNs instead of the conventional mechanoluminescence-based tactile sensors. Optical tactile sensor using UCNs have better sensitivity and response time compared with mechanoluminescence tactile sensors 41 , 42 . By comparing the luminescence shape from optical tactile sensor, we proposed a method to determine both the direction and magnitude of normal and shear forces in real-time using a machine learning algorithm for force discrimination (Fig. 1k). By analyzing the luminescence tendency such as axisymmetric and non-axisymmetric for static and dynamic optical tactile sensors, respectively, we proposed machine learning methodologies to discriminate normal and shear forces in a single image frame in real-time. To understand how normal force affects the luminescence signals, we analyzed the signal intensity at different applied normal forces. When the magnitude of the normal force increased from 0 to 5.0 N, both luminescence intensity and the pressure distribution area increased in an axisymmetric circle (Figs. 2a and b, Supplementary Figs. 14, and 15, Supplementary Movie 1, and Methods). We observed that the stress concentration layer exhibited a strong and wide luminescence area with enhanced intensity due to its modulus difference and undulating interface (Supplementary Fig. 16). To confirm the correlation between grayscale and pressure, we first observed the area beneath the prism (Supplementary Fig. 17a). By combining the grayscale of Fig. 2a corresponding to the observed area, we could exhibit the correlation between pressure and grayscale (Supplementary Fig. 18). We obtained an optimized tactile sensor (1000 µm thick) including the stress concentration layer (200 µm thick) that can detect normal force down to 0.05 N, and later utilized for the dynamic force decoupling process (Fig. 2b, Supplementary Note 3 for optimization, and Supplementary Figs. 19–21). The characteristic luminescence intensity of yellow UCNs at 530 and 660 nm also increased with increasing normal force (Fig. 2c). The relationship between luminescence intensity and the cumulative gray value for each pixel exhibits consistent slopes concerning force magnitudes, providing the possibility of presenting a pressure distribution in a color map (Fig. 2d and Supplementary Fig. 22). Under normal force, our sensor did not show hysteresis in the luminescence intensity, indicating that the adhesion between the PDMS micro-hemispherical array and the TIR prism is negligible (Fig. 2e). Further, our tactile sensor is highly stable; its luminescence intensity remained constant when a normal force of 5.0 N was repeatedly applied over 1,000 cycles (Fig. 2f), and even at the high temperature (Supplementary Fig. 23). Furthermore, we measured the response time by repeatedly tapping the sensor (Figs. 2g, and Supplementary Fig. 24). We found the average response time using Gaussian fitting and it was 9.12 ms based on multiple repeated measurements. We further examined the temporal change in luminescence signal under applied lateral shear force (Methods in details). We began by pressing a spherical indenter to apply a 1.0 N vertical preload on the tactile sensor pad (Supplementary Note 4). The indented area forms a circular axisymmetric luminescence intensity and pressure contour. Following this, we accelerated the pressed indenter along the top surface of the sensor pad at a constant rate of 1.0 mm/s 2 until the velocity reached 1.0 mm/s. As soon as the spherical indenter began gliding, the axisymmetric luminescence circle appeared to stretch into a non-axisymmetric oval along the direction of the lateral shear force (Fig. 3a, and Supplementary Fig. 16). This shape change was recorded as a video clip at 10 frames per second (Supplementary Movie 2 middle), and the luminescence image frames were extracted from the video for analysis (Fig. 3a left). Both the pressure contour color map (Fig. 3a middle) and the luminescence intensity plot (Fig. 3a right and detailed in Supplementary Fig. 25) also clearly showed that the luminescence intensity profile had changed from a circle to an oval while the center maintained an axisymmetric circular luminescence (Fig. 3a middle). The non-axisymmetric luminescence continued to form until the indenter reached a specific velocity (Supplementary Figs. 26, and 27, and Supplementary Movie 2). Using a customized force gauge, we measured the corresponding shear force and recorded the values on each non-axisymmetric luminescence image (Fig. 3a, Supplementary Fig. 28). Unlike conventional tactile sensors where the direction of movement is obtained through coordination analysis of multiple images, we can immediately tell the direction of the applied shear force from the non-axisymmetric luminescence profile alone (Fig. 3b) 43 , 44 . Because shear force induces friction force in the opposite direction 45 , we evaluated the effect of friction on the luminescence signal (Supplementary Note 4). When shear force is applied to the sensor pad by moving the spherical indenter, the micro-hemispheres are subjected to an asymmetric lateral deformation caused by friction between the micro-hemisphere array layer and the TIR prism below the sensor pad. This results in the observed non-axisymmetric luminescence. When we reduced the friction force by spreading silicone oil between the sensor pad and the prism, the applied shear force no longer changed the shape of the luminescence signal; an axisymmetric signal was obtained even during acceleration (Supplementary Fig. 29). We further analyzed the friction force using a surface forces apparatus (SFA) (Supplementary Fig. 30, and Supplementary Methods in detail). The kinetic friction coefficient (obtained from the slope of friction versus load curve) increased from 0.148 to 0.531 as the driving velocity increased from 11.6 to 58.0 µm/s (Supplementary Fig. 31). These results indicate that increasing applied velocity increases frictional forces (or friction coefficient) and micro-hemisphere deformation, which affects the overall non-axisymmetric shape of the luminescence signal. It is worth noting that the sliding velocity-dependent friction coefficients are commonly observed, especially in complex systems with soft polymers 46 or polymer-like surfaces 47 , which contradicts conventional Amontons’ Laws of Friction 48 , 49 . To further quantify this non-axisymmetric luminescence characteristic of shear force, the degree of non-axisymmetric luminescence was statistically described using the concept of skewness (γ), which is obtained by calculating the median and mean of the luminescence intensity profile (Supplementary Note 5). For a moving indenter, the calculated skewness matched well with the measured shear force (Fig. 3c), indicating that the magnitude of the shear force could be quantitatively estimated directly from the non-axisymmetric luminescence profile. When we analyzed the change in shear force at a given skewness for various normal forces, skewness decreased when the normal force increased from 0.5 to 1.5 N (Fig. 3d, and Supplementary Figs. 32–34). This is because an increase in normal force decreases the relative contribution of shear force (and thus, skewness value) to the luminescence signal, which is formed from both the normal force that creates the axisymmetric profile, and shear force that creates the non-axisymmetric profile (Supplementary Fig. 35). Based on the single image frame analysis of luminescence intensity profile for static normal force and the directional skewness for dynamic shear force, we successfully quantified in real time the direction and magnitude of both normal and shear forces from a single image using force discrimination machine learning algorithm (Fig. 3e, Supplementary Fig. 36, Supplementary Tables 1–3, Supplementary Movies 3, and 4, and Supplementary Notes 6, and 7). Our machine learning model was highly reliable with r 2 values of 0.965 and 0.947 for normal and shear forces, respectively. In detail, the r, g, and b values of the detected images were analyzed in histogram format to predict the applied normal force. By utilizing the predicted normal force value, the threshold for displaying the contour of the normal force region was also predicted. As a result, normal and shear forces were successfully decoupled based on the determined threshold value. Note that the trend of skewness change was consistent irrespective of the fps, indicating that the more cost-effective 10-fps camera is suitable for the normal and shear force decoupling (Supplementary Figs. 37, and 38). We further examined whether and how the undulated interface between the stiff TPU and soft PDMS in the stress concentration layer of the tactile sensor pad affects the non-axisymmetric luminescence (Supplementary Figs. 39, and 40). When an indenter moved at a velocity of 1.0 mm/s with 1.0 mm/s 2 acceleration across a pad without undulation at the TPU-PDMS interface, the maximum value of skewness was 0.1 (Supplementary Fig. 41). With undulation, the maximum value of skewness increased to 0.37. When the velocity of the indenter was decreased to velocity of 0.5 mm/s with 0.5 mm/s 2 , the skewness decreased further to 0.32. Our results indicate that the undulation at the interface and the differences in modulus between TPU and PDMS improved the transmission of the applied shear force to the signal-generating UMs layer, which, in turn, increased the non-axisymmetric of the luminescence signal. To further understand how the undulation at the TPU-PDMS interface enhances force transmission, we used finite element analysis (FEA) to examine the stress distribution of 2D model sensors with and without interface undulation under 1.0 N applied normal force (F n ) with and without shear force (F s ) ranging from 0 to 1.0 N (Fig. 3f, Supplementary Figs. 42–50, and Supplementary Note 8). Because luminescence signal in our tactile sensor is produced only when UMs contact the TIR prism, we expected the intensity of the luminescence signal in the model sensor to scale with the contact width between the individual UMs and the underlying substrate. Consistent with the experimental intensity profile, our FEA analysis showed that contact width is symmetric when only F n is applied but is skewed toward the direction of applied F s (Fig. 3g and Supplementary Figs. 46, and 47). The resulting calculated skewness, γ, increased with the magnitude of F s and was substantially higher in the model sensor with an undulating interface (Fig. 3h). These calculations corroborate the experimental results that showed higher F s transmission in sensors with an undulating interface (Supplementary Figs. 39, and 40). We visualized this enhancement by plotting the differences in stress distribution for sensors (σ w ) and without (σ wo ) an undulating interface, Δσ (Fig. 3i). Roughly 1 mm anterior to the center of the applied force – especially near the micro-hemispheres array – Δσ yx increased strongly, indicating that F s was more effectively transmitted to the micro-hemispherical array owing to the undulating interface. No such enhancement was observed for Δσ yy . We also found that σ yy remained symmetric even under F s , whereas σ xy became more asymmetric with increasing F s (Supplementary Fig. 48). A more careful examination of σ xy near the TPU-PDMS interface indicates that the regions beneath the interface have on average 50% higher value of σ xy for the undulated interface (Supplementary Fig. 49). This phenomenon was observed to be independent of the indenter shape (Supplementary Fig. 50). Furthermore, similar non-axisymmetric luminescence patterns were identified for indenters with different shape (spherical, elliptical, rectangular, and triangular), spherical with different curvature, and cut cone with different radius geometries (Supplementary Figs. 51–56). Consequently, this behavior can be observed regardless of the indenter shape. Although the stress concentration layer may hinder the precise determination of the tip shape, we employed deep learning techniques to facilitate shape identification (Supplementary Fig. 57 and detailed in Supplementary Note 9). Additionally, asymmetric fluorescence emission and the resulting change in skewness when shear force is applied have been confirmed using various tools within the laboratory (Supplementary Fig. 58). While the stress concentration layer exhibited an excellent characteristic of shear force, its resolution decreased at low normal forces (Supplementary Figs. 17, and 59). To accommodate various applications, we fabricated 200 µm thickness sensor pads without the stress concentration layer for precise vertical normal force measurements and surface inspection. Along with the high-resolution capability of detecting low pressures of 0.02 N (Supplementary Figs. 60–62), which an order of magnitude better than current optical tactile sensors that detect forces down to 0.2 N (Supplementary Table 4) 15 – 22 , 41 , 42 our optical sensor enabled high spatio-temporal tactile recognition of 2D surface textures including multiple objects (Fig. 4a-c, and Supplementary Fig. 63) and fingerprints of different personnel (Fig. 4d, and Supplementary Fig. 64). The maximum spatial resolution of our tactile sensor is 100 µm, which is on the upper bounds of the spatial resolution found in previously studied optical tactile sensors (Supplementary Fig. 65, and Supplementary Table 4) 15 – 22 , 41 , 42 . Our optical tactile sensor can also recognize and distinguish different surface textures. For example, small screws having different threads too fine for the human eye to see were identified in a high contrast manner using MATLAB-based automated image analysis code (Figs. 4e, and f, Supplementary Fig. 66, Supplementary Movie 5, and Supplementary Note 10) 50 . The size and shape of dimples (Fig. 4g) and signs of use (Fig. 4h) of different golf balls were also visualized with high spatial-temporal resolution. We built a small-scale BTS transformation system that can instantly translate microscopic braille patterns into the corresponding sounds using a machine learning algorithm (Supplementary Note 11). Topographic patterns of micro-sized braille for “Hello World” were converted to luminescence images (Fig. 5a), and instantly translated into the corresponding sounds by extracting 2D patterns of dots (Figs. 5b, and c, Supplementary Figs. 67, and 68, and Supplementary Movie 6). The diameter of individual braille dots in this experiment was two times smaller (500 µm) than standard dots (1 mm), increasing the density of storable information by an order of magnitude (~ 3 2 ). Most notably, using the ability to simultaneously visualize the direction of movement and the intensity change of both normal and shear force, we developed a machine learning (ML) framework that distinguishes dynamic biometric handwriting of individuals (Fig. 5d, Supplementary Movie 7, and detailed in Supplementary Notes 12–14). Decoupled images represented normal and shear forces, and related features extracted from videos of three different individuals writing the letter ‘e’ were fed into our ML-based linear discriminant analysis (LDA) for writing style classification. LDA clustering successfully discriminated the three different handwritten letters ‘e’ (Fig. 5e, Supplementary Figs. 69–73, Supplementary Tables 6–9, and Supplementary Note 15). We further extracted the averaged velocity-related features from multiple image frames and combined them with normal and shear force features for handwriting analysis. Our single-frame touch signal decoupling-based ML framework showed outstanding performance when compared to analyzing the handwriting using normal forces alone or normal forces with velocity-related features (Fig. 5f, Supplementary Figs. 74, and 75, Supplementary Table 8). While the average distance within a specific cluster was the same for all analysis, the average distance between different clusters was greater when the analysis included shear force features than when it included velocity features. Comparing the data cluster analyzed using both velocity and shear force features with those analyzed using velocity only and shear force only, we find that the analysis using combined features had a greater impact on the data clusters of velocity only analysis than on shear force only analysis. Further, between velocity only analysis and shear force only analysis, we find that shear force feature-based handwriting analysis forms a more clear-cut data cluster (Supplementary Fig. 72). To compare clustering performance, we calculated the Silhouette coefficient, which describes the average distance between points within the same cluster and the average distance between points in the nearest cluster to which the data point does not belong to, and the Calinski-Harabasz index, which is the ratio of dispersion within a cluster to dispersion between clusters. Of all the analysis combinations, the normal/shear force combination showed a Silhouette coefficient closest to 1 (0.867) and a remarkable value Calinski-Harabasz index (5.76×10 2 ) (Fig. 5g, Supplementary Fig. 73, and Supplementary Table 9). Adding velocity feature to the normal/shear force feature slightly decreased the Silhouette coefficients (0.858) and Calinski-Harabasz index (4.52×10 2 ) and there is no significant advantage in terms of computational cost. From this point of view, we believe that besides normal force, shear force feature is a dominant factor for distinguishing writing style. These results demonstrate that normal/shear force-based handwriting analysis represents the most clear-cut cluster and is well classified with clusters of different force combinations. In conventional handwriting authentication, verification is predominantly grounded in the consistency of static handwriting features such as letter size, inclination, and inter-character spacing, rather than dynamic attributes. Remarkably, our study introduces an innovative methodology for handwriting verification, focused on shear force—a parameter not previously explored in this context. This approach encapsulates a wealth of data, encompassing factors such as the exerted pressure during writing and the speed of inscription. These parameters raise the expectation that our framework can offer valuable insights into the writer’s identity, as they exhibit variations correlated with the writer’s age and gender 51 . Furthermore, our framework possesses the unique capability to decouple normal and shear forces within a single frame, allowing it to provide real-time feedback unlike previous studies that evaluated handwriting only after all letters were completed. Thus, our system holds the potential to significantly enhance the precision of handwriting authentication process. In conclusion, we present an upconversion luminescence-based behavioral biometric optical tactile sensor that simultaneously and quantitatively discriminates velocity, direction, normal and shear force in a single frame analysis. Such a unique axisymmetric and non-axisymmetric luminescence signal amplified through the human skin mimetic stress concentration microarchitecture was theoretically studied by FEA simulation, experimentally examined by SFA measurement, and quantitatively analyzed. Our optical sensor is simple to build, and we show they can be used for tactile recognition of surface textures and fingerprints. Using machine learning, we applied the sensor system in a small-scale BTS system. Most importantly, our system has enabled accurate dynamic biometric handwriting analysis. We believe our sensor system will open new avenues in facile optical tactile sensors that can intuitively and sensitively display multiple elements of dynamic forces in a single image frame, forming the basis of flexible behavioral biometric optical tactile sensors in wearable devices. Declarations Reprint and permissions information is available online at www.nature.com/reprints. Competing financial interests The authors declare no competing financial interests. References Alzubaidi, A. & Kalita, J. Authentication of smartphone users using behavioral biometrics. IEEE Communications Surveys & Tutorials 18 , 1998-2026 (2016). Frank, M., Biedert, R., Ma, E., Martinovic, I. & Song, D. Touchalytics: On the applicability of touchscreen input as a behavioral biometric for continuous authentication. IEEE transactions on information forensics and security 8 , 136-148 (2012). Ha, M. et al. Bioinspired interlocked and hierarchical design of ZnO nanowire arrays for static and dynamic pressure‐sensitive electronic skins. Advanced Functional Materials 25 , 2841-2849 (2015). Park, J., Kim, M., Lee, Y., Lee, H. S. & Ko, H. Fingertip skin–inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. Science advances 1 , e1500661 (2015). Lee, H.-K., Chung, J., Chang, S.-I. & Yoon, E. Normal and shear force measurement using a flexible polymer tactile sensor with embedded multiple capacitors. Journal of Microelectromechanical Systems 17 , 934-942 (2008). Li, R., Nie, B., Digiglio, P. & Pan, T. Microflotronics: A Flexible, Transparent, Pressure‐Sensitive Microfluidic Film. Advanced Functional Materials 24 , 6195-6203 (2014). Ramuz, M., Tee, B. C. K., Tok, J. B. H. & Bao, Z. Transparent, optical, pressure‐sensitive artificial skin for large‐area stretchable electronics. Advanced Materials 24 , 3223-3227 (2012). Stassi, S., Cauda, V., Canavese, G. & Pirri, C. F. Flexible tactile sensing based on piezoresistive composites: A review. Sensors 14 , 5296-5332 (2014). Viry, L. et al. Flexible three‐axial force sensor for soft and highly sensitive artificial touch. Advanced materials 26 , 2659-2664 (2014). Wang, L. & Beebe, D. J. A silicon-based shear force sensor: development and characterization. Sensors and Actuators A: Physical 84 , 33-44 (2000). Wang, X. et al. Self‐powered high‐resolution and pressure‐sensitive triboelectric sensor matrix for real‐time tactile mapping. Advanced materials 28 , 2896-2903 (2016). Yoon, S. G., Park, B. J. & Chang, S. T. Highly sensitive piezocapacitive sensor for detecting static and dynamic pressure using ion-gel thin films and conductive elastomeric composites. ACS applied materials & interfaces 9 , 36206-36219 (2017). Zheng, Q. et al. Dynamic real-time imaging of living cell traction force by piezo-phototronic light nano-antenna array. Science Advances 7 , eabe7738 (2021). https://doi.org/doi:10.1126/sciadv.abe7738 Jia, M. et al. Tactile tribotronic reconfigurable pn junctions for artificial synapses. Science Bulletin 67 , 803-812 (2022). Wei, X. Y. et al. Dynamic triboelectrification‐induced electroluminescence and its use in visualized sensing. Advanced Materials 28 , 6656-6664 (2016). Lee, B. et al. Ultraflexible and transparent electroluminescent skin for real-time and super-resolution imaging of pressure distribution. Nature Communications 11 , 663 (2020). Zhao, X. et al. Self-powered user-interactive electronic skin for programmable touch operation platform. Science advances 6 , eaba4294 (2020). Du, Y. et al. Mechanically excited multicolor luminescence in lanthanide ions. Advanced Materials 31 , 1807062 (2019). Chen, C. et al. Creating visible-to-near-infrared mechanoluminescence in mixed-anion compounds SrZn2S2O and SrZnSO. Nano Energy 68 , 104329 (2020). Wang, Y. et al. Enhanced high-resolution triboelectrification-induced electroluminescence for self-powered visualized interactive sensing. ACS applied materials & interfaces 11 , 13796-13802 (2019). Wei, X. Y. et al. High‐intensity triboelectrification‐induced electroluminescence by microsized contacts for self‐powered display and illumination. Advanced Materials Interfaces 5 , 1701063 (2018). Zhuang, Y. et al. Force-induced charge carrier storage: a new route for stress recording. Light: Science & Applications 9 , 182 (2020). Dvorak, N., Chung, K., Mueller, K. & Ku, P.-C. Ultrathin tactile sensors with directional sensitivity and a high spatial resolution. Nano Letters 21 , 8304-8310 (2021). Chou, H.-H. et al. A chameleon-inspired stretchable electronic skin with interactive colour changing controlled by tactile sensing. Nature communications 6 , 8011 (2015). Haque, M. A., Kurokawa, T., Kamita, G., Yue, Y. & Gong, J. P. Rapid and reversible tuning of structural color of a hydrogel over the entire visible spectrum by mechanical stimulation. Chemistry of Materials 23 , 5200-5207 (2011). Liu, H. et al. Pressure-induced blue-shifted and enhanced emission: a cooperative effect between aggregation-induced emission and energy-transfer suppression. Journal of the American Chemical Society 142 , 1153-1158 (2020). Kim, J. T., Choi, H., Shin, E., Park, S. & Kim, I. G. Graphene-based optical waveguide tactile sensor for dynamic response. Scientific reports 8 , 16118 (2018). Xu, Y. et al. Glowing stereocomplex biopolymers are generating power: polylactide/carbon quantum dot hybrid nanofibers with high piezoresponse and multicolor luminescence. Journal of Materials Chemistry A 7 , 1810-1823 (2019). Wang, Y. et al. Dynamic behavior of molecular switches in crystal under pressure and its reflection on tactile sensing. Journal of the American Chemical Society 137 , 931-939 (2015). Zhao, W. et al. Highly sensitive switching of solid-state luminescence by controlling intersystem crossing. Nature communications 9 , 3044 (2018). Chen, G., Qiu, H., Prasad, P. N. & Chen, X. Upconversion nanoparticles: design, nanochemistry, and applications in theranostics. Chemical reviews 114 , 5161-5214 (2014). Prevost, A., Scheibert, J. & Debrégeas, G. Effect of fingerprints orientation on skin vibrations during tactile exploration of textured surfaces. Communicative & integrative biology 2 , 422-424 (2009). Aleemardani, M., Trikić, M. Z., Green, N. H. & Claeyssens, F. The importance of mimicking dermal-epidermal junction for skin tissue engineering: a review. Bioengineering 8 , 148 (2021). Cauna, N. Nature and functions of the papillary ridges of the digital skin. The Anatomical Record 119 , 449-468 (1954). Maeno, T., Kobayashi, K. & Yamazaki, N. Relationship between the structure of human finger tissue and the location of tactile receptors. JSME International Journal Series C Mechanical Systems, Machine Elements and Manufacturing 41 , 94-100 (1998). Wang, F. et al. Simultaneous phase and size control of upconversion nanocrystals through lanthanide doping. nature 463 , 1061-1065 (2010). Liu, D. et al. Emission stability and reversibility of upconversion nanocrystals. Journal of Materials Chemistry C 4 , 9227-9234 (2016). Baek, D. et al. Multi‐Color Luminescence Transition of Upconversion Nanocrystals via Crystal Phase Control with SiO2 for High Temperature Thermal Labels. Advanced Science 7 , 2000104 (2020). Lee, J. et al. Universal process-inert encoding architecture for polymer microparticles. Nature materials 13 , 524-529 (2014). De Camillis, S. et al. Controlling the non-linear emission of upconversion nanoparticles to enhance super-resolution imaging performance. Nanoscale 12 , 20347-20355 (2020). Ma, Z. et al. Mechanics-induced triple-mode anticounterfeiting and moving tactile sensing by simultaneously utilizing instantaneous and persistent mechanoluminescence. Materials Horizons 6 , 2003-2008 (2019). Wang, X. et al. Dynamic pressure mapping of personalized handwriting by a flexible sensor matrix based on the mechanoluminescence process. Advanced Materials 27 , 2324-2331 (2015). Boutry, C. M. et al. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Science Robotics 3 , eaau6914 (2018). Tao, J. et al. Self‐powered tactile sensor array systems based on the triboelectric effect. Advanced Functional Materials 29 , 1806379 (2019). Mostaghel, N. & Davis, T. Representations of Coulomb friction for dynamic analysis. Earthquake engineering & structural dynamics 26 , 541-548 (1997). Lee, D. W., Banquy, X. & Israelachvili, J. N. Stick-slip friction and wear of articular joints. Proceedings of the National Academy of Sciences 110 , E567-E574 (2013). Maksuta, D. et al. Dependence of adhesive friction on surface roughness and elastic modulus. Soft Matter 18 , 5843-5849 (2022). Gao, J. et al. Vol. 108 3410-3425 (ACS Publications, 2004). Amontons, G. De la resistance cause'e dans les machines (1). JOURNAL-JAPANESE SOCIETY OF TRIBOLOGISTS 44 , 229-235 (1999). Guo, P., Jia, M., Guo, D., Wang, Z. L. & Zhai, J. Retina-inspired in-sensor broadband image preprocessing for accurate recognition via the flexophototronic effect. Matter 6 , 537-553 (2023). No, B. & Choi, N. Differences in graphomotor skills by the writing medium and children’s gender. Education Sciences 11 , 162 (2021). Methods Materials. NaOH (Aldrich), ethanol (Aldrich, 99 %), GdCl 3 ·6H 2 O (Aldrich, 99.999 %), YCl 3 ·6H 2 O (Aldrich, 99.999 %), YbCl 3 ·6H 2 O (Aldrich, 99.999 %), TmCl 3 ·6H 2 O (Aldrich, 99.9 %), NH 4 F (Aldrich, 99.9 %), ErCl 3 ·6H 2 O (Aldrich, 99.999 %), and oleic acid (Aldrich, 90 %). Fabrication of the UCN-embedded micro-hemisphere layer assembled with a human skin mimetic stress concentration layer. The synthesized UCNs were dispersed in n-hexane by ultrasonic sonication (VCX 750, Sonics) for 2 hours. The UCN solution was added to the PDMS base (Sylgard 184, Dow Corning) in a 2:3 ratio (UCN: PDMS). The UCN/PDMS mixture was dispersed using an ultrasonic processor for 1 hour. Subsequently, a PDMS curing agent (1:5 ratio of curing agent to base) was added to the UCN/PDMS mixture depending on the thickness of the sensor. The resulting UCN/PDMS mixture was dispersed using an ultrasonic process for 5 min before pouring into a micro-hemisphere silicon mold (diameter and pitch: 90 and 100 mm, respectively) and cured at 90 °C for 4 h. The dispersion of UCNs and PDMS for each step was observed (Supplementary Fig. 6). The top surface of the UMs array was coated with a Pt layer using an ion sputter (MC1000, Hitachi). To fabricate the stress concentration layer, the pre-cured PDMS solution was coated on a micro-hemisphere silicon mold (diameter and pitch: 20 and 40 mm, respectively) using a spin coater at 750 rpm for 1 min. After curing at 90 °C for 2 h, the obtained undulating microstructured PDMS was attached to a Pt layer. After 1 min of the O 2 plasma process (FEMTO SCIENCE, CUTE-1MPR, 100 W, 8 sccm), the TPU solution dissolved in DMF was poured onto the undulating microstructured PDMS surface, and the DMF solvent was evaporated at 60 °C for 12 h. Normal force and shear force detection device setting. To measure the optical properties of the optical tactile sensor under mechanical forces, the fabricated UMs integrated with a stress concentration layer were placed on the TIR dove prism where a 980 nm NIR laser (FC-EW-980-40W, Uniotech) was irradiated.A schematic of the normal-force detection system is shown in Supplementary Fig. 5a. The normal force was measured by a vertically attached force gauge with a spherical indenter to the Z-axis liner stage. The force gauge used in our study had a resolution of 0.01 N. A normal force of 0–5.0 N was measured by moving the Z-axis linear stage, and movie files of the luminescence profiles were recorded (Supplementary Movie 1). Supplementary Fig. 5b shows a schematic of the shear-force detection system. To measure the shear force, a right-angle spherical indenter was fabricated using a 90° tilted digital force gauge, and the shear force applied according to the velocity and acceleration of the spherical indenter was measured. Supplementary Movie 2 shows the change in the luminescence profile as the velocity reaches 1.5, 1.0, and 0.5 mm/s (acceleration: 1.5, 1.0, and 0.5 mm/s 2 , respectively); the velocity was maintained (1.5 mm/s: 2.3 s, 1.0 mm/s: 4 s, and 0.5 mm/s: 9 s respectively), and then decelerated to 0 mm/s in 1 s. To obtain the luminescence images and movie file, a CCD camera (DCC3260C, Thorlabs) was placed under a glass dove prism, and a 750 nm cut-off filter (FESH0750, Thorlabs) was used to eliminate 980 nm NIR light. Hand-writing data mining machine learning. Regression analysis from the normal and shear force predictions for each related feature extraction was conducted based on support vector regression (SVR). SVR is typically used for nonlinear systems and predicts the appropriate hyperplane in higher dimensions to fit the input data. Parameters C = 10, degree = 3, epsilon = 0.1, and kernel = 'rbf' were selected for the best performance results. The model was trained on 70 % of the data randomly split from the dataset, and the unused data were used to test the model performance. To evaluate the performance, r 2 , RMSE, and MAPE parameters were used. Linear discriminator analysis (LDA), which determines linear combinations of features that characterize or distinguish two or more classes, was used to classify handwriting by lowering the dimensions of the extracted force-related features. In addition, the clustering performance was calculated using the silhouette coefficient and the Calinski–Harabasz index for each class. Additional Declarations There is NO Competing Interest. Supplementary Files SupplementaryMovie1.mp4 Axisymmetric circular luminescence change when applied the normal force SupplementaryMovie2.mp4 Non-axisymmetric oval luminescence change when applied the shear force SupplementaryMovie3.mp4 Machine learning assisted quantitative analysis of the normal force SupplementaryMovie4.mp4 Decoupling of the applied dynamic force to the normal and shear force by spherical indenter using machine learning algorithm SupplementaryMovie5.mp4 Classification of multiple types of screws SupplementaryMovie6.mp4 Machine learning assisted braille-to-speech system SupplementaryMovie7.mp4 Decoupling of the applied dynamic force to the normal and shear force by pen using machine learning algorithm NCOMMS2342760Ars.pdf SupplementaryInformation.docx nreditorialpolicychecklist.pdf Editorial Policy Checklist nrreportingsummary.pdf Reporting Summary Cite Share Download PDF Status: Published Journal Publication published 12 Sep, 2024 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4427929","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":303641302,"identity":"48599434-ed56-48e0-8876-d0019eeecad7","order_by":0,"name":"Jiseok Lee","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0UlEQVRIie3OvQrCMBiF4VMKZvlwbinqNYSCk/RaDAWvwiFQ0EV0LXgTbuIWKLRL0VVwaSfXTC6CWH8Gt2R0yDsEEvLAAVyuP8zLXqeaAP73JbAksxfxpBX5pIo3tyP+krWNrk9iz9i1wTxBuFWmYRTz/HwRh4y4RJki6k+NpBeRvohdQd2wnsKQDMO8jF3vpI8dYa3Ew4pg7NNZdQRceguFyEwoDvM6jbthPBfrlMKVgfBN1WpdJoPdqWq0viXDoDYR+XubAqZZwMj4w+VyuVxP1ARBy3NLQzsAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-2968-2458","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":true,"prefix":"","firstName":"Jiseok","middleName":"","lastName":"Lee","suffix":""},{"id":303641303,"identity":"47c92ecf-2038-4086-94d8-4aa354605e80","order_by":1,"name":"Changil Son","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Changil","middleName":"","lastName":"Son","suffix":""},{"id":303641304,"identity":"5c43f8b3-05d3-479d-a6f9-4741aecb1949","order_by":2,"name":"Jinyoung Kim","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Jinyoung","middleName":"","lastName":"Kim","suffix":""},{"id":303641305,"identity":"5b0ebea2-5f3b-4ad4-b657-c4764de6c5e5","order_by":3,"name":"Dongwon Kang","email":"","orcid":"","institution":"Sogang University","correspondingAuthor":false,"prefix":"","firstName":"Dongwon","middleName":"","lastName":"Kang","suffix":""},{"id":303641306,"identity":"e5c75794-0769-435c-8780-f16cb8edb21b","order_by":4,"name":"Seojoung Park","email":"","orcid":"https://orcid.org/0000-0001-8889-1378","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Seojoung","middleName":"","lastName":"Park","suffix":""},{"id":303641307,"identity":"710c0519-7799-4231-8231-da15ed59d3bd","order_by":5,"name":"Sanggyun Jung","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Sanggyun","middleName":"","lastName":"Jung","suffix":""},{"id":303641308,"identity":"f3f1ecd3-4168-4600-bc09-77c9ddf41aa7","order_by":6,"name":"Seonghyeon Ahn","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Seonghyeon","middleName":"","lastName":"Ahn","suffix":""},{"id":303641309,"identity":"81150c3e-6e13-4483-bb35-f0946dc2bfa3","order_by":7,"name":"Chanoong Lim","email":"","orcid":"https://orcid.org/0000-0001-6903-0855","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Chanoong","middleName":"","lastName":"Lim","suffix":""},{"id":303641310,"identity":"96b228dd-e63c-48c4-8425-cb1a6bd875ba","order_by":8,"name":"Dong Woog Lee","email":"","orcid":"https://orcid.org/0000-0002-1572-9270","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Dong","middleName":"Woog","lastName":"Lee","suffix":""},{"id":303641311,"identity":"f987f6c3-ab40-457f-bcf4-83dea3a756be","order_by":9,"name":"Donghyuk Kim","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Donghyuk","middleName":"","lastName":"Kim","suffix":""},{"id":303641312,"identity":"41f2bb93-c60f-43a2-b914-a2ae9f1fe9de","order_by":10,"name":"Jungwook Kim","email":"","orcid":"https://orcid.org/0000-0002-0376-9080","institution":"Seoul National University","correspondingAuthor":false,"prefix":"","firstName":"Jungwook","middleName":"","lastName":"Kim","suffix":""},{"id":303641313,"identity":"f074ee18-aa0e-42c8-9cf3-033fbb205c72","order_by":11,"name":"Hyunhyub Ko","email":"","orcid":"https://orcid.org/0000-0003-2111-6101","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Hyunhyub","middleName":"","lastName":"Ko","suffix":""},{"id":303641314,"identity":"2f270fab-7e8f-4b8a-9ed2-32523b6c171e","order_by":12,"name":"Chaeyeong Ryu","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Chaeyeong","middleName":"","lastName":"Ryu","suffix":""},{"id":303641315,"identity":"e00e5a98-8d41-44ce-a10e-a3a4a03a10ad","order_by":13,"name":"Dahye Baek","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Dahye","middleName":"","lastName":"Baek","suffix":""},{"id":303641316,"identity":"b68403dc-1bc9-4602-9123-7e46fcaec5cb","order_by":14,"name":"Geonyoung Jung","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Geonyoung","middleName":"","lastName":"Jung","suffix":""},{"id":303641317,"identity":"0c82aa8d-235b-4c3d-b390-fab9eb25fa4c","order_by":15,"name":"Yundon Jeong","email":"","orcid":"","institution":"Ulsan National Institute of Science and Technology (UNIST)","correspondingAuthor":false,"prefix":"","firstName":"Yundon","middleName":"","lastName":"Jeong","suffix":""},{"id":303641318,"identity":"0c46d40c-81ef-44b2-83a3-f788c8047580","order_by":16,"name":"Jung-Hoon Park","email":"","orcid":"https://orcid.org/0000-0001-5316-1690","institution":"Ulsan National Institute of Science and Technology (UNIST)","correspondingAuthor":false,"prefix":"","firstName":"Jung-Hoon","middleName":"","lastName":"Park","suffix":""},{"id":303641319,"identity":"3d54dd1e-b913-4aa6-b3a5-04fb9ec695df","order_by":17,"name":"Jeongin Eom","email":"","orcid":"","institution":"Sogang University","correspondingAuthor":false,"prefix":"","firstName":"Jeongin","middleName":"","lastName":"Eom","suffix":""}],"badges":[],"createdAt":"2024-05-16 02:30:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4427929/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4427929/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-024-52331-4","type":"published","date":"2024-09-12T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":56973830,"identity":"eff25cae-84ef-4f09-b332-741a2b8a1395","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1174072,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eUCNs-based optical tactile sensor resembling human skin.\u003c/strong\u003e \u003cstrong\u003ea\u003c/strong\u003e, UCNs-based optical tactile sensor pad generating upconversion tactile signals \u003cem\u003evia \u003c/em\u003efrustrated TIR of 980 nm NIR illumination laser at the UM/prism interface. \u003cstrong\u003eb\u003c/strong\u003e, Schematic of the optical tactile sensor pad consisting of a stress concentration layer made up of stiff TPU interlocked with a soft PDMS patterned with micro-hemispheres, a thin (20 nm) anti-reflective Pt layer, and a signal-generating tactile layer containing an array of UMs. \u003cstrong\u003ec\u003c/strong\u003e, Cross-sectional SEM images of the optical tactile sensor pad. (Scale bar, 100 µm) \u003cstrong\u003ed\u003c/strong\u003e, TEM image of rod-like hexagonal phase lanthanide UCNs. (Scale bar, 100 nm) \u003cstrong\u003ee\u003c/strong\u003e, Photograph of UCNs in cyclohexane excited by 980 nm NIR light display yellow luminescence. (Scale bar, 1 mm) \u003cstrong\u003ef\u003c/strong\u003e, Luminescence image of UMs array upon 980 nm NIR excitation. (Scale bar, 200 µm) \u003cstrong\u003eg\u003c/strong\u003e, Emission spectra of yellow luminescence of UCNs and UCNs embedding in PDMS micro-hemispheres. \u003cstrong\u003eh\u003c/strong\u003e, Principle of upconversion luminescence generation depending on the contact area changes between UM and TIR prism from the applied force. \u003cstrong\u003ei-j\u003c/strong\u003e, Luminescence intensity profile under (\u003cstrong\u003ei\u003c/strong\u003e) static normal (axisymmetric luminescence) and (\u003cstrong\u003ej\u003c/strong\u003e) dynamic shear force (non-axisymmetric luminescence). \u003cstrong\u003ek\u003c/strong\u003e, Decoupling of normal and shear forces from a single image frame using Machine Learning-based feature engineering.\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/170150f08095b56ab3fed8c4.png"},{"id":56973838,"identity":"b09286fc-1e18-4bc7-bf2a-da72794c7ceb","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":495339,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eQuantitative measurement of static normal force. a, \u003c/strong\u003eThe axisymmetric UCN luminescence image (\u003cem\u003etop\u003c/em\u003e) and its pressure distribution profile (\u003cem\u003ebottom\u003c/em\u003e) under the normal force using a spherical indenter on the tactile sensor pad. (Scale bar, 2 mm) \u003cstrong\u003eb\u003c/strong\u003e, Normalized UCN intensity across the centered indented area of the tactile sensor pad with increasing normal force from 0 to 5.0 N. (Scale bar, 2 mm) \u003cstrong\u003ec\u003c/strong\u003e, Change of upconversion luminescence spectra with increasing normal force from 0 to 5.0 N. \u003cstrong\u003ed\u003c/strong\u003e, Double y-axis graph of integration luminescence intensity from 510 to 560 nm wavelength, and gray value of luminescence image. \u003cstrong\u003ee\u003c/strong\u003e, Loading-unloading hysteresis curves of luminescence signal.\u003cstrong\u003e f\u003c/strong\u003e, Repeated stability test of luminescence intensity of UCNs-based optical tactile sensor subjected to a 5.0 N normal force over ~ 5,000 cycles. \u003cstrong\u003eg\u003c/strong\u003e, Response time of our tactile sensor using Gaussian fitting (9.12 ms).\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/4bffe8a391b502c45c205642.png"},{"id":56973832,"identity":"91ce2a30-f84c-4c07-9114-14d0d7d714e5","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1136483,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eQuantitative analysis of dynamic shear force and importance of interlocking microarchitecture for force transmission. a,\u003c/strong\u003e Luminescence images (\u003cem\u003eleft\u003c/em\u003e), pressure distribution profile (\u003cem\u003emiddle\u003c/em\u003e), and luminescence intensity plots (\u003cem\u003eright\u003c/em\u003e) show quantitative changes in the shape of the luminescence signal as the velocity of the spherical indenter increases. (Scale bar, 2 mm) \u003cstrong\u003eb,\u003c/strong\u003e UCN luminescence image (\u003cem\u003eleft\u003c/em\u003e) and pressure contour map (\u003cem\u003eright\u003c/em\u003e) change as the indenter moves in eight different directions (Scale bar, 2 mm). \u003cstrong\u003ec,\u003c/strong\u003e Calculated skewness for a moving indenter matched the measured shear force. \u003cstrong\u003ed,\u003c/strong\u003e Overlay of calculated skewness with the experimentally measured shear force. \u003cstrong\u003ee,\u003c/strong\u003e Flowchart depicting the process of force decoupling in a single image frame using machine learning. Red and yellow lines represent normal and shear force, respectively. (Scale bar, 2 mm). \u003cstrong\u003ef,\u003c/strong\u003e FEA calculation of the local stress distribution at the undulating interface (\u003cem\u003emagnified dotted rectangle on bottom left\u003c/em\u003e) and contact width variations on the micro-hemisphere array (\u003cem\u003emagnified dashed rectangle on bottom right\u003c/em\u003e) under applied normal (F\u003csub\u003en\u003c/sub\u003e) and shear (F\u003csub\u003es\u003c/sub\u003e) forces in a 2D model sensor. \u003cstrong\u003eg,\u003c/strong\u003e Comparison of the experimental intensity profile (black) and calculated contact width (red), and calculated skewness (blue). Experimental data was collected from a sensor with an undulating interface subjected to a shear force of 0.49 N. \u003cstrong\u003eh,\u003c/strong\u003e Skewness values for sensor pads with and without undulating interface under applied normal (F\u003csub\u003en \u003c/sub\u003e= 1.0 N) and shear (F\u003csub\u003es \u003c/sub\u003e= 0.49 N for high F\u003csub\u003es\u003c/sub\u003e and 0.22\u003csub\u003e \u003c/sub\u003eN for low F\u003csub\u003es\u003c/sub\u003e) forces. Skewness increased with F\u003csub\u003es\u003c/sub\u003e and is higher in sensor pads with the undulating interface. \u003cstrong\u003ei,\u003c/strong\u003e Color map at the undulating interface of difference in First Piola-Kirchhoff stress distribution under applied shear force. Dash line described the center of applied force.\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/320480d8b6c8525cde4b2821.png"},{"id":56973834,"identity":"0cf7d747-8cb3-4166-9c51-da8b20dc1eba","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":1765115,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eDemonstration of high spatio-temporal optical tactile recognition (thickness: 200 μm). a,\u003c/strong\u003e Schematic of an optical tactile sensor recognizing the shape of a stamped object. \u003cstrong\u003eb,\u003c/strong\u003eLuminescence images obtained when the sensor pad is pressed with triangle, square, pentagon, and hexagon stamps (\u003cem\u003eclockwise\u003c/em\u003e). (Scale bar, 2 mm) \u003cstrong\u003ec,\u003c/strong\u003e Tactile recognition of alphabets (Scale bar, 2 mm). \u003cstrong\u003ed,\u003c/strong\u003e Optical (\u003cem\u003eleft\u003c/em\u003e) and luminescence (\u003cem\u003eright\u003c/em\u003e) images of fingerprints for high spatio-temporal tactile recognition. (Scale bar, 2 mm) \u003cstrong\u003ee,\u003c/strong\u003e Luminescence (\u003cem\u003etop\u003c/em\u003e) and 3D colormap (\u003cem\u003ebottom\u003c/em\u003e) recognition of multiple screw threads. (Scale bar, 1 mm) \u003cstrong\u003ef, \u003c/strong\u003eOptical (\u003cem\u003etop\u003c/em\u003e), luminescence and binary images (\u003cem\u003emiddle\u003c/em\u003e), and the corresponding intensity profile (\u003cem\u003ebottom\u003c/em\u003e) showing the threads of different types of screws. (Scale bar, 1 mm) \u003cstrong\u003eg, \u003c/strong\u003ePhotograph of a golf ball (\u003cem\u003eleft\u003c/em\u003e) and the dimples (\u003cem\u003eright\u003c/em\u003e) on its surface. (Scale bar, 5 mm) \u003cstrong\u003eh,\u003c/strong\u003e Luminescence images of two types of golf balls, characterized by differences in dimple size, shape, and signs of use. (Scale bar, 2 mm)\u003c/p\u003e","description":"","filename":"image4.png","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/2b1f12633b022266e46fe119.png"},{"id":56974521,"identity":"26429219-953b-46c3-893e-73686fd07f42","added_by":"auto","created_at":"2024-05-23 01:53:26","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":1066422,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eBehavioral biometric mining of dynamic forces from luminescence images.\u003c/strong\u003e \u003cstrong\u003ea,\u003c/strong\u003e Compilation of luminescence images obtained from stamping microscopic braille alphabets on the tactile sensor (\u003cem\u003etop\u003c/em\u003e). Extraction of braille dot features from contour maps (\u003cem\u003ebottom\u003c/em\u003e). \u003cstrong\u003eb, \u003c/strong\u003eSchematic diagram of a machine learning workflow for real-time BTS. \u003cstrong\u003ec, \u003c/strong\u003eBTS test using the “HELLO WORLD” sentence. (Scale bar, 2 mm) \u003cstrong\u003ed, \u003c/strong\u003eSchematic diagram of the workflow for discriminating handwriting using a machine learning-based analysis of shear and normal forces extracted from luminescence profiles. Normal force features (red panels) extracted for analysis include maximum intensity from the intensity profile and diameter of the clustering region with maximum intensity (blue circle). Extracted shear force features (green panels) including the length of the major/minor axis of the oval, the ratio of the major to minor axis of the oval, the angle of the major axis, and the statistically estimated shear force. (Scale bar, 2 mm) \u003cstrong\u003ee, \u003c/strong\u003eLDA clustering using different combinations of extracted features distinguish the handwritten letter ‘e’ of three individuals with different clarity. (Scale bar, 2 mm) \u003cstrong\u003ef,\u003c/strong\u003e Mean of intra-cluster distance (gray) and mean of inter-cluster distance (orange) for LDA clustering using different combinations of extracted features. \u003cstrong\u003eg,\u003c/strong\u003e Comparison of LDA clustering performances in terms of Calinski-Harabasz index and Silhouette coefficient for each case.\u003c/p\u003e","description":"","filename":"image5.png","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/484c58906f4e380bbe5f340a.png"},{"id":64435718,"identity":"6ee1e67a-9cc2-46c4-b95f-858a6c2b20c0","added_by":"auto","created_at":"2024-09-13 07:06:22","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":7308280,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/9e4e978b-936f-4cbe-bd81-83334d5fcb55.pdf"},{"id":56974519,"identity":"bb6e0051-1a03-487f-aeea-5819014c020d","added_by":"auto","created_at":"2024-05-23 01:53:26","extension":"mp4","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":15888520,"visible":true,"origin":"","legend":"Axisymmetric circular luminescence change when applied the normal force","description":"","filename":"SupplementaryMovie1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/183053f94758cf2ce8be03e4.mp4"},{"id":56973831,"identity":"2a0970c0-7dec-4dab-be5b-14cf42ddac20","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":12133510,"visible":true,"origin":"","legend":"Non-axisymmetric oval luminescence change when applied the shear force","description":"","filename":"SupplementaryMovie2.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/2ca277d3b4a92c3b7460ca8d.mp4"},{"id":56973844,"identity":"121a5328-6112-4bb6-85c6-e9ac6c887dc0","added_by":"auto","created_at":"2024-05-23 01:45:27","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":3468916,"visible":true,"origin":"","legend":"Machine learning assisted quantitative analysis of the normal force","description":"","filename":"SupplementaryMovie3.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/8bea2e257907095bb793b453.mp4"},{"id":56973833,"identity":"da7bd45e-8e64-4ea9-968d-4482b0a9ee71","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"mp4","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":4600334,"visible":true,"origin":"","legend":"Decoupling of the applied dynamic force to the normal and shear force by spherical indenter using machine learning algorithm","description":"","filename":"SupplementaryMovie4.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/ff9c7ae191ccf24a376192d0.mp4"},{"id":56973843,"identity":"28e5161d-8e94-40e9-8325-345a5ec11fa9","added_by":"auto","created_at":"2024-05-23 01:45:27","extension":"mp4","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":3657056,"visible":true,"origin":"","legend":"Classification of multiple types of screws","description":"","filename":"SupplementaryMovie5.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/2ac0676347295ebfc6e30bb1.mp4"},{"id":56973842,"identity":"1f089e41-0fe0-4cbf-aa66-299a26df4390","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"mp4","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":19969880,"visible":true,"origin":"","legend":"Machine learning assisted braille-to-speech system","description":"","filename":"SupplementaryMovie6.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/049cb074f6ba51e19f1d0e92.mp4"},{"id":56973840,"identity":"5f78e84c-2bd9-4ef1-b3ad-c89d2ea18722","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"mp4","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":3077584,"visible":true,"origin":"","legend":"Decoupling of the applied dynamic force to the normal and shear force by pen using machine learning algorithm","description":"","filename":"SupplementaryMovie7.mp4","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/e185e2c6a140cf059bcdb688.mp4"},{"id":56973841,"identity":"68b939d3-c842-4d13-adad-74d9830673de","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"pdf","order_by":8,"title":"","display":"","copyAsset":false,"role":"supplement","size":3207342,"visible":true,"origin":"","legend":"","description":"","filename":"NCOMMS2342760Ars.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/bb7bf1e67b3509afd4fbe6a4.pdf"},{"id":56973835,"identity":"b3509cdd-aa3a-4380-a7e8-b99ab0c89406","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"docx","order_by":9,"title":"","display":"","copyAsset":false,"role":"supplement","size":37997139,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformation.docx","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/fe21c352a6b7d55797aeeb2e.docx"},{"id":56973829,"identity":"ff60b411-acce-48a3-9334-54549d5a829f","added_by":"auto","created_at":"2024-05-23 01:45:25","extension":"pdf","order_by":10,"title":"","display":"","copyAsset":false,"role":"supplement","size":1682226,"visible":true,"origin":"","legend":"\u003cp\u003eEditorial Policy Checklist\u003c/p\u003e","description":"","filename":"nreditorialpolicychecklist.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/77dafae86aac87b1a75ad812.pdf"},{"id":56973837,"identity":"bb993e8c-13fe-4d9c-9f38-5801639b9d95","added_by":"auto","created_at":"2024-05-23 01:45:26","extension":"pdf","order_by":11,"title":"","display":"","copyAsset":false,"role":"supplement","size":1677158,"visible":true,"origin":"","legend":"\u003cp\u003eReporting Summary\u003c/p\u003e","description":"","filename":"nrreportingsummary.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4427929/v1/115771e289badd376d49f7a7.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time","fulltext":[{"header":"Full Text","content":"\u003cp\u003eBehavioral biometric tactile systems are emerging technologies that identify people by analyzing the way they interact with their devices. For example, by measuring how fast the person writes or moves the pen and how much pressure they exert when interacting with the device, behavioral biometric tactile algorithms can authenticate the person signing a document\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Because personal habits are unique, behavioral biometric data is difficult to replicate, making these technologies better at preventing theft and fraud. Consequently, there is significant interest in using dynamic pressure sensing technology to identify and measure different types of forces that can be used for behavioral recognition\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eOptical tactile sensors are an attractive low-cost solution for interactive devices and tactile displays. While typical optical tactile sensors can measure normal force, information on in-plane motion such as velocity can only be obtained by positional analysis of two or more sequential images\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. However, the resulting \u0026ldquo;time-averaged\u0026rdquo; velocity may not represent the true motion, especially when the directional change of motion is rapid with respect to the available rate of measurement, e.g., frames-per-second (fps) of a camera (Supplementary Fig.\u0026nbsp;1). For more accurate information on in-plane motion, the optical tactile sensor should ideally also measure \u0026ldquo;instantaneous\u0026rdquo; velocity of the in-plane motion. Because instantaneous slip velocity is highly correlated to the shear force applied to the solid-solid interface in contact mechanics, having an optical tactile sensor capable of instantaneously measuring both normal and shear force would be desirable for studying behavioral biometrics. Optical polymers\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e, photonic crystals\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e, optical light guide\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e, quantum dots\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e and organic dyes\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e29\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e have been used to build optical tactile sensors. However, these systems cannot decompose dynamic elements of forces because they have slow signal response, high limit of detection, and low spatio-temporal resolution. To analyze fast movements such as handwriting in behavioral biometrics, we need optical tactile sensors that can quantitatively decompose applied force into vertical normal and lateral shear forces from a single image.\u003c/p\u003e\n\u003cp\u003eHere, we report a highly sensitive optical tactile sensor system that can instantaneously decouple dynamic touch signals in a single image for behavioral biometric analysis. Our tactile sensor contains a stress concentration layer that resembles the sensory architecture of the human skin and an optical tactile layer that contains an array of upconversion nanocrystals (UCNs). When force is applied on the sensor, the stress concentration layer amplifies and transfers the dynamic force to the tactile layer that interfaces with a total internal reflection (TIR) dove prism. Our UCNs-based optical tactile sensor generates unique axisymmetric or non-axisymmetric upconversion luminescence signals with spatio-temporal patterns that contain information on the motion of the object such as direction, velocity, and magnitude of normal and shear force. Unlike conventional optical tactile sensors, our system can quantitatively decompose the applied force into individual components of vertical normal and lateral shear force from a single image in real-time. We used our sensor system for surface profiling of objects, high-resolution fingerprint recognition and anti-counterfeiting applications. Using a machine learning framework to analyze the luminescence tactile signals, we applied the sensor system in a small-scale Braille-to-Speech (BTS) translation system. Most importantly, our sensor accurately recognized behavioral biometric handwriting patterns. We expect such a simple sensor design to find ample applications in sophisticated behavioral biometric technologies.\u003c/p\u003e\n\u003cp\u003eOur tactile sensor system includes a tactile sensor pad, a TIR dove prism and a charge-coupled device (CCD) camera (Fig.\u0026nbsp;1a, and Supplementary Fig.\u0026nbsp;2). The tactile sensor pad is composed of a stress concentration layer at the top, a thin (20 nm) anti-reflective platinum (Pt) layer in between, and a signal generating tactile layer at the bottom (Fig.\u0026nbsp;1b). The stress concentration layer is designed to mimic the undulating interface of hard epidermis and soft dermis in human skin because such an interface is known to amplify applied forces by increasing interfacial area and localizing the applied force.\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e Further, mechanical mismatch between the rigid epidermis and soft dermis enhances force transmission from the skin to sensory mechanoreceptors\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e\u003c/sup\u003e. To recreate this microarchitecture, we used stiff thermoplastic polyurethane (TPU, 52.5 MPa) and soft polydimethylsiloxane (PDMS, 1.76 MPa) with an undulated two-dimensional (2D) array of micro-hemispheres patterned at the interface (Fig.\u0026nbsp;1c, and Supplementary Fig.\u0026nbsp;3). The Pt layer in between serves to suppress diffuse reflection and improve the accuracy of the images (Supplementary Fig.\u0026nbsp;4). The bottom tactile layer contains an array of PDMS micro-hemispheres embedded with UCNs (Supplementary Fig.\u0026nbsp;5 and Supplementary Note 1). For our tactile sensor, we synthesized a hexagonal phase yellow luminescent lanthanide-doped UCNs (\u0026beta;-NaYF\u003csub\u003e4\u003c/sub\u003e:Yb\u003csup\u003e3+\u003c/sup\u003e/Er\u003csup\u003e3+\u003c/sup\u003e/Gd\u003csup\u003e3+\u003c/sup\u003e; 30/2/30 mol%) using a conventional hydrothermal method (Fig.\u0026nbsp;1d, and 1e, and Supplementary Methods for details). When homogenously embedded into the PDMS micro-hemispheres (Fig.\u0026nbsp;1f and Supplementary Fig.\u0026nbsp;6), their luminescence remained unchanged (Fig.\u0026nbsp;1g, and Methods for details). UCNs are advantageous for our system because they have a high spectral resolution (Full Width at Half Maximum (FWHM) of ~\u0026thinsp;20 nm), low optical signal errors and large anti-Stoke shifts. Further, UCNs do not require optical filters for luminescence color purification, have negligible background auto-fluorescence\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e\u003c/sup\u003e and are optically stable upon environmental perturbations\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e. Moreover, their luminescence colors can be easily tuned\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e by simply adjusting the stoichiometry of lanthanide dopant ions (Supplementary Figs.\u0026nbsp;7, and 8, and Supplementary Methods for details). UCNs are also amenable to large-scale synthesis\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eIn our system, signal generation depends on the contact between the UCNs-embedded micro-hemispheres (UMs) in the tactile layer and the TIR prism. Irradiating the prism with 980 nm near-infrared (NIR) laser generates evanescent waves in the thin (\u0026lt;\u0026thinsp;100 nm) boundary layer close to the interface of the prism and UMs layer. When an object moves across our sensor pad, the stress concentration layer amplifies and transfers the applied dynamic forces to the UMs layer. As the contact area between the UM layer and TIR prism increases, the UMs produce a luminescence signal measurable by the CCD camera and luminescence spectrometer (Fig.\u0026nbsp;1h). This phenomenon can be ascribed to the transformation of the evanescence wave field into propagating far-fields at contact points between the UM and prism, where TIR is frustrated due to the decrease in the refractive index contrast according to the higher refractive index of PDMS (n\u0026thinsp;=\u0026thinsp;1.4) compared to air (n\u0026thinsp;=\u0026thinsp;1.0) (Supplementary Figs.\u0026nbsp;9, and 10 and Supplementary Note 2). At the contact points, the propagating far-field components can now penetrate the UM, effectively illuminating the UCNs within the UM volume. Consequently, locally compressed UMs with a more extensive contact area with the prism generate a stronger luminescence signal, while those without applied force do not yield measurable luminescence. It is noteworthy that the non-linear luminescence property of UCNs suppresses the initial luminescence caused by the evanescence wave field from the minimal contact between the UM layer and the TIR prism in the absence of applied force, consequently reducing background noise\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e. Locally pressed UMs with higher contact area with the prism produce higher luminescence signals while those without applied force do not contact the prism and therefore, do not produce luminescence. When we applied a normal force to the top layer of the tactile sensor by vertically pressing a spherical indenter (5 mm diameter), the resulting luminescence signal was axisymmetric (Fig.\u0026nbsp;1i). The tactile sensor we fabricated exhibited uniform signal generation at across the entire sensor pad or 10 different sensor pads, and demonstrated highly stable signals under continuous laser excitation (Supplementary Figs.\u0026nbsp;11\u0026ndash;13). However, when we simultaneously applied a lateral shear force in addition to the normal force by moving the pressed indenter along the surface of the top layer, the luminescence signal became non-axisymmetric \u0026ndash; the shape of the signal is skewed towards the direction of movement (Fig.\u0026nbsp;1j). This suggests that detailed information on the surface-parallel motions of the indenter such as the magnitude of normal force, and the velocity, direction and magnitude of lateral shear force can be extracted by analyzing the non-axisymmetric shape of the luminescence signal. These features induced subtle changes in luminescence intensity, and to effectively observe this aspect, we proposed a tactile sensor based on UCNs instead of the conventional mechanoluminescence-based tactile sensors. Optical tactile sensor using UCNs have better sensitivity and response time compared with mechanoluminescence tactile sensors\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e. By comparing the luminescence shape from optical tactile sensor, we proposed a method to determine both the direction and magnitude of normal and shear forces in real-time using a machine learning algorithm for force discrimination (Fig.\u0026nbsp;1k). By analyzing the luminescence tendency such as axisymmetric and non-axisymmetric for static and dynamic optical tactile sensors, respectively, we proposed machine learning methodologies to discriminate normal and shear forces in a single image frame in real-time.\u003c/p\u003e\n\u003cp\u003eTo understand how normal force affects the luminescence signals, we analyzed the signal intensity at different applied normal forces. When the magnitude of the normal force increased from 0 to 5.0 N, both luminescence intensity and the pressure distribution area increased in an axisymmetric circle (Figs.\u0026nbsp;2a and b, Supplementary Figs.\u0026nbsp;14, and 15, Supplementary Movie 1, and Methods). We observed that the stress concentration layer exhibited a strong and wide luminescence area with enhanced intensity due to its modulus difference and undulating interface (Supplementary Fig.\u0026nbsp;16). To confirm the correlation between grayscale and pressure, we first observed the area beneath the prism (Supplementary Fig.\u0026nbsp;17a). By combining the grayscale of Fig.\u0026nbsp;2a corresponding to the observed area, we could exhibit the correlation between pressure and grayscale (Supplementary Fig.\u0026nbsp;18). We obtained an optimized tactile sensor (1000 \u0026micro;m thick) including the stress concentration layer (200 \u0026micro;m thick) that can detect normal force down to 0.05 N, and later utilized for the dynamic force decoupling process (Fig.\u0026nbsp;2b, Supplementary Note 3 for optimization, and Supplementary Figs.\u0026nbsp;19\u0026ndash;21). The characteristic luminescence intensity of yellow UCNs at 530 and 660 nm also increased with increasing normal force (Fig.\u0026nbsp;2c). The relationship between luminescence intensity and the cumulative gray value for each pixel exhibits consistent slopes concerning force magnitudes, providing the possibility of presenting a pressure distribution in a color map (Fig.\u0026nbsp;2d and Supplementary Fig.\u0026nbsp;22). Under normal force, our sensor did not show hysteresis in the luminescence intensity, indicating that the adhesion between the PDMS micro-hemispherical array and the TIR prism is negligible (Fig.\u0026nbsp;2e). Further, our tactile sensor is highly stable; its luminescence intensity remained constant when a normal force of 5.0 N was repeatedly applied over 1,000 cycles (Fig.\u0026nbsp;2f), and even at the high temperature (Supplementary Fig.\u0026nbsp;23). Furthermore, we measured the response time by repeatedly tapping the sensor (Figs.\u0026nbsp;2g, and Supplementary Fig.\u0026nbsp;24). We found the average response time using Gaussian fitting and it was 9.12 ms based on multiple repeated measurements.\u003c/p\u003e\n\u003cp\u003eWe further examined the temporal change in luminescence signal under applied lateral shear force (Methods in details). We began by pressing a spherical indenter to apply a 1.0 N vertical preload on the tactile sensor pad (Supplementary Note 4). The indented area forms a circular axisymmetric luminescence intensity and pressure contour. Following this, we accelerated the pressed indenter along the top surface of the sensor pad at a constant rate of 1.0 mm/s\u003csup\u003e2\u003c/sup\u003e until the velocity reached 1.0 mm/s. As soon as the spherical indenter began gliding, the axisymmetric luminescence circle appeared to stretch into a non-axisymmetric oval along the direction of the lateral shear force (Fig.\u0026nbsp;3a, and Supplementary Fig.\u0026nbsp;16). This shape change was recorded as a video clip at 10 frames per second (Supplementary Movie 2 middle), and the luminescence image frames were extracted from the video for analysis (Fig.\u0026nbsp;3a left). Both the pressure contour color map (Fig.\u0026nbsp;3a middle) and the luminescence intensity plot (Fig.\u0026nbsp;3a right and detailed in Supplementary Fig.\u0026nbsp;25) also clearly showed that the luminescence intensity profile had changed from a circle to an oval while the center maintained an axisymmetric circular luminescence (Fig.\u0026nbsp;3a middle). The non-axisymmetric luminescence continued to form until the indenter reached a specific velocity (Supplementary Figs.\u0026nbsp;26, and 27, and Supplementary Movie 2). Using a customized force gauge, we measured the corresponding shear force and recorded the values on each non-axisymmetric luminescence image (Fig.\u0026nbsp;3a, Supplementary Fig.\u0026nbsp;28). Unlike conventional tactile sensors where the direction of movement is obtained through coordination analysis of multiple images, we can immediately tell the direction of the applied shear force from the non-axisymmetric luminescence profile alone (Fig.\u0026nbsp;3b)\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eBecause shear force induces friction force in the opposite direction\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e45\u003c/span\u003e\u003c/sup\u003e, we evaluated the effect of friction on the luminescence signal (Supplementary Note 4). When shear force is applied to the sensor pad by moving the spherical indenter, the micro-hemispheres are subjected to an asymmetric lateral deformation caused by friction between the micro-hemisphere array layer and the TIR prism below the sensor pad. This results in the observed non-axisymmetric luminescence. When we reduced the friction force by spreading silicone oil between the sensor pad and the prism, the applied shear force no longer changed the shape of the luminescence signal; an axisymmetric signal was obtained even during acceleration (Supplementary Fig.\u0026nbsp;29). We further analyzed the friction force using a surface forces apparatus (SFA) (Supplementary Fig.\u0026nbsp;30, and Supplementary Methods in detail). The kinetic friction coefficient (obtained from the slope of friction versus load curve) increased from 0.148 to 0.531 as the driving velocity increased from 11.6 to 58.0 \u0026micro;m/s (Supplementary Fig.\u0026nbsp;31). These results indicate that increasing applied velocity increases frictional forces (or friction coefficient) and micro-hemisphere deformation, which affects the overall non-axisymmetric shape of the luminescence signal. It is worth noting that the sliding velocity-dependent friction coefficients are commonly observed, especially in complex systems with soft polymers\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e46\u003c/span\u003e\u003c/sup\u003e or polymer-like surfaces\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e47\u003c/span\u003e\u003c/sup\u003e, which contradicts conventional Amontons\u0026rsquo; Laws of Friction\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e48\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eTo further quantify this non-axisymmetric luminescence characteristic of shear force, the degree of non-axisymmetric luminescence was statistically described using the concept of skewness (\u0026gamma;), which is obtained by calculating the median and mean of the luminescence intensity profile (Supplementary Note 5). For a moving indenter, the calculated skewness matched well with the measured shear force (Fig.\u0026nbsp;3c), indicating that the magnitude of the shear force could be quantitatively estimated directly from the non-axisymmetric luminescence profile. When we analyzed the change in shear force at a given skewness for various normal forces, skewness decreased when the normal force increased from 0.5 to 1.5 N (Fig.\u0026nbsp;3d, and Supplementary Figs.\u0026nbsp;32\u0026ndash;34). This is because an increase in normal force decreases the relative contribution of shear force (and thus, skewness value) to the luminescence signal, which is formed from both the normal force that creates the axisymmetric profile, and shear force that creates the non-axisymmetric profile (Supplementary Fig.\u0026nbsp;35). Based on the single image frame analysis of luminescence intensity profile for static normal force and the directional skewness for dynamic shear force, we successfully quantified in real time the direction and magnitude of both normal and shear forces from a single image using force discrimination machine learning algorithm (Fig.\u0026nbsp;3e, Supplementary Fig.\u0026nbsp;36, Supplementary Tables\u0026nbsp;1\u0026ndash;3, Supplementary Movies 3, and 4, and Supplementary Notes 6, and 7). Our machine learning model was highly reliable with r\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e values of 0.965 and 0.947 for normal and shear forces, respectively. In detail, the r, g, and b values of the detected images were analyzed in histogram format to predict the applied normal force. By utilizing the predicted normal force value, the threshold for displaying the contour of the normal force region was also predicted. As a result, normal and shear forces were successfully decoupled based on the determined threshold value. Note that the trend of skewness change was consistent irrespective of the fps, indicating that the more cost-effective 10-fps camera is suitable for the normal and shear force decoupling (Supplementary Figs.\u0026nbsp;37, and 38).\u003c/p\u003e\n\u003cp\u003eWe further examined whether and how the undulated interface between the stiff TPU and soft PDMS in the stress concentration layer of the tactile sensor pad affects the non-axisymmetric luminescence (Supplementary Figs.\u0026nbsp;39, and 40). When an indenter moved at a velocity of 1.0 mm/s with 1.0 mm/s\u003csup\u003e2\u003c/sup\u003e acceleration across a pad without undulation at the TPU-PDMS interface, the maximum value of skewness was 0.1 (Supplementary Fig.\u0026nbsp;41). With undulation, the maximum value of skewness increased to 0.37. When the velocity of the indenter was decreased to velocity of 0.5 mm/s with 0.5 mm/s\u003csup\u003e2\u003c/sup\u003e, the skewness decreased further to 0.32. Our results indicate that the undulation at the interface and the differences in modulus between TPU and PDMS improved the transmission of the applied shear force to the signal-generating UMs layer, which, in turn, increased the non-axisymmetric of the luminescence signal.\u003c/p\u003e\n\u003cp\u003eTo further understand how the undulation at the TPU-PDMS interface enhances force transmission, we used finite element analysis (FEA) to examine the stress distribution of 2D model sensors with and without interface undulation under 1.0 N applied normal force (F\u003csub\u003en\u003c/sub\u003e) with and without shear force (F\u003csub\u003es\u003c/sub\u003e) ranging from 0 to 1.0 N (Fig.\u0026nbsp;3f, Supplementary Figs.\u0026nbsp;42\u0026ndash;50, and Supplementary Note 8). Because luminescence signal in our tactile sensor is produced only when UMs contact the TIR prism, we expected the intensity of the luminescence signal in the model sensor to scale with the contact width between the individual UMs and the underlying substrate. Consistent with the experimental intensity profile, our FEA analysis showed that contact width is symmetric when only F\u003csub\u003en\u003c/sub\u003e is applied but is skewed toward the direction of applied F\u003csub\u003es\u003c/sub\u003e (Fig.\u0026nbsp;3g and Supplementary Figs.\u0026nbsp;46, and 47). The resulting calculated skewness, \u0026gamma;, increased with the magnitude of F\u003csub\u003es\u003c/sub\u003e and was substantially higher in the model sensor with an undulating interface (Fig.\u0026nbsp;3h). These calculations corroborate the experimental results that showed higher F\u003csub\u003es\u003c/sub\u003e transmission in sensors with an undulating interface (Supplementary Figs.\u0026nbsp;39, and 40). We visualized this enhancement by plotting the differences in stress distribution for sensors (\u0026sigma;\u003csub\u003ew\u003c/sub\u003e) and without (\u0026sigma;\u003csub\u003ewo\u003c/sub\u003e) an undulating interface, \u0026Delta;\u0026sigma; (Fig.\u0026nbsp;3i). Roughly 1 mm anterior to the center of the applied force \u0026ndash; especially near the micro-hemispheres array \u0026ndash; \u0026Delta;\u0026sigma;\u003csub\u003eyx\u003c/sub\u003e increased strongly, indicating that F\u003csub\u003es\u003c/sub\u003e was more effectively transmitted to the micro-hemispherical array owing to the undulating interface. No such enhancement was observed for \u0026Delta;\u0026sigma;\u003csub\u003eyy\u003c/sub\u003e. We also found that \u0026sigma;\u003csub\u003eyy\u003c/sub\u003e remained symmetric even under F\u003csub\u003es\u003c/sub\u003e, whereas \u0026sigma;\u003csub\u003exy\u003c/sub\u003e became more asymmetric with increasing F\u003csub\u003es\u003c/sub\u003e (Supplementary Fig.\u0026nbsp;48). A more careful examination of \u0026sigma;\u003csub\u003exy\u003c/sub\u003e near the TPU-PDMS interface indicates that the regions beneath the interface have on average 50% higher value of \u0026sigma;\u003csub\u003exy\u003c/sub\u003e for the undulated interface (Supplementary Fig.\u0026nbsp;49). This phenomenon was observed to be independent of the indenter shape (Supplementary Fig.\u0026nbsp;50). Furthermore, similar non-axisymmetric luminescence patterns were identified for indenters with different shape (spherical, elliptical, rectangular, and triangular), spherical with different curvature, and cut cone with different radius geometries (Supplementary Figs.\u0026nbsp;51\u0026ndash;56). Consequently, this behavior can be observed regardless of the indenter shape. Although the stress concentration layer may hinder the precise determination of the tip shape, we employed deep learning techniques to facilitate shape identification (Supplementary Fig.\u0026nbsp;57 and detailed in Supplementary Note 9). Additionally, asymmetric fluorescence emission and the resulting change in skewness when shear force is applied have been confirmed using various tools within the laboratory (Supplementary Fig.\u0026nbsp;58).\u003c/p\u003e\n\u003cp\u003eWhile the stress concentration layer exhibited an excellent characteristic of shear force, its resolution decreased at low normal forces (Supplementary Figs.\u0026nbsp;17, and 59). To accommodate various applications, we fabricated 200 \u0026micro;m thickness sensor pads without the stress concentration layer for precise vertical normal force measurements and surface inspection. Along with the high-resolution capability of detecting low pressures of 0.02 N (Supplementary Figs.\u0026nbsp;60\u0026ndash;62), which an order of magnitude better than current optical tactile sensors that detect forces down to 0.2 N (Supplementary Table\u0026nbsp;4)\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e our optical sensor enabled high spatio-temporal tactile recognition of 2D surface textures including multiple objects (Fig.\u0026nbsp;4a-c, and Supplementary Fig.\u0026nbsp;63) and fingerprints of different personnel (Fig.\u0026nbsp;4d, and Supplementary Fig.\u0026nbsp;64). The maximum spatial resolution of our tactile sensor is 100 \u0026micro;m, which is on the upper bounds of the spatial resolution found in previously studied optical tactile sensors (Supplementary Fig.\u0026nbsp;65, and Supplementary Table\u0026nbsp;4) \u003csup\u003e\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e. Our optical tactile sensor can also recognize and distinguish different surface textures. For example, small screws having different threads too fine for the human eye to see were identified in a high contrast manner using MATLAB-based automated image analysis code (Figs.\u0026nbsp;4e, and f, Supplementary Fig.\u0026nbsp;66, Supplementary Movie 5, and Supplementary Note 10)\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e50\u003c/span\u003e\u003c/sup\u003e. The size and shape of dimples (Fig.\u0026nbsp;4g) and signs of use (Fig.\u0026nbsp;4h) of different golf balls were also visualized with high spatial-temporal resolution.\u003c/p\u003e\n\u003cp\u003eWe built a small-scale BTS transformation system that can instantly translate microscopic braille patterns into the corresponding sounds using a machine learning algorithm (Supplementary Note 11). Topographic patterns of micro-sized braille for \u0026ldquo;Hello World\u0026rdquo; were converted to luminescence images (Fig.\u0026nbsp;5a), and instantly translated into the corresponding sounds by extracting 2D patterns of dots (Figs.\u0026nbsp;5b, and c, Supplementary Figs.\u0026nbsp;67, and 68, and Supplementary Movie 6). The diameter of individual braille dots in this experiment was two times smaller (500 \u0026micro;m) than standard dots (1 mm), increasing the density of storable information by an order of magnitude (~\u0026thinsp;3\u003csup\u003e2\u003c/sup\u003e).\u003c/p\u003e\n\u003cp\u003eMost notably, using the ability to simultaneously visualize the direction of movement and the intensity change of both normal and shear force, we developed a machine learning (ML) framework that distinguishes dynamic biometric handwriting of individuals (Fig.\u0026nbsp;5d, Supplementary Movie 7, and detailed in Supplementary Notes 12\u0026ndash;14). Decoupled images represented normal and shear forces, and related features extracted from videos of three different individuals writing the letter \u0026lsquo;e\u0026rsquo; were fed into our ML-based linear discriminant analysis (LDA) for writing style classification. LDA clustering successfully discriminated the three different handwritten letters \u0026lsquo;e\u0026rsquo; (Fig.\u0026nbsp;5e, Supplementary Figs.\u0026nbsp;69\u0026ndash;73, Supplementary Tables\u0026nbsp;6\u0026ndash;9, and Supplementary Note 15). We further extracted the averaged velocity-related features from multiple image frames and combined them with normal and shear force features for handwriting analysis. Our single-frame touch signal decoupling-based ML framework showed outstanding performance when compared to analyzing the handwriting using normal forces alone or normal forces with velocity-related features (Fig.\u0026nbsp;5f, Supplementary Figs.\u0026nbsp;74, and 75, Supplementary Table\u0026nbsp;8). While the average distance within a specific cluster was the same for all analysis, the average distance between different clusters was greater when the analysis included shear force features than when it included velocity features. Comparing the data cluster analyzed using both velocity and shear force features with those analyzed using velocity only and shear force only, we find that the analysis using combined features had a greater impact on the data clusters of velocity only analysis than on shear force only analysis. Further, between velocity only analysis and shear force only analysis, we find that shear force feature-based handwriting analysis forms a more clear-cut data cluster (Supplementary Fig.\u0026nbsp;72).\u003c/p\u003e\n\u003cp\u003eTo compare clustering performance, we calculated the Silhouette coefficient, which describes the average distance between points within the same cluster and the average distance between points in the nearest cluster to which the data point does not belong to, and the Calinski-Harabasz index, which is the ratio of dispersion within a cluster to dispersion between clusters. Of all the analysis combinations, the normal/shear force combination showed a Silhouette coefficient closest to 1 (0.867) and a remarkable value Calinski-Harabasz index (5.76\u0026times;10\u003csup\u003e2\u003c/sup\u003e) (Fig.\u0026nbsp;5g, Supplementary Fig.\u0026nbsp;73, and Supplementary Table\u0026nbsp;9). Adding velocity feature to the normal/shear force feature slightly decreased the Silhouette coefficients (0.858) and Calinski-Harabasz index (4.52\u0026times;10\u003csup\u003e2\u003c/sup\u003e) and there is no significant advantage in terms of computational cost. From this point of view, we believe that besides normal force, shear force feature is a dominant factor for distinguishing writing style. These results demonstrate that normal/shear force-based handwriting analysis represents the most clear-cut cluster and is well classified with clusters of different force combinations. In conventional handwriting authentication, verification is predominantly grounded in the consistency of static handwriting features such as letter size, inclination, and inter-character spacing, rather than dynamic attributes. Remarkably, our study introduces an innovative methodology for handwriting verification, focused on shear force\u0026mdash;a parameter not previously explored in this context. This approach encapsulates a wealth of data, encompassing factors such as the exerted pressure during writing and the speed of inscription. These parameters raise the expectation that our framework can offer valuable insights into the writer\u0026rsquo;s identity, as they exhibit variations correlated with the writer\u0026rsquo;s age and gender\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e. Furthermore, our framework possesses the unique capability to decouple normal and shear forces within a single frame, allowing it to provide real-time feedback unlike previous studies that evaluated handwriting only after all letters were completed. Thus, our system holds the potential to significantly enhance the precision of handwriting authentication process.\u003c/p\u003e\n\u003cp\u003eIn conclusion, we present an upconversion luminescence-based behavioral biometric optical tactile sensor that simultaneously and quantitatively discriminates velocity, direction, normal and shear force in a single frame analysis. Such a unique axisymmetric and non-axisymmetric luminescence signal amplified through the human skin mimetic stress concentration microarchitecture was theoretically studied by FEA simulation, experimentally examined by SFA measurement, and quantitatively analyzed. Our optical sensor is simple to build, and we show they can be used for tactile recognition of surface textures and fingerprints. Using machine learning, we applied the sensor system in a small-scale BTS system. Most importantly, our system has enabled accurate dynamic biometric handwriting analysis. We believe our sensor system will open new avenues in facile optical tactile sensors that can intuitively and sensitively display multiple elements of dynamic forces in a single image frame, forming the basis of flexible behavioral biometric optical tactile sensors in wearable devices.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eReprint and permissions information\u003c/strong\u003e is available online at www.nature.com/reprints.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting financial interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing financial interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAlzubaidi, A. \u0026amp; Kalita, J. Authentication of smartphone users using behavioral biometrics. \u003cem\u003eIEEE Communications Surveys \u0026amp; Tutorials\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, 1998-2026 (2016). \u003c/li\u003e\n\u003cli\u003eFrank, M., Biedert, R., Ma, E., Martinovic, I. \u0026amp; Song, D. Touchalytics: On the applicability of touchscreen input as a behavioral biometric for continuous authentication. \u003cem\u003eIEEE transactions on information forensics and security\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 136-148 (2012). \u003c/li\u003e\n\u003cli\u003eHa, M.\u003cem\u003e et al.\u003c/em\u003e Bioinspired interlocked and hierarchical design of ZnO nanowire arrays for static and dynamic pressure‐sensitive electronic skins. \u003cem\u003eAdvanced Functional Materials\u003c/em\u003e \u003cstrong\u003e25\u003c/strong\u003e, 2841-2849 (2015). \u003c/li\u003e\n\u003cli\u003ePark, J., Kim, M., Lee, Y., Lee, H. S. \u0026amp; Ko, H. Fingertip skin\u0026ndash;inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. \u003cem\u003eScience advances\u003c/em\u003e \u003cstrong\u003e1\u003c/strong\u003e, e1500661 (2015). \u003c/li\u003e\n\u003cli\u003eLee, H.-K., Chung, J., Chang, S.-I. \u0026amp; Yoon, E. Normal and shear force measurement using a flexible polymer tactile sensor with embedded multiple capacitors. \u003cem\u003eJournal of Microelectromechanical Systems\u003c/em\u003e \u003cstrong\u003e17\u003c/strong\u003e, 934-942 (2008). \u003c/li\u003e\n\u003cli\u003eLi, R., Nie, B., Digiglio, P. \u0026amp; Pan, T. Microflotronics: A Flexible, Transparent, Pressure‐Sensitive Microfluidic Film. \u003cem\u003eAdvanced Functional Materials\u003c/em\u003e \u003cstrong\u003e24\u003c/strong\u003e, 6195-6203 (2014). \u003c/li\u003e\n\u003cli\u003eRamuz, M., Tee, B. C. K., Tok, J. B. H. \u0026amp; Bao, Z. Transparent, optical, pressure‐sensitive artificial skin for large‐area stretchable electronics. \u003cem\u003eAdvanced Materials\u003c/em\u003e \u003cstrong\u003e24\u003c/strong\u003e, 3223-3227 (2012). \u003c/li\u003e\n\u003cli\u003eStassi, S., Cauda, V., Canavese, G. \u0026amp; Pirri, C. F. Flexible tactile sensing based on piezoresistive composites: A review. \u003cem\u003eSensors\u003c/em\u003e \u003cstrong\u003e14\u003c/strong\u003e, 5296-5332 (2014). \u003c/li\u003e\n\u003cli\u003eViry, L.\u003cem\u003e et al.\u003c/em\u003e Flexible three‐axial force sensor for soft and highly sensitive artificial touch. \u003cem\u003eAdvanced materials\u003c/em\u003e \u003cstrong\u003e26\u003c/strong\u003e, 2659-2664 (2014). \u003c/li\u003e\n\u003cli\u003eWang, L. \u0026amp; Beebe, D. J. A silicon-based shear force sensor: development and characterization. \u003cem\u003eSensors and Actuators A: Physical\u003c/em\u003e \u003cstrong\u003e84\u003c/strong\u003e, 33-44 (2000). \u003c/li\u003e\n\u003cli\u003eWang, X.\u003cem\u003e et al.\u003c/em\u003e Self‐powered high‐resolution and pressure‐sensitive triboelectric sensor matrix for real‐time tactile mapping. \u003cem\u003eAdvanced materials\u003c/em\u003e \u003cstrong\u003e28\u003c/strong\u003e, 2896-2903 (2016). \u003c/li\u003e\n\u003cli\u003eYoon, S. G., Park, B. J. \u0026amp; Chang, S. T. Highly sensitive piezocapacitive sensor for detecting static and dynamic pressure using ion-gel thin films and conductive elastomeric composites. \u003cem\u003eACS applied materials \u0026amp; interfaces\u003c/em\u003e \u003cstrong\u003e9\u003c/strong\u003e, 36206-36219 (2017). \u003c/li\u003e\n\u003cli\u003eZheng, Q.\u003cem\u003e et al.\u003c/em\u003e Dynamic real-time imaging of living cell traction force by piezo-phototronic light nano-antenna array. \u003cem\u003eScience Advances\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, eabe7738 (2021). https://doi.org/doi:10.1126/sciadv.abe7738\u003c/li\u003e\n\u003cli\u003eJia, M.\u003cem\u003e et al.\u003c/em\u003e Tactile tribotronic reconfigurable pn junctions for artificial synapses. \u003cem\u003eScience Bulletin\u003c/em\u003e \u003cstrong\u003e67\u003c/strong\u003e, 803-812 (2022). \u003c/li\u003e\n\u003cli\u003eWei, X. Y.\u003cem\u003e et al.\u003c/em\u003e Dynamic triboelectrification‐induced electroluminescence and its use in visualized sensing. \u003cem\u003eAdvanced Materials\u003c/em\u003e \u003cstrong\u003e28\u003c/strong\u003e, 6656-6664 (2016). \u003c/li\u003e\n\u003cli\u003eLee, B.\u003cem\u003e et al.\u003c/em\u003e Ultraflexible and transparent electroluminescent skin for real-time and super-resolution imaging of pressure distribution. \u003cem\u003eNature Communications\u003c/em\u003e \u003cstrong\u003e11\u003c/strong\u003e, 663 (2020). \u003c/li\u003e\n\u003cli\u003eZhao, X.\u003cem\u003e et al.\u003c/em\u003e Self-powered user-interactive electronic skin for programmable touch operation platform. \u003cem\u003eScience advances\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, eaba4294 (2020). \u003c/li\u003e\n\u003cli\u003eDu, Y.\u003cem\u003e et al.\u003c/em\u003e Mechanically excited multicolor luminescence in lanthanide ions. \u003cem\u003eAdvanced Materials\u003c/em\u003e \u003cstrong\u003e31\u003c/strong\u003e, 1807062 (2019). \u003c/li\u003e\n\u003cli\u003eChen, C.\u003cem\u003e et al.\u003c/em\u003e Creating visible-to-near-infrared mechanoluminescence in mixed-anion compounds SrZn2S2O and SrZnSO. \u003cem\u003eNano Energy\u003c/em\u003e \u003cstrong\u003e68\u003c/strong\u003e, 104329 (2020). \u003c/li\u003e\n\u003cli\u003eWang, Y.\u003cem\u003e et al.\u003c/em\u003e Enhanced high-resolution triboelectrification-induced electroluminescence for self-powered visualized interactive sensing. \u003cem\u003eACS applied materials \u0026amp; interfaces\u003c/em\u003e \u003cstrong\u003e11\u003c/strong\u003e, 13796-13802 (2019). \u003c/li\u003e\n\u003cli\u003eWei, X. Y.\u003cem\u003e et al.\u003c/em\u003e High‐intensity triboelectrification‐induced electroluminescence by microsized contacts for self‐powered display and illumination. \u003cem\u003eAdvanced Materials Interfaces\u003c/em\u003e \u003cstrong\u003e5\u003c/strong\u003e, 1701063 (2018). \u003c/li\u003e\n\u003cli\u003eZhuang, Y.\u003cem\u003e et al.\u003c/em\u003e Force-induced charge carrier storage: a new route for stress recording. \u003cem\u003eLight: Science \u0026amp; Applications\u003c/em\u003e \u003cstrong\u003e9\u003c/strong\u003e, 182 (2020). \u003c/li\u003e\n\u003cli\u003eDvorak, N., Chung, K., Mueller, K. \u0026amp; Ku, P.-C. Ultrathin tactile sensors with directional sensitivity and a high spatial resolution. \u003cem\u003eNano Letters\u003c/em\u003e \u003cstrong\u003e21\u003c/strong\u003e, 8304-8310 (2021). \u003c/li\u003e\n\u003cli\u003eChou, H.-H.\u003cem\u003e et al.\u003c/em\u003e A chameleon-inspired stretchable electronic skin with interactive colour changing controlled by tactile sensing. \u003cem\u003eNature communications\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 8011 (2015). \u003c/li\u003e\n\u003cli\u003eHaque, M. A., Kurokawa, T., Kamita, G., Yue, Y. \u0026amp; Gong, J. P. Rapid and reversible tuning of structural color of a hydrogel over the entire visible spectrum by mechanical stimulation. \u003cem\u003eChemistry of Materials\u003c/em\u003e \u003cstrong\u003e23\u003c/strong\u003e, 5200-5207 (2011). \u003c/li\u003e\n\u003cli\u003eLiu, H.\u003cem\u003e et al.\u003c/em\u003e Pressure-induced blue-shifted and enhanced emission: a cooperative effect between aggregation-induced emission and energy-transfer suppression. \u003cem\u003eJournal of the American Chemical Society\u003c/em\u003e \u003cstrong\u003e142\u003c/strong\u003e, 1153-1158 (2020). \u003c/li\u003e\n\u003cli\u003eKim, J. T., Choi, H., Shin, E., Park, S. \u0026amp; Kim, I. G. Graphene-based optical waveguide tactile sensor for dynamic response. \u003cem\u003eScientific reports\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 16118 (2018). \u003c/li\u003e\n\u003cli\u003eXu, Y.\u003cem\u003e et al.\u003c/em\u003e Glowing stereocomplex biopolymers are generating power: polylactide/carbon quantum dot hybrid nanofibers with high piezoresponse and multicolor luminescence. \u003cem\u003eJournal of Materials Chemistry A\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 1810-1823 (2019). \u003c/li\u003e\n\u003cli\u003eWang, Y.\u003cem\u003e et al.\u003c/em\u003e Dynamic behavior of molecular switches in crystal under pressure and its reflection on tactile sensing. \u003cem\u003eJournal of the American Chemical Society\u003c/em\u003e \u003cstrong\u003e137\u003c/strong\u003e, 931-939 (2015). \u003c/li\u003e\n\u003cli\u003eZhao, W.\u003cem\u003e et al.\u003c/em\u003e Highly sensitive switching of solid-state luminescence by controlling intersystem crossing. \u003cem\u003eNature communications\u003c/em\u003e \u003cstrong\u003e9\u003c/strong\u003e, 3044 (2018). \u003c/li\u003e\n\u003cli\u003eChen, G., Qiu, H., Prasad, P. N. \u0026amp; Chen, X. Upconversion nanoparticles: design, nanochemistry, and applications in theranostics. \u003cem\u003eChemical reviews\u003c/em\u003e \u003cstrong\u003e114\u003c/strong\u003e, 5161-5214 (2014). \u003c/li\u003e\n\u003cli\u003ePrevost, A., Scheibert, J. \u0026amp; Debr\u0026eacute;geas, G. Effect of fingerprints orientation on skin vibrations during tactile exploration of textured surfaces. \u003cem\u003eCommunicative \u0026amp; integrative biology\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 422-424 (2009). \u003c/li\u003e\n\u003cli\u003eAleemardani, M., Trikić, M. Z., Green, N. H. \u0026amp; Claeyssens, F. The importance of mimicking dermal-epidermal junction for skin tissue engineering: a review. \u003cem\u003eBioengineering\u003c/em\u003e \u003cstrong\u003e8\u003c/strong\u003e, 148 (2021). \u003c/li\u003e\n\u003cli\u003eCauna, N. Nature and functions of the papillary ridges of the digital skin. \u003cem\u003eThe Anatomical Record\u003c/em\u003e \u003cstrong\u003e119\u003c/strong\u003e, 449-468 (1954). \u003c/li\u003e\n\u003cli\u003eMaeno, T., Kobayashi, K. \u0026amp; Yamazaki, N. Relationship between the structure of human finger tissue and the location of tactile receptors. \u003cem\u003eJSME International Journal Series C Mechanical Systems, Machine Elements and Manufacturing\u003c/em\u003e \u003cstrong\u003e41\u003c/strong\u003e, 94-100 (1998). \u003c/li\u003e\n\u003cli\u003eWang, F.\u003cem\u003e et al.\u003c/em\u003e Simultaneous phase and size control of upconversion nanocrystals through lanthanide doping. \u003cem\u003enature\u003c/em\u003e \u003cstrong\u003e463\u003c/strong\u003e, 1061-1065 (2010). \u003c/li\u003e\n\u003cli\u003eLiu, D.\u003cem\u003e et al.\u003c/em\u003e Emission stability and reversibility of upconversion nanocrystals. \u003cem\u003eJournal of Materials Chemistry C\u003c/em\u003e \u003cstrong\u003e4\u003c/strong\u003e, 9227-9234 (2016). \u003c/li\u003e\n\u003cli\u003eBaek, D.\u003cem\u003e et al.\u003c/em\u003e Multi‐Color Luminescence Transition of Upconversion Nanocrystals via Crystal Phase Control with SiO2 for High Temperature Thermal Labels. \u003cem\u003eAdvanced Science\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 2000104 (2020). \u003c/li\u003e\n\u003cli\u003eLee, J.\u003cem\u003e et al.\u003c/em\u003e Universal process-inert encoding architecture for polymer microparticles. \u003cem\u003eNature materials\u003c/em\u003e \u003cstrong\u003e13\u003c/strong\u003e, 524-529 (2014). \u003c/li\u003e\n\u003cli\u003eDe Camillis, S.\u003cem\u003e et al.\u003c/em\u003e Controlling the non-linear emission of upconversion nanoparticles to enhance super-resolution imaging performance. \u003cem\u003eNanoscale\u003c/em\u003e \u003cstrong\u003e12\u003c/strong\u003e, 20347-20355 (2020). \u003c/li\u003e\n\u003cli\u003eMa, Z.\u003cem\u003e et al.\u003c/em\u003e Mechanics-induced triple-mode anticounterfeiting and moving tactile sensing by simultaneously utilizing instantaneous and persistent mechanoluminescence. \u003cem\u003eMaterials Horizons\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 2003-2008 (2019). \u003c/li\u003e\n\u003cli\u003eWang, X.\u003cem\u003e et al.\u003c/em\u003e Dynamic pressure mapping of personalized handwriting by a flexible sensor matrix based on the mechanoluminescence process. \u003cem\u003eAdvanced Materials\u003c/em\u003e \u003cstrong\u003e27\u003c/strong\u003e, 2324-2331 (2015). \u003c/li\u003e\n\u003cli\u003eBoutry, C. M.\u003cem\u003e et al.\u003c/em\u003e A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. \u003cem\u003eScience Robotics\u003c/em\u003e \u003cstrong\u003e3\u003c/strong\u003e, eaau6914 (2018). \u003c/li\u003e\n\u003cli\u003eTao, J.\u003cem\u003e et al.\u003c/em\u003e Self‐powered tactile sensor array systems based on the triboelectric effect. \u003cem\u003eAdvanced Functional Materials\u003c/em\u003e \u003cstrong\u003e29\u003c/strong\u003e, 1806379 (2019). \u003c/li\u003e\n\u003cli\u003eMostaghel, N. \u0026amp; Davis, T. Representations of Coulomb friction for dynamic analysis. \u003cem\u003eEarthquake engineering \u0026amp; structural dynamics\u003c/em\u003e \u003cstrong\u003e26\u003c/strong\u003e, 541-548 (1997). \u003c/li\u003e\n\u003cli\u003eLee, D. W., Banquy, X. \u0026amp; Israelachvili, J. N. Stick-slip friction and wear of articular joints. \u003cem\u003eProceedings of the National Academy of Sciences\u003c/em\u003e \u003cstrong\u003e110\u003c/strong\u003e, E567-E574 (2013). \u003c/li\u003e\n\u003cli\u003eMaksuta, D.\u003cem\u003e et al.\u003c/em\u003e Dependence of adhesive friction on surface roughness and elastic modulus. \u003cem\u003eSoft Matter\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, 5843-5849 (2022). \u003c/li\u003e\n\u003cli\u003eGao, J.\u003cem\u003e et al.\u003c/em\u003e Vol. 108 3410-3425 (ACS Publications, 2004).\u003c/li\u003e\n\u003cli\u003eAmontons, G. De la resistance cause\u0026apos;e dans les machines (1). \u003cem\u003eJOURNAL-JAPANESE SOCIETY OF TRIBOLOGISTS\u003c/em\u003e \u003cstrong\u003e44\u003c/strong\u003e, 229-235 (1999). \u003c/li\u003e\n\u003cli\u003eGuo, P., Jia, M., Guo, D., Wang, Z. L. \u0026amp; Zhai, J. Retina-inspired in-sensor broadband image preprocessing for accurate recognition via the flexophototronic effect. \u003cem\u003eMatter\u003c/em\u003e \u003cstrong\u003e6\u003c/strong\u003e, 537-553 (2023). \u003c/li\u003e\n\u003cli\u003eNo, B. \u0026amp; Choi, N. Differences in graphomotor skills by the writing medium and children\u0026rsquo;s gender. \u003cem\u003eEducation Sciences\u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 162 (2021). \u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eMaterials. \u003c/strong\u003eNaOH (Aldrich), ethanol (Aldrich, 99 %), GdCl\u003csub\u003e3\u003c/sub\u003e\u0026middot;6H\u003csub\u003e2\u003c/sub\u003eO (Aldrich, 99.999 %), YCl\u003csub\u003e3\u003c/sub\u003e\u0026middot;6H\u003csub\u003e2\u003c/sub\u003eO (Aldrich, 99.999 %), YbCl\u003csub\u003e3\u003c/sub\u003e\u0026middot;6H\u003csub\u003e2\u003c/sub\u003eO (Aldrich, 99.999 %), TmCl\u003csub\u003e3\u003c/sub\u003e\u0026middot;6H\u003csub\u003e2\u003c/sub\u003eO (Aldrich, 99.9 %), NH\u003csub\u003e4\u003c/sub\u003eF (Aldrich, 99.9 %), ErCl\u003csub\u003e3\u003c/sub\u003e\u0026middot;6H\u003csub\u003e2\u003c/sub\u003eO (Aldrich, 99.999 %), and oleic acid (Aldrich, 90 %).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFabrication of the UCN-embedded micro-hemisphere layer assembled with a human skin mimetic stress concentration layer. \u003c/strong\u003eThe synthesized UCNs were dispersed in n-hexane by ultrasonic sonication (VCX 750, Sonics) for 2 hours. The UCN solution was added to the PDMS base (Sylgard 184, Dow Corning) in a 2:3 ratio (UCN: PDMS). The UCN/PDMS mixture was dispersed using an ultrasonic processor for 1 hour. Subsequently, a PDMS curing agent (1:5 ratio of curing agent to base) was added to the UCN/PDMS mixture depending on the thickness of the sensor. The resulting UCN/PDMS mixture was dispersed using an ultrasonic process for 5 min before pouring into a micro-hemisphere silicon mold (diameter and pitch: 90 and 100 mm, respectively) and cured at 90 \u0026deg;C for 4 h. The dispersion of UCNs and PDMS for each step was observed (Supplementary Fig. 6). The top surface of the UMs array was coated with a Pt layer using an ion sputter (MC1000, Hitachi). To fabricate the stress concentration layer, the pre-cured PDMS solution was coated on a micro-hemisphere silicon mold (diameter and pitch: 20 and 40 mm, respectively) using a spin coater at 750 rpm for 1 min. After curing at 90 \u0026deg;C for 2 h, the obtained undulating microstructured PDMS was attached to a Pt layer. After 1 min of the O\u003csub\u003e2\u003c/sub\u003e plasma process (FEMTO SCIENCE, CUTE-1MPR, 100 W, 8 sccm), the TPU solution dissolved in DMF was poured onto the undulating microstructured PDMS surface, and the DMF solvent was evaporated at 60 \u0026deg;C for 12 h.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNormal force and shear force detection device setting. \u003c/strong\u003eTo measure the optical properties of the optical tactile sensor under mechanical forces, the fabricated UMs integrated with a stress concentration layer were placed on the TIR dove prism where a 980 nm NIR laser (FC-EW-980-40W, Uniotech) was irradiated.A schematic of the normal-force detection system is shown in Supplementary Fig. 5a. The normal force was measured by a vertically attached force gauge with a spherical indenter to the Z-axis liner stage. The force gauge used in our study had a resolution of 0.01 N. A normal force of 0\u0026ndash;5.0 N was measured by moving the Z-axis linear stage, and movie files of the luminescence profiles were recorded (Supplementary Movie 1). Supplementary Fig. 5b shows a schematic of the shear-force detection system. To measure the shear force, a right-angle spherical indenter was fabricated using a 90\u0026deg; tilted digital force gauge, and the shear force applied according to the velocity and acceleration of the spherical indenter was measured. Supplementary Movie 2 shows the change in the luminescence profile as the velocity reaches 1.5, 1.0, and 0.5 mm/s (acceleration: 1.5, 1.0, and 0.5 mm/s\u003csup\u003e2\u003c/sup\u003e, respectively); the velocity was maintained (1.5 mm/s: 2.3 s, 1.0 mm/s: 4 s, and 0.5 mm/s: 9 s respectively), and then decelerated to 0 mm/s in 1 s. To obtain the luminescence images and movie file, a CCD camera (DCC3260C, Thorlabs) was placed under a glass dove prism, and a 750 nm cut-off filter (FESH0750, Thorlabs) was used to eliminate 980 nm NIR light.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHand-writing data mining machine learning.\u003c/strong\u003e Regression analysis from the normal and shear force predictions for each related feature extraction was conducted based on support vector regression (SVR). SVR is typically used for nonlinear systems and predicts the appropriate hyperplane in higher dimensions to fit the input data. Parameters C = 10, degree = 3, epsilon = 0.1, and kernel = \u0026apos;rbf\u0026apos; were selected for the best performance results. The model was trained on 70 % of the data randomly split from the dataset, and the unused data were used to test the model performance. To evaluate the performance, r\u003csup\u003e2\u003c/sup\u003e, RMSE, and MAPE parameters were used. Linear discriminator analysis (LDA), which determines linear combinations of features that characterize or distinguish two or more classes, was used to classify handwriting by lowering the dimensions of the extracted force-related features. In addition, the clustering performance was calculated using the silhouette coefficient and the Calinski\u0026ndash;Harabasz index for each class.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4427929/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4427929/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Decoupling dynamic touch signals in the optical tactile sensors is highly desired for behavioral tactile applications yet challenging because typical optical sensors mostly measure only static normal force and use imprecise multi-image averaging for dynamic force sensing. Here, we report a highly sensitive upconversion nanocrystals-based behavioral biometric optical tactile sensor that instantaneously and quantitatively decomposes dynamic touch signals into individual components of vertical normal and lateral shear force from a single image in real-time. By mimicking the sensory architecture of human skin, the unique luminescence signal obtained is axisymmetric for static normal forces and non-axisymmetric for dynamic shear forces. Our sensor demonstrates high spatio-temporal screening of small objects and recognizes fingerprints for authentication with high spatial-temporal resolution. Using a dynamic force discrimination machine learning framework, we realized a Braille-to-Speech translation system and a next-generation dynamic biometric recognition system for handwriting.","manuscriptTitle":"Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-23 01:45:19","doi":"10.21203/rs.3.rs-4427929/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"5953faa4-9194-4fd5-9626-bd8c0b8f4c72","owner":[],"postedDate":"May 23rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":32055021,"name":"Physical sciences/Engineering/Chemical engineering"},{"id":32055022,"name":"Physical sciences/Materials science/Materials for devices/Sensors and biosensors"}],"tags":[],"updatedAt":"2024-09-13T07:06:10+00:00","versionOfRecord":{"articleIdentity":"rs-4427929","link":"https://doi.org/10.1038/s41467-024-52331-4","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2024-09-12 04:00:00","publishedOnDateReadable":"September 12th, 2024"},"versionCreatedAt":"2024-05-23 01:45:19","video":"","vorDoi":"10.1038/s41467-024-52331-4","vorDoiUrl":"https://doi.org/10.1038/s41467-024-52331-4","workflowStages":[]},"version":"v1","identity":"rs-4427929","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4427929","identity":"rs-4427929","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}