Pain Expression Detection System for Non-Communicable Patients

Pain Expression Detection System for Non-Communicable Patients | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Pain Expression Detection System for Non-Communicable Patients Bernardo Ternus de Abreu, Arthur Wagner This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7631185/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This paper presents a system for detecting facial expressions of pain using a computer vision algorithm and a camera connected to a microcontroller. The prototype was designed to alert physicians to facial expressions of pain in patients during surgical procedures. Currently, the detection of pain signals through hospital technologies has been done by monitoring variables linked to heart rate variability in combination with blood pressure. New technologies have been developed using cameras and algorithms that detect patient movements and facial expressions, aiming to obtain more data for patient assessment by the medical team. Maintaining the well-being of hospitalized individuals is a challenge in cases of non-communicative or poorly communicative patients, such as infants, adults with cognitive impairments that affect communication, or some elderly patients. The algorithm demonstrated high precision (0.99), indicating strong reliability in identifying pain without generating false positives. However, its recall or sensitivity was moderate (0.57), missing a significant number of real pain events (79 false negatives). The F1-score (0.72) and accuracy (0.78) reveal a reasonable balance between precision and recall, though with limitations in sensitivity. These results suggest that while the model is effective at confirming pain when detected, it fails to identify all occurrences. Despite this, the system holds potential as a didactic tool for educational purposes. Algorithm Communication Detection Pain Patients Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Introduction Pain is a biological response resulting from the interaction between physiological, psychological, and social mechanisms, and is therefore subjective and multifactorial. From a neurophysiological perspective, pain involves the activation of peripheral nociceptors, the transmission of afferent signals to the central nervous system, and the cortical interpretation of these stimuli. However, its perception varies among individuals, influenced by factors such as clinical history, emotional state, cultural context, and personal expectations, making clinical assessment a significant challenge. Currently, the detection of pain signals through hospital technologies has been achieved by monitoring variables linked to heart rate variability in combination with blood pressure. Currently, medical assistive technologies are being used for a variety of purposes, one of which is to support individuals who are temporarily unable to communicate or have communication difficulties. In a surgical setting, it is important for patients to demonstrate reactions in case of discomfort or to communicate kinesthetically or audibly if they are in pain. However, communication may be difficult, either due to a lack of speech or due to medication response. On the other hand, in many cases, the physical response of the facial muscles can be helpful in monitoring the patient's response. When an individual experiences pain, a set of muscles typically moves in the mouth, eyes, cheeks, and other parts of the face. A human's appearance is made up of a significant number of muscles that allow for the production of thousands of facial expressions. These expressions are gateways to information regarding a person's cognitive state and social intentions. Facial expression analysis allows for the automatic extraction of features from videos or images. With the development of computer vision, clinical studies began using algorithms of this type [2]. This article seeks to apply resources developed in the field of computer vision to tracking an individual's facial movements for pain detection. With the development of computer vision technologies, based on intelligent algorithms that monitor patients' conditions, the possibilities for maintaining patient health and well-being have increased. Monitoring pain in patients is a critical challenge, especially when direct communication is limited, such as during procedures under general anesthesia. In recent decades, comparative anatomical studies of different mammals have shown that facial expressions are one of the most common and specific indicators of pain in animals [5]. The incorporation of new technologies can be a way to reduce the occurrence of medical errors. Over the last decade, it is estimated that medical errors occurred in at least 3.5% of surgical procedures [11]. The use of technologies can help reduce this rate, which may be even higher than reported if possible underreporting rates are taken into account. Social communication deficits are a central feature of autism spectrum disorder (ASD). The clinical manifestations of ASD are heterogeneous, varying in intensity and form, with communication impairments ranging from mild to severe. It is estimated that approximately 30% of individuals diagnosed with ASD also have intellectual disability (ID), a neurological condition that affects approximately 1% of the general population. In cases of severe or profound ID, cerebral palsy often coexists, compromising control of the muscles involved in speech, gestures, and facial expressions. Individuals with both ASD and severe ID may therefore be unable to express the presence of pain, remaining essentially non-communicative. Given the intensive nursing care demands of this patient profile, there has been growing interest in technological solutions capable of assisting these individuals' communication and participation in their care settings. In general, the application of assistive technologies can contribute to promoting independence, inclusion, and quality of life in populations with complex needs [7]. Heart rate variability (HRV) monitoring has been used to identify pain signals in non-communicative patients and has the potential to guide targeted interventions. Kildal, Quintana, Szabo, Tronstad, Andreassen, Nærland, and Hassel (2023) conducted a randomized controlled trial that investigated the continuous use of heart rate as a marker of acute pain in non-verbal patients with autism and intellectual disability. In this study, non-communicative patients were monitored for up to 8 hours daily during routine care to identify episodes of acute pain. Data on heart rate variability and inflammatory biomarkers such as cytokines were collected to assess long-term pain. This approach demonstrated that physiological signals can be used to guide specific clinical interventions and reduce painful episodes in the daily lives of these patients [7]. Heart rate (HR) investigation as a tool for pain management in nonverbal patients should be performed in conjunction with other established biomarkers, such as heart rate variability (HRV) and circulating inflammatory markers. HRV is a noninvasive measure that reflects autonomic nervous system activity, with higher variability being associated with lower physiological stress. Evidence from psychology studies has identified HRV as a consistent biomarker of persistent pain, in addition to its association with emotional responses. Concurrently, recent research suggests that certain circulating inflammatory cytokines may constitute reliable biomarkers of chronic pain in various clinical conditions. Among the frequently described markers are MCP-1, IL-1RA, IL-8, TGFβ1, and IL-17, whose elevated levels have been associated with both experimental and self-reported pain, paralleling the temporal dynamics of pain episodes. Thus, the integration of HR, HRV, and inflammatory biomarkers may represent a relevant approach for the objective assessment of pain in groups of people with these characteristics [7]. This article presents the development of an early-stage technology for patient monitoring using a camera and a microcontroller that communicate with a computer. The literature review indicated that the application of sensors in the care of nonverbal patients is not yet widespread. Much of the research on the use of sensors for pain monitoring and facilitating communication has been conducted in groups of people with preserved verbal capacity. Noncommunicative patients are often excluded from these studies, although recent advances in medical technology may provide significant benefits to this segment of the clinical population [7]. According to the 2030 Agenda for Sustainable Development of United Nations, health and well-being are listed as one of the 17 key objectives for societies. This objective recognizes that access to health care is a fundamental human right and seeks to reduce mortality and promote health in general. Research that uses technologies for health contributes, therefore, to this sustainable objective. Theoretical Background Pain Detection Pain is a complex phenomenon, linked to an individual's sensory and emotional experience. It is the main reason people seek medical care. Treatments such as surgery, chemotherapy, and radiation therapy often cause pain. Furthermore, persistent pain has implications for individuals' quality of life [13]. Some patients have a higher pain tolerance than others. Despite accumulated knowledge about the characteristics of pain, it is currently poorly managed for certain groups of patients who live with this physiological manifestation. It most acutely affects patients with limited communication skills and who cannot express their pain experience. These include: infants, children, adults with cognitive impairments that affect communication, people with intellectual disabilities, seriously ill or unconscious individuals, and terminally ill individuals [13]. In response, automatic pain recognition systems are being developed based on "pain behaviors," which can be associated with facial expressions, vocalizations, and body movements. Physiological responses captured by signals monitored by equipment can complement current assessment methods, aiming for better continuous pain tracking. With more data, early intervention is possible, especially in patients in the aforementioned groups. A few years ago, an index of nociceptive analgesia, called the Analgesia Nociception Index, was proposed. The ANI is calculated from a frequency-domain analysis of the high-frequency component of heart rate variability. The method also considers respiratory rate as a potential confounding factor. The index is presented as a score from 0 to 100 and reflects parasympathetic activity; that is, lower values indicate low parasympathetic activity, and higher values indicate high parasympathetic activity. Pain itself results in the predominance of sympathetic activity, so ANI values decrease with pain [1]. Several areas of the brain are involved in the pain process. The process begins with the activation of sensory neuronal pathways by stimuli, which activate primary sensory neurons with surface receptors specialized for detecting harmful stimuli. Action potentials are conducted by nerve fibers, and the signals result in synapses in the spinal cord [13]. When a harmful event is identified, excitatory and inhibitory interneuronal circuits in the spinal cord are activated, leading to a protective reflex withdrawal event. Data processing occurs in several supraspinal structures, which lead to the sensory perception of pain. Although stimulation generally leads to pain perception, certain conditions can hinder perception. A person can also experience pain without activation of the nociceptive pathway, which occurs in psychogenic pain [13]. Postoperative pain remains a current problem, with an estimated incidence of 30% to 71%, depending on the surgical procedure performed [10]. The American Pain Society guidelines recommend assessing postoperative pain every 15 minutes initially and every 1 to 2 hours as the intensity decreases. Assessments are usually performed by a healthcare professional monitoring the patient. In sedated patients, assessment may be more difficult. Measurement difficulties also occur in patients with poor communication during the immediate postoperative period [10]. Among the existing scales, the ENR and the VAS can be mentioned. The ENR consists of 11 points and is used as a reference measure for pain intensity. The ENR has significant validated correlations with other pain classification measures, such as the visual analog scale or VAS [10]. To address this situation, changes in visual signals, such as facial expressions, and physiological indicators, such as vital signs and the analgesia index (ANI), and nociception, are measured. Currently, a combination of methods has been used, considering facial expressions, the analgesia index, and vital signs, to assess pain. The analgesia/nociception index is based on electrocardiographic data that reflect parasympathetic activity. Furthermore, the measurement does not require a healthcare professional; anyone can perform it as long as protocols are followed. Pain assessment is a critical component of healthcare and is essential for patient diagnosis and treatment. The subjective nature of pain poses challenges for healthcare teams, particularly in groups unable to communicate their discomfort effectively. Infants, nonverbal patients, and patients with cognitive impairments fall into this group [9]. Physiological and observational measurements can be used in certain situations. For adequate pain management, assessment must be performed frequently, especially if the patient cannot call for help. Therefore, assessment tools have been developed for more standardized use by healthcare teams around the world [9]. For expression analysis, it is not recommended to rely on "prototypical" expressions. A commonly used representation in the field of affective computing derives from Ekman's work on prototypical emotions, in which the author proposed a list of universally recognized emotions: joy, anger, sadness, fear, disgust, and surprise [3]. The main problem with this representation is that it is not suitable for characterizing spontaneous facial expressions, as most human affective behaviors cannot be translated into terms of prototypical emotions [2]. The Visual Analog Scale (VAS) is a tool used to measure the subjective intensity of sensations or feelings, especially pain. It consists of a straight line with descriptions indicating the extremes of the sensation being measured. The patient is instructed to mark a point along the line that best represents the intensity of their pain at that moment. Because it is a continuous measurement, it allows for the capture of small variations in pain perception, making it more sensitive than categorical scales such as numerical or verbal scales. The score is obtained by measuring the distance, in millimeters or centimeters, between the marked point and the extremity representing "no pain," resulting in a value between 0 and 10. The VAS scale is widely accepted and has been validated as a pain assessment tool. Despite its effectiveness, the scale has some significant limitations. The VAS requires cognitive and motor comprehension skills from the patient, which can make it difficult to use with young children, people with neurological impairments, or those with communication difficulties. Scales are problematic in pediatric, geriatric, or critically ill patients, as well as in patients with communication problems or who are sedated. There have also been some identified cases of temporary postoperative delirium, which can be a confounding factor in the assessment. Therefore, the use of methods capable of assessing pain more objectively is necessary [1]. Technologies to support pain detection In terms of technologies for detecting physiological pain signals, a device developed in France is a case in point, having already been used in over 60 countries. It's the PhysioDoloris monitor, also known as Metrodoloris, a non-invasive clinical device developed by MetroDoloris Medical Systems. It tracks heart rate signals and generates a real-time graph on the monitor screen indicating pain status. Integrated with the patient's medication administration system, it can be adapted for better administration of medication doses. The monitor displays indicators related to the Analgesia/Nociception Index (ANI) on its screen. It was developed to assess the balance between analgesia and nociception by analyzing heart rate variability (HRV). It captures the ECG signal through a special sensor and generates a series of intervals; the analysis of these high-frequency fluctuations (between 0.15 and 0.4 Hz), which mediate parasympathetic tone, results in a numerical index between 0 and 100. Values close to 0 indicate high nociception or intense pain, while values close to 100 reflect predominant analgesia or comfort. The ANI index is interpreted as follows: average values between 50 and 70 are considered adequate analgesia during the anesthetic procedure. When the index drops below 50 for more than five minutes, the risk of hemodynamic reactivity increases considerably, signaling the need for analgesia adjustments. On the other hand, persistent values above 70 suggest possible analgesic overdose. Several clinical studies have demonstrated the usefulness of the monitor in intrapartum and postoperative pain management. For example, a linear inverse relationship was observed between visual analog scale pain scores and the ANI, indicating its ability to reflect actual discomfort, even during uterine contractions. In pediatric patients, predefined lower ANI values correlated well with postoperative pain or elevated levels on scales such as FLACC. In situations of noxious intraoperative stimuli, the ANI proved more sensitive than traditional hemodynamic parameters such as heart rate or blood pressure. The use of the ANI to guide the administration of analgesics such as remifentanil resulted in a reduction in intraoperative opioid consumption and a decrease in immediate postoperative pain levels. It is important to note, however, that some conditions limit the reliability of the index, such as arrhythmias, patients with cardiac pacemakers, or the use of drugs that alter autonomic tone, such as agonists, beta-blockers, or antimuscarinics. Continuous analysis of cardiac variability allows for more precise titration of analgesics, aiming to avoid both dosages well below or above expected values, maintaining hemodynamic stability and potentially reducing postoperative complications. Despite the monitor's qualities, low-cost technologies can also be developed at the research level, aiming to reduce future technological costs for pain detection. While it is quite positive that pain detection using specific equipment is a reality in several countries, auxiliary prototypes are warranted. If successful, computer vision technologies could be tested in the future in integration with the existing monitor. Several technologies have been developed to capture nonverbal pain signals using cameras and artificial intelligence algorithms. One of the most promising approaches involves the use of computer vision to analyze facial expressions, involuntary movements, and microexpressions. Systems that taxonomize human facial expressions have been integrated with machine learning and deep learning algorithms to detect patterns associated with pain. Recently, deep neural networks (CNNs) have been applied to support medical assessment, designed to analyze pain-induced facial expressions. The developed models can extract relevant descriptors and optimize inference models based on input data. The field of affective computing has offered promising advances by integrating artificial intelligence (AI) with sensing technologies. This work uses facial videos to improve pain assessment, enabling better patient care with greater empathy. Traditional pain assessment methods rely primarily on self-reports, as pain is a subjective experience. However, self-reports are not always valid and reliable, which can occur when the individual is not fully aware of reality [9]. Nguyen, Yang, Kim, Shin, and Kim (2024) developed a model to analyze facial videos for pain assessment, contributing to better patient care and more empathetic clinical interventions. The authors used a Transformer-based architecture composed of an Autoencoder for temporal feature extraction and a Transformer-based classifier for Multivariate Time Series Classification. The authors applied two position coding techniques: Time Absolute Position Encoding (tAPE) and Efficient Relative Position Encoding (eRPE) [9]. Liu, Peng, Rudovic, and Picard (2017) proposed a deep learning model to estimate self-reported pain levels based on facial expressions. DeepFaceLift is a model that extracts facial landmarks, sends them through a neural network, and uses Gaussian Process Regression to estimate VAS scores [8]. Unlike previous approaches that use generic classifiers, DeepFaceLIFT introduces a personalized and interpretable model that takes into account individual variations in pain perception and expression. The proposal is based on a neural network architecture that incorporates person-specific information and offers clear explanations of how the estimates are generated. The system is evaluated on the UNBC-McMaster Shoulder Pain Expression Archive database, known for containing annotated videos of patients with chronic shoulder pain. The results show that the model outperforms traditional methods, especially when it comes to subjective pain assessments, such as the Visual Analogue Scale (VAS). The paper also highlights the importance of interpretability in machine learning models applied to healthcare, suggesting that transparency can increase clinical professionals' trust in these systems. Technologies based on convolutional neural networks (CNNs) and deep learning have shown great potential in analyzing and interpreting facial expressions associated with pain. One of the most common approaches involves the use of computer vision algorithms, which process facial images or videos to identify specific facial features associated with pain. Pattern recognition techniques are used to detect changes in facial expressions, such as movements in the muscles around the eyes, mouth, and eyebrows, which are common indicators of pain. Algorithms can provide continuous pain monitoring, identifying early signs of pain events. As a potential improvement, applications can monitor pain levels throughout the day and prevent pain-related complications. In the surgical setting, algorithms can be applied to predict persistent postoperative pain after surgery. Existing technologies can monitor pain levels throughout the day and prevent pain-related complications. In the surgical setting, algorithms can be applied to predict persistent postoperative pain after surgery [4]. The use of devices such as high-resolution cameras, infrared sensors, or 3D cameras can be integrated to improve detection accuracy by capturing subtle details of facial expressions. Combining different types of sensors and multimodal analysis are areas of interest to increase the robustness of systems. Algorithms such as support vector machines (SVMs) have also been applied to classify facial expressions of pain into different categories, providing a quantitative analysis of the detected emotions. Werner, Al Hamadi, Niese, Walter, Gruss, and Traue (2019) developed an automatic pain recognition system using multimodal analysis, combining video signals, facial expressions, and head movements with biomedical data such as galvanic skin response (GSR), electromyography (EMG), and electrocardiogram (ECG). Using the BioVid Heat Pain Database, developed by the University of Ulm, the algorithm focuses on identifying different pain intensities caused by thermal stimuli. The model is notable for its ability to capture subtle patterns in the user's facial expression and physiology, employing data fusion via the Transferable Belief Model, which integrates information over time to refine detection per frame. The study results demonstrate that the multimodal approach significantly outperforms methods that use only video or biomedical signals alone. The inclusion of head movements, when combined with facial expressions, results in statistically significant gains in accuracy, especially for moderate to high pain levels. The galvanic skin response (GSR) also stands out as an important physiological variable for discriminating different pain intensities. Furthermore, the authors note that personalized models, adjusted to each subject's individual response, tend to perform better than generic models, highlighting the importance of considering interpersonal variations in pain expression. This approach does not rely on contact sensors, using only video to extract relevant behavioral patterns. This allows pain intensity to be identified with accuracy comparable to or superior to that of human observers, especially in experimentally induced acute pain settings. On the other hand, studies such as those by Ashraf et al. (2020), explore multimodal models that combine visual data with physiological signals, such as electrocardiogram (ECG), skin conductance (SDI), electromyography (EMG), and remote photoplethysmography (rPPG). This fusion of modalities typically improves the accuracy of pain detection in real clinical settings. Nevertheless, the authors note that even purely visual analysis, using CNNs, can achieve high levels of accuracy in detecting acute pain, often sufficient in scenarios where the use of physical sensors is not feasible or desirable. Furthermore, they show that networks trained solely on facial signals can perform excellently when well-tuned, particularly in databases where nonverbal pain patterns are clearly evident. There are emerging commercial solutions on the market, such as the PainChek system. Approved in some countries for clinical use, this technology uses facial recognition with artificial intelligence support to assess signs of pain in patients with communication difficulties, such as elderly people with specific conditions of dementia or patients under sedation. Although still undergoing broader clinical validation, these technologies indicate a promising direction for the automation and standardization of intraoperative pain monitoring. They can complement clinical judgment and support anesthesiology, promoting greater safety and personalized pain control during surgical procedures. Continued research in this field, especially focusing on surgical center environments, should develop integration between video systems, artificial intelligence, and multiparameter monitors, aiming for better patient care. In the field of facial expression analysis, early deep learning-based methods followed the principle of cascade alignment. This approach involves iteratively estimating the positions of facial feature points based on displacements learned from an annotated training base. Each stage of the cascade aims to refine previous estimates, starting from a rough initial prediction and reaching higher accuracy. With the popularization of convolutional neural networks (CNNs), new models have been proposed to replace traditional cascade stages. Sun et al. (2013) presented a cascade model based on deep convolutional networks, in which each regression stage, composed of distinct feature extraction and mapping steps, was replaced by a specialized CNN. This proposal demonstrated superior performance to classical approaches, highlighting the potential of CNNs for the task of face alignment [12]. Subsequently, Zhang, Jie et al. (2014) proposed CFAN, a cascade of deep auto-encoders. The main idea was to use a sequence of trained autoencoders to progressively refine estimates of the position of facial points, allowing identification in different lighting, pose, and expression conditions [14]. The project developed by the authors consists in a model based on deep multi-task learning, in which the facial landmark detection process was coupled with the simultaneous recognition of facial attributes such as gender, age, and emotion. This integration helped the model learn more discriminative representations and accelerated inference time by sharing convolutional layers across tasks [14]. More recently, the same authors expanded their approach with an even more efficient and accurate model, combining multiple convolutional networks to more robustly predict the location of facial landmarks across different transitions. This approach also aimed to improve processing time by adjusting the approximations between cascade steps and the spatial relationship between landmarks [14]. Materials and Methods For this project, a Python algorithm was developed consisting of a pre-trained neural network adapted to capture facial features. The algorithm, run by the computer, receives data from a camera connected to a Raspberry Pi microcontroller. The project is based on analyzing variations in specific facial regions, using the OpenCV library to display text and draw information about the captured frames. The MediaPipe library is used to detect facial features or "landmarks." The algorithm begins with (1) the camera frame capture step. This captures a new frame from the video transmitted by the camera via IP. The second step is (2) frame preprocessing, which horizontally mirrors the frame and converts it from BGR, the OpenCV standard, to RGB, as required by MediaPipe. The third step is (3) processing with the FaceMesh model, which passes the frame to the "face_mesh" model, which returns the detected facial landmarks, with refined points for the eyes and iris. The fourth step is (4) result manipulation in case of detection, which accesses the detected points and stores the x and y coordinates of each landmark. Next, the algorithm (5) performs data processing, which involves calculating point distances, distance normalization, calibration logic, and pain detection. This step is within a video processing loop. The code was developed in Python and hosted on the Github repository. The algorithm structure can be found at the following link: https://github.com/beternus/pain-detector/blob/main/main.py . Figure 1 presents a flowchart of the algorithm structure, relating its main steps. Initially, the code defines and initializes the Flask server, which runs in a parallel thread. This configuration ensures that the server continues to run during the video processing stage. The "video_feed()" function creates a route that continuously sends processed frames to connected clients, using the multipart response type required for real-time video streaming. Additionally, there is a "/frame" route that allows access to the last captured frame in JPEG image format. To this end, the last processed frame is stored in a global variable ("last_frame") and converted to a JPEG buffer to be sent as an HTTP response. The code integrates a web interface using the Flask framework, allowing the processed video to be streamed to a browser or client application. The process occurs through a specific HTTP route that continuously sends the annotated frames, encoded in JPEG. Another endpoint allows the capture of a current static frame, useful for specific analysis. This entire system operates in parallel, with the graphical interface displayed locally by OpenCV and the web server running in a separate thread, allowing simultaneous analysis and visualization. The system can capture video from an IP source (such as a smartphone streaming over Wi-Fi), process the frames using MediaPipe's facial mesh model, perform a calibration of facial features at rest, measure variations in the positions of specific facial points over time, and, based on heuristic criteria, identify possible expressions of pain. It also makes the processed frames available through a Flask server, allowing remote access via a browser or app. The algorithm uses "IP streaming" video input via "cv2.VideoCapture" with a URL indicating that the video source could be a smartphone using applications such as IP Webcam. This eliminates the need for a camera connected to a computer and makes the system more flexible and portable, allowing facial capture to be performed remotely. The "cv2.flip(frame, 1)" function is used to horizontally flip the captured image. The code uses the "cv2.putText" function to overlay the percentage variations detected in the cheeks, lips, and mouth opening onto the frame. These values are dynamically extracted and formatted to one decimal place, and are drawn using a simple font "cv2.FONT_HERSHEY_SIMPLEX," with green indicating normality or absence of pain and a line width of 2. The data visually informs the user of the behavior of the monitored facial regions in real time. These values include the variation of the right eyebrow, left and right eyes, left and right cheeks, and two measurements related to the mouth: the distance between the lips and the mouth opening. The variations are converted to text and drawn on the image at specific positions. After capturing and converting the frames to RGB format, the system identifies the face and extracts the previously defined points of interest from a dictionary called "LANDMARKS." These points include pupils, eyebrows, eyes, cheeks, lips, mouth, and more. Using this data, normalized relative distances are calculated. In the system's operation, for each frame captured from the camera, the algorithm converts the image to RGB format and sends it to the "face_mesh.process()" model. If successful, this returns a structure containing the face's landmarks. These points are mapped to pixel coordinates based on the frame resolution. The code selects specific landmarks based on predefined parameters, identifying parts of the face. From these points, the system calculates several Euclidean measurements, such as the vertical distance between the eyebrow and forehead, eye opening, cheek retraction, lip opening, and the distance between the pupils. These measurements are normalized to facial dimensions such as height, width, and interpupillary distance, to ensure robustness across different face sizes and camera positions. The algorithm performs logical comparisons to determine whether the variations exceed pre-established threshold values, indicating patterns consistent with an expression of pain. Eyebrow raising is detected if both present a variation equal to or greater than 8%, while eye twitching is considered when their variations are less than − 15%. Cheek retraction is detected with values below − 8%, and lip twitching with a variation less than − 25%. Each of these conditions returns a Boolean value indicating whether the criterion is met. Mouth opening is analyzed as significant and potentially indicative of pain if its value exceeds 3,500 units, likely linked to the distance between landmarks. If at least three of these facial regions are "altered" at the same time, the system concludes that there is an "expression of pain present" and displays an indicative message on the video. The algorithm uses the "refine_landmarks = True" command to refine the location of critical landmarks, such as pupils. This improves the accuracy of interpupillary measurements and eye tracking, which is relevant for analyses related to eye closure. The algorithm uses normalization for different measurements. Vertical distances, such as eyebrows and mouth, are normalized by face height, while horizontal distances, such as cheeks, are divided by face width, and measurements between the eyes are normalized by interpupillary distance. Defining the normalization basis for each region ensures that the analysis is proportional and consistent with facial anatomy, avoiding distortions. The system builds its decision criteria based on a sort of "voting" among facial regions: if three or more of the five regions evaluated present variations consistent with the expression of pain, then the pain is identified as real. The voting-based approach makes the system less susceptible to isolated false positives, requiring a more robust standard for detection alerts. An important step in the algorithm is "calibration," which occurs within the first few seconds after the application begins. During this period, the user must remain still, allowing the system to record reference measurements of their neutral facial expression. These baseline average values are then used as a comparison point to detect expressive variations in subsequent frames. Over time, the system compares the new measurements with these initial averages, calculating "percentage changes" that indicate how far each part of the face has deviated from the neutral condition. The calibration process is automatically activated at the start of the runtime and aims to capture the neutral state of the user's face. Over a period of approximately 8 seconds, the system collects measurements of the mentioned facial regions and, at the end of the time, calculates the average of each of these distances. These averages are stored and used as a reference to identify variations over time. After calibration, the system enters detection mode. With each new frame, it calculates the current distances of the facial regions and compares them with the calibrated average values, generating variation percentages. If at least three of these five conditions are simultaneously true, the system interprets the situation as a possible expression of pain. In this case, it displays a red alert in the frame: "Expression of PAIN detected!" (“Expressão de DOR detectada!”). The alert message is superimposed on the image, informing that the expression has been detected. The text is displayed in red and thicker to stand out visually. The algorithm displays a side legend that individually shows each facial criterion, such as "Eyebrows," "Eyes," and "Mouth," colored red if the pain condition is present or green if it is not. If the option to display facial landmark indices is enabled, as expressed by the "mostrar_indices" variable, the algorithm iterates through all detected facial landmarks and plots their indices directly on the image, allowing visual and numerical verification of each landmark's position. The system displays the numerical indices of each facial landmark on the image, which is useful for calibrating the facial mesh. The final rendering of the facial mesh is done using the "mp_drawing.draw_landmarks" function, which traces contours and landmarks of the face with specific colors and styles. The frame with these annotations is stored as "last_frame," ensuring that the most recently processed image is available even if detection is interrupted. The facial contours are drawn onto the frame using MediaPipe's facial mesh model, with custom visual specifications to highlight the mesh points and lines. This provides a detailed graphical representation of the facial structure recognized by the system. MediaPipe uses a model that uses neural networks trained to locate 468 specific points on the face. The density of points allows for detailed measurements. Normalizing distances is a strategy to reduce the impact of different face sizes and camera distances while maintaining proportionality. MediaPipe doesn't perform this task deterministically or based on classical geometric algorithms. It uses "deep neural networks" previously trained on large databases of facial images. Internally, face detection and regression of facial mesh points are performed by a machine learning pipeline. This pipeline initially consists of a "face detection" step, responsible for identifying the presence and location of the face in the image, followed by a "landmark regression" step, which estimates the three-dimensional (x, y, z) position of facial points based on the cropped image of the detected face. The architecture of these neural networks is based on optimized convolutional models, specifically designed to be lightweight and fast enough to run even on mobile devices. When the "refine_landmarks = True" option is enabled, as in the code presented, an additional model comes into play to further improve the accuracy of points located in critical regions such as the eyes and irises. This feature enables refined pupil detection, an extremely important factor in the algorithm in question, which uses interpupillary distance as a reference metric for data normalization. At the end of each iteration, the annotated image is updated and stored as the last frame processed. In application control, the system responds to keyboard commands. The "ESC" key ends the program; "M" toggles the display of facial point indices; the space bar (“SPACE”) pauses or resumes processing; and "R" starts a new calibration. Finally, the application continuously displays the processed result in a window called "Expression Detection," updating with each captured frame, unless paused. At the end of the code, there is a command to release and close all capture resources and windows opened by OpenCV, aiming for the completion of execution. The algorithm is an application of computer vision which uses neural networks encapsulated in the MediaPipe library to extract facial representations. While the system does not train or fine-tune neural networks on its own, it directly benefits from the power of optimized pre-trained models, which enable the extraction of facial data in real time. From this data, the code performs a quantitative analysis of facial variations, creating a tool for detecting expressions of pain, with potential use in medical contexts. The code implements a neural network for facial expression detection using MediaPipe in conjunction with OpenCV and Flask. The overall structure is based on a Flask server that displays the processed video feed in real time, with facial expression analysis performed using MediaPipe's FaceMesh. The neural network used in MediaPipe for facial detection, specifically FaceMesh, is a pre-trained deep convolutional network provided as a resource in the library, which identifies a certain number of facial features. Initially, the code configures the MediaPipe library with the "FaceMesh" function, which detects and tracks specific facial features, such as eyebrows, eyes, mouth, cheeks, and more. The model uses a pre-trained neural network to identify up to 468 facial features on a single face, enabling detailed analysis of facial features. MediaPipe performs real-time detection with high accuracy using images captured by the camera. The code also includes an interface control to pause execution, show or hide the indices of the facial landmarks, and restart calibration. Facial expressions are continuously monitored, and detecting certain specific facial movement conditions, such as raised eyebrows or closed eyes, triggers alerts for expressions like pain. MediaPipe's neural network is efficiently used to provide highly accurate analysis of facial emotions, while Flask communicates the results to the user through a real-time web interface. By combining these technologies, the system enables dynamic assessment of facial expressions, with the ability to calibrate and adjust over time. The neural network used in MediaPipe for facial feature detection, called FaceMesh, is a convolutional neural network (CNN) specialized in tracking and identifying facial features in images and videos. MediaPipe is a framework developed by Google that uses deep learning and computer vision techniques to solve problems such as real-time facial feature tracking. FaceMesh's architecture is designed to be highly efficient, enabling real-time detection with low resource consumption, making it ideal for mobile devices and environments with hardware limitations. At the heart of FaceMesh is a convolutional neural network trained to detect 468 key points on the human face, distributed across specific facial regions, such as the eyebrows, eyes, nose, mouth, and chin. Each of these landmarks is mapped into 3D space, allowing the model to understand facial depth and capture nuances in the position of the points at different angles and expressions. The network is trained on large databases containing millions of facial images, providing good generalization across different face types, ages, ethnicities, and lighting conditions. The network structure consists of several convolutional layers responsible for extracting facial features at different scales. These convolutional layers are followed by pooling layers, which help reduce the dimensionality of the information and focus on the most important aspects of the image. After these convolutional and pooling layers, the network passes through fully connected layers, which are responsible for refining the detection and mapping the extracted information to the spatial coordinates of the facial features. A key feature of FaceMesh is its ability to operate very quickly and efficiently. This is possible thanks to the use of lightweight neural networks and optimization techniques such as quantization and layer fusion. Furthermore, MediaPipe uses a pipeline architecture that divides processing into stages, allowing detection to be performed in a modular and scalable manner. The network is also optimized to work on resource-constrained devices, such as smartphones and embedded systems, without compromising detection accuracy. FaceMesh is designed to be robust and adaptable to different conditions, such as varying lighting, rapid movements, and even partial occlusions of the face. The neural network is able to handle these challenges thanks to its large-scale training and its ability to learn complex facial representations, such as skin deformation, facial expressions, and viewing angles. This makes it a powerful tool for applications in areas such as augmented reality, emotion analysis, biometrics, and gesture-based interaction systems. Microcontroller and Camera Real-time data is captured by a camera connected to a Raspberry Pi microcontroller. The microcontroller is connected to a Noir V2 camera. Figure 3 shows the camera connected by a flat cable, which is inserted into a terminal on the Raspberry Pi electronics board. The Raspberry Pi 4 Model B is a single-board microcomputer developed by the Raspberry Pi Foundation, which features a Broadcom BCM2711 processor, a 64-bit quad-core Cortex-A72 (ARM v8), operating at 1.5 GHz, which provides significant improvements in speed and processing capacity compared to previous generations. The Raspberry Pi 4 is available in versions with 2 GB, 4 GB, or 8 GB of RAM, allowing you to run full Linux-based operating systems and multitasking applications. In terms of connectivity, the Raspberry Pi 4 includes two USB 3.0 ports, two micro-HDMI ports, gigabit Ethernet, Wi-Fi, and Bluetooth 5.0. The board has a 40-pin GPIO connector for integration with external sensors and modules, making it suitable for scientific prototyping. The Raspberry Pi 4 allows you to run computer vision algorithms locally with Python and OpenCV. The NoIR V2 camera is a Raspberry Pi module designed for applications involving night vision or low-light imaging. It uses a Sony IMX219 sensor with 8 megapixel resolution, capable of capturing photos up to 3280 x 2464 pixels and videos in Full HD (1080p) resolution at 30 fps. It also allows for 720p and 640x480 video capture at higher frame rates. The main feature that differentiates the NoIR V2 from the standard version is the absence of an infrared filter (IR-cut filter), which makes it sensitive to near-infrared light (near-IR). The camera connects directly to the Raspberry Pi 4's CSI (Camera Serial Interface) connector using a flat cable, making it a suitable option for embedded computer vision projects. The Raspberry Pi 4 was placed in a plastic case that was 3D printed to the appropriate dimensions to accommodate the microcontroller. A 3D printer was used to build the plastic case. This method consists of an additive manufacturing technology, in which objects are created layer by layer from a digital model. The FDM (Fused Deposition Modeling) method was used, which melts plastic filament to build the object. After modeling the plastic case in the design software, the plastic compounds were inserted into the system for melting and gradually forming the final shape of the case, which includes camera ports. 3D prototyping is recommended for experimental and educational projects, as was the case with this one. For large-scale production, the method may not be the most cost-effective, but it adequately met the project's needs in its initial design. Figure 3 shows the plastic box isolated from the other parts of the project. It has a flexible rod for inserting the camera, allowing it to be moved in different directions. Results and Discussion When executed, the algorithm running on the computer receives data from the camera. When a human face is located, it calibrates the system for a few seconds and then displays green text and numeric indications on the left and right sides of the screen. Figure 4 shows the face detection result after automatic calibration, which occurs within 8 seconds. After identifying the face, the algorithm tracks changes in distances between points and performs a summation. If three or more parameters exhibit the minimum changes defined by the user, a red message indicating pain detection is displayed on the screen. Figure 5 shows the detection result. Tests with the algorithm confirmed that it was possible to transmit camera images to peripheral monitors. A tablet was used in the project to receive video data, and data capture was possible with a low delay between the images displayed on the computer and the tablet. Figure 6 shows the images with the indications displayed on both devices. The computer processes the data captured from the camera and sends it to the tablet. This option may be recommended for monitoring a patient in a surgical room where more than one monitor is required, with the screens located in different locations. A test of the algorithm's decision logic was developed using numerical data defined for the test. This was an initial test of the algorithm's logic, and a future step will involve testing with labeled videos from a free database.[1] The metrics found with the numerical test were: Table 1 . Algorithm evaluation through metrics Metrics Result Precision 0.990476 Recall 0.568306 F1-score 0.722222 Accuracy 0.777778 True Positives 104.000000 False Positives 1.000000 True Negatives 176.000000 False Negatives 79.000000 The algorithm's decision logic was evaluated by monitoring its response over a period of frames during which there were pain intervals. The algorithm should track the pain intervals and be sensitive to these signals. The response is expressed in the following graph: The model achieved a precision of 0.99, indicating that 99% of the times the algorithm detected pain, it detected it correctly. This result suggests that the system is effective in avoiding false positives, meaning it rarely flags pain when it is not present. However, when analyzing sensitivity or recall, the value of 0.568 indicates that the algorithm correctly identified only about 56.8% of the instances in which pain actually occurred. The low recall value indicates that many actual pain episodes were missed, considering that the test yielded 79 false negatives. This can be critical in sensitive applications, such as clinical settings or automated patient monitoring. The F1-score metric, representing the harmonic mean between precision and recall, presented a value of 0.722, reflecting the balance between the two metrics and reinforcing the limitation in the complete detection of pain episodes. The model's overall accuracy was 0.778, revealing that 77.8% of all classifications, both positive and negative, were correct. The results indicate that the algorithm is highly reliable in detecting the presence of pain, but it still fails to recognize many instances of actual pain. The algorithm is a didactic study and is not applicable to clinical or critical cases, but can be used for educational purposes by students. Improvements to the algorithm are expected to improve detection criteria and decision thresholds to increase sensitivity, even if this results in an increase in the false positive rate. Final Considerations The algorithm was able to detect pain in initial tests as planned. However, the algorithm needs to be improved for further tests. The results indicate that the algorithm is highly reliable in detecting the presence of pain, but it still fails to recognize many instances of actual pain. Improvements to the algorithm are expected to improve detection criteria and decision thresholds to increase sensitivity, even if this results in an increase in the false positive rate. A literature review of existing technologies revealed the use, over a decade ago, of a monitor that detects pain in patients based on heart rate variables. This monitor is the PhysioDoloris monitor, also known as Metrodoloris, initially developed in France. Computational vision technologies can be integrated with existing technologies in future research contexts. On the other hand, there are situations where camera monitoring can be indicated, such as monitoring the health of babies in hospitals. However, for this, it is necessary to develop more robust technologies than the algorithm presented if the application occurs in real scenarios. The data processing of project was performed by the computer, not the Raspberry Pi 4 itself, which merely captures the signals and sends them to the computer. It has not yet been possible to use the Raspberry Pi 4 to process data captured by the attached camera. Future steps include implementing more executable functions on the Raspberry Pi 4's processor. The application has potential for future expansion, whether by incorporating machine learning models trained on expression datasets or introducing audio feedback or real-time alerts. The modular architecture and use of widely adopted libraries make this technical evolution feasible with few structural changes. The algorithm can be improved in the future to track the movement of certain body parts, such as hands. Studies indicate that, in addition to facial expressions, vocalizations and body part movements are also indicative of pain. Therefore, algorithms connected to cameras could be used to track patients' body movements in a way that's more sensitive to their physiological responses. One of the downsides of video-based monitoring is that it's considered more intrusive than heart rate or other variable monitoring. For it to be used, the patient must authorize the monitoring, meaning consent must be provided, and all caregivers must be informed if the patient is with caregivers. Therefore, there are ethical implications. On the other hand, just as heart rate monitoring is recommended in some situations, especially surgical ones, it also requires additional monitoring with cameras and/or sensors. The algorithm demonstrated high precision (0.99), indicating strong reliability in identifying pain without generating false positives. However, its recall was moderate (0.57), missing a significant number of real pain events (79 false negatives). The F1-score (0.72) and accuracy (0.78) reveal a reasonable balance between precision and recall, though with limitations in sensitivity. These results suggest that while the model is effective at confirming pain when detected, it fails to identify all occurrences. Despite this, the system holds potential as a didactic tool for educational purposes. [1] The PEDFE, or "Padova Emotional Dataset of Facial Expressions," is a database developed to provide standardized material for the study and recognition of facial expressions. PEDFE brings together videos that record various human facial expressions. The database contains 1,458 clips of facial expressions, which were recorded under controlled laboratory conditions. Participants displayed both spontaneous (genuine) expressions and simulated or acted-out expressions, covering the six universally recognized basic emotions: happiness, sadness, anger, fear, surprise, and disgust. Therefore, the videos present both truly felt and staged emotions. PEDFE data is systematically labeled, enabling its use in machine learning tasks such as classification, micro expression detection, and temporal analysis of emotional intensity. The database includes 1,458 clips, 707 genuine and 751 staged, obtained from 56 participants. Two versions are available: the original, with the participants, their bodies, and the background included, and the modified version, where only the face appears against a neutral background, removing distractions such as scenery and clothing. The dataset underwent extensive validation with human observers, 122 observers for the original clips and 280 for the modified ones. PEDFE is publicly available on the OSF platform under a public domain license, meaning it can be downloaded and used without any usage restrictions or registration. The database can be accessed by the link: https://osf.io/cynsx/files/osfstorage Declarations Author Contributions B.T.A.: conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing and editing. A. W.: methodology, software, validation, formal analysis, investigation, data curation, writing and editing. All authors have read and agreed to the published version of the manuscript. Funding This research received no external funding. Informed Consent Statement The study did not require informed consent, as the test data were numerical and defined for the tests according to the algorithm's characteristics, and did not include personal or patient data. No clinical or ethnic-demographic data were used. Conflicts of Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. References Abdullayev, R.; Uludag, O.; Celik, B. Analgesia Nociception Index: assessment of acute postoperative pain. Revista Brasileira de Anestesiologia, v. 69, n. 4, p. 396–402, 2019. https://doi.org/10.1016/j.bjan.2019.01.003 Bailly, K. Apprentissage automatique pour l’analyse des expressions faciales. Intelligence artificielle, Sorbonne Université, 2019. Available at: https://tel.archives-ouvertes.fr/tel-02489704. Accessed: 15 Sep. 2025. Ekman, P.; Friesen, W. V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, v. 17, n. 2, p. 124–129, 1971. https://doi.org/10.1037/h0030377 El‑Tallawy, S.; Pergolizzi, J. V.; Vasiliu‑Feltes, I.; Ahmed, R. S.; LeQuang, J. K.; El‑Tallawy, H. N.; Varrassi, G.; Nagiub, M. S. Incorporation of “Artificial Intelligence” for Objective Pain Assessment: A Comprehensive Review. Pain Therapy, v. 13, n. 3, p. 293–317, 2024. https://doi.org/10.1007/s40122-024-00584-8) Feighelstein, M.; Shimshoni, I.; Finka, L. R.; Luna, S. P. L.; Mills, D. S.; Zamansky, A. Automated recognition of pain in cats. Scientific Reports, v. 12, art. 9575, 2022. https://doi.org/10.1038/s41598-022-13348-1. Kelleher, E.; Kaplan, C. M.; Kheirabadi, D.; Schrepf, A.; Tracey, I.; Clauw, D. J.; Irani, A. The number of central nervous system-driven symptoms predicts subsequent chronic primary pain: evidence from UK Biobank. British Journal of Anaesthesia, v. 134, n. 3, p. 772–782, 2025. https://doi.org/10.1016/j.bja.2024.12.009 Kildal, E. S. M.; Quintana, D. S.; Szabo, A.; Tronstad, C.; Andreassen, O.; Nærland, T.; Hassel, B. Heart rate monitoring to detect acute pain in non-verbal patients: a study protocol for a randomized controlled clinical trial. BMC Psychiatry, Vol. 23, Art. 252 (2023). https://doi.org/10.1186/s12888-023-04757-1 Liu, D., Peng, F., Rudovic, O., Picard, R. DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain. Journal of Machine Learning Research, 18 (167), pp. 1-24, 2017. https://doi.org/10.5555/3122009.3242046 Nguyen, M.; Yang, H.; Kim, S.; Shin, J.; Kim, S. Transformer with Leveraged Masked Autoencoder for Video-Based Pain Assessment. arXiv preprint, arXiv:2409.05088, 2024. https://doi.org/10.48550/arXiv.2409.05088 Park, I.; Park, J. H.; Yoon, J.; Na, H. S.; Oh, A. Y.; Ryu, J. H.; Bon-Wook Koo. Machine learning model of facial expression outperforms models using analgesia nociception index and vital signs to predict postoperative pain intensity: a pilot study. Korean Journal of Anesthesiology, vol. 77, no. 2, pp. 195-204, 2024. https://doi.org/10.4097/kja.23583 Santos, A.; Pierobon, N.; Zarichen, F. A.; Wibbelt, G. L.; Bertoni, A. P. M.; Mota, C. C.; Pulcini, L. S. E.; Teixeira, S. P.; Lenhani, B. E.; Marcondes, L.; Batista, J. Eventos adversos em pacientes cirúrgicos: revisão integrativa. Research, Society and Development, vol. 10, no. 4, 2021. https://doi.org/10.33448/rsd-v10i4.13896 Sun, Y.; Wang, X.; Tang, X. Deep Convolutional Network Cascade for Facial Point Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013. https://doi.org/10.1109/CVPR.201.446 Werner, P.; Lopez-Martinez, D.; Walter, S.; Al-Hamadi, A.; Gruss, S.; Picard, R. W. Automatic Recognition Methods Supporting Pain Assessment: A Survey. IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 530-552, 2019. https://doi.org/10.1109/TAFFC.2019.2946774 Zhang, J., Shan, S., Kan, M., Chen, X. (2014). Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. Lecture Notes in Computer Science, vol 8690. (2014). https://doi.org/10.1007/978-3-319-10605-2_1 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7631185","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":526058537,"identity":"dc07a313-eb05-435e-8c8b-f50d77cad8e2","order_by":0,"name":"Bernardo Ternus de Abreu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA8UlEQVRIiWNgGAWjYBACAwjFDOPbADFj4wGitPBAGGkgLQ0kaTkMJvFqMWfvffbg5x5rOXsg4zHPn/N2a9sPA22psYnGpcWy57i5Yc+zdGMenuPmxrxtt5O3nUkEajmWltuAy2E30tgkeA4cTuyRSGOT5m24nWx2AKiFseEwbi33n7FJ/gFpkX/GJs3z51yy2fmHBLTcYAOqBNsCYrAdsDO7QcAWyx6ge2QOAP1yJo1Ncm5bcoLZDaAtCXj8Ys5+jE3yzQFrOfb2Y2wSb/7Y2ZudT3/44EONDU4tGCARrDKBWOUgYE+K4lEwCkbBKBgZAACQT1yC92EZsAAAAABJRU5ErkJggg==","orcid":"","institution":"University of Vale do Rio dos Sinos - UNISINOS","correspondingAuthor":true,"prefix":"","firstName":"Bernardo","middleName":"Ternus","lastName":"de Abreu","suffix":""},{"id":526058538,"identity":"0e2323c3-9f93-4b0c-a0b7-0cd9f0fa8efc","order_by":1,"name":"Arthur Wagner","email":"","orcid":"","institution":"University of Vale do Rio dos Sinos - UNISINOS","correspondingAuthor":false,"prefix":"","firstName":"Arthur","middleName":"","lastName":"Wagner","suffix":""}],"badges":[],"createdAt":"2025-09-16 13:53:21","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7631185/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7631185/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93843581,"identity":"b63dd2a4-7210-4a85-b20e-3c525e047515","added_by":"auto","created_at":"2025-10-18 14:29:39","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":868669,"visible":true,"origin":"","legend":"","description":"","filename":"Article16.09.docx","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/f3b0f54f88edb757362d048d.docx"},{"id":93842738,"identity":"d3d274d5-d21f-4397-9789-fcbfda1c597b","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4488,"visible":true,"origin":"","legend":"","description":"","filename":"17231e6297c542cab2f971828927d5c6.json","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/f9f13c53207593dbba9a50f7.json"},{"id":93842740,"identity":"ededd208-b20e-4125-b623-9ce1f7198f0b","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":95615,"visible":true,"origin":"","legend":"","description":"","filename":"17231e6297c542cab2f971828927d5c61enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/14f5b7ac421f8152daebf62a.xml"},{"id":93842129,"identity":"cef63fb2-3719-4a0a-bbf4-c3a3e7eb2568","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"jpeg","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":84337,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/58dff4b2f1c50842c2215b98.jpeg"},{"id":93842746,"identity":"9c42333c-d063-4219-8f30-c9d22c16330b","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":185196,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/a7aed1b178ddc932de9c4299.png"},{"id":93842137,"identity":"6b335ecb-6df4-4f9a-920e-2a3b5f263e84","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":70261,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/cb40d9ac601881e1c7675c21.png"},{"id":93843580,"identity":"99425f51-cc0e-49c5-b86d-9449a4631cb6","added_by":"auto","created_at":"2025-10-18 14:29:39","extension":"jpeg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":109728,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/1f2a03885433ac37628b0b0c.jpeg"},{"id":93842143,"identity":"975dd6d5-66f8-4460-9c55-4b7169bcc99e","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"jpeg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":114452,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/8e56e3b75882ebbb286f6acf.jpeg"},{"id":93842139,"identity":"da22f93a-24f3-4a02-95eb-cbc5f8436ae7","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"jpeg","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":154194,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage6.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/460c435a91cd276d2b604e16.jpeg"},{"id":93842742,"identity":"bdc638d0-4a43-4f9e-9461-c47be87c783e","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":58814,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/8b9b9dd381ae20243f2d90c2.png"},{"id":93842145,"identity":"262f9a2d-e346-4ea5-91ba-233211c6778f","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":17949,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/da952c8e36380fcd99f9dc60.png"},{"id":93842154,"identity":"8f27b298-9354-4909-ae2d-91676b70882e","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":69444,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/f0ec537d2fbcfbac2075370d.png"},{"id":93842144,"identity":"4007c578-8b90-434f-b3c5-9cdf0e76d039","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":27423,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/9f2ec1d18d93c91959a1bf2d.png"},{"id":93842151,"identity":"e42fe368-3eff-4db8-97b2-62a47e01d41a","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":13451,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/df2d73338c7c37121e80f4f0.png"},{"id":93842748,"identity":"b185b524-c39c-42e1-b431-1c12c034916b","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":81227,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/b700072364be98d30470e447.png"},{"id":93842147,"identity":"918dd163-779d-4946-b018-5d523b561af0","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":73394,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/7044b5919f1ac4d103c1238c.png"},{"id":93842149,"identity":"ad3de044-1558-4cca-a9ed-da711588bd54","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":216967,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/284f622516a78fdcd057e436.png"},{"id":93842146,"identity":"27494c15-f78f-4dd9-add8-0302b24d0f64","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23044,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/1d5bbb0e9afac6cdb641d6f5.png"},{"id":93842153,"identity":"d827e6ec-9989-45a7-a929-6838fb7e06a6","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5472,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/e9fb190fd2ea09bbb6d7ce58.png"},{"id":93842749,"identity":"95a2a3c7-1833-4d67-a645-129f8ea165dc","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"xml","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":93193,"visible":true,"origin":"","legend":"","description":"","filename":"17231e6297c542cab2f971828927d5c61structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/221aa5362e3feb8d755f4717.xml"},{"id":93842155,"identity":"259f5edd-8647-41b6-bb95-4870413a8651","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"html","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":102249,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/7032baf901af02855d5b5e59.html"},{"id":93843582,"identity":"2935e9f4-4493-40f0-bc8c-7f00221096bc","added_by":"auto","created_at":"2025-10-18 14:29:39","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":132049,"visible":true,"origin":"","legend":"\u003cp\u003eAlgorithm flowchart\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/8cc5c4edd3e774cd2e4f008c.png"},{"id":93842127,"identity":"1049af9c-8ace-4ef5-a3d9-0603d87cf8d8","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":190535,"visible":true,"origin":"","legend":"\u003cp\u003eNoIR V2 camera connected to Raspberry Pi 4\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/d342c9de8bff57b1ac47c7a5.png"},{"id":93842744,"identity":"1e4bf329-fb00-4e30-a0a1-a4a49690935d","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":97729,"visible":true,"origin":"","legend":"\u003cp\u003ePlastic box produced with 3D printing\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/f7090d594f94c19d1049d180.png"},{"id":93842134,"identity":"2311571b-68a2-4e91-8715-148802cde220","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":304352,"visible":true,"origin":"","legend":"\u003cp\u003eFace detection after automatic calibration\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/cb3ca5cda4a005961aac9164.png"},{"id":93842136,"identity":"63016fb8-82ed-4408-9019-3ffccf6fd767","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":291645,"visible":true,"origin":"","legend":"\u003cp\u003ePain detection by changing three or more parameters\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/68999a4f5e09d6ec9e289e47.png"},{"id":93842741,"identity":"24e9a9d4-1485-4c8d-9d0d-ce424046ac90","added_by":"auto","created_at":"2025-10-18 14:21:39","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":319477,"visible":true,"origin":"","legend":"\u003cp\u003eIntegrated system with tablet for viewing results\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/9393101cecc96fcbe55fec7c.png"},{"id":93842141,"identity":"bc056216-4c23-4384-b283-444cb1a3b79b","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":53237,"visible":true,"origin":"","legend":"\u003cp\u003eEvaluation of pain detection logic\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/cce493ab005e55dc7eef89e8.png"},{"id":93842152,"identity":"13aa4653-0de1-42b5-a929-476707979490","added_by":"auto","created_at":"2025-10-18 14:13:39","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":25923,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion Matrix\u003c/p\u003e","description":"","filename":"8.png","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/fbaa7fef40d6ceaab927f896.png"},{"id":108666736,"identity":"dad104cd-0011-406e-9735-c9f9b4e6e27d","added_by":"auto","created_at":"2026-05-07 06:42:14","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1960291,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7631185/v1/3a542e43-7f4e-4e18-88ec-84006f177d62.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Pain Expression Detection System for Non-Communicable Patients","fulltext":[{"header":"Introduction","content":"\u003cp\u003ePain is a biological response resulting from the interaction between physiological, psychological, and social mechanisms, and is therefore subjective and multifactorial. From a neurophysiological perspective, pain involves the activation of peripheral nociceptors, the transmission of afferent signals to the central nervous system, and the cortical interpretation of these stimuli. However, its perception varies among individuals, influenced by factors such as clinical history, emotional state, cultural context, and personal expectations, making clinical assessment a significant challenge. Currently, the detection of pain signals through hospital technologies has been achieved by monitoring variables linked to heart rate variability in combination with blood pressure.\u003c/p\u003e\n\u003cp\u003eCurrently, medical assistive technologies are being used for a variety of purposes, one of which is to support individuals who are temporarily unable to communicate or have communication difficulties. In a surgical setting, it is important for patients to demonstrate reactions in case of discomfort or to communicate kinesthetically or audibly if they are in pain. However, communication may be difficult, either due to a lack of speech or due to medication response. On the other hand, in many cases, the physical response of the facial muscles can be helpful in monitoring the patient\u0026apos;s response. When an individual experiences pain, a set of muscles typically moves in the mouth, eyes, cheeks, and other parts of the face.\u003c/p\u003e\n\u003cp\u003eA human\u0026apos;s appearance is made up of a significant number of muscles that allow for the production of thousands of facial expressions. These expressions are gateways to information regarding a person\u0026apos;s cognitive state and social intentions. Facial expression analysis allows for the automatic extraction of features from videos or images. With the development of computer vision, clinical studies began using algorithms of this type [2]. This article seeks to apply resources developed in the field of computer vision to tracking an individual\u0026apos;s facial movements for pain detection.\u003c/p\u003e\n\u003cp\u003eWith the development of computer vision technologies, based on intelligent algorithms that monitor patients\u0026apos; conditions, the possibilities for maintaining patient health and well-being have increased. Monitoring pain in patients is a critical challenge, especially when direct communication is limited, such as during procedures under general anesthesia. In recent decades, comparative anatomical studies of different mammals have shown that facial expressions are one of the most common and specific indicators of pain in animals [5].\u003c/p\u003e\n\u003cp\u003eThe incorporation of new technologies can be a way to reduce the occurrence of medical errors. Over the last decade, it is estimated that medical errors occurred in at least 3.5% of surgical procedures [11]. The use of technologies can help reduce this rate, which may be even higher than reported if possible underreporting rates are taken into account.\u003c/p\u003e\n\u003cp\u003eSocial communication deficits are a central feature of autism spectrum disorder (ASD). The clinical manifestations of ASD are heterogeneous, varying in intensity and form, with communication impairments ranging from mild to severe. It is estimated that approximately 30% of individuals diagnosed with ASD also have intellectual disability (ID), a neurological condition that affects approximately 1% of the general population. In cases of severe or profound ID, cerebral palsy often coexists, compromising control of the muscles involved in speech, gestures, and facial expressions. Individuals with both ASD and severe ID may therefore be unable to express the presence of pain, remaining essentially non-communicative. Given the intensive nursing care demands of this patient profile, there has been growing interest in technological solutions capable of assisting these individuals\u0026apos; communication and participation in their care settings. In general, the application of assistive technologies can contribute to promoting independence, inclusion, and quality of life in populations with complex needs [7].\u003c/p\u003e\n\u003cp\u003eHeart rate variability (HRV) monitoring has been used to identify pain signals in non-communicative patients and has the potential to guide targeted interventions. Kildal, Quintana, Szabo, Tronstad, Andreassen, N\u0026aelig;rland, and Hassel (2023) conducted a randomized controlled trial that investigated the continuous use of heart rate as a marker of acute pain in non-verbal patients with autism and intellectual disability. In this study, non-communicative patients were monitored for up to 8 hours daily during routine care to identify episodes of acute pain. Data on heart rate variability and inflammatory biomarkers such as cytokines were collected to assess long-term pain. This approach demonstrated that physiological signals can be used to guide specific clinical interventions and reduce painful episodes in the daily lives of these patients [7].\u003c/p\u003e\n\u003cp\u003eHeart rate (HR) investigation as a tool for pain management in nonverbal patients should be performed in conjunction with other established biomarkers, such as heart rate variability (HRV) and circulating inflammatory markers. HRV is a noninvasive measure that reflects autonomic nervous system activity, with higher variability being associated with lower physiological stress. Evidence from psychology studies has identified HRV as a consistent biomarker of persistent pain, in addition to its association with emotional responses. Concurrently, recent research suggests that certain circulating inflammatory cytokines may constitute reliable biomarkers of chronic pain in various clinical conditions. Among the frequently described markers are MCP-1, IL-1RA, IL-8, TGF\u0026beta;1, and IL-17, whose elevated levels have been associated with both experimental and self-reported pain, paralleling the temporal dynamics of pain episodes. Thus, the integration of HR, HRV, and inflammatory biomarkers may represent a relevant approach for the objective assessment of pain in groups of people with these characteristics [7].\u003c/p\u003e\n\u003cp\u003eThis article presents the development of an early-stage technology for patient monitoring using a camera and a microcontroller that communicate with a computer. The literature review indicated that the application of sensors in the care of nonverbal patients is not yet widespread. Much of the research on the use of sensors for pain monitoring and facilitating communication has been conducted in groups of people with preserved verbal capacity. Noncommunicative patients are often excluded from these studies, although recent advances in medical technology may provide significant benefits to this segment of the clinical population [7].\u003c/p\u003e\n\u003cp\u003eAccording to the 2030 Agenda for Sustainable Development of United Nations, health and well-being are listed as one of the 17 key objectives for societies. This objective recognizes that access to health care is a fundamental human right and seeks to reduce mortality and promote health in general. Research that uses technologies for health contributes, therefore, to this sustainable objective.\u003c/p\u003e"},{"header":"Theoretical Background","content":"\u003cp\u003e\u003cstrong\u003ePain Detection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePain is a complex phenomenon, linked to an individual\u0026apos;s sensory and emotional experience. It is the main reason people seek medical care. Treatments such as surgery, chemotherapy, and radiation therapy often cause pain. Furthermore, persistent pain has implications for individuals\u0026apos; quality of life [13].\u003c/p\u003e\n\u003cp\u003eSome patients have a higher pain tolerance than others. Despite accumulated knowledge about the characteristics of pain, it is currently poorly managed for certain groups of patients who live with this physiological manifestation. It most acutely affects patients with limited communication skills and who cannot express their pain experience. These include: infants, children, adults with cognitive impairments that affect communication, people with intellectual disabilities, seriously ill or unconscious individuals, and terminally ill individuals [13].\u003c/p\u003e\n\u003cp\u003eIn response, automatic pain recognition systems are being developed based on \u0026quot;pain behaviors,\u0026quot; which can be associated with facial expressions, vocalizations, and body movements. Physiological responses captured by signals monitored by equipment can complement current assessment methods, aiming for better continuous pain tracking. With more data, early intervention is possible, especially in patients in the aforementioned groups.\u003c/p\u003e\n\u003cp\u003eA few years ago, an index of nociceptive analgesia, called the Analgesia Nociception Index, was proposed. The ANI is calculated from a frequency-domain analysis of the high-frequency component of heart rate variability. The method also considers respiratory rate as a potential confounding factor. The index is presented as a score from 0 to 100 and reflects parasympathetic activity; that is, lower values indicate low parasympathetic activity, and higher values indicate high parasympathetic activity. Pain itself results in the predominance of sympathetic activity, so ANI values decrease with pain [1].\u003c/p\u003e\n\u003cp\u003eSeveral areas of the brain are involved in the pain process. The process begins with the activation of sensory neuronal pathways by stimuli, which activate primary sensory neurons with surface receptors specialized for detecting harmful stimuli. Action potentials are conducted by nerve fibers, and the signals result in synapses in the spinal cord [13]. When a harmful event is identified, excitatory and inhibitory interneuronal circuits in the spinal cord are activated, leading to a protective reflex withdrawal event.\u003c/p\u003e\n\u003cp\u003eData processing occurs in several supraspinal structures, which lead to the sensory perception of pain. Although stimulation generally leads to pain perception, certain conditions can hinder perception. A person can also experience pain without activation of the nociceptive pathway, which occurs in psychogenic pain [13]. Postoperative pain remains a current problem, with an estimated incidence of 30% to 71%, depending on the surgical procedure performed [10].\u003c/p\u003e\n\u003cp\u003eThe American Pain Society guidelines recommend assessing postoperative pain every 15 minutes initially and every 1 to 2 hours as the intensity decreases. Assessments are usually performed by a healthcare professional monitoring the patient. In sedated patients, assessment may be more difficult. Measurement difficulties also occur in patients with poor communication during the immediate postoperative period [10].\u003c/p\u003e\n\u003cp\u003eAmong the existing scales, the ENR and the VAS can be mentioned. The ENR consists of 11 points and is used as a reference measure for pain intensity. The ENR has significant validated correlations with other pain classification measures, such as the visual analog scale or VAS [10].\u003c/p\u003e\n\u003cp\u003eTo address this situation, changes in visual signals, such as facial expressions, and physiological indicators, such as vital signs and the analgesia index (ANI), and nociception, are measured. Currently, a combination of methods has been used, considering facial expressions, the analgesia index, and vital signs, to assess pain. The analgesia/nociception index is based on electrocardiographic data that reflect parasympathetic activity. Furthermore, the measurement does not require a healthcare professional; anyone can perform it as long as protocols are followed.\u003c/p\u003e\n\u003cp\u003ePain assessment is a critical component of healthcare and is essential for patient diagnosis and treatment. The subjective nature of pain poses challenges for healthcare teams, particularly in groups unable to communicate their discomfort effectively. Infants, nonverbal patients, and patients with cognitive impairments fall into this group [9].\u003c/p\u003e\n\u003cp\u003ePhysiological and observational measurements can be used in certain situations. For adequate pain management, assessment must be performed frequently, especially if the patient cannot call for help. Therefore, assessment tools have been developed for more standardized use by healthcare teams around the world [9].\u003c/p\u003e\n\u003cp\u003eFor expression analysis, it is not recommended to rely on \u0026quot;prototypical\u0026quot; expressions. A commonly used representation in the field of affective computing derives from Ekman\u0026apos;s work on prototypical emotions, in which the author proposed a list of universally recognized emotions: joy, anger, sadness, fear, disgust, and surprise [3]. The main problem with this representation is that it is not suitable for characterizing spontaneous facial expressions, as most human affective behaviors cannot be translated into terms of prototypical emotions [2].\u003c/p\u003e\n\u003cp\u003eThe Visual Analog Scale (VAS) is a tool used to measure the subjective intensity of sensations or feelings, especially pain. It consists of a straight line with descriptions indicating the extremes of the sensation being measured. The patient is instructed to mark a point along the line that best represents the intensity of their pain at that moment. Because it is a continuous measurement, it allows for the capture of small variations in pain perception, making it more sensitive than categorical scales such as numerical or verbal scales. The score is obtained by measuring the distance, in millimeters or centimeters, between the marked point and the extremity representing \u0026quot;no pain,\u0026quot; resulting in a value between 0 and 10.\u003c/p\u003e\n\u003cp\u003eThe VAS scale is widely accepted and has been validated as a pain assessment tool. Despite its effectiveness, the scale has some significant limitations. The VAS requires cognitive and motor comprehension skills from the patient, which can make it difficult to use with young children, people with neurological impairments, or those with communication difficulties. \u003c/p\u003e\n\u003cp\u003eScales are problematic in pediatric, geriatric, or critically ill patients, as well as in patients with communication problems or who are sedated. There have also been some identified cases of temporary postoperative delirium, which can be a confounding factor in the assessment. Therefore, the use of methods capable of assessing pain more objectively is necessary [1].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTechnologies to support pain detection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn terms of technologies for detecting physiological pain signals, a device developed in France is a case in point, having already been used in over 60 countries. It\u0026apos;s the PhysioDoloris monitor, also known as Metrodoloris, a non-invasive clinical device developed by MetroDoloris Medical Systems. It tracks heart rate signals and generates a real-time graph on the monitor screen indicating pain status. Integrated with the patient\u0026apos;s medication administration system, it can be adapted for better administration of medication doses.\u003c/p\u003e\n\u003cp\u003eThe monitor displays indicators related to the Analgesia/Nociception Index (ANI) on its screen. It was developed to assess the balance between analgesia and nociception by analyzing heart rate variability (HRV). It captures the ECG signal through a special sensor and generates a series of intervals; the analysis of these high-frequency fluctuations (between 0.15 and 0.4 Hz), which mediate parasympathetic tone, results in a numerical index between 0 and 100. Values close to 0 indicate high nociception or intense pain, while values close to 100 reflect predominant analgesia or comfort.\u003c/p\u003e\n\u003cp\u003eThe ANI index is interpreted as follows: average values between 50 and 70 are considered adequate analgesia during the anesthetic procedure. When the index drops below 50 for more than five minutes, the risk of hemodynamic reactivity increases considerably, signaling the need for analgesia adjustments. On the other hand, persistent values above 70 suggest possible analgesic overdose.\u003c/p\u003e\n\u003cp\u003eSeveral clinical studies have demonstrated the usefulness of the monitor in intrapartum and postoperative pain management. For example, a linear inverse relationship was observed between visual analog scale pain scores and the ANI, indicating its ability to reflect actual discomfort, even during uterine contractions. In pediatric patients, predefined lower ANI values correlated well with postoperative pain or elevated levels on scales such as FLACC. In situations of noxious intraoperative stimuli, the ANI proved more sensitive than traditional hemodynamic parameters such as heart rate or blood pressure.\u003c/p\u003e\n\u003cp\u003eThe use of the ANI to guide the administration of analgesics such as remifentanil resulted in a reduction in intraoperative opioid consumption and a decrease in immediate postoperative pain levels. It is important to note, however, that some conditions limit the reliability of the index, such as arrhythmias, patients with cardiac pacemakers, or the use of drugs that alter autonomic tone, such as agonists, beta-blockers, or antimuscarinics. Continuous analysis of cardiac variability allows for more precise titration of analgesics, aiming to avoid both dosages well below or above expected values, maintaining hemodynamic stability and potentially reducing postoperative complications.\u003c/p\u003e\n\u003cp\u003eDespite the monitor\u0026apos;s qualities, low-cost technologies can also be developed at the research level, aiming to reduce future technological costs for pain detection. While it is quite positive that pain detection using specific equipment is a reality in several countries, auxiliary prototypes are warranted. If successful, computer vision technologies could be tested in the future in integration with the existing monitor.\u003c/p\u003e\n\u003cp\u003eSeveral technologies have been developed to capture nonverbal pain signals using cameras and artificial intelligence algorithms. One of the most promising approaches involves the use of computer vision to analyze facial expressions, involuntary movements, and microexpressions. Systems that taxonomize human facial expressions have been integrated with machine learning and deep learning algorithms to detect patterns associated with pain.\u003c/p\u003e\n\u003cp\u003eRecently, deep neural networks (CNNs) have been applied to support medical assessment, designed to analyze pain-induced facial expressions. The developed models can extract relevant descriptors and optimize inference models based on input data.\u003c/p\u003e\n\u003cp\u003eThe field of affective computing has offered promising advances by integrating artificial intelligence (AI) with sensing technologies. This work uses facial videos to improve pain assessment, enabling better patient care with greater empathy. Traditional pain assessment methods rely primarily on self-reports, as pain is a subjective experience. However, self-reports are not always valid and reliable, which can occur when the individual is not fully aware of reality [9].\u003c/p\u003e\n\u003cp\u003eNguyen, Yang, Kim, Shin, and Kim (2024) developed a model to analyze facial videos for pain assessment, contributing to better patient care and more empathetic clinical interventions. The authors used a Transformer-based architecture composed of an Autoencoder for temporal feature extraction and a Transformer-based classifier for Multivariate Time Series Classification. The authors applied two position coding techniques: Time Absolute Position Encoding (tAPE) and Efficient Relative Position Encoding (eRPE) [9].\u003c/p\u003e\n\u003cp\u003eLiu, Peng, Rudovic, and Picard (2017) proposed a deep learning model to estimate self-reported pain levels based on facial expressions. DeepFaceLift is a model that extracts facial landmarks, sends them through a neural network, and uses Gaussian Process Regression to estimate VAS scores [8].\u003c/p\u003e\n\u003cp\u003eUnlike previous approaches that use generic classifiers, DeepFaceLIFT introduces a personalized and interpretable model that takes into account individual variations in pain perception and expression. The proposal is based on a neural network architecture that incorporates person-specific information and offers clear explanations of how the estimates are generated.\u003c/p\u003e\n\u003cp\u003eThe system is evaluated on the UNBC-McMaster Shoulder Pain Expression Archive database, known for containing annotated videos of patients with chronic shoulder pain. The results show that the model outperforms traditional methods, especially when it comes to subjective pain assessments, such as the Visual Analogue Scale (VAS). The paper also highlights the importance of interpretability in machine learning models applied to healthcare, suggesting that transparency can increase clinical professionals\u0026apos; trust in these systems.\u003c/p\u003e\n\u003cp\u003eTechnologies based on convolutional neural networks (CNNs) and deep learning have shown great potential in analyzing and interpreting facial expressions associated with pain. One of the most common approaches involves the use of computer vision algorithms, which process facial images or videos to identify specific facial features associated with pain. Pattern recognition techniques are used to detect changes in facial expressions, such as movements in the muscles around the eyes, mouth, and eyebrows, which are common indicators of pain.\u003c/p\u003e\n\u003cp\u003eAlgorithms can provide continuous pain monitoring, identifying early signs of pain events. As a potential improvement, applications can monitor pain levels throughout the day and prevent pain-related complications. In the surgical setting, algorithms can be applied to predict persistent postoperative pain after surgery. Existing technologies can monitor pain levels throughout the day and prevent pain-related complications. In the surgical setting, algorithms can be applied to predict persistent postoperative pain after surgery [4].\u003c/p\u003e\n\u003cp\u003eThe use of devices such as high-resolution cameras, infrared sensors, or 3D cameras can be integrated to improve detection accuracy by capturing subtle details of facial expressions. Combining different types of sensors and multimodal analysis are areas of interest to increase the robustness of systems. Algorithms such as support vector machines (SVMs) have also been applied to classify facial expressions of pain into different categories, providing a quantitative analysis of the detected emotions.\u003c/p\u003e\n\u003cp\u003eWerner, Al Hamadi, Niese, Walter, Gruss, and Traue (2019) developed an automatic pain recognition system using multimodal analysis, combining video signals, facial expressions, and head movements with biomedical data such as galvanic skin response (GSR), electromyography (EMG), and electrocardiogram (ECG). Using the BioVid Heat Pain Database, developed by the University of Ulm, the algorithm focuses on identifying different pain intensities caused by thermal stimuli. The model is notable for its ability to capture subtle patterns in the user\u0026apos;s facial expression and physiology, employing data fusion via the Transferable Belief Model, which integrates information over time to refine detection per frame.\u003c/p\u003e\n\u003cp\u003eThe study results demonstrate that the multimodal approach significantly outperforms methods that use only video or biomedical signals alone. The inclusion of head movements, when combined with facial expressions, results in statistically significant gains in accuracy, especially for moderate to high pain levels. The galvanic skin response (GSR) also stands out as an important physiological variable for discriminating different pain intensities. Furthermore, the authors note that personalized models, adjusted to each subject\u0026apos;s individual response, tend to perform better than generic models, highlighting the importance of considering interpersonal variations in pain expression.\u003c/p\u003e\n\u003cp\u003eThis approach does not rely on contact sensors, using only video to extract relevant behavioral patterns. This allows pain intensity to be identified with accuracy comparable to or superior to that of human observers, especially in experimentally induced acute pain settings.\u003c/p\u003e\n\u003cp\u003eOn the other hand, studies such as those by Ashraf et al. (2020), explore multimodal models that combine visual data with physiological signals, such as electrocardiogram (ECG), skin conductance (SDI), electromyography (EMG), and remote photoplethysmography (rPPG). This fusion of modalities typically improves the accuracy of pain detection in real clinical settings. Nevertheless, the authors note that even purely visual analysis, using CNNs, can achieve high levels of accuracy in detecting acute pain, often sufficient in scenarios where the use of physical sensors is not feasible or desirable. Furthermore, they show that networks trained solely on facial signals can perform excellently when well-tuned, particularly in databases where nonverbal pain patterns are clearly evident.\u003c/p\u003e\n\u003cp\u003eThere are emerging commercial solutions on the market, such as the PainChek system. Approved in some countries for clinical use, this technology uses facial recognition with artificial intelligence support to assess signs of pain in patients with communication difficulties, such as elderly people with specific conditions of dementia or patients under sedation.\u003c/p\u003e\n\u003cp\u003eAlthough still undergoing broader clinical validation, these technologies indicate a promising direction for the automation and standardization of intraoperative pain monitoring. They can complement clinical judgment and support anesthesiology, promoting greater safety and personalized pain control during surgical procedures. Continued research in this field, especially focusing on surgical center environments, should develop integration between video systems, artificial intelligence, and multiparameter monitors, aiming for better patient care.\u003c/p\u003e\n\u003cp\u003eIn the field of facial expression analysis, early deep learning-based methods followed the principle of cascade alignment. This approach involves iteratively estimating the positions of facial feature points based on displacements learned from an annotated training base. Each stage of the cascade aims to refine previous estimates, starting from a rough initial prediction and reaching higher accuracy.\u003c/p\u003e\n\u003cp\u003eWith the popularization of convolutional neural networks (CNNs), new models have been proposed to replace traditional cascade stages. Sun et al. (2013) presented a cascade model based on deep convolutional networks, in which each regression stage, composed of distinct feature extraction and mapping steps, was replaced by a specialized CNN. This proposal demonstrated superior performance to classical approaches, highlighting the potential of CNNs for the task of face alignment [12].\u003c/p\u003e\n\u003cp\u003eSubsequently, Zhang, Jie et al. (2014) proposed CFAN, a cascade of deep auto-encoders. The main idea was to use a sequence of trained autoencoders to progressively refine estimates of the position of facial points, allowing identification in different lighting, pose, and expression conditions [14]. The project developed by the authors consists in a model based on deep multi-task learning, in which the facial landmark detection process was coupled with the simultaneous recognition of facial attributes such as gender, age, and emotion. This integration helped the model learn more discriminative representations and accelerated inference time by sharing convolutional layers across tasks [14]. More recently, the same authors expanded their approach with an even more efficient and accurate model, combining multiple convolutional networks to more robustly predict the location of facial landmarks across different transitions. This approach also aimed to improve processing time by adjusting the approximations between cascade steps and the spatial relationship between landmarks [14].\u003c/p\u003e\n"},{"header":"Materials and Methods","content":"\u003cp\u003eFor this project, a Python algorithm was developed consisting of a pre-trained neural network adapted to capture facial features. The algorithm, run by the computer, receives data from a camera connected to a Raspberry Pi microcontroller. The project is based on analyzing variations in specific facial regions, using the OpenCV library to display text and draw information about the captured frames. The MediaPipe library is used to detect facial features or \"landmarks.\"\u003c/p\u003e\u003cp\u003eThe algorithm begins with (1) the camera frame capture step. This captures a new frame from the video transmitted by the camera via IP. The second step is (2) frame preprocessing, which horizontally mirrors the frame and converts it from BGR, the OpenCV standard, to RGB, as required by MediaPipe. The third step is (3) processing with the FaceMesh model, which passes the frame to the \"face_mesh\" model, which returns the detected facial landmarks, with refined points for the eyes and iris. The fourth step is (4) result manipulation in case of detection, which accesses the detected points and stores the x and y coordinates of each landmark. Next, the algorithm (5) performs data processing, which involves calculating point distances, distance normalization, calibration logic, and pain detection. This step is within a video processing loop.\u003c/p\u003e\u003cp\u003eThe code was developed in Python and hosted on the Github repository. The algorithm structure can be found at the following link: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/beternus/pain-detector/blob/main/main.py\u003c/span\u003e\u003cspan address=\"https://github.com/beternus/pain-detector/blob/main/main.py\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e presents a flowchart of the algorithm structure, relating its main steps.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eInitially, the code defines and initializes the Flask server, which runs in a parallel thread. This configuration ensures that the server continues to run during the video processing stage. The \"video_feed()\" function creates a route that continuously sends processed frames to connected clients, using the multipart response type required for real-time video streaming. Additionally, there is a \"/frame\" route that allows access to the last captured frame in JPEG image format. To this end, the last processed frame is stored in a global variable (\"last_frame\") and converted to a JPEG buffer to be sent as an HTTP response.\u003c/p\u003e\u003cp\u003eThe code integrates a web interface using the Flask framework, allowing the processed video to be streamed to a browser or client application. The process occurs through a specific HTTP route that continuously sends the annotated frames, encoded in JPEG. Another endpoint allows the capture of a current static frame, useful for specific analysis. This entire system operates in parallel, with the graphical interface displayed locally by OpenCV and the web server running in a separate thread, allowing simultaneous analysis and visualization.\u003c/p\u003e\u003cp\u003eThe system can capture video from an IP source (such as a smartphone streaming over Wi-Fi), process the frames using MediaPipe's facial mesh model, perform a calibration of facial features at rest, measure variations in the positions of specific facial points over time, and, based on heuristic criteria, identify possible expressions of pain. It also makes the processed frames available through a Flask server, allowing remote access via a browser or app.\u003c/p\u003e\u003cp\u003eThe algorithm uses \"IP streaming\" video input via \"cv2.VideoCapture\" with a URL indicating that the video source could be a smartphone using applications such as IP Webcam. This eliminates the need for a camera connected to a computer and makes the system more flexible and portable, allowing facial capture to be performed remotely. The \"cv2.flip(frame, 1)\" function is used to horizontally flip the captured image.\u003c/p\u003e\u003cp\u003eThe code uses the \"cv2.putText\" function to overlay the percentage variations detected in the cheeks, lips, and mouth opening onto the frame. These values are dynamically extracted and formatted to one decimal place, and are drawn using a simple font \"cv2.FONT_HERSHEY_SIMPLEX,\" with green indicating normality or absence of pain and a line width of 2. The data visually informs the user of the behavior of the monitored facial regions in real time.\u003c/p\u003e\u003cp\u003eThese values include the variation of the right eyebrow, left and right eyes, left and right cheeks, and two measurements related to the mouth: the distance between the lips and the mouth opening. The variations are converted to text and drawn on the image at specific positions.\u003c/p\u003e\u003cp\u003eAfter capturing and converting the frames to RGB format, the system identifies the face and extracts the previously defined points of interest from a dictionary called \"LANDMARKS.\" These points include pupils, eyebrows, eyes, cheeks, lips, mouth, and more. Using this data, normalized relative distances are calculated.\u003c/p\u003e\u003cp\u003eIn the system's operation, for each frame captured from the camera, the algorithm converts the image to RGB format and sends it to the \"face_mesh.process()\" model. If successful, this returns a structure containing the face's landmarks. These points are mapped to pixel coordinates based on the frame resolution. The code selects specific landmarks based on predefined parameters, identifying parts of the face. From these points, the system calculates several Euclidean measurements, such as the vertical distance between the eyebrow and forehead, eye opening, cheek retraction, lip opening, and the distance between the pupils. These measurements are normalized to facial dimensions such as height, width, and interpupillary distance, to ensure robustness across different face sizes and camera positions.\u003c/p\u003e\u003cp\u003eThe algorithm performs logical comparisons to determine whether the variations exceed pre-established threshold values, indicating patterns consistent with an expression of pain. Eyebrow raising is detected if both present a variation equal to or greater than 8%, while eye twitching is considered when their variations are less than \u0026minus;\u0026thinsp;15%. Cheek retraction is detected with values below \u0026minus;\u0026thinsp;8%, and lip twitching with a variation less than \u0026minus;\u0026thinsp;25%. Each of these conditions returns a Boolean value indicating whether the criterion is met. Mouth opening is analyzed as significant and potentially indicative of pain if its value exceeds 3,500 units, likely linked to the distance between landmarks.\u003c/p\u003e\u003cp\u003eIf at least three of these facial regions are \"altered\" at the same time, the system concludes that there is an \"expression of pain present\" and displays an indicative message on the video. The algorithm uses the \"refine_landmarks\u0026thinsp;=\u0026thinsp;True\" command to refine the location of critical landmarks, such as pupils. This improves the accuracy of interpupillary measurements and eye tracking, which is relevant for analyses related to eye closure.\u003c/p\u003e\u003cp\u003eThe algorithm uses normalization for different measurements. Vertical distances, such as eyebrows and mouth, are normalized by face height, while horizontal distances, such as cheeks, are divided by face width, and measurements between the eyes are normalized by interpupillary distance. Defining the normalization basis for each region ensures that the analysis is proportional and consistent with facial anatomy, avoiding distortions.\u003c/p\u003e\u003cp\u003eThe system builds its decision criteria based on a sort of \"voting\" among facial regions: if three or more of the five regions evaluated present variations consistent with the expression of pain, then the pain is identified as real. The voting-based approach makes the system less susceptible to isolated false positives, requiring a more robust standard for detection alerts.\u003c/p\u003e\u003cp\u003eAn important step in the algorithm is \"calibration,\" which occurs within the first few seconds after the application begins. During this period, the user must remain still, allowing the system to record reference measurements of their neutral facial expression. These baseline average values are then used as a comparison point to detect expressive variations in subsequent frames. Over time, the system compares the new measurements with these initial averages, calculating \"percentage changes\" that indicate how far each part of the face has deviated from the neutral condition.\u003c/p\u003e\u003cp\u003eThe calibration process is automatically activated at the start of the runtime and aims to capture the neutral state of the user's face. Over a period of approximately 8 seconds, the system collects measurements of the mentioned facial regions and, at the end of the time, calculates the average of each of these distances. These averages are stored and used as a reference to identify variations over time. After calibration, the system enters detection mode. With each new frame, it calculates the current distances of the facial regions and compares them with the calibrated average values, generating variation percentages.\u003c/p\u003e\u003cp\u003eIf at least three of these five conditions are simultaneously true, the system interprets the situation as a possible expression of pain. In this case, it displays a red alert in the frame: \"Expression of PAIN detected!\" (\u0026ldquo;Express\u0026atilde;o de DOR detectada!\u0026rdquo;). The alert message is superimposed on the image, informing that the expression has been detected. The text is displayed in red and thicker to stand out visually. The algorithm displays a side legend that individually shows each facial criterion, such as \"Eyebrows,\" \"Eyes,\" and \"Mouth,\" colored red if the pain condition is present or green if it is not.\u003c/p\u003e\u003cp\u003eIf the option to display facial landmark indices is enabled, as expressed by the \"mostrar_indices\" variable, the algorithm iterates through all detected facial landmarks and plots their indices directly on the image, allowing visual and numerical verification of each landmark's position. The system displays the numerical indices of each facial landmark on the image, which is useful for calibrating the facial mesh.\u003c/p\u003e\u003cp\u003eThe final rendering of the facial mesh is done using the \"mp_drawing.draw_landmarks\" function, which traces contours and landmarks of the face with specific colors and styles. The frame with these annotations is stored as \"last_frame,\" ensuring that the most recently processed image is available even if detection is interrupted.\u003c/p\u003e\u003cp\u003eThe facial contours are drawn onto the frame using MediaPipe's facial mesh model, with custom visual specifications to highlight the mesh points and lines. This provides a detailed graphical representation of the facial structure recognized by the system.\u003c/p\u003e\u003cp\u003eMediaPipe uses a model that uses neural networks trained to locate 468 specific points on the face. The density of points allows for detailed measurements. Normalizing distances is a strategy to reduce the impact of different face sizes and camera distances while maintaining proportionality. MediaPipe doesn't perform this task deterministically or based on classical geometric algorithms. It uses \"deep neural networks\" previously trained on large databases of facial images.\u003c/p\u003e\u003cp\u003eInternally, face detection and regression of facial mesh points are performed by a machine learning pipeline. This pipeline initially consists of a \"face detection\" step, responsible for identifying the presence and location of the face in the image, followed by a \"landmark regression\" step, which estimates the three-dimensional (x, y, z) position of facial points based on the cropped image of the detected face.\u003c/p\u003e\u003cp\u003eThe architecture of these neural networks is based on optimized convolutional models, specifically designed to be lightweight and fast enough to run even on mobile devices. When the \"refine_landmarks\u0026thinsp;=\u0026thinsp;True\" option is enabled, as in the code presented, an additional model comes into play to further improve the accuracy of points located in critical regions such as the eyes and irises. This feature enables refined pupil detection, an extremely important factor in the algorithm in question, which uses interpupillary distance as a reference metric for data normalization.\u003c/p\u003e\u003cp\u003eAt the end of each iteration, the annotated image is updated and stored as the last frame processed. In application control, the system responds to keyboard commands. The \"ESC\" key ends the program; \"M\" toggles the display of facial point indices; the space bar (\u0026ldquo;SPACE\u0026rdquo;) pauses or resumes processing; and \"R\" starts a new calibration. Finally, the application continuously displays the processed result in a window called \"Expression Detection,\" updating with each captured frame, unless paused. At the end of the code, there is a command to release and close all capture resources and windows opened by OpenCV, aiming for the completion of execution.\u003c/p\u003e\u003cp\u003eThe algorithm is an application of computer vision which uses neural networks encapsulated in the MediaPipe library to extract facial representations. While the system does not train or fine-tune neural networks on its own, it directly benefits from the power of optimized pre-trained models, which enable the extraction of facial data in real time. From this data, the code performs a quantitative analysis of facial variations, creating a tool for detecting expressions of pain, with potential use in medical contexts.\u003c/p\u003e\u003cp\u003eThe code implements a neural network for facial expression detection using MediaPipe in conjunction with OpenCV and Flask. The overall structure is based on a Flask server that displays the processed video feed in real time, with facial expression analysis performed using MediaPipe's FaceMesh. The neural network used in MediaPipe for facial detection, specifically FaceMesh, is a pre-trained deep convolutional network provided as a resource in the library, which identifies a certain number of facial features.\u003c/p\u003e\u003cp\u003eInitially, the code configures the MediaPipe library with the \"FaceMesh\" function, which detects and tracks specific facial features, such as eyebrows, eyes, mouth, cheeks, and more. The model uses a pre-trained neural network to identify up to 468 facial features on a single face, enabling detailed analysis of facial features. MediaPipe performs real-time detection with high accuracy using images captured by the camera.\u003c/p\u003e\u003cp\u003eThe code also includes an interface control to pause execution, show or hide the indices of the facial landmarks, and restart calibration. Facial expressions are continuously monitored, and detecting certain specific facial movement conditions, such as raised eyebrows or closed eyes, triggers alerts for expressions like pain.\u003c/p\u003e\u003cp\u003eMediaPipe's neural network is efficiently used to provide highly accurate analysis of facial emotions, while Flask communicates the results to the user through a real-time web interface. By combining these technologies, the system enables dynamic assessment of facial expressions, with the ability to calibrate and adjust over time.\u003c/p\u003e\u003cp\u003eThe neural network used in MediaPipe for facial feature detection, called FaceMesh, is a convolutional neural network (CNN) specialized in tracking and identifying facial features in images and videos. MediaPipe is a framework developed by Google that uses deep learning and computer vision techniques to solve problems such as real-time facial feature tracking. FaceMesh's architecture is designed to be highly efficient, enabling real-time detection with low resource consumption, making it ideal for mobile devices and environments with hardware limitations.\u003c/p\u003e\u003cp\u003eAt the heart of FaceMesh is a convolutional neural network trained to detect 468 key points on the human face, distributed across specific facial regions, such as the eyebrows, eyes, nose, mouth, and chin. Each of these landmarks is mapped into 3D space, allowing the model to understand facial depth and capture nuances in the position of the points at different angles and expressions. The network is trained on large databases containing millions of facial images, providing good generalization across different face types, ages, ethnicities, and lighting conditions.\u003c/p\u003e\u003cp\u003eThe network structure consists of several convolutional layers responsible for extracting facial features at different scales. These convolutional layers are followed by pooling layers, which help reduce the dimensionality of the information and focus on the most important aspects of the image. After these convolutional and pooling layers, the network passes through fully connected layers, which are responsible for refining the detection and mapping the extracted information to the spatial coordinates of the facial features.\u003c/p\u003e\u003cp\u003eA key feature of FaceMesh is its ability to operate very quickly and efficiently. This is possible thanks to the use of lightweight neural networks and optimization techniques such as quantization and layer fusion. Furthermore, MediaPipe uses a pipeline architecture that divides processing into stages, allowing detection to be performed in a modular and scalable manner. The network is also optimized to work on resource-constrained devices, such as smartphones and embedded systems, without compromising detection accuracy.\u003c/p\u003e\u003cp\u003eFaceMesh is designed to be robust and adaptable to different conditions, such as varying lighting, rapid movements, and even partial occlusions of the face. The neural network is able to handle these challenges thanks to its large-scale training and its ability to learn complex facial representations, such as skin deformation, facial expressions, and viewing angles. This makes it a powerful tool for applications in areas such as augmented reality, emotion analysis, biometrics, and gesture-based interaction systems.\u003c/p\u003e\n\u003ch3\u003eMicrocontroller and Camera\u003c/h3\u003e\n\u003cp\u003eReal-time data is captured by a camera connected to a Raspberry Pi microcontroller. The microcontroller is connected to a Noir V2 camera. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the camera connected by a flat cable, which is inserted into a terminal on the Raspberry Pi electronics board.\u003c/p\u003e\u003cp\u003eThe Raspberry Pi 4 Model B is a single-board microcomputer developed by the Raspberry Pi Foundation, which features a Broadcom BCM2711 processor, a 64-bit quad-core Cortex-A72 (ARM v8), operating at 1.5 GHz, which provides significant improvements in speed and processing capacity compared to previous generations.\u003c/p\u003e\u003cp\u003eThe Raspberry Pi 4 is available in versions with 2 GB, 4 GB, or 8 GB of RAM, allowing you to run full Linux-based operating systems and multitasking applications. In terms of connectivity, the Raspberry Pi 4 includes two USB 3.0 ports, two micro-HDMI ports, gigabit Ethernet, Wi-Fi, and Bluetooth 5.0. The board has a 40-pin GPIO connector for integration with external sensors and modules, making it suitable for scientific prototyping. The Raspberry Pi 4 allows you to run computer vision algorithms locally with Python and OpenCV.\u003c/p\u003e\u003cp\u003eThe NoIR V2 camera is a Raspberry Pi module designed for applications involving night vision or low-light imaging. It uses a Sony IMX219 sensor with 8 megapixel resolution, capable of capturing photos up to 3280 x 2464 pixels and videos in Full HD (1080p) resolution at 30 fps. It also allows for 720p and 640x480 video capture at higher frame rates. The main feature that differentiates the NoIR V2 from the standard version is the absence of an infrared filter (IR-cut filter), which makes it sensitive to near-infrared light (near-IR). The camera connects directly to the Raspberry Pi 4's CSI (Camera Serial Interface) connector using a flat cable, making it a suitable option for embedded computer vision projects.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThe Raspberry Pi 4 was placed in a plastic case that was 3D printed to the appropriate dimensions to accommodate the microcontroller. A 3D printer was used to build the plastic case. This method consists of an additive manufacturing technology, in which objects are created layer by layer from a digital model. The FDM (Fused Deposition Modeling) method was used, which melts plastic filament to build the object. After modeling the plastic case in the design software, the plastic compounds were inserted into the system for melting and gradually forming the final shape of the case, which includes camera ports.\u003c/p\u003e\u003cp\u003e3D prototyping is recommended for experimental and educational projects, as was the case with this one. For large-scale production, the method may not be the most cost-effective, but it adequately met the project's needs in its initial design. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the plastic box isolated from the other parts of the project. It has a flexible rod for inserting the camera, allowing it to be moved in different directions.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e"},{"header":"Results and Discussion","content":"\u003cp\u003eWhen executed, the algorithm running on the computer receives data from the camera. When a human face is located, it calibrates the system for a few seconds and then displays green text and numeric indications on the left and right sides of the screen. Figure 4 shows the face detection result after automatic calibration, which occurs within 8 seconds.\u003c/p\u003e\n\u003cp\u003eAfter identifying the face, the algorithm tracks changes in distances between points and performs a summation. If three or more parameters exhibit the minimum changes defined by the user, a red message indicating pain detection is displayed on the screen. Figure 5 shows the detection result.\u003c/p\u003e\n\u003cp\u003eTests with the algorithm confirmed that it was possible to transmit camera images to peripheral monitors. A tablet was used in the project to receive video data, and data capture was possible with a low delay between the images displayed on the computer and the tablet. Figure 6 shows the images with the indications displayed on both devices. The computer processes the data captured from the camera and sends it to the tablet. This option may be recommended for monitoring a patient in a surgical room where more than one monitor is required, with the screens located in different locations.\u003c/p\u003e\n\u003cp\u003eA test of the algorithm\u0026apos;s decision logic was developed using numerical data defined for the test. This was an initial test of the algorithm\u0026apos;s logic, and a future step will involve testing with labeled videos from a free database.[1] The metrics found with the numerical test were:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e1\u003c/strong\u003e\u003cstrong\u003e.\u003c/strong\u003e Algorithm evaluation through metrics\u003c/p\u003e\n\u003cdiv align=\"Left\"\u003e\n \u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"249\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 109px;\"\u003e\n \u003cp\u003eMetrics\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 140px;\"\u003e\n \u003cp\u003eResult\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e0.990476\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e0.568306\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eF1-score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e0.722222\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e0.777778\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eTrue Positives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e104.000000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eFalse Positives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e1.000000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eTrue Negatives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e176.000000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 109px;\"\u003e\n \u003cp\u003eFalse Negatives\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 140px;\"\u003e\n \u003cp\u003e79.000000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eThe algorithm\u0026apos;s decision logic was evaluated by monitoring its response over a period of frames during which there were pain intervals. The algorithm should track the pain intervals and be sensitive to these signals. The response is expressed in the following graph:\u003c/p\u003e\n\u003cp\u003eThe model achieved a precision of 0.99, indicating that 99% of the times the algorithm detected pain, it detected it correctly. This result suggests that the system is effective in avoiding false positives, meaning it rarely flags pain when it is not present. However, when analyzing sensitivity or recall, the value of 0.568 indicates that the algorithm correctly identified only about 56.8% of the instances in which pain actually occurred.\u003c/p\u003e\n\u003cp\u003eThe low recall value indicates that many actual pain episodes were missed, considering that the test yielded 79 false negatives. This can be critical in sensitive applications, such as clinical settings or automated patient monitoring. The F1-score metric, representing the harmonic mean between precision and recall, presented a value of 0.722, reflecting the balance between the two metrics and reinforcing the limitation in the complete detection of pain episodes. The model\u0026apos;s overall accuracy was 0.778, revealing that 77.8% of all classifications, both positive and negative, were correct.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe results indicate that the algorithm is highly reliable in detecting the presence of pain, but it still fails to recognize many instances of actual pain. The algorithm is a didactic study and is not applicable to clinical or critical cases, but can be used for educational purposes by students. Improvements to the algorithm are expected to improve detection criteria and decision thresholds to increase sensitivity, even if this results in an increase in the false positive rate.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFinal Considerations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe algorithm was able to detect pain in initial tests as planned. However, the algorithm needs to be improved for further tests. The results indicate that the algorithm is highly reliable in detecting the presence of pain, but it still fails to recognize many instances of actual pain. Improvements to the algorithm are expected to improve detection criteria and decision thresholds to increase sensitivity, even if this results in an increase in the false positive rate.\u003c/p\u003e\n\u003cp\u003eA literature review of existing technologies revealed the use, over a decade ago, of a monitor that detects pain in patients based on heart rate variables. This monitor is the PhysioDoloris monitor, also known as Metrodoloris, initially developed in France. Computational vision technologies can be integrated with existing technologies in future research contexts. On the other hand, there are situations where camera monitoring can be indicated, such as monitoring the health of babies in hospitals. However, for this, it is necessary to develop more robust technologies than the algorithm presented if the application occurs in real scenarios.\u003c/p\u003e\n\u003cp\u003eThe data processing of project was performed by the computer, not the Raspberry Pi 4 itself, which merely captures the signals and sends them to the computer. It has not yet been possible to use the Raspberry Pi 4 to process data captured by the attached camera. Future steps include implementing more executable functions on the Raspberry Pi 4\u0026apos;s processor.\u003c/p\u003e\n\u003cp\u003eThe application has potential for future expansion, whether by incorporating machine learning models trained on expression datasets or introducing audio feedback or real-time alerts. The modular architecture and use of widely adopted libraries make this technical evolution feasible with few structural changes. The algorithm can be improved in the future to track the movement of certain body parts, such as hands. Studies indicate that, in addition to facial expressions, vocalizations and body part movements are also indicative of pain. Therefore, algorithms connected to cameras could be used to track patients\u0026apos; body movements in a way that\u0026apos;s more sensitive to their physiological responses.\u003c/p\u003e\n\u003cp\u003eOne of the downsides of video-based monitoring is that it\u0026apos;s considered more intrusive than heart rate or other variable monitoring. For it to be used, the patient must authorize the monitoring, meaning consent must be provided, and all caregivers must be informed if the patient is with caregivers. Therefore, there are ethical implications. On the other hand, just as heart rate monitoring is recommended in some situations, especially surgical ones, it also requires additional monitoring with cameras and/or sensors.\u003c/p\u003e\n\u003cp\u003eThe algorithm demonstrated high precision (0.99), indicating strong reliability in identifying pain without generating false positives. However, its recall was moderate (0.57), missing a significant number of real pain events (79 false negatives). The F1-score (0.72) and accuracy (0.78) reveal a reasonable balance between precision and recall, though with limitations in sensitivity. These results suggest that while the model is effective at confirming pain when detected, it fails to identify all occurrences. Despite this, the system holds potential as a didactic tool for educational purposes.\u003c/p\u003e\n\u003cdiv id=\"ftn1\"\u003e\n \u003cp\u003e[1] The PEDFE, or \u0026quot;Padova Emotional Dataset of Facial Expressions,\u0026quot; is a database developed to provide standardized material for the study and recognition of facial expressions. PEDFE brings together videos that record various human facial expressions. The database contains 1,458 clips of facial expressions, which were recorded under controlled laboratory conditions. Participants displayed both spontaneous (genuine) expressions and simulated or acted-out expressions, covering the six universally recognized basic emotions: happiness, sadness, anger, fear, surprise, and disgust. Therefore, the videos present both truly felt and staged emotions. PEDFE data is systematically labeled, enabling its use in machine learning tasks such as classification, micro expression detection, and temporal analysis of emotional intensity. The database includes 1,458 clips, 707 genuine and 751 staged, obtained from 56 participants. Two versions are available: the original, with the participants, their bodies, and the background included, and the modified version, where only the face appears against a neutral background, removing distractions such as scenery and clothing. \u0026nbsp;The dataset underwent extensive validation with human observers, 122 observers for the original clips and 280 for the modified ones. PEDFE is publicly available on the OSF platform under a public domain license, meaning it can be downloaded and used without any usage restrictions or registration. The database can be accessed by the link: https://osf.io/cynsx/files/osfstorage\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAuthor Contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eB.T.A.: conceptualization, methodology, software, validation, formal analysis, investigation, data curation, writing and editing. A. W.: methodology, software, validation, formal analysis, investigation, data curation, writing and editing. All authors have read and agreed to the published version of the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis research received no external funding.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInformed Consent Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study did not require informed consent, as the test data were numerical and defined for the tests according to the algorithm's characteristics, and did not include personal or patient data. No clinical or ethnic-demographic data were used.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of Interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAbdullayev, R.; Uludag, O.; Celik, B. Analgesia Nociception Index: assessment of acute postoperative pain. Revista Brasileira de Anestesiologia, v. 69, n. 4, p. 396\u0026ndash;402, 2019. https://doi.org/10.1016/j.bjan.2019.01.003\u003c/li\u003e\n\u003cli\u003eBailly, K. Apprentissage automatique pour l\u0026rsquo;analyse des expressions faciales. Intelligence artificielle, Sorbonne Universit\u0026eacute;, 2019. Available at: https://tel.archives-ouvertes.fr/tel-02489704. Accessed: 15 Sep. 2025.\u003c/li\u003e\n\u003cli\u003eEkman, P.; Friesen, W. V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, v. 17, n. 2, p. 124\u0026ndash;129, 1971. https://doi.org/10.1037/h0030377\u003c/li\u003e\n\u003cli\u003eEl‑Tallawy, S.; Pergolizzi, J. V.; Vasiliu‑Feltes, I.; Ahmed, R. S.; LeQuang, J. K.; El‑Tallawy, H. N.; Varrassi, G.; Nagiub, M. S. Incorporation of \u0026ldquo;Artificial Intelligence\u0026rdquo; for Objective Pain Assessment: A Comprehensive Review. Pain Therapy, v. 13, n. 3, p. 293\u0026ndash;317, 2024. https://doi.org/10.1007/s40122-024-00584-8)\u003c/li\u003e\n\u003cli\u003eFeighelstein, M.; Shimshoni, I.; Finka, L. R.; Luna, S. P. L.; Mills, D. S.; Zamansky, A. Automated recognition of pain in cats. Scientific Reports, v. 12, art. 9575, 2022. https://doi.org/10.1038/s41598-022-13348-1.\u003c/li\u003e\n\u003cli\u003eKelleher, E.; Kaplan, C. M.; Kheirabadi, D.; Schrepf, A.; Tracey, I.; Clauw, D. J.; Irani, A. The number of central nervous system-driven symptoms predicts subsequent chronic primary pain: evidence from UK Biobank. British Journal of Anaesthesia, v. 134, n. 3, p. 772\u0026ndash;782, 2025. https://doi.org/10.1016/j.bja.2024.12.009\u003c/li\u003e\n\u003cli\u003eKildal, E. S. M.; Quintana, D. S.; Szabo, A.; Tronstad, C.; Andreassen, O.; N\u0026aelig;rland, T.; Hassel, B. Heart rate monitoring to detect acute pain in non-verbal patients: a study protocol for a randomized controlled clinical trial. BMC Psychiatry, Vol. 23, Art. 252 (2023). https://doi.org/10.1186/s12888-023-04757-1\u003c/li\u003e\n\u003cli\u003eLiu, D., Peng, F., Rudovic, O., Picard, R. DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain. Journal of Machine Learning Research, 18 (167), pp. 1-24, 2017. https://doi.org/10.5555/3122009.3242046\u003c/li\u003e\n\u003cli\u003eNguyen, M.; Yang, H.; Kim, S.; Shin, J.; Kim, S. Transformer with Leveraged Masked Autoencoder for Video-Based Pain Assessment. arXiv preprint, arXiv:2409.05088, 2024. https://doi.org/10.48550/arXiv.2409.05088\u003c/li\u003e\n\u003cli\u003ePark, I.; Park, J. H.; Yoon, J.; Na, H. S.; Oh, A. Y.; Ryu, J. H.; Bon-Wook Koo. Machine learning model of facial expression outperforms models using analgesia nociception index and vital signs to predict postoperative pain intensity: a pilot study. Korean Journal of Anesthesiology, vol. 77, no. 2, pp. 195-204, 2024. https://doi.org/10.4097/kja.23583\u003c/li\u003e\n\u003cli\u003eSantos, A.; Pierobon, N.; Zarichen, F. A.; Wibbelt, G. L.; Bertoni, A. P. M.; Mota, C. C.; Pulcini, L. S. E.; Teixeira, S. P.; Lenhani, B. E.; Marcondes, L.; Batista, J. Eventos adversos em pacientes cir\u0026uacute;rgicos: revis\u0026atilde;o integrativa. Research, Society and Development, vol. 10, no. 4, 2021. https://doi.org/10.33448/rsd-v10i4.13896\u003c/li\u003e\n\u003cli\u003eSun, Y.; Wang, X.; Tang, X. Deep Convolutional Network Cascade for Facial Point Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013. https://doi.org/10.1109/CVPR.201.446\u003c/li\u003e\n\u003cli\u003eWerner, P.; Lopez-Martinez, D.; Walter, S.; Al-Hamadi, A.; Gruss, S.; Picard, R. W. Automatic Recognition Methods Supporting Pain Assessment: A Survey. IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 530-552, 2019. https://doi.org/10.1109/TAFFC.2019.2946774\u003c/li\u003e\n\u003cli\u003eZhang, J., Shan, S., Kan, M., Chen, X. (2014). Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision \u0026ndash; ECCV 2014. Lecture Notes in Computer Science, vol 8690. (2014). https://doi.org/10.1007/978-3-319-10605-2_1\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Algorithm, Communication, Detection, Pain, Patients","lastPublishedDoi":"10.21203/rs.3.rs-7631185/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7631185/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis paper presents a system for detecting facial expressions of pain using a computer vision algorithm and a camera connected to a microcontroller. The prototype was designed to alert physicians to facial expressions of pain in patients during surgical procedures. Currently, the detection of pain signals through hospital technologies has been done by monitoring variables linked to heart rate variability in combination with blood pressure. New technologies have been developed using cameras and algorithms that detect patient movements and facial expressions, aiming to obtain more data for patient assessment by the medical team. Maintaining the well-being of hospitalized individuals is a challenge in cases of non-communicative or poorly communicative patients, such as infants, adults with cognitive impairments that affect communication, or some elderly patients. The algorithm demonstrated high precision (0.99), indicating strong reliability in identifying pain without generating false positives. However, its recall or sensitivity was moderate (0.57), missing a significant number of real pain events (79 false negatives). The F1-score (0.72) and accuracy (0.78) reveal a reasonable balance between precision and recall, though with limitations in sensitivity. These results suggest that while the model is effective at confirming pain when detected, it fails to identify all occurrences. Despite this, the system holds potential as a didactic tool for educational purposes.\u003c/p\u003e","manuscriptTitle":"Pain Expression Detection System for Non-Communicable Patients","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-18 14:13:34","doi":"10.21203/rs.3.rs-7631185/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"fe771aac-c583-48e8-bd62-9d8c06f26525","owner":[],"postedDate":"October 18th, 2025","published":true,"recentEditorialEvents":[{"type":"decision","content":"Rejected","date":"2026-05-07T06:28:35+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-07T06:40:34+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-18 14:13:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7631185","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7631185","identity":"rs-7631185","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}