Fourier-enhanced multi-modal 3D small object optical mark recognition and positioning method for percutaneous abdominal puncture surgical navigation

Fourier-enhanced multi-modal 3D small object optical mark recognition and positioning method for percutaneous abdominal puncture surgical navigation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Fourier-enhanced multi-modal 3D small object optical mark recognition and positioning method for percutaneous abdominal puncture surgical navigation Zezhao Guo, Yanzhong Guo, Zhanfang Zhao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4488932/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Navigation for thoracoabdominal puncture surgery is used to locate the needle entry point on the patient's body surface. The traditional reflective ball navigation method is difficult to position the needle entry point on the soft, irregular, smooth chest and abdomen. Due to the lack of clear characteristic points on the body surface using structured light technology, it is difficult to identify and locate arbitrary needle insertion points. Based on the high stability and high accuracy requirements of surgical navigation, this paper proposed a novel method, a muti-modal 3D small object medical marker detection method, which identifies the center of a small single ring as the needle insertion point. Moreover, this novel method leverages Fourier transform enhancement technology to augment the dataset, enrich image details, and enhance the network's capability. The method extracts the Region of Interest (ROI) of the feature image from both enhanced and original images, followed by generating a mask map. Subsequently, the point cloud of the ROI from the depth map is obtained through the registration of ROI point cloud contour fitting. In addition, this method employs Tukey loss for optimal precision. The experimental results show this novel method proposed in this paper not only achieves high-precision and high-stability positioning, but also enables the positioning of any needle insertion point. Biological sciences/Cancer/Cancer therapy Biological sciences/Cancer Health sciences/Oncology Abdominal Percutaneous Ablation 3D Small Object Detection Surgical Navigation Machine Vision Medical Marking Surgical Robot Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 1. Introduction Navigation for thoracoabdominal puncture surgery is used to locate the needle entry point on the patient's body surface. The traditional reflective ball navigation method is difficult to position the needle entry point on the soft, irregular and smooth chest and abdomen. Due to the lack of clear feature points on the body surface using structured light technology, it is difficult to identify and locate any needle insertion point. In order to solve this problem, we use a white 24mm diameter hollow ring to identify medical markers. The ceramic material can be imaged well under CT, and has excellent color-based optical recognition and tomography-based imaging recognition performance. It can fit well on the human body surface and accurately position itself in the body surface environment. However, 3D small object detection is an industry challenge. 3D object detection has been widely used in actual scenes. With the development of deep learning technology in imaging[ 1][2][3][4] and With the progress of deep learning technology in point clouds[ 5][6][7 ][ 8] , 3D object detection methods have shown significant progress[ 9 ][ 10 ][ 11 ][ 12] . However, small object detection is still a challenge in the 3D field. In autonomous driving scenarios[ 13] , the performance gap between cars and pedestrians is obvious. In indoor scenes[ 14 ][ 15 ], the difference in size is larger (e.g. a wardrobe is 1000 times bigger than a cup), so the situation is even worse. For indoor 3D object detection, although the speed and accuracy of furniture-level benchmarks[10][ 16][17] have been greatly improved, they are still far from practical applications due to the limited range of object sizes they can handle. For example, it is difficult for previous methods to detect small desktop objects[ 18] . This paper proposes a multi-modal fusion perception method of Fourier enhancement, as shown in Figure 1. Fast Fourier transform is used to enhance the data to obtain richer details and clarity, and then the enhanced image and the original image are compared through a single-step object detection network. We pre-select the ROI area, perform segmentation and extraction based on the pre-selected ROI area, and then use histogram equalization and binarization operations to extract the outline and make a mask, use the mask to extract the ROI area of the depth map. The depth map ROI is converted into a point cloud format to obtain an approximation of reality. The shape point cloud of the value is then extracted by point cloud fitting and registration methods to obtain high-precision and stable single small object positioning and pose. The sub-diagram of the instantiated medical markup graphics processing steps is shown in Figure 2. Related work: There are generally two definition methods for the definition of 3D small objects (1) Based on relative scale:The ratio of the width and height of the object bounding box to the width and height of the image is less than a certain value, and a more common ratio value is 0.1; The square root of the ratio of the object bounding box area to the image area is less than a certain value, and a more common value is 0.03; .Small objects are defined based on the ratio between the actual covered pixels of the object and the total pixels of the image. (2) Based on absolute scale: The most common definition currently comes from the MS COCO dataset, a common dataset in the field of object detection, which defines small objects as objects with a resolution less than 32×32 pixels. The problems existing in small object detection are the few available features, high positioning accuracy requirements, the small proportion of small objects in the existing dataset, sample imbalance, small object aggregation problems, and network structure reasons. General research ideas for small object detection: (1) Data enhancement: the simplest and most effective method to improve small object detection performance[ 19 ]. Enhancement methods include: copy enhancement, adaptive sampling, scale matching, scaling and splicing, and self-learning data Enhance. Data enhancement solves the problem of small object information, such as lack of appearance features and texture to a certain extent, improves the generalization ability of the network, and achieves better results in the final detection performance, but at the same time it brings about computational cost. Moreover, it is often necessary to optimize the object characteristics in practical applications, but improperly designed data enhancement strategies may introduce new noise. (2) Multi-scale learning: Small objects require both deep semantic information and shallow representation information, and multi-scale Learning to combine the two is an effective strategy to improve small object detection performance. Four methods currently used: Image pyramid; Use shallower feature maps to detect smaller objects, and use deeper feature maps to detect larger objects; Multi-scale fusion network;Feature Pyramid FPN (Feature Pyramid Network) multi-scale feature fusion takes into account both shallow representation information and deep semantic information, which is conducive to feature extraction of small objects and can effectively improve small object detection performance. However, existing multi-scale learning methods not only improve detection performance, but also increase the amount of additional calculations, and it is difficult to avoid the influence of interference noise during the feature fusion process. These problems make it difficult to obtain small object detection performance based on multi-scale learning. further improvement. (3) Context learning:The method based on context learning makes full use of the object-related information in the image and can effectively improve the performance of small object detection. However, existing methods do not take into account the possible lack of contextual information in the scene, and do not specifically use easy-to-detect results in the scene to assist in the detection of small objects. Future research directions generally start from the following two perspectives:Construct a context memory model based on category semantic pooling to alleviate the problem of lack of context information in the current image by using the context of historical memory;Small object detection based on graph reasoning, through the combination of graph model and object detection model can specifically improve the detection performance of small objects. (4) Generative adversarial learning: By mapping the features of low-resolution small objects into features equivalent to those of high-resolution objects, it can achieve the same detection performance as larger objects. Currently, it still faces two unavoidable problems: Generative adversarial networks are difficult to train, and it is difficult to achieve a good balance between the generator and the discriminator; The diversity of samples generated by the generator during the training process is limited, and the performance improvement after training to a certain extent is limited. (5) Anchor-free mechanism: An idea to get rid of the anchor frame mechanism is to convert the object detection task into the estimation of key points, that is, an object detection method based on key points. object detection methods based on key points mainly include two categories: corner-based detection and center-based detection. Corner-based detectors predict object bounding boxes by grouping corner points learned from convolution feature maps. The object detection framework based on center prediction is called Center-Net.Representative point (RepPoints) detection method. This method can automatically learn the spatial information and local semantic features of the object, which improves the accuracy of small object detection to a certain extent.The fully convolution one-stage object detector FCOS (Fully convolution one-stage) avoids the problem of too many hyper-parameters and difficulty in training in methods based on the anchor box mechanism. The mainstream research directions in the future include three aspects: feature fusion, context learning, and super-resolution reconstruction. 2. Method and material The medical mark to be recognized is a white 24mm diameter hollow ring with a central hollow diameter of 10mm and a thickness of 3mm. The ceramic material can be imaged well under CT and has excellent color-based optical recognition and tomography-based imaging recognition performance. It can fit well on the human body surface and accurately position the function in the body surface environment. The recognition device used is a speckle method structured light camera, which can generate 2D, depth map and point cloud information and is an RGB-D information camera. The specific identification and positioning method proposed in this paper is to first obtain the 2D image and then obtain the depth map of the point cloud, so that a clear 2D image can be obtained. If the depth image of the 2D image and the point cloud information is obtained at the same time, the 2D image acquisition will obtain a noise image full of snowflake spots. This snowflake noise is actually the projection of a random speckle disparity map. Therefore, the difference between one simultaneous acquisition and two separate acquisitions lies in the speed and processing difficulty. The disparity image may be noisy, but it is faster to collect it once. The separate acquisition image is clear and easy to process, but it takes a little longer. According to the traditional algorithm, it is recommended to collect it twice. If high speed requirements are required for image detection, one-time acquisition can be used. In addition, the speed can be adjusted in the mode of image collection clarity from another angle, but the image quality will be reduced. 2.1 Comparative Analysis and Existing Issues with Traditional Methodologies The steps are as follows: First, In the image pre-processing stage, this method proposed in this paper compares pre-processing filters, high-pass filtering, histogram equalization, median filtering, mean filtering, and Gaussian filtering. Each has its own advantages and disadvantages in the comparison, but they all lead to the same results. The local effect is good and the other situation is unstable, so it is relatively recommended that histogram equalization and Gaussian filtering be used as noise reduction methods. The use of multi-classifier extraction is the core of this method. The second is to use open operation; the third is to use threshold segmentation; the fourth is to use connected areas; the fifth is to use area screening; and the sixth is to use circular similarity analysis. The seventh is obtain the 3Dxyz value center based on the fitted circle center mapping depth map. Circle extraction is based on the least squares calculation: the formula is as follows. For specific calculations, refer to the extraction formula below. But this is only the algorithm stage. In practice, external assistance is very important. The steps are: Step 1: Optical stability light environment observation. First of all, there is a light environment parameter table in the medical environment. Secondly, various on-site light environment sampling. Step 2: Contrast enhancement of the observed object. It is a good way to use a surrounding black edge ring with strong black and white contrast to highlight the marker, which can ensure stable and clear extraction. The material selection of this contrast object is very important. The mirror metal material is not recommended for selection, because it seems that the light perception contrast is very strong, but in a certain some angles have high contrast, but some angles have very low contrast. To achieve good optical contrast enhancement materials, it is recognized that the best light-absorbing material is carbon nanotubes. However, this material is not only expensive and has a long cycle, but also has the characteristics of being difficult to obtain. Therefore, the simple and easy solution used in the experiment is to use matte material, which is often a rough and porous surface. In practice, the optical performance of medical scene markers is almost the same. Step 3: Limit the scope of observation and the angle of observation. The observation angle is limited. It is normal for the observation angle to be within 30 degrees. The recognition deviation is large at an excessively large angle, so it is better to limit it to a certain angle. Otherwise, the set circle fitting deviation including ellipse fitting will have poor fitting. For example, it will be very stable if the observation is limited to a relative angle of 30 degrees and a height of 400mm. The observation effect is shown in Figure 3. It can be seen that this method can only observe at a small angle and has poor stability, and is also sensitive to ambient light. The advantage is that it is simple to deploy, lightweight and quick to use, but it is difficult to avoid the problems of depth map mapping loss and optical blur. The optical blur value will cause the coordinate positioning accuracy to fluctuate below the stable sub-millimeter level. Depending on the product, it will be between 0.2-0.7mm. The ideal observation posture and observation distance are one solution, but it can only avoid part of it and limit the scope of flexible application in practice. For problems based on optical blur coefficients such as light interference, the optical blur changes with distance, as shown in Figure 4 below. If direct RANSAC and DBSACN point cloud fitting are omitted, the instability will increase due to the sparse point cloud density at different distances. Algorithms such as RANSAC themselves are called uncertainty algorithms, resulting in unstable extraction or failure. Common methods of point cloud fitting often have uncertainties. The RANSAC algorithm itself is a non-deterministic algorithm. The clustering algorithm will also become less stable as the point cloud sparse values are used. Other effective methods such as arbitrary surface cutting circles are used. Ring extraction and non-directional random cutting will lose the direction accuracy, lose the original intention of improving accuracy, and become meaningless. 2.2 Fourier-enhanced multi-modal point cloud refinement method For structured light cameras, point clouds are characterized by collecting multiple disparity maps. The information is richer than a single shot of 2D images. Compared with optics, it is more accurate in producing camera optical blur values at different distances and can also avoid depth mapping. Mapping loss occurs because this method is not used, so the point cloud extraction effect in this scene is theoretically more ideal. However, on-site extraction of small object point clouds of 24mm rings is a challenge. Based on this, a multi-modal method of using fast Fourier transform to enhance image details is proposed. Indeed, enhancing the resolution and meticulously extracting the predefined image contour, followed by the retrieval of a 24-60mm point cloud, facilitates the refinement of point cloud fitting and registration, thereby elevating precision. This approach circumvents the pitfalls of optical blur and depth map mapping loss. Moreover, it exhibits superior resilience to optical disturbances, ensuring the stability and accuracy in feature enumeration and classification—a quintessential attribute for applications demanding medical-grade precision and stability. First, we use fast Fourier transform to enhance the image details, and then use a single-step object detector (such as YOLO, SSD, etc.) combined with the depth map mapping ROI method. In this paper, we adopt YOLOv5 as the baseline model to extract the region of interest (ROI) through a single-step object detection network and the identified ROI is subsequently segmented from the image. Second, a BLOB operation is conducted on the segmented area to delineate its contours, which are then rendered onto an empty array corresponding to the original image dimensions, creating a mask image. This mask facilitates the cropping of the depth map, yielding the target area's ROI depth map. The depth map is then transformed into a point cloud, producing the target point cloud. Fitting extraction based on the point cloud's shape and registration culminates in the retrieval of precise, stable, and minimally interfered position information for small targets. Third, We determine the center point of the target based on the size of the rectangular frame, and use the coordinates of the center point to find the corresponding mapping point in the depth map to obtain the coordinate value of the target point. It is worth noting that the single-step detection network usually processes three-channel images. For single-channel images, they need to be converted into three-channel images. When the target cannot be recognized, adding unrecognized samples for retraining can solve the problem. In practice, sparse sample situations are extremely common. By augmenting the dataset and fine-tuning the model, the model performance can ultimately be improved to a higher level. The specific processing flow is shown in Figure 5. 2.2.1 Data acquisition and pre-processing A high-quality dataset is the foundation and premise for a superior model, crucial for enhancing optical performance and point cloud effects. In private domain scenarios, we collected and created a diverse range of real-shot data under laboratory and hospital CT room environments, featuring various angles and simulated situations. Adhering to medical building standards, the lighting conditions in our experimental setups ranged from 130 to 750LX, incorporating common environmental interference tools during data collection. We captured 2D and 3D images using structured light cameras to accommodate multi-modal experiments, thus employing a fusion labeling approach for both 2D and 3D data. Specifically, 350 2D images were annotated using the LabelImg tool, while 330 multi-modal fused 3D point clouds and 370 point cloud and image fusion samples were annotated with the Xtreme1 tool. To expand the dataset in later stages, we utilized model-based line data augmentation methods, potentially increasing the dataset size by 2 to 5 times. 2.2.2 Fast Fourier transform enhancement Performance on images is often affected by the aspect ratio being too small and the image pixels not being obvious. It is a common practice to use image enhancement and linear difference. image enhancement methods include common contrast sharpening, contrast enhancement, brightness enhancement, etc. However, image enhancement will also bring about the problem of larger low-order values and poor stability. Fast Fourier transform is a good method, but it has not been applied to 3D small objects before. Fast Fourier transform (FFT) is a An algorithm in digital signal processing changes in the frequency domain: Fourier transform splits the signal in the spatial domain into signals in different frequency bands, and converts the pixel image into a spectrogram. The idea of frequency domain detection is to detect from the spatial domain to Frequency domain, perform appropriate filtering in the frequency domain, select the frequency band you want, and then return to the spatial domain. There are two key factors, one is to generate a suitable filter, and the other is the conversion between the spatial domain and the frequency domain. This paper uses Gaussian filtering. The sigma value of Gaussian filtering is the standard deviation of the Gaussian kernel function, which determines the width and shape of the Gaussian kernel function. The larger the sigma value, the wider the width of the Gaussian kernel function, and the smoother the image; the smaller the sigma value, the narrower the width of the Gaussian kernel function, and the more prominent the details of the image. The method is to set two Gaussian filter kernels, one with a sigma value equal to 3 and the other with a sigma value equal to 15. After passing through two Gaussian filters respectively, the two images are subtracted, so that they are converted to frequency domain processing, and then after converting it back, we will find that the image with a very low threshold difference becomes a large gap and the outline is clear and obvious. The pseudo code is as follows: Algorithm 1: Enhanced 3D Object Recognition using Fourier Transform and Gaussian Filters Input: Directory of image files Output: Display images and image variable 1: for each image in the directory do 2: # Convert image to grayscale 3: # Apply Fourier Transform to convert the image to frequency domain 4: fft_generic(GrayImage, ImageFFT, 'to_freq', -1, 'n', 'dc_center', 'complex') 5: 6: # Generate Gaussian filters for background weakening and defect enhancement 7: gen_gauss_filter(ImageGauss, 15, 15, 0, 'none', 'dc_center', Width, Height) 8: gen_gauss_filter(ImageGauss1, 3, 3, 0, 'none', 'dc_center', Width, Height) 9: 10: # Subtract the generated Gaussian filters to highlight defects 11: sub_image(ImageGauss1, ImageGauss, ImageSub, 1, 128) 12: 13: # Apply convolution in the frequency domain using the subtracted image 14: convol_fft(ImageFFT, ImageSub, ImageConvol) 15: 16: # Convert the convoluted image back to the spatial domain 17: fft_generic(ImageConvol, ImageFFT1, 'from_freq', 1, 'n', 'dc_center', 'real') 18: 19: # Analyze the image for object recognition (eg, Blob analysis) 20: Perform Blob Analysis on ImageFFT1 21: 22: Display the enhanced image How to view the spectrogram description (information includes: phase, frequency, and frequency band content): (1) Each point is not a pixel, but represents a different frequency or frequency band, and the center of the spectrogram is low frequency, which gradually increases from the center to the surroundings. Sometimes the center of the spectrum is not at the center of the window. (2) The spectrum chart is symmetrical up, down, left and right. (3) The brightness of each point in the spectrogram represents the amplitude of the frequency band (and the degree of grayscale transformation). If the bright spot has a certain width, it means that the frequency content at this point is relatively rich. (4) There are bright spots or bright lines in a certain direction in the spectrogram, indicating that there are edges with drastic grayscale changes in that direction. 2.2.3 Training details We use a GeForce RTX 4080 for training, and the batch size is set to 128. Usually each training is 500 epoch, and the accuracy rate is 99.8% as the inference model. Two parameters that are often adjusted are ROI pooing and Anchor box. Setting of hyperparameters: In the loss design, α and γ focal loss are set to [0.75, 0.75, 0.25] and [2., 2., 2.] respectively. The three loss weights are set to λ loc = 1 , λ cls = 1 and λ h = 1.5. The Adma optimizer[ 20] is used to train for 70 epochs with an initial learning rate of lr = 2 e -4 , and the weight decay is set to 1 e − 4 . We first adopt the warm-up strategy[ 21] , with 300 warm-up iterations and a warm-up ratio of 1./3 . In addition, the decay ratio of lr at the 40th epoch and 60th ecpoch is 0.1. During the inference phase, the prediction score threshold is set to 0.2 and the rotated NMS[ 22] threshold is set to [0.02, 0.02, 0.4]. Configuration of single-step detector on data augmentation: flip lr: 0.5 (probability of left and right flipping); mosaic: 1.0 (probability of mosaic data enhancement ); mix up: 0.0 (probability of mixed images); hsv_h: 0.015 (probability of hue dithering); hsv_s: 0.7 (probability of saturation dithering) ; hsv_v: 0.4 (probability of luminance dithering); degrees: 0.7 (angle range of rotation); translate: 0.1 (panning range); scale: 0.5 (zooming range) , which can be increased by 2-3 times by configuring the actual image. 2.2.4 The process of masking the ROI obtained by cropping the depth map: It is to create an empty array with the same size as the depth map, and then only have the mask map in it. Set all the mask numbers to 1, and the remaining blank parts to 0. Then the mask map is actually a multi-dimensional array, which is also a matrix. Multiplying the array (matrix) of the depth map, you get the mask map. The mask part is the depth map, the depth information part, and the rest of the non-roi area is 0. Then remove all the 0 values in this array, and you get The depth map is cropped by the mask image. This depth map is the depth map of the area of interest, and the rest has been deleted. 2.3 Point cloud refinement: fitting and registration This paper adopts the refinement part of the progressive method of two superposition classifiers, and obtains strong robustness and high accuracy results through fitting and registration superposition combination. Since blob analysis is used to extract contours, it is often a fixed value, resulting in the extracted point cloud containing noise information in a few cases. Therefore, the fitting will lead to an increase in error, so a registration is added to improve the accuracy, because the registration is Extracted from the standard template, coupled with the effect of Tu-based loss, it has less impact on noise. In fact, fitting can extract the center of the circle. However, in order to prevent the error from increasing and unstable recognition, registration is added. These two are in a progressive relationship. If the fitting and registration errors are large enough, they will exit to ensure accurate positioning and small errors every time during the operation. This is due to the high stability and accuracy requirements of surgical positioning. In the process of extracting ROI point clouds, Gaussian filtering and binarization are initially applied to obtain the ROI contours, which are then used to generate a new image serving as an image mask. This mask can also be created by drawing contours. Subsequently, this mask is utilized to crop the depth map, producing the ROI depth map. For depth maps acquired with structured light cameras, this method can be directly applied. In cases where depth maps are captured using LiDAR and cameras, multi-camera calibration is necessary to establish the mapping relationship, typically requiring mapping to images of the same size. However, employing bitmap methods for mask cropping is impractical because such methods usually operate on UNIT8, and converting depth maps to UNIT8 format would result in loss of depth information. Instead, multi-loop techniques or array multiplication can be used for mask mapping, facilitating the extraction of the ROI from the depth map. Afterward, the depth map can be converted into a point cloud for subsequent refinement operations on the ROI point cloud. By removing noise and environmental information, the target point cloud is simplified into a simple annular shape, making RANSAC fitting easier. Direct extraction with RANSAC at the outset may fail due to small target size or environmental noise. In addition, Point cloud information is often richer than image information and is more stable and accurate for collecting precise locations. 2.3.1 Point cloud volume fitting method Extracting refined point clouds is relatively simple, accurate, and stable. Since the mark is a donut shape, it is connected to the environment to form a cone. Therefore, the shape of the point cloud can be relatively ideal using the cone fitting. The specific method Refer to the cone fitting formula of the following fitting: Explanation: (1) Geometric properties of the cone: For a cone with (x0, y0, z0) as the vertex and the axis aligned with the z-axis, any point (x, y, z) located on its surface satisfies the above relationship, here, the numerator is the horizontal distance from the point (x, y, z) on the xy plane to the vertex of the cone, and the denominator is the vertical distance from the point to the vertex. This ratio is equal to the tangent of the cone's opening angle. (2) Calculate the theoretical z coordinate: According to the above formula, we can rearrange the formula to calculate the theoretical z coordinate. The pseudo code is as follows: # Algorithm 2 : Calculate the theoretical z using the cone equationtan Tan_theta = np.tan(theta) Theoretical_z =z0 +np.sqrt((x- ×0)**2+(y - y0)**2)/tan_theta Where, theoretical_z is the predicted z coordinate value based on the cone model. (3) Calculate the residual. The residual is the difference between the z coordinate of each point in the actual point cloud data and the theoretical predicted value. During the optimization process, the goal is to minimize the sum of squares of these residuals for all points. 2.3.2 Tukey loss registration method ICP (Iterative Closest Point) has many registration types. For medical labeling scenarios, a robust registration method with high stability is required. This paper uses the Tukey loss robust registration method, which has good robust effects. Meanwhile, it can output very stable results for the point cloud registration method. The formula is as follows: The loss ρ(r) for a given residual r is: The weight w(r) for given residual r is given by ： this formula is the Tukey loss function and its corresponding weight function, which are used to calculate the loss value and weight of the residual. When the residual r is less than or equal to the threshold k, the shape of the loss function ρ(r) is similar to the square loss, but when the absolute value of the residual increases to close to k, the growth rate slows down and remains unchanged after exceeding k. This design reduces the weight of large residuals in the total loss, thereby improving robustness to outliers. The weight function w(r) is the derivative of the loss function to the residual r, which is used to adjust the influence of each data point during the iterative optimization process. When the residual is less than or equal to k, the weight decreases as the residual increases; when the residual is greater than k, the weight is zero, that is, these possible outliers are ignored. This helps the optimization process focus more on data points that fit the model. The specific method is to use the estimated normal vector method to calculate the normal for the two point clouds, which is necessary for the ICP alignment of the point to the plane. The initial alignment uses the identity matrix np.eye(4) as the initial transformation. This means that the source and object point clouds are initially aligned without any rotational or translational alignment. Point-to-plane ICP alignment, use the point-to-plane ICP method to initially align the source point cloud and the object point cloud, apply the resulting transformation to a copy of the source point cloud and display the result, and then try to use the Tukey loss function Variant ICP method. The Tukey loss function is a robust loss function that is less sensitive to outliers and works well for processing data containing noise or outliers. The result after segmentation is shown in Figure 6. 3. Result Accuracy is a multiple factor, including algorithm error, sensor error, mechanical assembly error, CT bed displacement offset value, ground vibration value, etc. It can be analyzed by category to produce a combined theoretical value, but the real value accumulated in practice in the actual environment is more worthy of investment. For analysis, especially in CT rooms where heavy equipment is running at all times, the before-and-after comparison algorithm can use numerical comparisons under actual measured vibrations. For a clear description, Table 1 is used as the depth map mapping result, and Table 2 is used as the multi-modal point cloud. Refine the results. Table 1: Comparative acquisition accuracy is as follows, previously using depth map mapping method Traditional mapping algorithm places A-Mark recognition results serial number Recognition results Straight line distance difference (±) XYZ 1 60.36693 -4.17822 363.7445 1.332731176 2 61.35583 -2.3430135 364.15797 0.821445875 3 60.34972 -4.1770315 363.6408 1.369640695 4 60.4186 -3.2620718 364.05582 0.515140576 5 61.39088 -2.344352 364.36777 0.880754035 6 Recognition failed 7 60.36693 -4.178227 363.7445 1.33273741 8 61.408417 -2.3450217 364.47183 0.927111199 9 61.39088 -2.344352 364.36777 0.880754035 10 60.34972 -4.1770315 363.6408 1.369640695 11 Recognition failed 12 61.373352 -2.3436825 364.2637 0.845102971 13 60.34972 -4.1770315 363.6408 1.369640695 14 60.487637 -2.3450217 364.47183 0.828496361 15 Recognition failed 16 61.408416 -2.3450217 364.47183 0.927110601 17 60.384144 -3.2602117 363.8482 0.599204743 mean 60.85401488 -2.991194841 364.1049053 0.973682037 Standard deviation σ 0.501098721 0.817832392 0.314885796 Table 2: Actual performance test table using this algorithm Multi-modal point cloud refinement placement A-Mark recognition results serial number Recognition results Straight line distance difference (±) XYZ 1 57.93445 1.32185 360.04922 0.10323271 2 57.95081 1.32222 360.15085 0.136184428 3 57.93445 1.32185 360.04922 0.10323271 4 57.93445 1.32185 360.04922 0.10323271 5 57.93445 1.32185 360.04922 0.10323271 6 57.53772 1.14727 360.06187 0.716752595 7 57.93445 1.32185 360.04922 0.10323271 8 57.91811 1.32147 359.94763 0.154767426 9 57.87316 1.23348 360.35431 0.285762242 10 57.73445 1.32185 360.04922 0.110382359 11 57.93445 1.32185 360.04922 0.10323271 mean 57.83826818 1.297944545 360.0781091 0.133580156 Standard deviation σ 0.121477809 0.053937284 0.097565242 Through the above operation algorithm, the repetition accuracy is identified from the standard deviation value of the accuracy of 0.973682037 mm to the repetition accuracy error value of 0.133580156 . Recognition was originally unrecognizable due to misoperation, but it will no longer occur after multi-modal point cloud refinement is used. 4. Discussion The Fourier-enhanced multi-modal point cloud refinement method proposed in this paper provides a novel solution for 3D small object detection at the medical labeling level. Although existing 3D object detection methods have achieved significant progress on large objects, but the detection methods of small objects, especially small medical markers for use in surgical navigation, are still under development. This paper effectively improves the detection accuracy and stability of small objects in this application case by combining a single-step object detection network and point cloud processing technology. This method has a positive impact on medical mark recognition. Previously, the recognition method relied on optical reflective balls, which were difficult to fit on the human body surface. There was a level of positioning of surgical instruments. For body surface points, this method predominantly used three reflective balls, posing numerous challenges in their placement. Inferring from a certain distance from the epidermis further complicated matters. However, the use of white ring markers can accurately reflect skin points, particularly beneficial for surgeries requiring precise fitting, such as those involving the chest and abdomen. This method has undergone practical testing and is point-based. Based on point cloud 3D general neural networks are mostly for large objects, making the application of this small object method still relatively uncommon in practice. Meanwhile, this method will increase the amount of calculation and the calculation time, but the overall time is influenced by various factors due to the robustness of the algorithm. Therefore, it is feasible to process lower quality images and sparse point clouds, which can offset the time required for camera acquisition. Cameras are generally divided into 5 modes from high to low based on parameter adjustments (levels of quality and speed): static quality, static balance, static fast, dynamic balance, and dynamic fast. The difference is the clarity of captured images and the frequency of projected images. Higher modes provide faster imaging but with relatively more blur, resulting in larger accuracy errors. Conversely, slower modes yield better quality and higher accuracy. Previously, the acquisition time required in static balance and static quality modes was 1.7 and 2.1 seconds. However, using the dynamic fast mode can complete acquisition in just 0.5 seconds. Hence, by enhancing the graphics card configuration and adjusting camera speed settings, this time can be significantly reduced. The main advantage of this method is its high-precision positioning capability for small objects. Traditional 3D object detection methods often suffer from performance degradation when dealing with small-sized objects due to insufficient features and high positioning accuracy requirements. By utilizing 2D images to extract ROI areas and combining them with point cloud technology, this method can handle small-sized objects more accurately, which has positive significance for surgical navigation. However, this paper also has some limitations. Firstly, the accuracy of the method is highly dependent on the quality of the 2D image and the accuracy of the mask. Any errors during the 2D image processing stage may lead to a decrease in the accuracy of the final point cloud. Secondly, the fitting and registration processes in the point cloud processing step require precise tuning and optimization to ensure optimal performance. Despite these challenges, the method proposed in this paper showed excellent performance in experiments. Especially in repeatability tests, this method can significantly reduce positioning errors, which is extremely important for practical surgical applications. 5. Conclusion This paper successfully developed a multi-modal point cloud refinement method for 3D small object detection, particularly focusing on the precise positioning of medical markers. For medical surgical navigation, body surface positioning is found to be more suitable for accurately placing needles on lesion areas of the body surface. This method combines fast Fourier transform, single-step object detection network and point cloud technology to effectively improve the detection accuracy and stability of small objects in complex medical environments. Detailed experiments and analysis demonstrate the practicality and effectiveness of this method in medical surgical navigation. This method can position any needle insertion point based on the body surface. In the future, it holds promise for applications in body surface positioning for puncture ablation across diversified surgical scenarios. As a novel technique in the field of surgical navigation, it has positive significance for puncture ablation surgical navigation of tumors. In summary, this paper provides an effective method for 3D small object detection in surgical navigation, and contributes positively towards enhancing the safety and accuracy of thoracoabdominal surgical navigation. Declarations Ethics approval and consent to participate This article uses a laboratory plastic human breast model as the test object, and no animals or human participants are designed. No unethical measures or practices were adopted,using the experimental part of the prosthesis, we ensure that all materials meet biocompatibility and safety standards. Consent for publication Not applicable. Data availability statement The datasets generated and/or analyzed during the current study are not publicly available due [for patent and trade secret reasons] but are available from the corresponding author on reasonable request. Declaration of interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Author contribution statement. Zezhao Guo contributed most of the creativity/code/experiments, Yanzhong Guo had equal contributions,did a lot of solid work, and Zhanfang Zhao reviewed the manuscript as a supervisor. Funding The research work of this article received financial support from the Hebei Provincial Social Science Fund project. No. (HB20TQ003) Acknowledgement This work was supported by the Hebei Provincial Social Science Foundation Project (No.HB20TQ003). We thank Beijing Yingrui Pioneer Medical Technology Company for experimental data support and Professor Zhao Zhanfang of Hebei GEO University for correcting the writing of this article. References Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25. Wang, Lingyun, et al. "Axial super-resolution optical coherence tomography via complex-valued network." Physics in Medicine & Biology 68.23 (2023): 235016. Aswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017). He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652-660). Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30. B.Graham, M. Engelcke, and L. Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In CVPR, pages 9224–9232, 2018. C.Choy, J. Gwak, and S. Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR, pages 3075–3084, 2019. S.Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In CVPR, pages 10529–10538, 2020. W.Zheng, W. Tang, L. Jiang, and C.-W. Fu. Se-ssd: Self-ensembling single-stage object detector from point cloud. In CVPR, pages 14494–14503, 2021. H.Wang, L. Ding, S. Dong, S. Shi, A. Li, J. Li, Z. Li, and L. Wang. Cagroup3d: Class-aware grouping for 3d object detection on point clouds. arXiv preprint arXiv:2210.04264, 2022. D.Rukhovich, A. Vorontsova, and A. Konushin. Tr3d: Towards real-time indoor 3d object detection. arXiv preprint arXiv:2302.02858, 2023. A.Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving the kitti vision benchmark suite. In CVPR, pages 3354–3361, 2012. A.Dai, AX Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. Scannet: Richly annotated 3d reconstructions of indoor scenes. In CVPR, pages 5828—-5839, 2017. A.Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y. Zhang. Matterport3d: Learning from rgb-d data in indoor environments . 3DV, 2017. I. Armeni, O. Sener, AR Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic of large-scale indoor spaces. In ICCV, pages 1534–1543, 2016. S.Song, SP Lichtenberg, and J. Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, pages 567–576, 2015. X.Xu, Y. Wang, Y. Zheng, Y. Rao, J. Zhou, and J. Lu. Back to reality: Weakly-supervised 3d object detection with shape-guided label enhancement. In CVPR, pages 8438– 8447, 2022. ]Rabbi, J., Ray, N., Schubert, M., Chowdhury, S., & Chao, D. (2020). Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network. Remote Sensing, 12(9), 1432. Da, K. (2014). A method for stochastic optimization. arXiv preprint arXiv:1412.6980. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). Neubeck, A., & Van Gool, L. (2006, August). Efficient non-maximum suppression. In 18th international conference on pattern recognition (ICPR'06) (Vol. 3, pp. 850-855). IEEE. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4488932","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":312856243,"identity":"a15cf170-8230-4fb5-a877-9d4b3c7eac99","order_by":0,"name":"Zezhao Guo","email":"","orcid":"","institution":"Hebei GEO University","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zezhao","middleName":"","lastName":"Guo","suffix":""},{"id":312856244,"identity":"9ac6a765-70af-4f0a-a359-7be4015f6b1e","order_by":1,"name":"Yanzhong Guo","email":"","orcid":"","institution":"beijing yingrui pioneer Inc","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Yanzhong","middleName":"","lastName":"Guo","suffix":""},{"id":312856245,"identity":"b802a41d-b9a7-48c7-9923-141b7f246848","order_by":2,"name":"Zhanfang Zhao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABA0lEQVRIiWNgGAWjYDACCSBmbGBgYAdiho8NYDED4rTwHADSM0nWwsxLjBb52c0PH/7cYZfHw3728GvbHbWJDezN2yQYau7g1MI455ixgeSZ5GIenrw069wzxxMbeI6VSTAce4ZTC7NEgpmEYRtz4n6GHDPj3LZjiQ0SOWYSjA2HcWphk0j/JpHYVp/Yw//GzNgSpEX+DX4tPCAzD7YdTuyRyDF+zNhWA7SFB78WCYmcYsPGtuNALW/MGHvbDhi38aQVWyQcw61Ffkb6xoc/26qBDssx/vCzrU62n/3wxhsfanBrQfUXA8NhBjYQM4EoDcDQ+8DAUEek2lEwCkbBKBhJAAAmOlZ2ER3RpQAAAABJRU5ErkJggg==","orcid":"","institution":"Hebei GEO University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Zhanfang","middleName":"","lastName":"Zhao","suffix":""}],"badges":[],"createdAt":"2024-05-28 07:33:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4488932/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4488932/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":58248082,"identity":"b981d62f-49b9-4a9e-bc47-bdfa13f6208e","added_by":"auto","created_at":"2024-06-13 02:44:55","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":55971,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eImage processing and object detection flow chart. The input data includes images and depth maps, which are successively processed by Fourier transform enhancement and ROI masking of the object detection network and point cloud refinement in the second step. First, Fourier transform is used for dataset enhancement. The enhanced dataset is fed into the object detection network, and the detection results are generated to perform mask operations to extract the object and generate ROI areas. The depth map is then used for shape fitting and a robust iterative closest point (ICP) algorithm to Achieve high-precision and stable object positioning.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/f7cfb41a9f37a870453f1924.png"},{"id":58248086,"identity":"24605035-eb1c-4088-ae6b-d1d4bda37b6a","added_by":"auto","created_at":"2024-06-13 02:44:55","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":49221,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eExample showing image processing and feature extraction sequence. The sub-images are processed in each stage in order from left to right: the original image first undergoes Fourier transform enhancement, and is combined with the original image to become a mixed dataset adding model. Then, through the 2D object detection network segment, the mask contour is extracted and entered into the depth map to obtain the ROI point cloud. Next, fitting and robust registration are applied to extract poses and center points. Based on these analyses, specific features are identified and located in the image. Finally, the location of the feature is marked prominently.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/fb6ca1a07cd14cf59a90ffd1.png"},{"id":58249070,"identity":"37931152-c838-4ac0-a158-bcde36b2044c","added_by":"auto","created_at":"2024-06-13 02:52:55","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":223822,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eRecognition and positioning effect diagram. It shows the stable recognition status that limited to a relative angle of 30 degrees and a height of 400mm.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/2485783df9a0655edb80fb44.png"},{"id":58248083,"identity":"daea403f-b983-4656-b021-0c7973c82969","added_by":"auto","created_at":"2024-06-13 02:44:55","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":62399,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eThe trends of four different measurement parameters as a function of working distance. Each parameter is plotted in its own subplot.Z-Accuracy [mm]: This image description shows the trend of Z-axis (depth) accuracy gradually getting worse with increasing working distance. As the working distance increases, the depth accuracy error of the measurement also increases.XY Pixel Size [mm]: This graph depicts the tendency of pixel size in the XY plane to increase with increasing working distance. The farther the working distance, the larger the actual area covered by each pixel, resulting in reduced resolution.View Field Width [mm]: This graph shows the linear increase in field width with increasing working distance. This means that as the camera moves further away from the subject, its field of view covers a wider area.Optics Blur [px]: This graph shows the optical blur (in pixels) trend, which reaches a minimum value within a certain working distance range, and then increases again as the working distance continues to increase. This indicates that there is an optimal working distance at which the optical system provides the sharpest image\u003c/em\u003e.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/cdb2cfd01eb2695fdd951eab.png"},{"id":58249069,"identity":"ce3084ec-103f-488d-844f-80b445506bbb","added_by":"auto","created_at":"2024-06-13 02:52:55","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":234459,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eThe grayscale image and the depth map are collected twice. They have the same size and mapping relationship. Gray is the grayscale image, and red is the depth map. The grayscale image is enhanced by Fourier transform, and the data is not enhanced. The sets form a hybrid enhanced dataset and are trained together in the model. The trained model is used to infer the grayscale image and generate a 2D target area. Then BLOB analysis is performed to obtain the contour mask to obtain the cropping depth map, and the contour point cloud is generated based on the contour depth map. Finally, it can obtain stable and accurate positioning points through a robust registration method with geometric shape fitting with Tukey loss.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/04fe1334f66cb27727ad821b.png"},{"id":58248085,"identity":"21c44295-9d02-48c4-bdd5-11ce0ef571a1","added_by":"auto","created_at":"2024-06-13 02:44:55","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":106402,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eROI point cloud obtained from the whole scene, the purple point cloud segmentation part, and the gray part is the background part.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/b687c579a0e2f8a75afb8a19.png"},{"id":60131142,"identity":"a771a2f2-bd84-4967-be25-c400f60c162a","added_by":"auto","created_at":"2024-07-12 07:06:50","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1225925,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4488932/v1/280d58ae-103f-438d-aa6e-b626bee4ef03.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Fourier-enhanced multi-modal 3D small object optical mark recognition and positioning method for percutaneous abdominal puncture surgical navigation","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eNavigation for thoracoabdominal puncture surgery is used to locate the needle entry point on the patient\u0026apos;s body surface. The traditional reflective ball navigation method is difficult to position the needle entry point on the soft, irregular and smooth chest and abdomen. Due to the lack of clear feature points on the body surface using structured light technology, it is difficult to identify and locate any needle insertion point. In order to solve this problem, we use a white 24mm diameter hollow ring to identify medical markers. The ceramic material can be imaged well under CT, and has excellent color-based optical recognition and tomography-based imaging recognition performance. It can fit well on the human body surface and accurately position itself in the body surface environment. However, 3D small object detection is an industry challenge.\u003c/p\u003e\n\u003cp\u003e3D object detection has been widely used in actual scenes. With the development of deep learning technology in imaging[\u003csup\u003e1][2][3][4]\u003c/sup\u003e and With the progress of deep learning technology in point clouds[\u003csup\u003e5][6][7\u003c/sup\u003e][\u003csup\u003e8]\u003c/sup\u003e, 3D object\u0026nbsp;detection methods have shown significant\u0026nbsp;progress[\u003csup\u003e9\u003c/sup\u003e][\u003csup\u003e10\u003c/sup\u003e][\u003csup\u003e11\u003c/sup\u003e][\u003csup\u003e12]\u003c/sup\u003e. However, small\u0026nbsp;object\u0026nbsp;detection is still a challenge in the 3D field. In autonomous driving scenarios[\u003csup\u003e13]\u003c/sup\u003e, the performance gap between cars and pedestrians\u0026nbsp;is obvious. In indoor scenes[\u003csup\u003e14\u003c/sup\u003e][\u003csup\u003e15\u003c/sup\u003e], the difference in size\u0026nbsp;is larger\u0026nbsp;(e.g. a wardrobe is 1000 times bigger than a cup),\u0026nbsp;so\u0026nbsp;the situation\u0026nbsp;is even worse.\u0026nbsp;For indoor 3D object\u0026nbsp;detection, although the speed and accuracy of furniture-level benchmarks[10][\u003csup\u003e16][17]\u0026nbsp;\u003c/sup\u003ehave been greatly improved, they are still far from practical applications due to the limited range of object sizes they can handle. For example, it is difficult for previous methods to detect small desktop objects[\u003csup\u003e18]\u003c/sup\u003e.\u003c/p\u003e\n\u003cp\u003eThis paper proposes a multi-modal fusion perception method of Fourier enhancement, as shown in Figure 1. Fast Fourier transform is used to enhance the data to obtain richer details and clarity, and then the enhanced image and the original image are compared through a single-step object detection network. We pre-select the ROI area, perform segmentation and extraction based on the pre-selected ROI area, and then use histogram equalization and binarization operations to extract the outline and make a mask, use the mask to extract the ROI area of the depth map. The depth map ROI is converted into a point cloud format to obtain an approximation of reality. The shape point cloud of the value is then extracted by point cloud fitting and registration methods to obtain high-precision and stable single small object positioning and pose. The sub-diagram of the instantiated medical markup graphics processing steps is shown in Figure 2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRelated work:\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThere are generally two definition methods for the definition of 3D small objects (1) Based on relative scale:The ratio of the width and height of the object bounding box to the width and height of the image is less than a certain value, and a more common ratio value is 0.1; The square root of the ratio of the object bounding box area to the image area is less than a certain value, and a more common value is 0.03; .Small objects are defined based on the ratio between the actual covered pixels of the object and the total pixels of the image. (2) Based on absolute scale: The most common definition currently comes from the MS COCO dataset, a common dataset in the field of object detection, which defines small objects as objects with a resolution less than 32\u0026times;32 pixels.\u0026nbsp;The problems\u0026nbsp;existing in\u0026nbsp;small object detection\u0026nbsp;are\u0026nbsp;the few available features, high positioning accuracy requirements, the small proportion of small objects in the existing dataset, sample imbalance, small object aggregation problems, and network structure reasons.\u003c/p\u003e\n\u003cp\u003eGeneral research ideas for small object detection: (1) Data enhancement: the simplest and most effective method to improve small object detection performance[\u003csup\u003e19\u003c/sup\u003e]. Enhancement methods include: copy enhancement, adaptive sampling, scale matching, scaling and splicing, and self-learning data Enhance. Data enhancement solves the problem of small object information, such as lack of appearance features and texture to a certain extent, improves the generalization ability of the network, and achieves better results in the final detection performance, but at the same time it brings about computational cost. Moreover, it is often necessary to optimize the object characteristics in practical applications, but improperly designed data enhancement strategies may introduce new noise. (2) Multi-scale learning: Small objects require both deep semantic information and shallow representation information, and multi-scale Learning to combine the two is an effective strategy to improve small object detection performance. Four methods currently used: Image pyramid; Use shallower feature maps to detect smaller objects, and use deeper feature maps to detect larger objects; Multi-scale fusion network;Feature Pyramid FPN (Feature Pyramid Network) multi-scale feature fusion takes into account both shallow representation information and deep semantic information, which is conducive to feature extraction of small objects and can effectively improve small object detection performance. However, existing multi-scale learning methods not only improve detection performance, but also increase the amount of additional calculations, and it is difficult to avoid the influence of interference noise during the feature fusion process. These problems make it difficult to obtain small object detection performance based on multi-scale learning. further improvement. (3) Context learning:The method based on context learning makes full use of the object-related information in the image and can effectively improve the performance of small object detection. However, existing methods do not take into account the possible lack of contextual information in the scene, and do not specifically use easy-to-detect results in the scene to assist in the detection of small objects. Future research directions generally start from the following two perspectives:Construct a context memory model based on category semantic pooling to alleviate the problem of lack of context information in the current image by using the context of historical memory;Small object detection based on graph reasoning, through the combination of graph model and object detection model can specifically improve the detection performance of small objects. (4) Generative adversarial learning: By mapping the features of low-resolution small objects into features equivalent to those of high-resolution objects, it can achieve the same detection performance as larger objects. Currently, it still faces two unavoidable problems: Generative adversarial networks are difficult to train, and it is difficult to achieve a good balance between the generator and the discriminator; The diversity of samples generated by the generator during the training process is limited, and the performance improvement after training to a certain extent is limited. (5) Anchor-free mechanism: An idea to get rid of the anchor frame mechanism is to convert the object detection task into the estimation of key points, that is, an object detection method based on key points. object detection methods based on key points mainly include two categories: corner-based detection and center-based detection. Corner-based detectors predict object bounding boxes by grouping corner points learned from convolution feature maps. The object detection framework based on center prediction is called Center-Net.Representative point (RepPoints) detection method. This method can automatically learn the spatial information and local semantic features of the object, which improves the accuracy of small object detection to a certain extent.The fully convolution one-stage object detector FCOS (Fully convolution one-stage) avoids the problem of too many hyper-parameters and difficulty in training in methods based on the anchor box mechanism. The mainstream research directions in the future include three aspects: feature fusion, context learning, and super-resolution reconstruction.\u003c/p\u003e"},{"header":"2. Method and material","content":"\u003cp\u003eThe medical mark to be recognized is a white 24mm diameter hollow ring with a central hollow diameter of 10mm and a thickness of 3mm. The ceramic material can be imaged well under CT and has excellent color-based optical recognition and tomography-based imaging recognition performance. It can fit well on the human body surface and accurately position the function in the body surface environment. The recognition device used is a speckle method structured light camera, which can generate 2D, depth map and point cloud information and is an RGB-D information camera. The specific identification and positioning method proposed in this paper is to first obtain the 2D image and then obtain the depth map of the point cloud, so that a clear 2D image can be obtained. If the depth image of the 2D image and the point cloud information is obtained at the same time, the 2D image acquisition will obtain a noise image full of snowflake spots. This snowflake noise is actually the projection of a random speckle disparity map. Therefore, the difference between one simultaneous acquisition and two separate acquisitions lies in the speed and processing difficulty. The disparity image may be noisy, but it is faster to collect it once. The separate acquisition image is clear and easy to process, but it takes a little longer. According to the traditional algorithm, it is recommended to collect it twice. If high speed requirements are required for image detection, one-time acquisition can be used. In addition, the speed can be adjusted in the mode of image collection clarity from another angle, but the image quality will be reduced.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.1\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eComparative Analysis and Existing Issues with Traditional Methodologies\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe steps are as follows: First, In the image pre-processing stage, this method proposed in this paper compares pre-processing filters, high-pass filtering, histogram equalization, median filtering, mean filtering, and Gaussian filtering. Each has its own advantages and disadvantages in the comparison, but they all lead to the same results. The local effect is good and the other situation is unstable, so it is relatively recommended that histogram equalization and Gaussian filtering be used as noise reduction methods. The use of multi-classifier extraction is the core of this method. The second is to use open operation; the third is to use threshold segmentation; the fourth is to use connected areas; the fifth is to use area screening; and the sixth is to use circular similarity analysis. The seventh is obtain the 3Dxyz value center based on the fitted circle center mapping depth map. Circle extraction is based on the least squares calculation: the formula is as follows. For specific calculations, refer to the extraction formula below.\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAVcAAAAlCAIAAAB5zj0xAAAHz0lEQVR4Ae1cYZHuKgzFQjXUAh6QUA1YwEEd4IBroAowUAN1gAfe3M1MJhMopezX+7qz6S+WQkhOwiFQvlVZHkFAEPjdCKjfbb5YLwgIAllYQIJAEPjtCAgL/PYIEPsFAWEBiQFB4Lcj8AoWWNdV3Xx+u9/EfkHgcwi8ggVSSvM8K6Wcc23TUkpaa6VeoXZbVXkrCPwUBN4ynfZ9h2xg27Y2dtu2CQu0IZK3gsAtBN7CAjnnEIJSapqmfd/bNggLtPGRt4LALQRexAI5Z2utUkprnVK6ZYY0FgQEgWEExlnAOaeUWpZleOyyI277rbXlW6kRBASBJxAYZIEQwrIsKSVjzGfVOo5jmialVAjhI5Kf2D6s6zp9PZ9S8puWVvV5wvBv6lntvu+7MUYpZYz533PAlBIkpPM8053pvwdzAJZqGFQxZ5WDLLAsy3MTAM7/lFLUDUzvzj/PnBdC6Ew3YozWWqpJjBHi1Xv/fRI8jsM5d3kmivaC5nS2NPQ5Mx+lfaRwywTn3LqudFxjzLZt8J0oxkhfDZT7PQtHUdZaCua6rvChalkWpucTYMJZGH4l11pjpLVh+WxYDrLANE3fd1jDx3CDYJ5n6qFG++qrM7dZa5mDq92xElIedA/WhxDas9cY00Zp33et9XEcKLOnAKtEiUxVnzMQ2gP19xow4WyitneXQHZtte96Nud8BqZzrvRLPyxtPelbYwxE43EcWutpmujbnPMZLMNhyeTnPPo7AqVUO77Lke7WQJZ4BkGPtKrPeoKpFL7v+zzPtH7/emhNWb5kgcsGpUyo8d6zXOZMnyoIZ2Kxvr/XmAksl0wpxRhLXkN9cs6XjrtsQKXRcglmjLGkgL+z5YGLKsgCYCMd4hKWsbCktkN5JBeIMf4DFkgpwQGB977U+7KGokkbG2PYXsYYM03TcRxwNslmF/bVWuPKv20bpAZnjaFXe4bEGBmzQAYEo1hr2VvUJOecUlJK4bRp63MGBRXIyp1dmAnHcczzrLUGacuynMXJtm3YLKUELo4xMtdQrS4nOfMsBRNuo5xt3xiY3vv09VTvsHUiQzVvlykL0LswnbAMhGWpz4MsAIs57nmwcOaMUjm8SlRm42VjVnPmLTp5YGcIOS0sCI2B1nWFOX8cR6ctbRbALShofhyH9945tyyL974xH6A9Bv2lPmdQMMTon51dmAnee1AG8kTgcSqWlpVSsN7CaRxA2kgwL1mAehamkHMOg80511hOEEy6Ua9uGzuRoZa2y8gCsCPAQTthGQjLUp8HWaAcbKDGez92TFj1FmQxpRrrus7z3AjBnPO6rhhSpQSsafwmAttAAd1P62F1xVCgr1i52p21gT+rUJQtB1i7qoMxBtf2hiFnaQJTDAmXFZjkqmdjjLjNhk9aTDj+WTUE39LCx8GksDd4iupAy51hSbuU5VezAKTo1cSstITVVL1VjRXYj7H0e9s2diQxAHc7FziLPFwkmUXsz7PurNnYbraKXim5qoP7enLOzjncs9DTb5DTyQI4aDsXqHoWU/3jOBhroFgoVA1hbeDPTmSqfauVOLRzbpomRKzauKwcCMtSyDgLVI9P6ACU5CiR96yoIMdai7tHKrmzXDqsGit4Z7HtgAG4B1gAshI8gGhYitHTaDNGAf29qjqEEIwxlxPvH7BAzhlSPMpHVbiqhpQty4gq29ytoUPP89w+aSqFD4RlKWScBUpZn60JIcCh3bDYqs/KlRauA2itQwiQdwBZsDMC3ID169NmgVJgCAGOBqy1IQRQAO+xMHNwK9vWh/VqN8a3nb1KE+DbG4tmYwxLrIBoLhcS1KfnG0HpWfjMBmCCKHoVhwp/FEw6UFk2xmC2C6cSLPDKLrSm6gLaoKc8yAIsf+4Z6Vabfd8/ciWhjGbqbzh9hHM42M/jmVyZgwBN3LKizQLsgH2aJlgHgINwTbBfT86ZqgS57uUsKs3v1L+zIzMBhbN1HtgN3+act227G0LtHUHOmXoWx2JL5TzPIQR25/VpMFGZsoCHkTjz4coAxmHZhdUMhCWT8JeRy6rLmnVdS2q/7NXfAI4D2hu5fmksoLdtu9ySsCipHhz0K9BoqbVuH0nmnPF8i8rpWQGY4bT7B8ulCTFG5jvnHDOzOmO/qVXVs/RQMMZYjdv3gHkXgTMWvivnHgvAzdknXEj1ttZeTlTa/m7ZWos5WLXvtm3rumLgnl3Sqva9VQm3PtpLOlygpN/Sz6673Rr6U42ZCdWZBj8SxeXu7O7g91WinoWdFDoRqBxYwHuPyrwKzFsIfDAs77EA3ANpT6FblpSN4TigfVBX9rpb0w5EuLwBWVl5YfvuWO32l5fwAXNc00Dzp/Fp68zeggl//vyB35iWuumvB+rL3xEwad/8E/CB83Z2yIr/0gpTlReC2Wn+Z8PyHgt0qjjcDDbqyNPDcqSjICAI9CPwIhYAqu65ONE+des3XloKAoLA4OngQ8AtX0+PcHYE3dNF2ggCgsAZAm/JBbz3nb8jhk969NTnzDapFwQEgR4EXsEC+KshesWwXe6xTdoIAoJADwKvYIGzu8YNIuixTdoIAoJADwKvYIEeRaWNICAIPISAsMBDwIpYQeDHICAs8GNcJYoKAg8hICzwELAiVhD4MQgIC/wYV4migsBDCAgLPASsiBUEfgwC/wEusbAvvdVh+wAAAABJRU5ErkJggg==\" width=\"343\" height=\"37\"\u003e\u003c/p\u003e\n\u003cp\u003eBut this is only the algorithm stage. In practice, external assistance is very important. The steps are:\u003c/p\u003e\n\u003cp\u003eStep 1: Optical stability light environment observation. First of all, there is a light environment parameter table in the medical environment. Secondly, various on-site light environment sampling.\u003c/p\u003e\n\u003cp\u003eStep 2: Contrast enhancement of the observed object. It is a good way to use a surrounding black edge ring with strong black and white contrast to highlight the marker, which can ensure stable and clear extraction. The material selection of this contrast object is very important. The mirror metal material is not recommended for selection, because it seems that the light perception contrast is very strong, but in a certain some angles have high contrast, but some angles have very low contrast. To achieve good optical contrast enhancement materials, it is recognized that the best light-absorbing material is carbon nanotubes. However, this material is not only expensive and has a long cycle, but also has the characteristics of being difficult to obtain. Therefore, the simple and easy solution used in the experiment is to use matte material, which is often a rough and porous surface. In practice, the optical performance of medical scene markers is almost the same.\u003c/p\u003e\n\u003cp\u003eStep 3: Limit the scope of observation and the angle of observation. The observation angle is limited. It is normal for the observation angle to be within 30 degrees. The recognition deviation is large at an excessively large angle, so it is better to limit it to a certain angle. Otherwise, the set circle fitting deviation including ellipse fitting will have poor fitting. For example, it will be very stable if the observation is limited to a relative angle of 30 degrees and a height of 400mm. The observation effect is shown in Figure 3.\u003c/p\u003e\n\u003cp\u003eIt can be seen that this method can only observe at a small angle and has poor stability, and is also sensitive to ambient light. The advantage is that it is simple to deploy, lightweight and quick to use, but it is difficult to avoid the problems of depth map mapping loss and optical blur.\u003c/p\u003e\n\u003cp\u003eThe optical blur value will cause the coordinate positioning accuracy to fluctuate below the stable sub-millimeter level. Depending on the product, it will be between 0.2-0.7mm. The ideal observation posture and observation distance are one solution, but it can only avoid part of it and limit the scope of flexible application in practice. For problems based on optical blur coefficients such as light interference, the optical blur changes with distance, as shown in Figure 4 below.\u003c/p\u003e\n\u003cp\u003eIf direct RANSAC and DBSACN point cloud fitting are omitted, the instability will increase due to the sparse point cloud density at different distances. Algorithms such as RANSAC themselves are called uncertainty algorithms, resulting in unstable extraction or failure. Common methods of point cloud fitting often have uncertainties.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe RANSAC algorithm itself is a non-deterministic algorithm. The clustering algorithm will also become less stable as the point cloud sparse values are used. Other effective methods such as arbitrary surface cutting circles are used. Ring extraction and non-directional random cutting will lose the direction accuracy, lose the original intention of improving accuracy, and become meaningless.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2 Fourier-enhanced multi-modal point cloud refinement method\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor structured light cameras, point clouds are characterized by collecting multiple disparity maps. The information is richer than a single shot of 2D images. Compared with optics, it is more accurate in producing camera optical blur values at different distances and can also avoid depth mapping. Mapping loss occurs because this method is not used, so the point cloud extraction effect in this scene is theoretically more ideal. However, on-site extraction of small object point clouds of 24mm rings is a challenge.\u003c/p\u003e\n\u003cp\u003eBased on this, a multi-modal method of using fast Fourier transform to enhance image details is proposed. Indeed, enhancing the resolution and meticulously extracting the predefined image contour, followed by the retrieval of a 24-60mm point cloud, facilitates the refinement of point cloud fitting and registration, thereby elevating precision. This approach circumvents the pitfalls of optical blur and depth map mapping loss. Moreover, it exhibits superior resilience to optical disturbances, ensuring the stability and accuracy in feature enumeration and classification\u0026mdash;a quintessential attribute for applications demanding medical-grade precision and stability.\u003c/p\u003e\n\u003cp\u003eFirst, we use fast Fourier transform to enhance the image details, and then use a single-step object detector (such as YOLO, SSD, etc.) combined with the depth map mapping ROI method. In this paper, we adopt YOLOv5 as the baseline model to extract the region of interest (ROI) through a single-step object detection network and the identified ROI is subsequently segmented from the image. Second, a BLOB operation is conducted on the segmented area to delineate its contours, which are then rendered onto an empty array corresponding to the original image dimensions, creating a mask image. This mask facilitates the cropping of the depth map, yielding the target area\u0026apos;s ROI depth map. The depth map is then transformed into a point cloud, producing the target point cloud. Fitting extraction based on the point cloud\u0026apos;s shape and registration culminates in the retrieval of precise, stable, and minimally interfered position information for small targets. Third, We determine the center point of the target based on the size of the rectangular frame, and use the coordinates of the center point to find the corresponding mapping point in the depth map to obtain the coordinate value of the target point. It is worth noting that the single-step detection network usually processes three-channel images. For single-channel images, they need to be converted into three-channel images. When the target cannot be recognized, adding unrecognized samples for retraining can solve the problem. In practice, sparse sample situations are extremely common. By augmenting the dataset and fine-tuning the model, the model performance can ultimately be improved to a higher level. The specific processing flow is shown in Figure 5.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2.1 Data acquisition and pre-processing\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA high-quality dataset is the foundation and premise for a superior model, crucial for enhancing optical performance and point cloud effects. In private domain scenarios, we collected and created a diverse range of real-shot data under laboratory and hospital CT room environments, featuring various angles and simulated situations. Adhering to medical building standards, the lighting conditions in our experimental setups ranged from 130 to 750LX, incorporating common environmental interference tools during data collection. We captured 2D and 3D images using structured light cameras to accommodate multi-modal experiments, thus employing a fusion labeling approach for both 2D and 3D data. Specifically, 350 2D images were annotated using the LabelImg tool, while 330 multi-modal fused 3D point clouds and 370 point cloud and image fusion samples were annotated with the Xtreme1 tool. To expand the dataset in later stages, we utilized model-based line data augmentation methods, potentially increasing the dataset size by 2 to 5 times.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2.2 Fast Fourier transform enhancement\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePerformance on images is often affected by the aspect ratio being too small and the image pixels not being obvious. It is a common practice to use image enhancement and linear difference. image enhancement methods include common contrast sharpening, contrast enhancement, brightness enhancement, etc. However, image enhancement will also bring about the problem of larger low-order values and poor stability. Fast Fourier transform is a good method, but it has not been applied to 3D small objects before. Fast Fourier transform (FFT) is a An algorithm in digital signal processing changes in the frequency domain: Fourier transform splits the signal in the spatial domain into signals in different frequency bands, and converts the pixel image into a spectrogram.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe idea of frequency domain detection is to detect from the spatial domain to Frequency domain, perform appropriate filtering in the frequency domain, select the frequency band you want, and then return to the spatial domain. There are two key factors, one is to generate a suitable filter, and the other is the conversion between the spatial domain and the frequency domain. This paper uses Gaussian filtering. The sigma value of Gaussian filtering is the standard deviation of the Gaussian kernel function, which determines the width and shape of the Gaussian kernel function. The larger the sigma value, the wider the width of the Gaussian kernel function, and the smoother the image; the smaller the sigma value, the narrower the width of the Gaussian kernel function, and the more prominent the details of the image. The method is to set two Gaussian filter kernels, one with a sigma value equal to 3 and the other with a sigma value equal to 15. After passing through two Gaussian filters respectively, the two images are subtracted, so that they are converted to frequency domain processing, and then after converting it back, we will find that the image with a very low threshold difference becomes a large gap and the outline is clear and obvious. The pseudo code is as follows:\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAlgorithm 1: Enhanced 3D Object Recognition using Fourier Transform and Gaussian Filters\u003c/u\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eInput: Directory of image files\u003c/p\u003e\n\u003cp\u003eOutput: Display images and image variable\u003c/p\u003e\n\u003cp\u003e1: for each image in the directory do\u003c/p\u003e\n\u003cp\u003e2: # Convert image to grayscale\u003c/p\u003e\n\u003cp\u003e3: # Apply Fourier Transform to convert the image to frequency domain\u003c/p\u003e\n\u003cp\u003e4: fft_generic(GrayImage, ImageFFT, \u0026apos;to_freq\u0026apos;, -1, \u0026apos;n\u0026apos;, \u0026apos;dc_center\u0026apos;, \u0026apos;complex\u0026apos;)\u003c/p\u003e\n\u003cp\u003e5:\u003c/p\u003e\n\u003cp\u003e6: # Generate Gaussian filters for background weakening and defect enhancement\u003c/p\u003e\n\u003cp\u003e7: gen_gauss_filter(ImageGauss, 15, 15, 0, \u0026apos;none\u0026apos;, \u0026apos;dc_center\u0026apos;, Width, Height)\u003c/p\u003e\n\u003cp\u003e8: gen_gauss_filter(ImageGauss1, 3, 3, 0, \u0026apos;none\u0026apos;, \u0026apos;dc_center\u0026apos;, Width, Height)\u003c/p\u003e\n\u003cp\u003e9:\u003c/p\u003e\n\u003cp\u003e10: # Subtract the generated Gaussian filters to highlight defects\u003c/p\u003e\n\u003cp\u003e11: sub_image(ImageGauss1, ImageGauss, ImageSub, 1, 128)\u003c/p\u003e\n\u003cp\u003e12:\u003c/p\u003e\n\u003cp\u003e13: # Apply convolution in the frequency domain using the subtracted image\u003c/p\u003e\n\u003cp\u003e14: convol_fft(ImageFFT, ImageSub, ImageConvol)\u003c/p\u003e\n\u003cp\u003e15:\u003c/p\u003e\n\u003cp\u003e16: # Convert the convoluted image back to the spatial domain\u003c/p\u003e\n\u003cp\u003e17: fft_generic(ImageConvol, ImageFFT1, \u0026apos;from_freq\u0026apos;, 1, \u0026apos;n\u0026apos;, \u0026apos;dc_center\u0026apos;, \u0026apos;real\u0026apos;)\u003c/p\u003e\n\u003cp\u003e18:\u003c/p\u003e\n\u003cp\u003e19: # Analyze the image for object recognition (eg, Blob analysis)\u003c/p\u003e\n\u003cp\u003e20: Perform Blob Analysis on ImageFFT1\u003c/p\u003e\n\u003cp\u003e21:\u003c/p\u003e\n\u003cp\u003e22: \u003cu\u003eDisplay the enhanced image \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eHow to view the spectrogram description (information includes: phase, frequency, and frequency band content):\u003c/p\u003e\n\u003cp\u003e(1) Each point is not a pixel, but represents a different frequency or frequency band, and the center of the spectrogram is low frequency, which gradually increases from the center to the surroundings. Sometimes the center of the spectrum is not at the center of the window.\u003c/p\u003e\n\u003cp\u003e(2) The spectrum chart is symmetrical up, down, left and right.\u003c/p\u003e\n\u003cp\u003e(3) The brightness of each point in the spectrogram represents the amplitude of the frequency band (and the degree of grayscale transformation). If the bright spot has a certain width, it means that the frequency content at this point is relatively rich.\u003c/p\u003e\n\u003cp\u003e(4) There are bright spots or bright lines in a certain direction in the spectrogram, indicating that there are edges with drastic grayscale changes in that direction. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2.3 Training details\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe use a GeForce RTX 4080 for training, and the batch size is set to 128. Usually each training is 500 epoch, and the accuracy rate is 99.8% as the inference model. Two parameters that are often adjusted are ROI pooing and Anchor box.\u003c/p\u003e\n\u003cp\u003eSetting of hyperparameters: In the loss design, \u0026alpha; and \u0026gamma; focal loss are set to [0.75, 0.75, 0.25] and [2., 2., 2.] respectively. The three loss weights are set to\u0026nbsp;\u0026lambda;\u003csub\u003eloc\u003c/sub\u003e = 1 , \u0026lambda;\u003csub\u003ecls\u003c/sub\u003e = 1 and \u0026lambda;\u003csub\u003eh\u003c/sub\u003e = 1.5. The Adma optimizer[\u003csup\u003e20] is used\u0026nbsp;\u003c/sup\u003eto train for 70\u0026nbsp;epochs\u0026nbsp;with an initial learning rate of\u0026nbsp;lr = 2\u003cem\u003ee\u003c/em\u003e\u003csup\u003e\u0026nbsp;-4\u003c/sup\u003e, and the weight decay is set to 1\u003cem\u003ee\u003csup\u003e\u0026minus;\u0026nbsp;\u003c/sup\u003e\u003c/em\u003e\u003csup\u003e4\u003c/sup\u003e. We first adopt the warm-up strategy[\u003csup\u003e21]\u003c/sup\u003e, with 300 warm-up iterations and a warm-up ratio of 1./3 . In addition, the decay ratio of lr at the 40th epoch and 60th ecpoch is 0.1. During the inference phase, the prediction score threshold is set to 0.2 and the rotated NMS[\u003csup\u003e22]\u0026nbsp;\u003c/sup\u003ethreshold is set to [0.02, 0.02, 0.4].\u003c/p\u003e\n\u003cp\u003eConfiguration of single-step detector on data augmentation: flip lr: 0.5 (probability of left and right flipping); mosaic: 1.0 (probability of mosaic data enhancement ); mix up: 0.0 (probability of mixed images); hsv_h: 0.015 (probability of hue dithering); hsv_s: 0.7 (probability of saturation dithering) ; hsv_v: 0.4 (probability of luminance dithering); degrees: 0.7 (angle range of rotation); translate: 0.1 (panning range); scale: 0.5 (zooming range) , which can be increased by 2-3 times by configuring the actual image.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2.4 The process of masking the ROI obtained by cropping the depth map:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIt is to create an empty array with the same size as the depth map, and then only have the mask map in it. Set all the mask numbers to 1, and the remaining blank parts to 0. Then the mask map is actually a multi-dimensional array, which is also a matrix. Multiplying the array (matrix) of the depth map, you get the mask map. The mask part is the depth map, the depth information part, and the rest of the non-roi area is 0. Then remove all the 0 values in this array, and you get The depth map is cropped by the mask image. This depth map is the depth map of the area of interest, and the rest has been deleted.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.3 Point cloud refinement: fitting and registration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis paper adopts the refinement part of the progressive method of two superposition classifiers, and obtains strong robustness and high accuracy results through fitting and registration superposition combination. Since blob analysis is used to extract contours, it is often a fixed value, resulting in the extracted point cloud containing noise information in a few cases. Therefore, the fitting will lead to an increase in error, so a registration is added to improve the accuracy, because the registration is Extracted from the standard template, coupled with the effect of Tu-based loss, it has less impact on noise. In fact, fitting can extract the center of the circle. However, in order to prevent the error from increasing and unstable recognition, registration is added. These two are in a progressive relationship. If the fitting and registration errors are large enough, they will exit to ensure accurate positioning and small errors every time during the operation. This is due to the high stability and accuracy requirements of surgical positioning.\u003c/p\u003e\n\u003cp\u003eIn the process of extracting ROI point clouds, Gaussian filtering and binarization are initially applied to obtain the ROI contours, which are then used to generate a new image serving as an image mask. This mask can also be created by drawing contours. Subsequently, this mask is utilized to crop the depth map, producing the ROI depth map. For depth maps acquired with structured light cameras, this method can be directly applied. In cases where depth maps are captured using LiDAR and cameras, multi-camera calibration is necessary to establish the mapping relationship, typically requiring mapping to images of the same size. However, employing bitmap methods for mask cropping is impractical because such methods usually operate on UNIT8, and converting depth maps to UNIT8 format would result in loss of depth information. Instead, multi-loop techniques or array multiplication can be used for mask mapping, facilitating the extraction of the ROI from the depth map. Afterward, the depth map can be converted into a point cloud for subsequent refinement operations on the ROI point cloud. By removing noise and environmental information, the target point cloud is simplified into a simple annular shape, making RANSAC fitting easier. Direct extraction with RANSAC at the outset may fail due to small target size or environmental noise. In addition, Point cloud information is often richer than image information and is more stable and accurate for collecting precise locations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.3.1 Point cloud volume fitting method\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eExtracting refined point clouds is relatively simple, accurate, and stable. Since the mark is a donut shape, it is connected to the environment to form a cone. Therefore, the shape of the point cloud can be relatively ideal using the cone fitting. The specific method Refer to the cone fitting formula of the following fitting:\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOEAAAA8CAIAAAAfcUrcAAAGnElEQVR4Ae1czZHyPAx2D+kgBzqgAvZABcuNGw2wDTB7dgdwz3D2QAFbgAvI3vMWkAL8vYNmNHrtxLEdB0I+5eTEsiw9fiL/KCAMX4zAvBEQ8zaPrWMEDHOUSTB3BJijcx8hto85yhyYOwLM0bmPENvHHGUOzB2BnBwVfL0OgbkTbYR9mTk6whJuygh0I8Ac7caFn84HgWwcFSKbqozoNE1TlqUQQkqZUe1rVS3SKQ+k2Yg1T45er9fmcX18fDRN4wHijaoW6ZQH/4VzFD0/Ho9aa7ydc0EpVTyuQYPfyKkxgP9fOHq5XNq2HYPUc9pqrT8/P+u61lqfz2d/p+/ilN+LwdpJONo0zfPnVk+n6nENYtEn4NHc1yT5uZQSwqdSSkrp6XqkU8kWPr9hfo4qpcqyfMniT2u9Wq2sKVIptd/vk5F9sjun06lpmrqud7sdYDiFU8lovKRhHo7ihsnz3j/HPa31er3GN0RKicfqSqlOG9q2PRwO2ITKPN8dWIyu12v6piU4Rb1493Jmju73+5ef8kgpowKnh6NzcAcYFuvUu/OS2p+To03TYABo23az2QghiqK43W5lWRZFQWMDNcJfVkqBHmheVVVRFJvNpm8PZEUdv3JjTB9HqTuoZL/fCyGwd6UUllFmsJCgJNapQRveSCAnR92lm5QShhCWWWNwgUByPp8HiQ5H3H0zu2tDH0ddd6Atfd627ff3d9/b4vaFT2KVxDqFHS2gkJOjUkprt4TRtI8xEFFwyYgFd7Lu3Dp0DgAM5+CSY7Br1x3ojlrSNM31eqVmDKoNUUIVQjnQKbfhAp5EcxS3R+g8PukcVDqi2CShAHTv4zpVGDucfXG00x1jDA1pkPKhvQeWY5XEOhVoxluIxXEU6IikBA/xls5fUNU0zeFwOJ1OVnxNgOZ8Pu92Oze+uqro8Lu17pM+jrruQFvUrx+XqzDkSawSlA9RvjCZOI6C80hK69baZNCx3z+uZOyqqlJKQQ6mbVt/9IrdXlA7qYWWO1gFEf3r68ua5VEgpBCrJNapEBveRSaFo8YYSlNaxsMauhLVWhdFQbfD4ehIKYuigKwg6MSjg768duwxTR9HjTHoDjUYzAiJ6LSVVY5VEuuU1d1b32bmaF/syY5RX14776l7nzv3+z1hL2+BEK4kr1OWGSNvO22r6/rvGk8IAQGlUya830SO0lBK46gxJtcmye+DldcG4SkWbegOvBVa6zFfciQomcIpP7bhtZ1Ldq31drut69oYg+EfYQxXjpJjOWoRFPSOfG/QOE/BzWtP1ylovt1uq9Vq8Fskj834Aocrmc4ptBNWHSEHJtgECp22WTMPzXEkL6nTOQqhtJOjljNT3HbmtafoaPE6k+N052IdAyfgRjlKw2oUqqM4Smf8qF5ZeCYIAEEhdQLng/QrHNgXgkxVVZDcxoe4eUVfrCDqbjrTQulYjqJ9XJgbAuEZr6IoYK6nq0aYqeBbC/xeAh5qrTtXopTfmDKkq4i0mM0cnRu1nm0PnAxSJkH8Q15SASSxm4SDdS1NQVsTPaboqEyItxEcxTeDC7NFIGTILRlKQVgyCiHgI3/4VI0KYNnlqBUjXco+g6OWb3w7cwQS5no6g+O0jryEcwlYGFBJwMFajLpBFDlqxexBGCPi6KAuFnhHBCgFIasHXz9imQpg2WIkfIa72WyqqjLGVFW13W7dnzbwnukdGfJ6m2FSFkKUZam1hr/MEELsdjvIYMPCpiiKy+UCT2AN4J49QegVQhyPx848nHUyFeg8x9FAoFjMRsANpbbEv/edZ/7/inTfMUe7ceGnIQjgNn9Q2NpRDcpTAeYoRYPL0QiERMcQGU/HzFEPOFw1CwSYo7MYBjbCgwBz1AMOV80CgeVwFM9Q4Kwk9jt5+BU/JpBim89iMBdqxHI4+vd8+Ha7wTD9/v52ns95BlFrDWfObgbF04qrnoDAcjiKYP38/FgEDcwKJifrsGsuTIHA0jha17WbggsHbuTvV8M7YslwBBbF0T9//lgRNBwIYwzP8lFwPU14ORytHxcAd7lcYqOplQi53+9PGwPuyI/AQjgKDBuzK7e+Icd9PXwnkfynf370uTYEgYVwNMTVBJm+X/EnqOImyQgwR33Qdf6K39eA6yZAgDnqA9X9Fb9PmuumQYA56sMVFqPuj3R9bbguNwLM0dyIsr7cCDBHcyPK+nIjwBzNjSjry40AczQ3oqwvNwLM0dyIsr7cCDBHcyPK+nIjwBzNjSjry43Af79VkS62WxneAAAAAElFTkSuQmCC\" width=\"225\" height=\"60\"\u003e\u003c/p\u003e\n\u003cp\u003eExplanation: (1) Geometric properties of the cone: For a cone with (x0, y0, z0) as the vertex and the axis aligned with the z-axis, any point (x, y, z) located on its surface satisfies the above relationship, here, the numerator is the horizontal distance from the point (x, y, z) on the xy plane to the vertex of the cone, and the denominator is the vertical distance from the point to the vertex. This ratio is equal to the tangent of the cone\u0026apos;s opening angle. (2) Calculate the theoretical z coordinate: According to the above formula, we can rearrange the formula to calculate the theoretical z coordinate. \u0026nbsp; The pseudo code is as follows:\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e# Algorithm 2 : \u0026nbsp; \u0026nbsp; \u0026nbsp;Calculate the theoretical z using the cone equationtan\u003c/p\u003e\n\u003cp\u003eTan_theta = np.tan(theta)\u003c/p\u003e\n\u003cp\u003eTheoretical_z =z0 +np.sqrt((x- \u0026times;0)**2+(y - y0)**2)/tan_theta\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWhere, theoretical_z is the predicted z coordinate value based on the cone model.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e(3) Calculate the residual. The residual is the difference between the z coordinate of each point in the actual point cloud data and the theoretical predicted value. During the optimization process, the goal is to minimize the sum of squares of these residuals for all points.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.3.2 Tukey loss registration method\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eICP (Iterative Closest Point) has many registration types. For medical labeling scenarios, a robust registration method with high stability is required. This paper uses the\u0026nbsp;Tukey\u0026nbsp;loss robust registration method, which has good robust effects. Meanwhile, it can output very stable results for the point cloud registration method. The formula is as follows:\u003c/p\u003e\n\u003cp\u003eThe loss \u003cem\u003e\u0026rho;(r)\u003c/em\u003e for a given residual r is:\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAATkAAACUCAIAAABndskiAAAP/klEQVR4Ae1dO3LbOhTlHtQlTUaZUVagmUxap1DtmcgdO2cBTBagcc14A3KvSZMGw972AtC5UeGSTs8F8CW5CR5E8AMCoAiQR4UN4nNxcS4O8SE+UYkfEAACISAQhaAkdAQCQKAEV1EJgEAYCICrYdgJWgIBcBV1AAiEgQC4GoadoCUQAFdRB4BAGAiAq2HYCVoCgWlylXO+2WyWy2We57AxEJgGAhPkapIk6/U6yzJLC+V5vlwuo9PfYrHgnFtKRnKHCKRpemqi309pmjrMwhNRU+MqY2y73RZFYY9vnucfP35Ey2yPpBMJxEmddyVjDFx1gvmwQuI4Zow5yQNcdQKjEyGc8/1+X5ZlmqZxHLfLBFfb8fEitCiK7Xbrqo8Krnph1FMldHioE+dUahhPk+oDF0VxfX3tqtcKrvpWhYuiuLu769QKXO2EaPwI4Or4NhhMAzHV1zkWBVcHM4I7weFyNcuy79+/O5kScwfnWSVlf36dWaZp2vkpDlzthHH8CCFylXQ+HA7jwze2Bpzzzjl8nYEJuDq2JTXyD46rbifDNBDyPQrn/OrqqmXGgTGGeWDfraijX3BcTZLE1RcmHXyCiFPbuooFD51ELcsS7WoAhg6Lq+mfXwCwnl1FS2TA1bNbrH+GAXE1z/P2zl7/0k8nheXQAFwNoCoExFXLpiMAY9ipqDMubcoBXG1CxiP/ULhq2W54hPhgqujM9zZlDq42IeORfyhcrZ0+8QhHP1QxXtoNrvphwFYtQuEqOsCtZvwbaNwNBld14B05TihcNW4xRsa3LvvD4bBYLIbYMmrc+wBX6wzlmZ8BV4uiuLi4iKJI3RhpM2RqAcZAyRZp4walaZokSVEUnPMkSdqVaYG6NqEx/uBqLZ5+eT4/P7958+bp6Ulfrf1+T3vo4ji+uLiQV+Qa15X23AcS257pEKHEz5eXlyzLNptN51bEFqhr1TOegQNXa/H0y9OGBpzz9Xotr257enp69epVL+brwHF/f//u3bufP3/qRPY5Dq0lWq1Wt7e38juuU2cV6qYk79+/19kEV0n+6dOnzka+kiSIx0ntX7XkamXhuI20Ftv/+PHj7du30+AqbU87Ho+9tvjrD0TNuDql6QC5IoGrf9FQO07gqlxRVHee5+v1Ooqi9Xp9PB7VCE0+KtRNMd+/f//t27em0CZ/cLUJGY/8jdmV5/n19XWlI2csrR2RybSr7cVsCq2FuikyuCojg3b1Nxq73U4eqRJA4KpcUVy5a6FuEg6uysiAq7+PxqONaeKwPAIIXJUrihN3E9RNwsFVGZm5czWOY3ESdOUTK7gqVxR7dwvUTcLBVRmZGXGVvsVXCCljUXG3c1XnoNqKQHrsnAU1lkzyZbVlt+bhurU6q5Lbo6mD/5b4LUFms0RmqVrU8CRoRlwty7Ioipubm8ocUpMlKhVdjkafFnXOKJBTkbudqzaSy7KkuVlaliC7iahRFJnpXJGsFqrio/8FtZKw8mjGOrNUlaw9fJwXVysj0nZ7tHDVpo1q56qN5LIs5Woqu6mkNi22Kq0dPeOV97LYvplSWrNUcr5+uufF1TRNr66uxAC1/aTZ4LgqvwVkt6h5xlytlSbE1jqMVwjK0sxYZ5ZKztdP94y4WhTFly9fNM+hpV5fy91Tw9V7Y8lyQtktal6tpwhtcZgltOeMmQSzVC3F9yRoRlxljH348GGxWGgeHWjQroqtJKLpVoeInW1ULTF0JMvbYmW3qGqqZB2x1C1v74OILGRHrQ5yhE63GevMUnUqM3qEGXE1SZLDn5/m/IoBV3XMacZVHckyN2S3SKtyVQS1O1Rp8suo4hai1FQiSNNhxjqzVJoqjRhtLlwVS9toivLx8fH29rYd9xC5Kl5DtbSs9WwHgULNEtpzxkyCWSodHMaNMxeuivXi1OurbFWttUELVxlj1JgIbtRKqPVsXw9sI1lusWU3qeFKcm2hVE8nc0tYCyEDOxeuymXWdLdwVVNCbbR2rtYm0feUmxTZrS+hKWZfaU6+2YCrsjnAVRmNE3eIXJXXP8juk4IZPfSS5motBLgq2wpcldE4cQ90LsSg7SqtzRIXRrs920lTmsN3nBlXzVKd2N7LB3C10SyBcrWxPAEGmLHOLJX/8ICrjTZy2D7IeaizPnIo3DICSZJ0Hrkmxyd336G1KsFPH3C10S4Dtavi61Fjxgj4h4AZV3E22j/8PP7vtiUcjqstSxc9RrdGNfoApr/NsEZEs5fxVx9wtRlUb0KC4KrmDI03oHYo0mubYYes02Bja4Krp0B6+WRs3drSDNSulmVp1rWrVXJ0z17bDHtpazywd8XVh4eH2tFyk//hcNjtdlTGIXocGK821p/huGq/ULZR6bMH9Npm2Es749UU5+fqr81b6/WargsRZXTe4wBXBbZVx3Bc1bn9paqNl899txn2KoTxdO45uXo8Htfr9fX1tXpCsvMeB7jaWH+G4+pkhqx9txk2Yq0E2AxnnHOVc75arW5ubqIoYoyJPnCapi3nmDvvcYCrSjX55zEcV413hP5TzZf/fbcZ6utt3AEuy9ItV/M8Xy6X8j5kwdWyLDnnnz9/zrKsUrQhehzgagXk/x8H5apNu/G/iqO6xIdi/W2Gmvoaf60h+W65SoRcrVZinknmKuWYZVmFsUP0OMDVxvozKFepaQ36OjODbYaNWJ8GWM69nZ+rgrFiU/QQPQ5w9bSaSE9Dc5U+3mgeKCPpNXGn8acagctYXBUKDNTjAFcFwlXHGbhq2dmrahz+M+f86upKvVuoV8ncclUcSbVcLkkx0QeWbxKQzx6o7XFwzvXP+qotL7haC8tvzzNwlbawUX+pUY/ZBNi3qASVW66q8AuuqkGD+oCrjfCeh6uUfZZl379/17wQoFHjkAP0z4LtLCW42gnR+BHcTq6ek6vjYzchDcDVAIwJrgZgpOFVdMXV4TXtlwP6wI14oV1thMbvAHDVb/v80e75+fn169f39/dOdHXbSjtRCUJ0EBDTsDqRA4ozqXbV7ScQcDWgeiyrCq7KaHjqds5VWggq3wEx0BkIngIaglp0Y61soyiKDG7f8b+saFf9txE0BAK/EQBXUQ+AQBgIDM7VllGfWDbpCqqWvFxlATmzQoBWEXqyZntYrnZerODqMgWqQK4Wqc2qOqKwKgL7/Z6W/jp/+6dpqnPvmarS4H1gnWM4bHYVy0WiiSVPXoGyYnCHhUCWZWKvoqvK6QQBZ+0qHTwTRZHYTFBp5WifwXK5zLJM3rnrZPL2cDhsNht1e/7vtxF+c0WgkyGHw2GxWERRdH19TYuxxdYZxhjtsGF/fovFQtzfKVKlaSrXahJF57yUZSmGeGI7Ll2rSXL2+z3FF1PWsthazd1wlXO+2+1+nVtBxaNWXr1glzF2eXn5+PgoLkcinWqbX7EXSa5pAi+5MNinIqMBtyYCSZJcXFwcj8eiKEQNlDu9NECjTQWi4UnTlM4rFAfcVWp1HMdEvziOiQjH4/Hh4YH2VF1cXNBbYLvdioMmxMEDRVEIsWopHHC10jAKiorXichVFEP4kEONWYnQ/lgUBbWrcuHbkyB0VgjQN1h5fFSZKBFclTu9aZpeXl5SpaIqSkR6eXnJsmyz2VBQpVZT/WeMff78ebvdFkVxd3dHjbacKTVvZIVasaqBHHBVvHJIepIkBEqFgfIbq6JHJSaF6rerFD/PcwFfRT4egQDnnBo3gkKucnRMIedcdHpFG7jf76k3u16vOefE+dVqdXt7S/RTazVj7OPHj1+/fn18fNxut4fDQeQrZ1qWJWOM5Ktia+3lgKtiaqsoit1uJ/rfooGljOU3VkUV8Var+Pd9bMmiryjEnxICjLHK7Ct1ZcuypHkWqrTU7hFp5RaIerkvLy+CbMfjkTqxapVjjNHiNhrKimkq6n5mWfby8vL169c8z0UWqtha8G25ShpcXV1FUbRarQ6Hg8hGqEI+TYSsdKFFcgOH+pIzEIIkc0BA9NrkSssYW6/Xd3d3lUNh4zim07rpG2QUReJYYLVWc86pXynmlghPar3paxCtXRWzsKrYWhPYclV9Y8nZqCWRQ8mtvpnUOJo+kzkjW7O8iDYrBGy5WhlYV7A781oIh010pSB4BAKjI2DFVZ0+Z0tbp5O8F0Dgai+4EDksBKy46ltRwVXfLAJ9HCIArjoE0xdRYuIEu219MYkLPSbFVbdnuLiAdxwZ+/1efKavfKsYRyHk6gKBSXHV+QDYBcJjypAXyoypB/J2gQC46gJFX2VUPnH7qib00kIAXNWCKdBIUz0lLFBzWKoNrloC6G/yyroZfxWFZnoIgKt6OAUYa7fbWV64FmChp6wyuDpN66ZpSqtSOee0mWOa5ZxTqcDVCVpbHG5Ax3RgW+80bAyuTsOOKMX0EQBXp29jlHAaCICr07BjYymw3rARmtACwNXQLNZTX6w37AmYv9HBVX9t41YzrDd0i+f5pYGr58d8nByx3nAc3N3lCq66w9JvSVhv6Ld9urUDV7sxmkAMrDecgBHB1QkYsbsIWG/YjZH3McBV701krSDWG1pD6IUAcNULMwynBNYbDoftmSVPiqtPT0+vXr16eno6M4g+ZEdrHnDAkg+2GEiHSXF15me4FEVxc3ND96wMVF0gdkQEwNURwXecNba/OQbUM3HgqmcGsVAnTVO6WIhurBWXgFmIRFKPEABXPTKGjSq/rqn+8uULXexrIwdpvUUAXPXWNP0UY4x9+PBBXD3WLzFih4AAuBqClTR0TJLk8OcXx7FGdEQJDwFwNTybqRqLJYS0mebx8fH29laNBp+gEQBXgzbfX+XFunz6yop7MaZgVKUM4KoCCTyAgJcIgKtemgVKAQEFAXBVgQQeQMBLBMBVL80CpYCAggC4qkACDyDgJQLgqpdmgVJAQEEAXFUggQcQ8BIBcNVLs0ApIKAgAK4qkMADCHiJwKS4+vz8/Pr16/v7ey+hhlJAwAoBcNUKvtET01ZV47+j6w8F9BGYFFeLothut7huVN/8iBkQAuBqQMYaUNU0TaMooqvQB8wGoi0QAFctwJtWUs75w8PDtMo0qdKAq5Myp3FhGGPYSWeM3nkSgqvnwRm5AAFbBMBVWwSRHgicBwFw9Tw4IxcgYIsAuGqLINIDgfMgAK6eB+ez5kKnLkVRhOttzor7wJmBqwMDPIb4/X5PC0LiOMbs7hgWGCRPcHUQWD0RSkeQ5nnuiT5QwwYBcNUGPd/Tcs632y1ujvPdTnr6gat6OIUZS5wbHKb60PoEAXD1BI4pPYjD+KdUqDmXZVJcLcsyjmMsQKcKvdvtMFKdErenxlWM0Kh2pmlK7yxcoDwZuk6Nq2VZpmm62WzmvIs1jmOx+xyfWMFVrxHgnG82m+VyiU6g13aCcn0QmGC72qf4iAsEgkEAXA3GVFB05giAqzOvACh+MAiAq8GYCorOHAFwdeYVAMUPBgFwNRhTQdGZI/AfWVtxPj3QNKYAAAAASUVORK5CYII=\" width=\"313\" height=\"148\"\u003e\u003c/p\u003e\n\u003cp\u003eThe weight\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003cem\u003ew(r)\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003efor given residual\u0026nbsp;\u003c/em\u003er\u003cem\u003e\u0026nbsp;is given by\u003c/em\u003e\u003cem\u003e：\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAS8AAABzCAIAAABGhxO3AAAOX0lEQVR4Ae1dPZKjPBPmAhtscQMHvoGTTb2B4wkcks0FOMEcgPUF7JxyMgnFATwHUDaJgw3xATgA337uqn4bIQkB4s/qCaYkoZ/Wo36klhqLoPL4ryzL/X4fhuHpdCrL0mMkuOuLQCBYhBRzCAFUjKKIeTgQfkAyeP6FYSiEGFiht8X9ZWOWZZvNpigKb8feScfLsjwej8BAtDWYkP2w9ZeN0fOvH2pcChEQQtxuNxoNwzBJEkzhgD0CnrIRZnFWGntFscxZFMVms8myzDI/Z6MIeM1GVhqqCk7CQojj8chb8X5gMhttccvz/HK5+KNnZVleLpc8z20BeuaL45g3jZ0Qo5mZjRQNdbgsyziOz+ez+vFLp6Zp+v7+bjkHZc+/l8Zj3M4xG1vwLYricDikadqSb22P7d0SQojD4XC/381dFELEcWzOw0/NCDAbTfgAFV9ve9nVLdFKSGm7eLvd2F41KZbmGbNRA0xVvfC5aw+3RJZl+/1eabIKIcIwBO8//Nfl1GLND54IMBu1ipAkiSdaZemWUHposyyjPIQwu460WmV8wGxUwyOE2G63nphbkp2pRqSqiqLY7XaeYKIDYdR0ZqMaXuU6oM66/lR7t4Q/9sIso8psVMC+hEWgh3uzn4ewk1tiCcgoBuxVkpiNipGcdwUY6N7s5CHs4ZaIooi3hQqlcZHEbJRRnPco1Yl7s9UhAX2WtouWbgnD4aoMJcc7IsBslAGb0Rhz6N5sJWRvt8SM+MhD9XJxZqM8pHP97tH5mmxYxIa4JUDO13sjQtaDOeLMRhn1JEmiKJJTx4+PsVkd6WR4LojGH4SZW2A2ygMwyynFSO7NkaxKw6oro8nxLggwG2toOTcXa7XrIyMtYlVVjbHkMhv1IznoCbOxBt/fv3/DMPzz508tdeTISCsYSD1G5Z+fnz9//vz+/h4ZGO+q95SNwLrL5SIN+CxsHGP5ov1ybnszGym8DsOeslG3Ynx/f//48WPKtXEC29i5YclsdMhAWhWzkaJRTc9G3bxQE2tYxHkT87KxKIqPjw8lJIZHND/MgLOcnFMxmmFmYw2T6dnY1b0ZRdF+v0/TNAgCS31y7iFcKRvTNI3jGNCwR6+mHyNHmI01gKdno73vDtVou93Gcdxpt2nfSg0OTWR1bMzzfLfbARWhT+MdYmsws0pmNtZgmp6NXY9YkiTpcUW6263jitgIPHx/f5fu9WE21vR+3ohuKzUxG2G56/STiE5LIoL8qmyU7Ha6b7zf77vdrslDwATZiC8JWpr9COkYAV4ba6hOzMYeDpV+bHS7mrmtrTYAFhGgnNJuv9/vcMCTJEkYhoaXaZGNZVm+v78v5HMszMba+FveEFMrMyCyUjYKIXa73VwaTBdAyW6nj2zWRp2JNGBIBxVlNtbgm5iNPZbiJayNi2IjvUmMshHGFc9vHo8HHekoirbbrXn9pPmnCTMbazgvnI1RFMGlbF0PctzalitiI4xunueHw0E6U93v929vb12RrKmL6wizsYbowtlYk7VLxHM2AlTgb4Qw7BthuJdwfgNSMRtrSs1srMGhiaxubWz2A09x4EzVcN7TLDteCrOxhi2zsQaHJrIQNjbtdtw3JknSvHMZ37/Bgvv9/uvrCy5K32w2QojN82+uAypmY03jmI01ODSRhbCxKR2ysfloFSnMxtowMRtrcGgizEYNMEOTmY01BO3ZCLbQwP2GfXM1KbtH3PLHbW1de2NYAA2PurYyS35mYw12GwcgvgUSBIGfbHR7QlsbAL8jzMba+NuwEQrAfaTMxhp8HBmGgKds/P7+/vnz5+fnp4Qes1ECRBnltVEJy/BEZmMNQ2ZjDQ5NhNmoAWZoMrOxhiCzsQaHJsJs1AAzNJnZWEPQCRvRuSw5oJtvYK30TJXZWFMadxFmYw1LJ2ys1WiMWLJRYrUyamyncuuTYDaa0e79lNlYg86SHlX1f/0e/nsc++ZqUnaPuGWj29q69+ZlS3jKRt13L+zpwWyc62XOl+ViVc3DxiiKzJ66LMuauyyHw6Cb3S3ZSHeG9NeuXSW0N4y71izld2tb6tCTGh0pCuCPqh4jSd5a7dRstL+Xqd+P3Fs7DBl0lVuy0bKV1mzMxlaIlBmSJDHP5spSXRNBV4fMtl1bnJqNnS4sdHsLKEJjoJzhERZ3GGA2WoKZZZkQAjKXZXk8HjFqWcMqsk3Kxq73CMLk5HYWhDp1dg6z0UZrJ7ZU4W4b3KYKIY7HY1mWNqKuK48DNoJ+B0FA1/SPj4+iKOAR3BdKw4gRaH8QBEmSwF16Ek90JiXWYB8oy/J0Om232/P5rCvFbNQhQ9NHYmOaptvtNggCPKxG1QKnDszLyfOvqqo0TcMwlG6vA+piJU0Fe3t72+/3reoKv9Ght+bEcYy+JbpCSC1SoLqGHbARmqR36WVZFoYh2BJCiDiOq6rSKTognqbp19cXAo3doFVhYo8AUP10OpnnVJ2QPVq0KcKWKqBUlmUURbvdDm4ET9MUaQDqQbUfjgDTND2dTo/Hg1qtSZJgJbjNUSpYq7pWVUWPEvEQvizLOI7RTla2aDP0yjzO2EjFhSUI2Xi73cBBt91usRsoDUy01+tVyRODI4EebOKkhbctYP0YwLXRwMmJ2ThZc25XM7e1wQeYkX7SxI16BeNYFMXv37//LUd5noNS4dpI90FwUSpwWKlgWC1qhaSuVVXRk3/Mj+oEdEV7kLZI83QKu2ejEOLr6wuJd7lcgGa6VY7OLk3RDWxsZrZJgZlSsoex4GT0gBabEz9K4jbglj9uawPM6ecPqOpTjgEBdrsdfjEOF0CoJE3Tx+Mh7UeUCoZN6NS1efGx9DUxQ4u9x84ZG6lwEM6y7Ha74WKI/Zdkxdu7pHSIOmcjTr10+LHpyUxHaHGlbBzovQRDBjGXtBwWJVwqpUN46RwBly/8nMavX7/AgsX6lQrWqq7SLAC10dYNLWLTXQOO2Xg+n8uyhK6ez+fr9YoCIUUxBYlBdwX0KcyFuAWVHnW1VGlxCitNXzgbgb0Ga5z2hYaVukUzdAq7ZSPdwuErhzBXSguU5NsAqxXOWqVKqqq6Xq/F82+z2TQVDFVUp670CAOOJKuqorOnrsVOSEqZHbMRug1dfX9/p1tB6Im0IjXnRUk+HW2kbF2jOltrYjbCOiBhYu4LVQhzTvrULYwD2VhVVRD8p3hgNKVpWlVVnudhGMKxHzDzeDw+Hg/Y70i+DThlyfMcTLB/FzF+fHyUz7/4+VeWpU7B6NrQVFek/fV6FULs93s4h6eWGoSbLVLYu4b/A6VrSSl/WZaXywUSdd/9ac7QaPdLtWE9+/2+ObEpM3dK1JnN07PRjECzU3RBaD7VpSitNV3m1vSBbKRUhLbAVxEEwW63g+MZSIezBpzW6XoFZz//PByY/3w+w82otBIdvGZ1BX4eDgchBITBug7DkLrHlC22omfI4IyNhjboI2kbQB81w/SIufl0SIpkAmFV07OxOUOhMMpA1/xoX3VagZVNY+JANmI9HJAQmJqNSntVkgmibo0rqYnlsFEniSQwRmE6o3dp686HsUjXJrCgLsBs1CEzMH1qNoK4rW/9jrcqggA6BZ1+bbSfnuihF5SCTUurBvRYTs11MhvN+PR+Og8be4vrqqCOjT0+bzpcJHsrAA6f7vc7fQGlVQC3m8aqqpiNrZj3y8BsrOHW78SyVkX3iG5qaNaUJMn2+ddqnWJZ+8qxSGtAdyLdWpAzmBHwlI26+1RnYSP4OVoJBrLBZ5Xsf8Rgv/CaFYU+dW760sp9DnvKRsOK0enU15Xq6DwutH6UGQP0qTJsn1NZXJc4BsN1bXmVzmyUh1vnoZLzuY63qjh1ZMMbYXmeU/dXUyLnO0ZoYqRqm/L7lsJslEd8LjMMDFGDV5ByANziZiqO1JFWOWVAOW6NALNRhmok605uRhUviuJwODh590gIcTgc4OeCqqb6p82IT3+hV1KS2SgP1LxzPxASXtqUJbOOj0dF6Ud91hJxRisEmI0KmFq3cIoy7pLg1+VmK9TQWpqm+GKnIVvvR9Rg7l0JF1QiwGxUwLIEY0wIgT/UVoioSoI3ofEtalWWoWlLQGZoHxZcntmoHhxeAZS4zGs1KEV6pURmo3o0bRyA6pKvm8oL49hjy2zUIszrgAQN2wsSIM6jzEYtpPMermrFmunBSN7LmXqz0GaZjaaBcegANDWz+GejukwW3/vpBGQ2tmDtxAHY0sayHzMVJxsfZmM71AMdgO0NLDjH2N7LBXd9BtGYjbag93AA2la9yHwTeC8X2e85hWI2zok+t80IUASYjRQNDjMCcyLAbJwTfW6bEaAIMBspGhxmBOZEwFM26u7FmXMouG3vEWA2eq8CDMBiEGA2LmYoWBDvEWA2eq8CDMBiEGA2LmYoWBDvEfCUjfxTPe81f4kAMBuXOCo6mfBbgvghbl1OTl8jAszG1Ywa/bAuvel4NR1gQdsQYDa2IbSY59JP76XoYsRkQfojwGzsj92UJcFGpRcf8/I4Jf7TtMVsnAbnoa1QMxXqgsv/hRBDq+byi0GA2biYoTAK0rzDjtloBGyVD5mN6xg2ZuM6xmmYlMzGYfhNVbq5b0yShP0cU8E/UTvMxomAHt6MdIgqRYfXzzXMjgCzcfYhsBWAGqt8oGqL2qryMRvXNFxwshoEAbVRIZGmrKlLLCtBgNlIwFhzUAhxu93W3AOWvWI2vogSxHHMvse1j6W/bNxsNvTVlvUOZPO4db198VxyT9nIX7zxXO+X2X1P2VhVFX8Qbpka6bNU/rIRDLwkSXwefu77ohDwl41VVd3v993zL8/zsiwXNTAsjIcIeM1GGO88zw+HA/vrPNT+pXWZ2bi0EWF5/EWA2ejv2HPPl4YAs3FpI8Ly+IsAs9HfseeeLw2B/wEGRBHM2mXHVQAAAABJRU5ErkJggg==\" width=\"303\" height=\"115\"\u003e\u003c/em\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003ethis\u0026nbsp;formula\u0026nbsp;is the Tukey loss function and its corresponding weight function, which are used to calculate the loss value and weight of the residual.\u003c/p\u003e\n\u003cp\u003eWhen the residual r is less than or equal to the threshold k, the shape of the loss function\u0026nbsp;\u003cem\u003e\u0026rho;(r)\u0026nbsp;\u003c/em\u003eis similar to the square loss, but when the absolute value of the residual increases to close to k, the growth rate slows down and remains unchanged after exceeding k. This design reduces the weight of large residuals in the total loss, thereby improving robustness to outliers.\u003c/p\u003e\n\u003cp\u003eThe weight function\u0026nbsp;\u003cem\u003ew(r)\u0026nbsp;\u003c/em\u003eis the derivative of the loss function to the residual r, which is used to adjust the influence of each data point during the iterative optimization process. When the residual is less than or equal to k, the weight decreases as the residual increases; when the residual is greater than k, the weight is zero, that is, these possible outliers are ignored. This helps the optimization process focus more on data points that fit the model.\u003c/p\u003e\n\u003cp\u003eThe specific method is to use the estimated normal vector method to calculate the normal for the two point clouds, which is necessary for the ICP alignment of the point to the plane. The initial alignment uses the identity matrix np.eye(4) as the initial transformation. This means that the source and object point clouds are initially aligned without any rotational or translational alignment. Point-to-plane ICP alignment, use the point-to-plane ICP method to initially align the source point cloud and the object point cloud, apply the resulting transformation to a copy of the source point cloud and display the result, and then try to use the Tukey loss function Variant ICP method. The Tukey loss function is a robust loss function that is less sensitive to outliers and works well for processing data containing noise or outliers. The result after segmentation is shown in Figure 6.\u003c/p\u003e"},{"header":"3. Result","content":"\u003cp\u003eAccuracy is a multiple factor, including algorithm error, sensor error, mechanical assembly error, CT bed displacement offset value, ground vibration value, etc. It can be analyzed by category to produce a combined theoretical value, but the real value accumulated in practice in the actual environment is more worthy of investment. For analysis, especially in CT rooms where heavy equipment is running at all times, the before-and-after comparison algorithm can use numerical comparisons under actual measured vibrations. For a clear description, Table 1 is used as the depth map mapping result, and Table 2 is used as the multi-modal point cloud. Refine the results.\u003c/p\u003e\n\u003cp\u003eTable\u0026nbsp;1: Comparative acquisition accuracy is as follows, previously using depth map mapping method\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"560\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"5\"\u003e\n \u003cp\u003eTraditional mapping algorithm places A-Mark recognition results\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003eserial number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"61.853832442067734%\" colspan=\"3\"\u003e\n \u003cp\u003eRecognition results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003eStraight line distance difference (\u0026plusmn;)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"61.853832442067734%\" colspan=\"3\"\u003e\n \u003cp\u003eXYZ\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.36693\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-4.17822\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.7445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e1.332731176\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.35583\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.3430135\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.15797\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.821445875\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.34972\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-4.1770315\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.6408\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e1.369640695\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.4186\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-3.2620718\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.05582\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.515140576\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.39088\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.344352\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.36777\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.880754035\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"85.20499108734403%\" colspan=\"4\"\u003e\n \u003cp\u003eRecognition failed\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.36693\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-4.178227\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.7445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e1.33273741\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.408417\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.3450217\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.47183\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.927111199\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.39088\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.344352\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.36777\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.880754035\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.34972\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-4.1770315\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.6408\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e1.369640695\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"85.20499108734403%\" colspan=\"4\"\u003e\n \u003cp\u003eRecognition failed\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.373352\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.3436825\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.2637\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.845102971\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.34972\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-4.1770315\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.6408\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e1.369640695\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.487637\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.3450217\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.47183\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.828496361\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"85.20499108734403%\" colspan=\"4\"\u003e\n \u003cp\u003eRecognition failed\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e61.408416\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-2.3450217\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e364.47183\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.927110601\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\"\u003e\n \u003cp\u003e60.384144\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\"\u003e\n \u003cp\u003e-3.2602117\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\"\u003e\n \u003cp\u003e363.8482\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\"\u003e\n \u003cp\u003e0.599204743\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\" valign=\"bottom\"\u003e\n \u003cp\u003emean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\" valign=\"bottom\"\u003e\n \u003cp\u003e60.85401488\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\" valign=\"bottom\"\u003e\n \u003cp\u003e-2.991194841\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\" valign=\"bottom\"\u003e\n \u003cp\u003e364.1049053\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\" valign=\"bottom\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.973682037\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.795008912655971%\" valign=\"bottom\"\u003e\n \u003cp\u003eStandard deviation \u0026sigma;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.429590017825312%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.501098721\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.607843137254903%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.817832392\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.816399286987522%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.314885796\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.351158645276293%\" valign=\"bottom\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable\u0026nbsp;2: Actual performance test table using this algorithm\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"559\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"5\"\u003e\n \u003cp\u003eMulti-modal\u0026nbsp;point cloud refinement\u0026nbsp;placement A-Mark recognition results\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.311270125223613%\"\u003e\n \u003cp\u003eserial number\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"62.9695885509839%\" colspan=\"3\"\u003e\n \u003cp\u003eRecognition results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.719141323792485%\"\u003e\n \u003cp\u003eStraight line distance difference (\u0026plusmn;)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.311270125223613%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"62.9695885509839%\" colspan=\"3\"\u003e\n \u003cp\u003eXYZ\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.719141323792485%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.95081\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32222\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.15085\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.136184428\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.53772\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.14727\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.06187\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.716752595\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.91811\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32147\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e359.94763\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.154767426\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.87316\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.23348\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.35431\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.285762242\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.73445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.110382359\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.93445\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.32185\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.04922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e0.10323271\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\"\u003e\n \u003cp\u003emean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\"\u003e\n \u003cp\u003e57.83826818\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\"\u003e\n \u003cp\u003e1.297944545\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\"\u003e\n \u003cp\u003e360.0781091\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.133580156\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14.336917562724015%\" valign=\"bottom\"\u003e\n \u003cp\u003eStandard deviation \u0026sigma;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.892473118279568%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.121477809\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.07168458781362%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.053937284\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.939068100358423%\" valign=\"bottom\"\u003e\n \u003cp\u003e0.097565242\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.759856630824373%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThrough the above operation algorithm, the repetition accuracy is identified from the standard deviation value of the accuracy of \u003cstrong\u003e0.973682037\u0026nbsp;\u003c/strong\u003emm to the repetition accuracy error value of \u003cstrong\u003e0.133580156\u003c/strong\u003e.\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eRecognition was originally unrecognizable due to misoperation, but it will no longer occur after multi-modal point cloud refinement is used. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u003c/p\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eThe Fourier-enhanced multi-modal point cloud refinement method proposed in this paper provides a novel solution for 3D small object detection at the medical labeling level. Although existing 3D object detection methods have achieved significant progress on large objects, but the detection methods of small objects, especially small medical markers for use in surgical navigation, are still under development. This paper effectively improves the detection accuracy and stability of small objects in this application case by combining a single-step object detection network and point cloud processing technology.\u003c/p\u003e\n\u003cp\u003eThis method has a positive impact on medical mark recognition. Previously, the recognition method relied on optical reflective balls, which were difficult to fit on the human body surface. There was a level of positioning of surgical instruments. For body surface points, this method predominantly used three reflective balls, posing numerous challenges in their placement. Inferring from a certain distance from the epidermis further complicated matters. However, the use of white ring markers can accurately reflect skin points, particularly beneficial for surgeries requiring precise fitting, such as those involving the chest and abdomen. This method has undergone practical testing and is point-based.\u003c/p\u003e\n\u003cp\u003eBased on point cloud 3D general neural networks are mostly for large objects, making the application of this small object method still relatively uncommon in practice. Meanwhile, this method will increase the amount of calculation and the calculation time, but the overall time is influenced by various factors due to the robustness of the algorithm. Therefore, it is feasible to process lower quality images and sparse point clouds, which can offset the time required for camera acquisition.\u003c/p\u003e\n\u003cp\u003eCameras are generally divided into 5 modes from high to low based on parameter adjustments (levels of quality and speed): static quality, static balance, static fast, dynamic balance, and dynamic fast. The difference is the clarity of captured images and the frequency of projected images. Higher modes provide faster imaging but with relatively more blur, resulting in larger accuracy errors. Conversely, slower modes yield better quality and higher accuracy. Previously, the acquisition time required in static balance and static quality modes was 1.7 and 2.1 seconds. \u0026nbsp; However, using the dynamic fast mode can complete acquisition in just 0.5 seconds. Hence, by enhancing the graphics card configuration and adjusting camera speed settings, this time can be significantly reduced.\u003c/p\u003e\n\u003cp\u003eThe main advantage of this method is its high-precision positioning capability for small objects. Traditional 3D object detection methods often suffer from performance degradation when dealing with small-sized objects due to insufficient features and high positioning accuracy requirements. By utilizing 2D images to extract ROI areas and combining them with point cloud technology, this method can handle small-sized objects more accurately, which has positive significance for surgical navigation.\u003c/p\u003e\n\u003cp\u003eHowever, this paper also has some limitations. Firstly, the accuracy of the method is highly dependent on the quality of the 2D image and the accuracy of the mask. Any errors during the 2D image processing stage may lead to a decrease in the accuracy of the final point cloud. Secondly, the fitting and registration processes in the point cloud processing step require precise tuning and optimization to ensure optimal performance. Despite these challenges, the method proposed in this paper showed excellent performance in experiments. Especially in repeatability tests, this method can significantly reduce positioning errors, which is extremely important for practical surgical applications.\u003c/p\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eThis paper successfully developed a multi-modal point cloud refinement method for 3D small object detection, particularly focusing on the precise positioning of medical markers. For medical surgical navigation, body surface positioning is found to be more suitable for accurately placing needles on lesion areas of the body surface. This method combines fast Fourier transform, single-step object detection network and point cloud technology to effectively improve the detection accuracy and stability of small objects in complex medical environments. Detailed experiments and analysis demonstrate the practicality and effectiveness of this method in medical surgical navigation.\u003c/p\u003e \u003cp\u003eThis method can position any needle insertion point based on the body surface. In the future, it holds promise for applications in body surface positioning for puncture ablation across diversified surgical scenarios. As a novel technique in the field of surgical navigation, it has positive significance for puncture ablation surgical navigation of tumors.\u003c/p\u003e \u003cp\u003eIn summary, this paper provides an effective method for 3D small object detection in surgical navigation, and contributes positively towards enhancing the safety and accuracy of thoracoabdominal surgical navigation.\u003c/p\u003e "},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis article uses a laboratory plastic human breast model as the test object, and no animals or human participants are designed. No unethical measures or practices were adopted,using the experimental part of the prosthesis, we ensure that all materials meet biocompatibility and safety standards.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and/or analyzed during the current study are not publicly available due [for patent and trade secret reasons] but are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDeclaration of interests:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contribution statement.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eZezhao Guo contributed most of the creativity/code/experiments, Yanzhong Guo had equal contributions,did a lot of solid work, and Zhanfang Zhao reviewed the manuscript as a supervisor.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe research work of this article received financial support from the Hebei Provincial Social Science Fund project. No. (HB20TQ003)\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgement\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported by the Hebei Provincial Social Science Foundation Project (No.HB20TQ003). We thank Beijing Yingrui Pioneer Medical Technology Company for experimental data support and Professor Zhao Zhanfang of Hebei GEO University for correcting the writing of this article.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eKrizhevsky, A., Sutskever, I., \u0026amp; Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25. \u003c/li\u003e\n\u003cli\u003eWang, Lingyun, et al. \u0026quot;Axial super-resolution optical coherence tomography via complex-valued network.\u0026quot; Physics in Medicine \u0026amp; Biology 68.23 (2023): 235016.\u003c/li\u003e\n\u003cli\u003eAswani, Ashish, et al. \u0026quot;Attention is all you need.\u0026quot; Advances in neural information processing systems 30 (2017).\u003c/li\u003e\n\u003cli\u003eHe, Kaiming, et al. \u0026quot;Deep residual learning for image recognition.\u0026quot; Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.\u003c/li\u003e\n\u003cli\u003eQi, C. R., Su, H., Mo, K., \u0026amp; Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 652-660).\u003c/li\u003e\n\u003cli\u003eQi, C. R., Yi, L., Su, H., \u0026amp; Guibas, L. J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30.\u003c/li\u003e\n\u003cli\u003eB.Graham, M. Engelcke, and L. Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In CVPR, pages 9224\u0026ndash;9232, 2018.\u003c/li\u003e\n\u003cli\u003eC.Choy, J. Gwak, and S. Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR, pages 3075\u0026ndash;3084, 2019.\u003c/li\u003e\n\u003cli\u003eS.Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In CVPR, pages 10529\u0026ndash;10538, 2020.\u003c/li\u003e\n\u003cli\u003eW.Zheng, W. Tang, L. Jiang, and C.-W. Fu. Se-ssd: Self-ensembling single-stage object detector from point cloud. In CVPR, pages 14494\u0026ndash;14503, 2021.\u003c/li\u003e\n\u003cli\u003eH.Wang, L. Ding, S. Dong, S. Shi, A. Li, J. Li, Z. Li, and L. Wang. Cagroup3d: Class-aware grouping for 3d object detection on point clouds. arXiv preprint arXiv:2210.04264, 2022.\u003c/li\u003e\n\u003cli\u003eD.Rukhovich, A. Vorontsova, and A. Konushin. Tr3d: Towards real-time indoor 3d object detection. arXiv preprint arXiv:2302.02858, 2023.\u003c/li\u003e\n\u003cli\u003eA.Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving the kitti vision benchmark suite. In CVPR, pages 3354\u0026ndash;3361, 2012.\u003c/li\u003e\n\u003cli\u003eA.Dai, AX Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nie\u0026szlig;ner. Scannet: Richly annotated 3d reconstructions of indoor scenes. In CVPR, pages 5828\u0026mdash;-5839, 2017.\u003c/li\u003e\n\u003cli\u003eA.Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y. Zhang. Matterport3d: Learning from rgb-d data in indoor environments . 3DV, 2017.\u003c/li\u003e\n\u003cli\u003eI. Armeni, O. Sener, AR Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic of large-scale indoor spaces. In ICCV, pages 1534\u0026ndash;1543, 2016.\u003c/li\u003e\n\u003cli\u003eS.Song, SP Lichtenberg, and J. Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, pages 567\u0026ndash;576, 2015.\u003c/li\u003e\n\u003cli\u003eX.Xu, Y. Wang, Y. Zheng, Y. Rao, J. Zhou, and J. Lu. Back to reality: Weakly-supervised 3d object detection with shape-guided label enhancement. In CVPR, pages 8438\u0026ndash; 8447, 2022.\u003c/li\u003e\n\u003cli\u003e]Rabbi, J., Ray, N., Schubert, M., Chowdhury, S., \u0026amp; Chao, D. (2020). Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network. Remote Sensing, 12(9), 1432.\u003c/li\u003e\n\u003cli\u003eDa, K. (2014). A method for stochastic optimization. arXiv preprint arXiv:1412.6980.\u003c/li\u003e\n\u003cli\u003eHe, K., Zhang, X., Ren, S., \u0026amp; Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).\u003c/li\u003e\n\u003cli\u003eNeubeck, A., \u0026amp; Van Gool, L. (2006, August). Efficient non-maximum suppression. In 18th international conference on pattern recognition (ICPR\u0026apos;06) (Vol. 3, pp. 850-855). IEEE.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Abdominal Percutaneous Ablation, 3D Small Object Detection, Surgical Navigation, Machine Vision, Medical Marking, Surgical Robot","lastPublishedDoi":"10.21203/rs.3.rs-4488932/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4488932/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eNavigation for thoracoabdominal puncture surgery is used to locate the needle entry point on the patient's body surface. The traditional reflective ball navigation method is difficult to position the needle entry point on the soft, irregular, smooth chest and abdomen. Due to the lack of clear characteristic points on the body surface using structured light technology, it is difficult to identify and locate arbitrary needle insertion points. Based on the high stability and high accuracy requirements of surgical navigation, this paper proposed a novel method, a muti-modal 3D small object medical marker detection method, which identifies the center of a small single ring as the needle insertion point. Moreover, this novel method leverages Fourier transform enhancement technology to augment the dataset, enrich image details, and enhance the network's capability. The method extracts the Region of Interest (ROI) of the feature image from both enhanced and original images, followed by generating a mask map. Subsequently, the point cloud of the ROI from the depth map is obtained through the registration of ROI point cloud contour fitting. In addition, this method employs Tukey loss for optimal precision. The experimental results show this novel method proposed in this paper not only achieves high-precision and high-stability positioning, but also enables the positioning of any needle insertion point.\u003c/p\u003e","manuscriptTitle":"Fourier-enhanced multi-modal 3D small object optical mark recognition and positioning method for percutaneous abdominal puncture surgical navigation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-13 02:44:50","doi":"10.21203/rs.3.rs-4488932/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"5f57414a-5e9d-476e-941f-d739da100139","owner":[],"postedDate":"June 13th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":33073445,"name":"Biological sciences/Cancer/Cancer therapy"},{"id":33073446,"name":"Biological sciences/Cancer"},{"id":33073447,"name":"Health sciences/Oncology"}],"tags":[],"updatedAt":"2024-07-12T07:06:38+00:00","versionOfRecord":[],"versionCreatedAt":"2024-06-13 02:44:50","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4488932","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4488932","identity":"rs-4488932","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}