Machine Vision with CMOS based Hyperspectral Image Sensor Enables Meat Freshness Sensing

Machine Vision with CMOS based Hyperspectral Image Sensor Enables Meat Freshness Sensing | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Machine Vision with CMOS based Hyperspectral Image Sensor Enables Meat Freshness Sensing Suyeon Lee, Hyochul Kim, Seokin Kim, Jeong Su Han, Un Jeong Kim This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5551638/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Imaging spectral information and analyzing its properties of materials have become intriguing for consumer electronics toward food inspection, beauty care and etc. Those sensory physical quantities are difficult to quantify. Hyperspectral cameras, which capture its figure and spectral information simultaneously, can be a good candidate for non-destructive remote sensing. In this study, with the aid of a hyperspectral imaging system (HIS) and machine learning (ML), meat freshness is converted into a measurable physical quantity, i.e., freshness index (FI). FI is defined from meat fluorescence, which has a strong correlation with bacterial density. Combined with ML techniques, hyperspectral data are processed more efficiently. By employing linear discriminant and quadratic component analyses, FI can be estimated from its decision boundary after hyperspectral data are obtained at an unknown freshness state. We demonstrate HIS grafted with ML performs as artificial eye and brain which is advanced machine vision for consumer electronics including refrigerators and smartphones. Advanced sensing versatility utilized by computational sensing systems allows hyper-personalization and hyper-customization of human life. Photonics/optics Spectroscopy Hyperspectral Imaging Freshness Sensing Machine Learning Advanced Machine Vision Fluorescence Imaging Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 INTRODUCTION Imaging spectral information and analyzing its properties of materials have become intriguing for consumer electronics toward food inspection, beauty care and etc. Those sensory physical quantities are difficult to quantify. Hyperspectral cameras, which capture its figure and spectral information simultaneously, can be a good candidate for non-destructive remote sensing. In this study, with the aid of a hyperspectral imaging system (HIS) and machine learning (ML), meat freshness is converted into a measurable physical quantity, i.e., freshness index (FI). FI is defined from meat fluorescence, which has a strong correlation with bacterial density. Combined with ML techniques, hyperspectral data are processed more efficiently. By employing linear discriminant and quadratic component analyses, FI can be estimated from its decision boundary after hyperspectral data are obtained at an unknown freshness state. We demonstrate HIS grafted with ML performs as artificial eye and brain which is advanced machine vision for consumer electronics including refrigerators and smartphones. Advanced sensing versatility utilized by computational sensing systems allows hyper-personalization and hyper-customization of human life. Consumer electronics have been progressively developed to satisfy hyper-personalization or hyper-customization of human everyday life. 1,2 Utilizing proper sensors and analysis algorithm becomes one of the key factors to make them more attractive for consumers. Hyperspectral imaging system (HIS) can be the most promising since it measures plural quantities, i.e . morphology and spectrum, of objects, simultaneously. 3 Moreover, it becomes very powerful for consumer electronics since it captures the image of its figure and finger print non-destructively, remotely and etc. Recently, with machine learning (ML) based statistical analysis, by establishing a correlation between raw sensing data and ambiguously defined physical quantities, researchers have recently reported a growing versatility in the functionality and enhanced performance of traditional sensors when combined. 4–10 Furthermore, the utilization of advanced sensing versatility by computational sensing systems through different home appliances enables various functions including food inspection, healthcare, and beauty care. 11–13 With the steadily rising consumption of meat along with gastronomic flavor, its freshness and aging have emerged as the most intriguing quality control factors for food inspection. Defining the freshness or aging degree of meat as a physical quantity is intriguing, ambiguous, and strongly influenced by personal preference. However, traditional techniques for the evaluation of meat freshness or its aging degree are mostly destructive, non-portable, expensive, and difficult to access by general users. 14,15 On the other hand, optical inspection techniques can be a proper candidate for an alternative such as measurement of reflectance from meat surfaces as investigated over several decades. 16–18 Even though several approaches are suitable for non-destructive or portable system, these are lack of the underlying mechanism. For reliable and reproducible results, specific molecular changes should be considered. To investigate the freshness, biomarkers such as collagen, reduced nicotinamide adenine dinucleotide (NADH), and flavin have been deemed suitable. 19,20 These studies have monitored the ~390, 460, and 525 nm fluorescence as a function of time from pork specimens stored at room and refrigerator temperatures (4 o C), when stimulated by a 340 nm light emitting diode (LED) for up to three days. Also, other study tried to analyze hyperspectral image of meat fluorescence by 365nm excitation for meat freshness based on color analysis. 21 Thus, hyperspectral imaging system is considered suitable for extracting invisible information on meat freshness by imaging morphological and spectroscopic data collected from macroscale meat samples through ML grafting. In this study, meat freshness was clearly distinguished not by an RGB camera but spectral information from HIS. With high spectral resolution hyperspectral data from line-scan type with grating, freshness index (FI) is derived based on chemical change depending on the storage time of meat. Not only the storage condition utilized for defining FI, but we also show that FI can be extended towards various kinds of situations, frozen state, vacuum packaged state, and etc. Based on the knowledge from high resolution hyperspectral data, data size can be reduced both in the HW aspect, selecting suitable filters instead of grating, and SW aspect, adopting ML for data reduction. Hyperspectral data from filter array-based snapshot type with ML successfully demonstrates FI with the comparison with hyperspectral data form line-scan type and physical quantity-based formula. Considering form factors of HIS for home appliances, snapshot type and line-scan type with filter array can be applicable to long and short distance imaging for large area substances. By combining HIS with ML, meat freshness can become a tangible physical quantity for applications in daily life. Our findings demonstrate that machine vision with HIS enable meat freshness sensing in consumer electronics including refrigerators and smartphones. RESULTS AND DISCUSSION Meat freshness can be monitored in the refrigerator or by smartphone using CMOS based hyperspectral imaging system and machine learning based algorithm. Overview of meat freshness sensing by machine vision for consumer electronics. Figure 1 illustrate the user scenarios of meat freshness sensors realized by machine vision with hyperspectral imaging system (HIS) and machine learning. Complementary metal-oxide-semiconductor (CMOS) based HIS can be either line-scan or snapshot type. Line-scan type can be powerful when the distance between the object of interest and the sensor is limited, and its capturing area is very large to be captured by snapshot. For example, as shown in Fig. 1 , line-scan type storage compartment in the refrigerator can be more suitable than snapshot type because of limited field of view (FOV). Instead, snapshot type can be installed as an additional smartphone camera which performs diagnosis of its freshness at any places. Meat freshness can be classified by machine learning based algorithm prebuilt in application processor (AP) in the consumer electronics. Acquisition of Hyperspectral Data and Evaluation of Meat Freshness by ML Figure 2 shows the entire process behind obtaining hyperspectral data using HIS and identifying meat freshness with ML. In this study, the line-scan and snapshot types of HIS are used to image the morphology and spectral information of meat specimens. A line-scan type HIS is designed to move 365 nm LED arrays using a conventional grating for collected light dispersion, and a long rectangular window to excite and collect fluorescence signals over the meat specimens. The fluorescence signal transported to a commercially available grating is collected in the 3 dimensional (one spectral dimension and two spatial dimensions) hyperspectral data, as shown in Fig. 2 . For the snapshot type HIS, Fabry-Perot filters are fabricated periodically on a CMOS image sensor, working in the range 380 ~ 840 nm. The schematics for line-scan and snapshot types of HIS are shown in Supplementary Figs. 1 and 2, respectively. For details on the scanning area and spatial resolution for both types of HIS, see Methods. High spectral and spatial resolution of the grating-based line-scan type HIS is beneficial for investigating parameters correlated with meat freshness from the fluorescence spectrum. Meat freshness can be extracted from the hyperspectral data by processing data efficiently. Particularly, ML on hyperspectral data with the merit of data size obtained by snapshot type HIS, based on fundamental studies by line-scan type HIS, is conducted to evaluate the information on meat freshness. Hyperspectral images of the meat surface are decomposed into a series of spectral bands (λ 1 , λ 2 , λ 3 , λ 4, ...), forming hyperspectral data. The full spectrum of each local point is constructed by combining intensity and wavelength in the vertical direction. Typical fluorescence spectra of fresh and rotten meat specimens are shown in Supplementary Fig. 3a and b. The broadband from NADH at 490 nm and its enhancement in intensity with lowered freshness agreed with those observed in previous studies. 19 , 22 Conversely, the sharp peak at ~ 600 nm appeared to have a relation with myoglobin, which exists inside cells of mammals and is related to their breathing activity. The reference spectra of NADH and myoglobin purchased from Aldrich Inc. are exceptionally close to those of meat and are shown in Supplementary Fig. 3c and d, respectively. Traditional Analysis of Meat Freshness and 2D Freshness Index Map To understand the morphological and chemical changes in meat specimens as a function of refrigerator storage time (T ≈ 4 o C), RGB images illuminated with white LED and hyperspectral images of fluorescence excited by 365 nm were captured by a digital camera and line-scan type HIS, respectively. The process was conducted on the same piece of specimen under the same conditions for 17 days, and the results are shown in Fig. 3 a and b. The meat specimens were packaged in polyethylene (PE) wraps to avoid contamination and other handling issues. The RGB images of the meat specimens as a function of storage time were analyzed using the CIELAB color space, referred to as L * a * b *. L* represents lightness, a* represents green–red opponent colors with negative and positive values toward green and red, and b* denotes blue–yellow opponents with negative and positive values toward blue and yellow, respectively. The distributions of a * and b * values, represented by red and yellow bars, are displayed on the right side for each RGB image, while their average values are indicated by red and yellow dots in the guideline, respectively. With increase in storage time, both a * and b * values merge to 10 starting from the 7th day, implying that the color of meat turned less reddish and yellowish. In addition, distinguishing the state of meat freshness after the 7th day is impossible. Another type of RGB images defined by “RGB hyper ” was obtained by transforming hyperspectral images of meat fluorescence, as shown in Fig. 3 b, where the representative fluorescence spectrum of each storage day is displayed on the right side. With longer storage time, the relative intensity of NADH located at ~ 490 nm against 600 nm increased. Based on this spectral change, FI is defined as $$\:\text{F}\text{I}=\left({I}_{N}-{I}_{M}\right)\:/\:\left({I}_{N}+{I}_{M}\right)$$ 1 where \(\:{I}_{N}\) and \(\:{I}_{M}\) are the intensities of NADH and myoglobin peaks deconvoluted from the fluorescence spectrum, shown by the blue and pink shaded areas, respectively. Figure 3 c shows a 2D map of FI for each storage day, while Fig. 3 d shows the average FI value from each hyperspectral data, which increased monotonously up to the 10th day and finally saturated. The number of bacteria per unit area (CFU/cm 2 ) ( \(\:{N}_{bac}\) ) measured using the standard method (see Methods) gradually increased on a logarithmic scale, as shown in Fig. 3 e, where the pink shaded area indicates the inedible state of meat at \(\:{N}_{bac}\) > 10 7 (CFU/cm 2 ) 17 . Thus, FI was confirmed to have a strong correlation with meat freshness. However, the CIELAB color space analysis of the RGB images confirmed that meat freshness could not be distinguished after the 7th day of storage, although meat remained edible at \(\:{N}_{bac}\) < 10 7 (CFU/cm 2 ). The correlation among \(\:{N}_{bac}\) , storage date ( \(\:d\) ), and FI was obtained by fitting the experimental data shown in Fig. 3 d and e using a relation defined as $$\:\text{log}\left({N}_{bac}\right)=2.6\sqrt{d+6.4}-3.4$$ 2 $$\:\text{F}\text{I}\:=0.066\left(\text{log}\left({N}_{bac}\right)+2.5\right)$$ 3 Using these functions, guidelines were added to the experimental data by applying a width of ± 1.0 and ± 0.1 in Fig. 3 d and e. According to Eq. ( 3 ), FI > 0.63 for the inedible state and is indicated as the pink shaded area in Fig. 3 d. The preparation process and number of samples are stated explicitly in Methods. The FI of the meat specimens, one stored in a refrigerator and the other in a freezer, was monitored as a function of storage time to prove the representativeness of meat freshness at unknown storage or commercial distribution channel history. Predictably, the FI values of the sample stored in the refrigerator increased gradually to approximately 0.7, but those of the sample stored in the freezer remained at approximately 0.55, as shown in Fig. 3 f. At the first stage, the initial increase in the FI value for the frozen meat could be attributed to the structural changes in the meat specimen or an actual increase in \(\:{N}_{bac}\) before freezing. The color of meat specimens is particularly related to the oxidation of myoglobin. Meat is bright red, which looks fresh to the human eyes, owing to iron oxidation at the center of the heme ring in myoglobin molecules by attaching oxygen or water molecules. When the meat specimen is vacuum-packaged, its color turns dark brown, which looks stale to the human eye regardless of its freshness. The vacuum-packaged meat specimens are investigated as a function of storage time using the FI and a *, b * values from hyperspectral and RGB images, respectively. The color of the meat specimen was darker even on the 0th day, as shown in Fig. 4 , due to the desorption of oxygen molecules from myoglobin on the meat surface during vacuum packaging. Figure 4 a shows an RGB image under white light captured by a digital camera, an RGB hyper image, and the FI map of a vacuum-packaged meat specimen on the 0th day. In Fig. 4 b, a* and b * for the vacuum-packaged meat on the 0th day were distributed in a range similar to that of inedible meats shown in Fig. 3 a, and they remained similar until the 12th day, as shown in Supplementary Fig. 4a. The values of a *, b * and FI were monitored as a function of storage time for both vacuum- and PE-packaged samples, and the results are presented in Fig. 4 c and d, respectively. On the 0th day, the vacuum-packaged meat exhibited relatively lower average values of a * and b * when compared to those of the PE-packaged specimen. With prolonged storage time, the mean values of a * and b * of the PE-packaged meat tended to decrease, while they remained almost constant for the vacuum-packaged specimen, as shown in Fig. 4 c. From the 0th to the 12th day, the mean values of a * and b * changed slightly from 13.7 to 12.5 and from 7.6 to 4.7 for the vacuum-packaged meat, respectively. Furthermore, the color of meat could be influenced by various factors such as its freshness, pH, part of meat, and nutritional state. However, larger differences between the FI value of the PE- and vacuum-packaged specimens than a * and b * were observed. \(\:{N}_{bac}\) was monitored in the PE-packaged meat specimen stored together with the vacuum-packaged meat specimen (Supplementary Fig. 4b). \(\:{N}_{bac}\) for the present batch increased rapidly when compared to that shown in Fig. 2 because of different environmental conditions, including weather, preparation process, and distribution channel history. The representative fluorescence spectra on the 0th and 12th day are shown in Supplementary Fig. 4c. The PE-packaged meat exhibited a gradual increase in the FI value, which is proportional to that of \(\:\text{log}\left({N}_{bac}\right)\) , as shown in Fig. 4 d due to exposure to air. However, the FI value of the vacuum-packaged meat was maintained approximately to the 12th day because the meat stayed fresh by preventing oxygen adsorption on its surface. The freshness of the vacuum packaged meat was verified through the hyperspectral image of the meat fluorescence and its FI, as shown in Supplementary Fig. 4d. The gray shaded area in Fig. 4 d represent the guidelines that are calculated using Eqs. ( 2 ) and ( 3 ). The slight discrepancy between the experimental data (black dots) illustrated may have originated from the different relative ratios of fat and flesh contents in the meat specimens used. Since the characteristic band of fatty acids is located in a wavelength range similar to that of NADH 23 , Eqs. ( 2 ) and ( 3 ) can be modified to fit FI with a higher accuracy by excluding the areas of fat while calculating FI. Thus, the results suggest that meat freshness can be measured more clearly by using hyperspectral image sensors than by RGB image sensors. Expectedly, RGB imaging of fluorescence signal might also be considered to distinguish meat freshness. We simulated the RGB image of meat fluorescence by converting the hyperspectral image into RGB hyper images as a function of the storage time. We observed that the a * and b * values of the RGB hyper images did not change sensitively within the edible state (≤ 7th day of storage), as shown in Supplementary Fig. 5. Thus, hyperspectral imaging is confirmed to be a powerful tool for discerning meat freshness against RGB images taken under white or 365 nm LED. Since the freshness stage of meat is not easily accessible to consumers, the proper analysis tool needs to be devised for precise freshness evaluation. In an experiment, four types of packaging materials were investigated to understand their influence on the fluorescence spectrum listed in Supplementary Table 1. The results are shown in Supplementary Fig. 6a. PE and PVC wrap or zipper bags produced unnoticeable fluorescence signals, while vacuum packaging material produced a non-negligible intensity of broad fluorescence at approximately 470 nm. However, the condition was magnified more than 20 times, where 5 s integration with two layers was performed as compared with the experimental condition of 400 ms to obtain fluorescence from the meat specimen. On packaging the meat in PE or PVC wrap and zipper bags, the shape of the fluorescence spectrum was maintained while its intensity decreased, as shown in Supplementary Fig. 6b. The overall shape of the fluorescence spectrum was independent of the packaging material. This result indicates that fluorescence spectroscopy can be applied in daily life to analyze meat freshness when commercial food packaging materials are used. Hyperspectral Imaging and ML for Meat Freshness by Snapshot type HIS Furthermore, a 16-channel (CH) snapshot type HIS was fabricated as Fabry-Perot filters were formed periodically on a CMOS image sensor operating in the 380 ~ 840 nm range including a blank channel. To optimize transmission and each filter’s resonance wavelength, SiN films with variable thicknesses were stacked vertically along with Cu or Al reflectors at the top and bottom. The final characteristic transmission curves shown in Fig. 5 a were obtained by multiplying the quantum efficiency (QE) of the CMOS image sensor and transmission of each filter. Despite relatively lower spatial and spectral resolution compared to line-scan type HIS, the snapshot type HIS offered competitive advantages in several aspects, such as cost effectiveness and efficiency of computing resources. Using this methodology, a smartphone installed with a hyperspectral camera can be used to capture both outer appearance and spectral information of meat at any place to determine meat freshness. Figure 5 b shows the hyperspectral image of fluorescence from the meat specimen supported by a black Styrofoam tray, excited by 365 nm LEDs. The image was demosaiced into each CH, as shown on the right side of Fig. 5 b, where the enlarged CH distribution of the small yellow square in the hyperspectral image is shown on the right-hand side. Hyperspectral images from the same meat specimen were taken as a function of storage time. For the 0th and 15th day, two representative fluorescence spectra from the fresh and rotten states of the meat specimen taken by the snapshot type HIS, respectively, are shown in Fig. 5 c. Comparatively, a large enhancement at approximately 500 nm was observed on the 15th day as opposed to the 0th day. Thus, the results were consistent with those obtained from the line-scan type HIS, as illustrated in Fig. 3 . ML can be applied to reduce the dimensions of hyperspectral data and to extract features related to meat freshness from hyperspectral images. The reduction in data dimension can maximize the efficiency of computing resources, such as computing time and memory. Moreover, the risk of overfitting data resulting from a complicated analysis model can be reduced, and the dimensionality reduction can prove to be advantageous for the ease of data interpretation. Among various algorithms available for dimensionality reduction, principal component analysis (PCA) and linear discriminant analysis (LDA) are commonly used. 24 , 25 As LDA is suitable for supervised learning and classification performance 26 , LDA is opted for dimensionality reduction of hyperspectral data from 11 to 2 dimensions (indicated by \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{1}\) and \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{2}\) in Fig. 6 a). Even though the sum of percentage of variance explained will be increased up to 1 with a greater number of dimensions, we choose 2 LDA components as the sum of percentage of variance explained is already 0.989 and fluorescence signal to determine freshness of meat is originated from two chemicals. Four long wavelength CHs and one blank CH were initially excluded out of the 16 CHs that did not convey information on the fluorescence signal from the meat specimen. The raw intensity profiles of the remaining 11 CHs as a function of wavelength were standardized by their mean value and standard deviation. Furthermore, dimensionality reduction was conducted by subtracting the channel-dependent constant ( \(\:\stackrel{-}{{x}_{j}}\) ) from value of each CH ( \(\:{x}_{j})\) , \(\:j=1,\:\dots\:11\) , followed by the multiplication of two coefficients ( \(\:{\text{s}\text{c}\text{a}\text{l}\text{i}\text{n}\text{g}\:\text{f}\text{a}\text{c}\text{t}\text{o}\text{r}}_{1}\) and \(\:{\text{s}\text{c}\text{a}\text{l}\text{i}\text{n}\text{g}\:\text{f}\text{a}\text{c}\text{t}\text{o}\text{r}}_{2}\) ), which were determined using the LDA method. The data reduction process and coefficients are shown in Supplementary Fig. 7. Here, $$\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{i}=\:\sum\:_{j=1}^{11}{\text{s}\text{c}\text{a}\text{l}\text{i}\text{n}\text{g}\:\text{f}\text{a}\text{c}\text{t}\text{o}\text{r}}_{j}\bullet\:\left({x}_{j}-\stackrel{-}{{x}_{j}}\right),\:\:\:i=1,\:2$$ 4 \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{1}\) was plotted against \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{2}\) of the hyperspectral images of meat as a function of storage time, as illustrated in Fig. 6 a. With longer durations of storage time, highly scattered data were gradually merged to the negative values of the \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{1}\) , where each dot was extracted from dimensionality reduction using LDA. Even on the 0th day, the negative values of \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{1}\) appeared to originate from the fat tissues whose fluorescence was similar to that of NADH. The decision or evaluation boundaries of FI produced by QDA are shown in Fig. 6 b as contour plots. 27 Using information on FI, averaged over each hyperspectral image by line-scan type HIS as reference data, QDA was adopted to obtain decision boundaries of the FI for the two LDA components under supervised learning. By hyperspectral imaging of meat fluorescence by snap-shop type camera and extracting two components through LDA, the value of FI was estimated using the decision boundary contour plot shown in Fig. 6 b. RGB hyper images were constructed from snapshot hyperspectral images, as shown in the upper row of Fig. 6 c. Gradually, the image turned bluish. The values of R, G, and B were calculated by averaging the intensities of the three channels at (630 nm, 670 nm, 690 nm), (520 nm, 540 nm, 575 nm), and (430 nm, 460 nm, 495 nm), respectively. The FI maps constructed from the hyperspectral images by the LDA and QDA are shown in the lower panel of Fig. 6 c. Thus, gradual changes in the FI values of the meat specimens were visualized as a function of storage time. Besides QDA, there exist various algorithms to determine decision boundaries, and it is possible to have different decision boundaries. Supplementary Fig. 8 shows decision boundaries and FI with other 3 machine learning algorithms, LDA, decision trees, Gaussian Naive Bayes. Even though decision boundaries with other algorithms are not identical, they shared similar characteristics, determining FI as lower/higher value (fresh/rotten state) with higher/lower value of \(\:{\text{L}\text{D}\text{A}\:\text{c}\text{o}\text{m}\text{p}\text{o}\text{n}\text{e}\text{n}\text{t}}_{1}\) , which result in similar value of FI. There may not be an absolute solution, but we can choose proper algorithm, considering properties of data and computation resources. Furthermore, ML analysis with snapshot type hyperspectral images was applied to vacuum-packaged meat for comparison with the PE wrap-packaged meat. The results are presented in Supplementary Fig. 9. For the vacuum-packaged meat, the RGBhyper images converted from the hyperspectral images and its FI map were compared on the 0th and the 28th day. The FI distribution up to the 28th day remained similar to the initial state, as shown in Supplementary Fig. 9a. The dependency of the FI values of PE- and vacuum-packaged meat specimens on storage time, obtained by line-scan type and snapshot type HIS, is shown in Supplementary Fig. 9b and 9c, respectively. Irrespective of the HIS type, i.e. either line-scan or snapshot type the time dependent behavior of FI was quite similar, both qualitatively and quantitatively. Machine learning algorithm is developed using hyperspectral images from 5 different storage times, shown in Fig. 6 a and Supplementary Fig. 9c. Hyperspectral images from the same meat sample at 6 different storage times have been used to confirm validity of the developed algorithm. PE wrap-packaged samples, which are not included in the training set, produce FI values that align with the overall trend as a function of storage time. Also, vacuum-packaged samples, which are also excluded during training, produce the reasonable results for FI by the algorithm as shown in Supplementary Fig. 9. This methodology could be applied to other kinds of muscle food for meat freshness sensing because it is based on fluorescence signal change from transition of chemical composition. This allows the combinations of HIS and ML to produce the reliable and reproducible results on meat freshness sensing. Fabry-Perot filters on CMOS image sensor can be extended to line-scan type as shown in Supplementary Fig. 10 to take advantage of its competitive advantages explained above. Line-scan type can be more suitable for large areal measurement locating at its close distance from sample surface since the imaging area by snapshot type is limited by its FoV. As a demonstration, three band pass filters in line shaped arrays (495 nm, 563 nm, 595 nm) are integrated on the CMOS image sensor to form a line-scan type HIS. The wavelength of three bands were selected for the maximum fluorescence intensity of NADH, porphyrin and a reference point to calculate freshness index as shown in Eq. ( 1 ). Its detailed information is described in Methods and Supplementary Fig. 10. The same meat specimen as PE-wrapped one in Fig. 4 was investigated as a function of storage day shown in Supplementary Fig. 10. These results well match also with those in Fig. 4 measured by grating based line-scan type HIS and provides opportunities to applying HIS and ML based algorithm in various home appliances including refrigerator and smartphone in more efficient manner. Demonstration of line-scan type meat freshness sensor in the refrigerator can be found in supplementary movie. CONCLUSION In this study, hyperspectral imaging technique has been demonstrated as a possible candidate of machine vision for consumer electronics such as smartphone and refrigerator. HIS was successfully applied to confirm meat freshness and to map freshness index by adopting ML. The FI value of the meat specimen was defined from the fluorescence spectrum and the value was compared with bacterial density. The features correlated with the freshness of meat were studied by line-scan type HIS incorporated with conventional gratings having high spatial and spectral resolution. Based on these fundamental studies, ML was applied to the hyperspectral images captured by the snapshot type HIS to identify meat freshness. LDA was adopted to efficiently reduce data dimensionality for extracting the features of meat freshness. Furthermore, QDA was applied to construct the decision boundaries of FI under supervised learning against LDA components. This work demonstrates that ambiguous physical quantities such as meat freshness can be substantialized by HIS and ML. HSI based freshness sensing can be competitive compared with not only conventional destructive methods but also even e-noses and opto-noses, mimicking olfactory to detect food freshness in the aid of smartphone. 28 – 30 There are disadvantages on selectivity of sensing molecules and recycling of the sensors in case of e-noses and opto-noses. To make it more reliable, its measurement condition needs to be well defined since density of molecule floating in the ambient condition could change the result dramatically. Our method is advantageous in this sense since it is irrespective of chemical binding of sensing molecules to the sensor, and detects optical signals from finger prints regarding meat freshness. Moreover, our sensing scheme detects only the density of target molecule on the meat surface which enhance the applicability in the numerous electronic appliances. Also, there might be methods to increase the prediction accuracy applying additional information such as separating fluorescence signal of NADH from fat, excluding fat area with RGB image, and etc. This technique will thrive in everyday life through investigation in reflectance of specimen under ambient light with advantages without additional light source, collection optics and safety issue compared to Raman scattering and fluorescence-based measurement. Besides meat freshness sensing, the physically undefined indices such as freshness of other kind of foods, moistness of skin and so on might be evaluated through artificial intelligence assisted computational power alongside with HIS. Our findings can be applied to advanced machine vision for consumer electronics including refrigerator, smartphone and etc. which will open a new business model toward hyper-personalization and hyper-customization of human life. EXPERIMENTAL AND ANALYSIS METHODS Hyperspectral imaging system and measurement condition Grating based Line-scan type: A commercial spectrometer composed of a grating and an image sensor from Ximea Corp. was installed in the line-scan type HIS. The distance ( h ) between the meat specimen and the 365 nm LEDs was maintained at 2.5 cm. Using 28 LEDs, approximately 37 mW was uniformly applied over the meat specimen. A two-dimensional hyperspectral image was obtained by collecting the spectrum from the thin rectangular window along the x-axis with an exposure time of 400 ms at each step. The length in the perpendicular direction ( l ) against the thin rectangular window along the y-axis is determined by configuration of the system. The lateral resolution of the x-axis was strongly dependent on h . The y-axis resolution could be varied by controlling the mechanical movement. The size of the images was 972×55 pixels, while the spatial resolution was found to be 0.0988 and 0.408 mm/pixel along the x- and y-axes at h = 2.5 cm, respectively. Snapshot type with Fabry-Perot filters: The distance between the meat specimen and hyperspectral image sensor was maintained around 50 cm, while distance between the meat specimen and 365 nm LEDs was 23 cm. A lens of F/1.2 and f = 6 mm, purchased from Fujinon (DF6HA-1B), was used to image an area of approximately 104×8.4 mm. The CMOS image sensor had 2608×1960 pixels with a size of 1.4 µm per pixel, containing 10 bits of information. Each channel was composed of two pixels. Therefore, the spatial size of the data was 326×245 after the demosaicing process. Figure 4 shows an image of a specimen with 140×140 pixels. The measurement conditions were set to 1 s exposure time, 16 analog gain, and 1 digital gain. Line-scan type with Fabry-Perot filters: The same model of CMOS sensor described in snapshot type is used. Arrays of three band pass filters (at 495, 563 and 595 nm with 5, 7 and 9 nm of full width of half maximum (FWHM), respectively) are integrated on the CMOS image sensor with line shape. Each filter covers 14 pixels in the lateral direction (19.6 µm in width). Data of 2 pixels at the edge of each filter are discarded for data analysis to avoid cross talk signals. Imaging RGB by digital camera The RGB images of the meat specimen were obtained in a small box using a digital camera (DSC-RX100, Sony Corp.). Simultaneously, a reference white plate (MINOLTA calibration plate) was placed right next to the meat specimen. A white LED was illuminated through a window placed 50 cm from the meat specimen. Preparation of meat specimen Beef sirloin was purchased from a store in Suwon, Korea Federation of Livestock Cooperatives. A loaf was sliced into pieces of 5×5×1.5 cm. Each piece of meat specimen was placed on a black Styrofoam tray, which was sanitized with alcohol and packaged in a PE wrap. Five pieces of the meat specimen were prepared for each storage time. Two of the five pieces were checked with hyperspectral imaging at each storage time and consumed for 𝑁𝑏𝑎𝑐 measurement. Three additional pieces of the meat specimen packaged in a PE wrap were kept in the refrigerator and measured by hyperspectral imaging as a function of storage time. Additional pieces of the meat specimen were vacuum packaged. Measuring bacterial density ( (CFU/unit)) and temperature Measuring bacterial density ( \(\:{N}_{bac}\) (CFU/unit)) and temperature General bacteria were collected by rubbing both sides of the meat specimen surface with a swab (3M Pipette Swab) and given number of rubbing times. The rubbed swab was rinsed in 1 mg of phosphate-buffered saline (PBS), which was further diluted 10 times. Two sets of 1 ml of the diluted solution containing the general bacteria were dropped onto a petrifilm, which was dried at 35 ± 1°C for 48 ± 2 h. The number of red colonies was then counted. The final \(\:{N}_{bac}\) value was obtained by multiplying the counted number by 10. As a reference, a clean PBS solution was compared with a PBS solution containing the general bacteria collected from the meat surface. The temperatures of the refrigerator (T ref ), freezer (T fre ), and national weather service (T nat ) were registered to understand the possible correlation with the increasing value of \(\:{N}_{bac}\) . The results shown in Fig. 3 a ~ e were measured on May 12 ~ 29, 2020, at T ref = 2.2°C and T nat =13.5 ~ 20.0°C, and those in Fig. 4 were measured from September 9 to 21, 2020, at T ref = 2.9℃ and T nat =18.3 ~ 22.6℃. The results shown in Fig. 3 f were measured from January 22 to February 10, 2021, at T ref = 3.5℃, T fre = − 4℃, and T nat = − 8.1 ~ 7.9℃. The results shown in Figs. 5 and 6 were measured from May 14 to 29, 2021, at T nat = 14.8 ~ 20.9℃ for the PE-wrapped specimen, while for the vacuum-packaged specimen, the results were measured from May 14 to June 11, 2021, at T nat = 14.8 ~ 20.9℃. Abbreviations HIS hyperspectral imaging system; ML machine learning; FI fresh index Declarations ASSOCIATED CONTENT Supporting Information . The supporting information is available free of charge at XXX. Schematic of grating based line-scan type HIS; Schematic of snapshot type HIS with Fabry-Perot filters; Fluorescence spectrum of the meat specimen, NAD and myoglobin; Freshness of vacuum-packaged meat specimen on the 12 th day; Storage time dependence of RGB hyper ; Influence of the packaging material on the acquisition of fluorescence spectrum; Dimensionality reduction from 11 to 2 dimensions in the hyperspectral data by LDA; Decision boundaries and FI depending on machine learning algorithms; Storage time dependent FI of the PE- and vacuum-packaged meat specimens from snapshot type HIS; Line-scan type HIS with Fabry-Perot filters and storage time dependent FI of vacuum-packaged meat specimens; List of packaging materials and their chemical components (PDF) Demonstration of meat freshness sensor installed in Samsung Family Hub T9000 (Video file) AUTHOR CONTRIBUTIONS U.J.K., S.L., and H.S. conceived and designed the research. U.J.K. and S.K. conducted optical measurements. H.K. fabricated the 16CH filter arrays on CMOS image sensor and measured the corresponding transmissions. S.L. devised and carried out the machine algorithm. H.S. designed and fabricated the line-scan type HIS, suggested experimental procedures and methods. J.S.H. had valuable discussions on meat storage in refrigerators. U.J.K. and S.L. wrote the manuscript. All authors reviewed the manuscript. The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript. ACKNOWLDEGEMENTS National Research Foundation of Korea (NRF) grant NRF-2021R1F1A1062182 (PV, YP) National Research Foundation of Korea (NRF) grant NRF-020R1A6A1A03047771 (PV, YP) Korea Institute for Advancement of Technology (KIAT) grant (PV, YP) References Jain G, Paul J, Shrivastava A, Hyper-Personalization (2021) Co-Creation, Digital Clienteling and Transformation. J Bus Res 124:12–23. https://doi.org/10.1016/j.jbusres.2020.11.034 Junmo Kim (2023) Digital Business Requirements in the Era of Hyper-Personalization. SamsungSDS July 28 Lee S, Kim H, Kim G, Son H, Kim UJ (2024) Spectral Analysis on Color Detection Sharpness of Animal Vision toward Polychromatic Vision System. Adv Mater Technol. https://doi.org/10.1002/admt.202400671 Jiang T, Li C, He Q, Peng ZK (2020) Randomized Resonant Metamaterials for Single-Sensor Identification of Elastic Vibrations. Nat Commun 11(1). https://doi.org/10.1038/s41467-020-15950-1 Feng C, Au WSA, Valaee S, Tan Z (2012) Received-Signal-Strength-Based Indoor Positioning Using Compressive Sensing. IEEE Trans Mob Comput 11(12):1983–1993. https://doi.org/10.1109/TMC.2011.216 Zhang X, Xie J, Li C, Xu R, Zhang Y, Liu S, Wang J (2018) MEMS-Based Super-Resolution Remote Sensing System Using Compressive Sensing. Opt Commun 426:410–417. https://doi.org/10.1016/j.optcom.2018.05.046 Shi Q, Zhang Z, He T, Sun Z, Wang B, Feng Y, Shan X, Salam B, Lee C (2020) Deep Learning Enabled Smart Mats as a Scalable Floor Monitoring System. Nat Commun 11(1). https://doi.org/10.1038/s41467-020-18471-z Golestani N, Moghaddam M (2020) Human Activity Recognition Using Magnetic Induction-Based Motion Signals and Deep Recurrent Neural Networks. Nat Commun 11(1). https://doi.org/10.1038/s41467-020-15086-2 Ballard Z, Brown C, Madni AM, Ozcan A (2021) Machine Learning and Computation-Enabled Intelligent Sensor Design. Nat Mach Intell 3(7):556–565. https://doi.org/10.1038/s42256-021-00360-9 Lee Y, Park J, Choe A, Shin YE, Kim J, Myoung J, Lee S, Lee Y, Kim YK, Yi SW, Nam J, Seo J, Ko H (2022) Flexible Pyroresistive Graphene Composites for Artificial Thermosensation Differentiating Materials and Solvent Types. ACS Nano 16(1):1208–1219. https://doi.org/10.1021/acsnano.1c08993 Hadoux X, Hui F, Lim JKH, Masters CL, Pébay A, Chevalier S, Ha J, Loi S, Fowler CJ, Rowe C, Villemagne VL, Taylor EN, Fluke C, Soucy JP, Lesage F, Sylvestre JP, Rosa-Neto P, Mathotaarachchi S, Gauthier S, Nasreddine ZS, Arbour JD, Rhéaume MA, Beaulieu S, Dirani M, Nguyen CTO, Bui BV, Williamson R, Crowston JG, van Wijngaarden P (2019) Non-Invasive in Vivo Hyperspectral Imaging of the Retina for Potential Biomarker Use in Alzheimer’s Disease. Nat Commun 10(1). https://doi.org/10.1038/s41467-019-12242-1 Joung HA, Ballard ZS, Wu J, Tseng DK, Teshome H, Zhang L, Horn EJ, Arnaboldi PM, Dattwyler RJ, Garner OB, Di Carlo D, Ozcan A (2020) Point-of-Care Serodiagnostic Test for Early-Stage Lyme Disease Using a Multiplexed Paper-Based Immunoassay and Machine Learning. ACS Nano 14(1):229–240. https://doi.org/10.1021/acsnano.9b08151 Kim UJ, Lee S, Kim H, Roh Y, Han S, Kim H, Park Y, Kim S, Chung MJ, Son H, Choo H (2023) Drug Classification with a Spectral Barcode Obtained with a Smartphone Raman Spectrometer. Nat Commun 14(1). https://doi.org/10.1038/s41467-023-40925-3 Lee YJ, Ko KS (2016) Effects of Extract of Lactic Acid Bacteria Culture Media on Quality Characteristics of Pork Loin and Antimicrobial Activity against Pathogenic Bacteria during Cold Storage. J Korean Soc Food Sci Nutr 45(10):1476–1480. https://doi.org/10.3746/jkfn.2016.45.10.1476 Seol K-H, Kim KH, Kim YH, Youm KE, Lee M (2014) Effect of Temperature and Relative Humidity in Refrigerator on Quality Traits and Storage Characteristics of Pre-Packed Hanwoo Loin. Korean J Agricultural Sci 41(4):415–424. https://doi.org/10.7744/cnujas.2014.41.4.415 Andrés S, Murray I, Navajas EA, Fisher AV, Lambe NR, Bünger L (2007) Prediction of Sensory Characteristics of Lamb Meat Samples by near Infrared Reflectance Spectroscopy. Meat Sci 76(3):509–516. https://doi.org/10.1016/j.meatsci.2007.01.011 McManus C, Tanure CB, Peripolli V, Seixas L, Fischer V, Gabbi AM, Menegassi SRO, Stumpf MT, Kolling GJ, Dias E, Costa JBG (2016) Infrared Thermography in Animal Production: An Overview. Computers and Electronics in Agriculture. Elsevier B V April 1:10–16. https://doi.org/10.1016/j.compag.2016.01.027 Kucha CT, Liu L, Ngadi MO (2018) Non-Destructive Spectroscopic Techniques and Multivariate Analysis for Assessment of Fat Quality in Pork and Pork Products: A Review. Sensors (Switzerland). MDPI AG Febr 1 https://doi.org/10.3390/s18020377 Wu B, Dahlberg K, Gao X, Smith J, Bailin J Rapid Measurement of Meat Spoilage Using Fluorescence Spectroscopy. In Imaging, Manipulation, and Analysis of Biomolecules, Cells, and, Tissues XV, SPIE (2017), ; Vol. 10068, p 1006820. https://doi.org/10.1117/12.2253526 Pu Y, Wang W, Alfano RR (2013) Optical Detection of Meat Spoilage Using Fluorescence Spectroscopy with Selective Excitation Wavelength. Appl Spectrosc 67(2):210–213. https://doi.org/10.1366/12-06653 Zhuang Q, Peng Y, Yang D, Nie S, Guo Q, Wang Y, Zhao R (2022) UV-Fluorescence Imaging for Real-Time Non-Destructive Monitoring of Pork Freshness. Food Chem 396. https://doi.org/10.1016/j.foodchem.2022.133673 Ministry of Food and Drug Safety of Korea. KFDA. Food Code Aït-Kaddour A, Thomas A, Mardon J, Jacquot S, Ferlay A, Gruffat D (2016) Potential of Fluorescence Spectroscopy to Predict Fatty Acid Composition of Beef. Meat Sci 113:124–131. https://doi.org/10.1016/j.meatsci.2015.11.020 Su J, Yi D, Liu C, Guo L, Chen WH (2017) Dimension Reduction Aided Hyperspectral Image Classification with a Small-Sized Training Dataset: Experimental Comparisons. Sens (Switzerland) 17(12). https://doi.org/10.3390/s17122726 Beatriz PP, Garcia-Salgado; Volodymyr I, Ponomaryov; Sergiy NS (2020) Rogelio Reyes-Reyes. Efficient Dimension Reduction of Hyperspectral Images for Big Data Remote Sensing Applications. J Appl Remote Sens 14(3). https://doi.org/10.1002/adma.202002854 Ayesha S, Hanif MK, Talib R (2020) Overview and Comparative Study of Dimensionality Reduction Techniques for High Dimensional Data. Inform Fusion 59:44–58. https://doi.org/10.1016/j.inffus.2020.01.005 Zhao B, Ulfarsson MO, Sveinsson JR, Chanussot J (2020) Unsupervised and Supervised Feature Extraction Methods for Hyperspectral Images Based on Mixtures of Factor Analyzers. Remote Sens (Basel) 12(7). https://doi.org/10.3390/rs12071179 Guo L, Wang T, Wu Z, Wang J, Wang M, Cui Z, Ji S, Cai J, Xu C, Chen X (2020) Portable Food-Freshness Prediction Platform Based on Colorimetric Barcode Combinatorics and Deep Convolutional Neural Networks. Adv Mater 32(45). https://doi.org/10.1002/adma.202004805 Anisimov DS, Abramov AA, Gaidarzhi VP, Kaplun DS, Agina EV, Ponomarenko SA (2023) Food Freshness Measurements and Product Distinguishing by a Portable Electronic Nose Based on Organic Field-Effect Transistors. ACS Omega 8(5):4649–4654. https://doi.org/10.1021/acsomega.2c06386 Istif E, Mirzajani H, Dağ Ç, Mirlou F, Ozuaciksoz EY, Cakır C, Koydemir HC, Yilgor I, Yilgor E, Beker L (2023) Miniaturized Wireless Sensor Enables Real-Time Monitoring of Food Spoilage. Nat Food 4(5):427–436. https://doi.org/10.1038/s43016-023-00750-9 Additional Declarations The authors declare no competing interests. Supplementary Files ACSSIMeatFreshness.docx SupplementaryMovie.mp4 Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5551638","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":384446339,"identity":"81c16b33-0057-4eda-b5a9-f0034d801a68","order_by":0,"name":"Suyeon Lee","email":"","orcid":"","institution":"Samsung Advanced Institute of Technology; Suwon, Gyeonggi-do 16678, Republic of Korea","correspondingAuthor":false,"prefix":"","firstName":"Suyeon","middleName":"","lastName":"Lee","suffix":""},{"id":384446340,"identity":"0d6c7b37-b461-41f8-a9ad-d76f71011a2f","order_by":1,"name":"Hyochul Kim","email":"","orcid":"","institution":"Samsung Advanced Institute of Technology; Suwon, Gyeonggi-do 16678, Republic of Korea","correspondingAuthor":false,"prefix":"","firstName":"Hyochul","middleName":"","lastName":"Kim","suffix":""},{"id":384446341,"identity":"2f7849d1-efca-4c7e-a877-55adc84e4074","order_by":2,"name":"Seokin Kim","email":"","orcid":"","institution":"School of Integrative Engineering, Chung-Ang University; Seoul 06974, Republic of Korea","correspondingAuthor":false,"prefix":"","firstName":"Seokin","middleName":"","lastName":"Kim","suffix":""},{"id":384446342,"identity":"a4149186-b942-47ed-a477-d2f457b7a914","order_by":3,"name":"Jeong Su Han","email":"","orcid":"","institution":"School of Integrative Engineering, Chung-Ang University; Seoul 06974, Republic of Korea","correspondingAuthor":false,"prefix":"","firstName":"Jeong","middleName":"Su","lastName":"Han","suffix":""},{"id":384446343,"identity":"93342f41-d719-4803-8795-3bfcdded58c0","order_by":4,"name":"Un Jeong Kim","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYFCCAwlAIoGHgb3BAMyXgGIitPAcIFoLGAC1SSQQqcXg4IGHnwsq0mR0Zz7e+LjgV23izAbmg7d58Gk5cCBZesaZHB6z22nFxjP7jifOZmBLtiagJUGat60CqCXHTJq351jiPAYeM2lCtvzm/QfUcvMMTAv/N0Ja0qR5G4AOuwEy/EcN0GE8bHi1SAK1WPMcS+MxOwP0C2/DAeOZzWzGlnPwaOG7cSb5Nk9Nsr3Z8cMbH/P8qZOdcbz54Y03eLQwSJxJQHAY2w4zMDDjUw4C/O0HkHh/6gipHwWjYBSMghEIANAWUwwoPLygAAAAAElFTkSuQmCC","orcid":"","institution":"Department of Physics, Dongguk university, Seoul 04620, Republic of Korea","correspondingAuthor":true,"prefix":"","firstName":"Un","middleName":"Jeong","lastName":"Kim","suffix":""}],"badges":[],"createdAt":"2024-11-29 23:50:42","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":true,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":true},"doi":"10.21203/rs.3.rs-5551638/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5551638/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":70586936,"identity":"57e5b9b6-b73c-4f12-98ba-061a281474f3","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":19139627,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eOverview of meat freshness sensing by machine vision for electronic appliances.\u003cbr\u003e\n \u003c/strong\u003eMeat freshness can be monitored in the refrigerator or by smartphone using CMOS based hyperspectral imaging system and machine learning based algorithm.\u003c/p\u003e","description":"","filename":"Figure1.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/d506e417adca83345ab2c2e1.png"},{"id":70586921,"identity":"b6e99f2e-723b-4c34-b5fa-a7fdc28687db","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":4568807,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSchematics of the full process required to obtain hyperspectral data and evaluation using ML for food inspection.\u003c/strong\u003eTwo hyperspectral imaging systems are incorporated for hyperspectral data acquisition: line-scan type HIS installed with commercial grating for high spatial and spectral resolution and snapshot type HIS for compact form factor and efficient computation resources. ML is conducted to extract FI values from the hyperspectral data for food inspection.\u003c/p\u003e","description":"","filename":"Figure2.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/27847481ccfb97a6c418246a.png"},{"id":70586925,"identity":"faba3171-61ab-40f7-88aa-7a26db255789","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":9932132,"visible":true,"origin":"","legend":"\u003cp\u003eTracking meat freshness and two-dimensional maps of FI of meat specimen. a RGB images of meat specimen taken by a digital camera under white light illumination and the distribution of a* and b*. Red and yellow solid lines are used to track the average values at each stage. b RGB images converted from hyperspectral images of fluorescence signals collected by line-scan type HIS and its representative spectrum on the right are displayed as a function of storage time. The same area of the meat specimen is monitored for the RGB and hyperspectral images of fluorescence. c Two-dimensional map of the FI of the meat specimen is reconstructed based on the hyperspectral data. d Average FI values for each hyperspectral image and e bacteria density (\u003cem\u003eN\u003c/em\u003e\u003csub\u003ebac\u003c/sub\u003e) are plotted as a function of storage time in a logarithmic scale. The pink shaded area indicates inedible state of meat \u003cem\u003eN\u003c/em\u003e\u003csub\u003ebac\u003c/sub\u003e \u0026gt; 107 (CFU/cm2) and corresponding FI \u0026gt; 0.63. f Storage time dependent FI of meat specimens stored in refrigerator at T = 4oC (black dot) and freezer (red dot).\u003c/p\u003e","description":"","filename":"Figure3.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/11a5ec4b0d1d2c3e0bf781f8.png"},{"id":70586918,"identity":"b8a4a1c2-4533-4944-a620-dce298db1b5a","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":2107518,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eFreshness evaluation of the vacuum-packaged meat specimen. a\u003c/strong\u003e RGB image taken by a digital camera, RGB\u003csup\u003ehyper\u003c/sup\u003e image and its FI map from the hyperspectral image of the vacuum-packaged meat specimen on the 0th day from top to bottom. \u003cstrong\u003eb\u003c/strong\u003e Distribution of \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* values of the RGB image of the vacuum-packaged meat specimen on the 0th day. \u003cstrong\u003ec\u003c/strong\u003e Mean \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* values of the vacuum-packaged and PE wrap-packaged meat specimen as a function of storage time. \u003cstrong\u003ed\u003c/strong\u003e FI of the vacuum-packaged (red dots) and PE wrap-packaged (black dots) meat specimen as a function of storage time. Six and two pieces of vacuum-packaged and PE wrap-packaged meat specimen are investigated, respectively.\u003c/p\u003e","description":"","filename":"Figure4.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/2147be8b8334225e0f08a4e1.png"},{"id":70586930,"identity":"0c72dec1-c0b6-4119-8e54-30d84e9c518e","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":2516174,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eHyperspectral image of meat fluorescence by snapshot type HIS\u003c/strong\u003e. \u003cstrong\u003ea\u003c/strong\u003e Characteristic curve of transmission of each channel multiplied by quantum efficiency. The inset is the schematics of the channel in the Fabry-Perot filter structure. \u003cstrong\u003eb\u003c/strong\u003e Hyperspectral image of the meat fluorescence excited by 365 nm LED. Fifteen channels of peak wavelength in the 380~850 nm range and one metal block (4´4 channels) are periodically arranged as indicated in the illustration on the right-hand side. Each color represents the channel’s peak wavelength in accordance with the color of the characteristic curves in \u003cstrong\u003ea\u003c/strong\u003e. Three representative demosaiced images from hyperspectral images on the 1st, 6th, and 11th channel of which center wavelengths are 390, 524, and 675 nm are shown. \u003cstrong\u003ec\u003c/strong\u003e Spectrum extracted from the hyperspectral data obtained on the 0th and the 15th day for the PE wrap-packaged specimen, which were in the fresh and rotten state, respectively.\u003c/p\u003e","description":"","filename":"Figure5.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/6f6893b0a2ad0d240d39d842.png"},{"id":70586926,"identity":"30bbdd69-57b9-449b-b3b7-a37f370ad757","added_by":"auto","created_at":"2024-12-04 16:05:08","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":6401555,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eML for evaluation of meat freshness.\u003c/strong\u003e \u003cstrong\u003ea\u003c/strong\u003e LDA components, which are a result of dimensionality reduction from 11 to 2 for analyzing meat freshness from hyperspectral images of the meat fluorescence. The images are obtained by snapshot type HIS and two-dimensional maps at five representative storage times. \u003cstrong\u003eb\u003c/strong\u003eDecision boundary contour determined by QDA to find FI at unknown state was evaluated by LDA and QDA of the hyperspectral images of the meat fluorescence. Numbers inside the contour plot are from the FI values obtained from the line-scan type hyperspectral image analysis of the same meat specimen. \u003cstrong\u003ec\u003c/strong\u003e Upper panels are the RGB images converted from the hyperspectral images at five representative storage times. The two-dimensional maps of FI are displayed at the lower panel.\u003c/p\u003e","description":"","filename":"Figure6.png","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/bc33f67e54eac8c4e0561830.png"},{"id":70589142,"identity":"3b893a82-19a9-421b-a5df-85a5f5943bb5","added_by":"auto","created_at":"2024-12-04 16:29:26","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":42624338,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/74e25ab2-f4ea-40d9-a778-6806fb7999ca.pdf"},{"id":70588057,"identity":"c5ecfe42-de25-42fb-9602-b149d25d9806","added_by":"auto","created_at":"2024-12-04 16:13:08","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1288267,"visible":true,"origin":"","legend":"","description":"","filename":"ACSSIMeatFreshness.docx","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/87b556118a3cec408cd41b60.docx"},{"id":70586941,"identity":"6b5e3f46-270e-4b23-9d8f-9c349d5c2dbb","added_by":"auto","created_at":"2024-12-04 16:05:09","extension":"mp4","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":50906791,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryMovie.mp4","url":"https://assets-eu.researchsquare.com/files/rs-5551638/v1/7825b604de6f273a3faccda1.mp4"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eMachine Vision with CMOS based Hyperspectral Image Sensor Enables Meat Freshness Sensing\u003c/p\u003e","fulltext":[{"header":"INTRODUCTION","content":"\u003cp\u003eImaging spectral information and analyzing its properties of materials have become intriguing for consumer electronics toward food inspection, beauty care and etc. Those sensory physical quantities are difficult to quantify. Hyperspectral cameras, which capture its figure and spectral information simultaneously, can be a good candidate for non-destructive remote sensing. In this study, with the aid of a hyperspectral imaging system (HIS) and machine learning (ML), meat freshness is converted into a measurable physical quantity, i.e., freshness index (FI). FI is defined from meat fluorescence, which has a strong correlation with bacterial density. Combined with ML techniques, hyperspectral data are processed more efficiently. By employing linear discriminant and quadratic component analyses, FI can be estimated from its decision boundary after hyperspectral data are obtained at an unknown freshness state. We demonstrate HIS grafted with ML performs as artificial eye and brain which is advanced machine vision for consumer electronics including refrigerators and smartphones. Advanced sensing versatility utilized by computational sensing systems allows hyper-personalization and hyper-customization of human life.\u003c/p\u003e\n\u003cp\u003eConsumer electronics have been progressively developed to satisfy hyper-personalization or hyper-customization of human everyday life.\u003csup\u003e1,2\u003c/sup\u003e Utilizing proper sensors and analysis algorithm becomes one of the key factors to make them more attractive for consumers. Hyperspectral imaging system (HIS) can be the most promising since it measures plural quantities, i.e\u003cem\u003e.\u003c/em\u003e morphology and spectrum, of objects, simultaneously.\u003csup\u003e3\u003c/sup\u003e Moreover, it becomes very powerful for consumer electronics since it captures the image of its figure and finger print non-destructively, remotely and etc. Recently, with machine learning (ML) based statistical analysis, by establishing a correlation between raw sensing data and ambiguously defined physical quantities, researchers have recently reported a growing versatility in the functionality and enhanced performance of traditional sensors when combined.\u003csup\u003e4\u0026ndash;10\u003c/sup\u003e Furthermore, the utilization of advanced sensing versatility by computational sensing systems through different home appliances enables various functions including food inspection, healthcare, and beauty care.\u003csup\u003e11\u0026ndash;13\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eWith the steadily rising consumption of meat along with gastronomic flavor, its freshness and aging have emerged as the most intriguing quality control factors for food inspection. Defining the freshness or aging degree of meat as a physical quantity is intriguing, ambiguous, and strongly influenced by personal preference. However, traditional techniques for the evaluation of meat freshness or its aging degree are mostly destructive, non-portable, expensive, and difficult to access by general users.\u003csup\u003e14,15\u003c/sup\u003e On the other hand, optical inspection techniques can be a proper candidate for an alternative such as measurement of reflectance from meat surfaces as investigated over several decades.\u003csup\u003e16\u0026ndash;18\u003c/sup\u003e Even though several approaches are suitable for non-destructive or portable system, these are lack of the underlying mechanism. For reliable and reproducible results, specific molecular changes should be considered. To investigate the freshness, biomarkers such as collagen, reduced nicotinamide adenine dinucleotide (NADH), and flavin have been deemed suitable.\u003csup\u003e19,20\u003c/sup\u003e These studies have monitored the ~390, 460, and 525 nm fluorescence as a function of time from pork specimens stored at room and refrigerator temperatures (4\u003csup\u003eo\u003c/sup\u003eC), when stimulated by a 340 nm light emitting diode (LED) for up to three days. Also, other study tried to analyze hyperspectral image of meat fluorescence by 365nm excitation for meat freshness based on color analysis.\u003csup\u003e21\u003c/sup\u003e Thus, hyperspectral imaging system is considered suitable for extracting invisible information on meat freshness by imaging morphological and spectroscopic data collected from macroscale meat samples through ML grafting.\u003c/p\u003e\n\u003cp\u003eIn this study, meat freshness was clearly distinguished not by an RGB camera but spectral information from HIS. With high spectral resolution hyperspectral data from line-scan type with grating, freshness index (FI) is derived based on chemical change depending on the storage time of meat. Not only the storage condition utilized for defining FI, but we also show that FI can be extended towards various kinds of situations, frozen state, vacuum packaged state, and etc. Based on the knowledge from high resolution hyperspectral data, data size can be reduced both in the HW aspect, selecting suitable filters instead of grating, and SW aspect, adopting ML for data reduction. Hyperspectral data from filter array-based snapshot type with ML successfully demonstrates FI with the comparison with hyperspectral data form line-scan type and physical quantity-based formula. Considering form factors of HIS for home appliances, snapshot type and line-scan type with filter array can be applicable to long and short distance imaging for large area substances. By combining HIS with ML, meat freshness can become a tangible physical quantity for applications in daily life. Our findings demonstrate that machine vision with HIS enable meat freshness sensing in consumer electronics including refrigerators and smartphones.\u0026nbsp;\u003c/p\u003e"},{"header":"RESULTS AND DISCUSSION","content":"\u003cp\u003e \u003c/p\u003e \u003cp\u003eMeat freshness can be monitored in the refrigerator or by smartphone using CMOS based hyperspectral imaging system and machine learning based algorithm.\u003c/p\u003e \u003cp\u003e \u003cb\u003eOverview of meat freshness sensing by machine vision for consumer electronics.\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrate the user scenarios of meat freshness sensors realized by machine vision with hyperspectral imaging system (HIS) and machine learning. Complementary metal-oxide-semiconductor (CMOS) based HIS can be either line-scan or snapshot type. Line-scan type can be powerful when the distance between the object of interest and the sensor is limited, and its capturing area is very large to be captured by snapshot. For example, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, line-scan type storage compartment in the refrigerator can be more suitable than snapshot type because of limited field of view (FOV). Instead, snapshot type can be installed as an additional smartphone camera which performs diagnosis of its freshness at any places. Meat freshness can be classified by machine learning based algorithm prebuilt in application processor (AP) in the consumer electronics.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\n\u003ch3\u003eAcquisition of Hyperspectral Data and Evaluation of Meat Freshness by ML\u003c/h3\u003e\n\u003cp\u003eFigure \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the entire process behind obtaining hyperspectral data using HIS and identifying meat freshness with ML. In this study, the line-scan and snapshot types of HIS are used to image the morphology and spectral information of meat specimens. A line-scan type HIS is designed to move 365 nm LED arrays using a conventional grating for collected light dispersion, and a long rectangular window to excite and collect fluorescence signals over the meat specimens. The fluorescence signal transported to a commercially available grating is collected in the 3 dimensional (one spectral dimension and two spatial dimensions) hyperspectral data, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. For the snapshot type HIS, Fabry-Perot filters are fabricated periodically on a CMOS image sensor, working in the range 380\u0026thinsp;~\u0026thinsp;840 nm. The schematics for line-scan and snapshot types of HIS are shown in Supplementary Figs.\u0026nbsp;1 and 2, respectively. For details on the scanning area and spatial resolution for both types of HIS, see Methods. High spectral and spatial resolution of the grating-based line-scan type HIS is beneficial for investigating parameters correlated with meat freshness from the fluorescence spectrum. Meat freshness can be extracted from the hyperspectral data by processing data efficiently. Particularly, ML on hyperspectral data with the merit of data size obtained by snapshot type HIS, based on fundamental studies by line-scan type HIS, is conducted to evaluate the information on meat freshness.\u003c/p\u003e \u003cp\u003eHyperspectral images of the meat surface are decomposed into a series of spectral bands (λ\u003csub\u003e1\u003c/sub\u003e, λ\u003csub\u003e2\u003c/sub\u003e, λ\u003csub\u003e3\u003c/sub\u003e, λ\u003csub\u003e4,\u003c/sub\u003e ...), forming hyperspectral data. The full spectrum of each local point is constructed by combining intensity and wavelength in the vertical direction. Typical fluorescence spectra of fresh and rotten meat specimens are shown in Supplementary Fig.\u0026nbsp;3a and b. The broadband from NADH at 490 nm and its enhancement in intensity with lowered freshness agreed with those observed in previous studies.\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e,\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e Conversely, the sharp peak at ~\u0026thinsp;600 nm appeared to have a relation with myoglobin, which exists inside cells of mammals and is related to their breathing activity. The reference spectra of NADH and myoglobin purchased from Aldrich Inc. are exceptionally close to those of meat and are shown in Supplementary Fig.\u0026nbsp;3c and d, respectively.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eTraditional Analysis of Meat Freshness and 2D Freshness Index Map\u003c/h2\u003e \u003cp\u003eTo understand the morphological and chemical changes in meat specimens as a function of refrigerator storage time (T\u0026thinsp;\u0026asymp;\u0026thinsp;4\u003csup\u003eo\u003c/sup\u003eC), RGB images illuminated with white LED and hyperspectral images of fluorescence excited by 365 nm were captured by a digital camera and line-scan type HIS, respectively. The process was conducted on the same piece of specimen under the same conditions for 17 days, and the results are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea and b. The meat specimens were packaged in polyethylene (PE) wraps to avoid contamination and other handling issues. The RGB images of the meat specimens as a function of storage time were analyzed using the CIELAB color space, referred to as \u003cem\u003eL\u003c/em\u003e*\u003cem\u003ea\u003c/em\u003e*\u003cem\u003eb\u003c/em\u003e*. \u003cem\u003eL*\u003c/em\u003e represents lightness, \u003cem\u003ea*\u003c/em\u003e represents green\u0026ndash;red opponent colors with negative and positive values toward green and red, and \u003cem\u003eb*\u003c/em\u003e denotes blue\u0026ndash;yellow opponents with negative and positive values toward blue and yellow, respectively. The distributions of \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* values, represented by red and yellow bars, are displayed on the right side for each RGB image, while their average values are indicated by red and yellow dots in the guideline, respectively. With increase in storage time, both \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* values merge to 10 starting from the 7th day, implying that the color of meat turned less reddish and yellowish. In addition, distinguishing the state of meat freshness after the 7th day is impossible. Another type of RGB images defined by \u0026ldquo;RGB\u003csup\u003ehyper\u003c/sup\u003e\u0026rdquo; was obtained by transforming hyperspectral images of meat fluorescence, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb, where the representative fluorescence spectrum of each storage day is displayed on the right side. With longer storage time, the relative intensity of NADH located at ~\u0026thinsp;490 nm against 600 nm increased. Based on this spectral change, FI is defined as\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:\\text{F}\\text{I}=\\left({I}_{N}-{I}_{M}\\right)\\:/\\:\\left({I}_{N}+{I}_{M}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{I}_{N}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{I}_{M}\\)\u003c/span\u003e\u003c/span\u003e are the intensities of NADH and myoglobin peaks deconvoluted from the fluorescence spectrum, shown by the blue and pink shaded areas, respectively. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec shows a 2D map of FI for each storage day, while Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed shows the average FI value from each hyperspectral data, which increased monotonously up to the 10th day and finally saturated. The number of bacteria per unit area (CFU/cm\u003csup\u003e2\u003c/sup\u003e) (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e) measured using the standard method (see Methods) gradually increased on a logarithmic scale, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ee, where the pink shaded area indicates the inedible state of meat at \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e \u0026gt; 10\u003csup\u003e7\u003c/sup\u003e (CFU/cm\u003csup\u003e2\u003c/sup\u003e)\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e. Thus, FI was confirmed to have a strong correlation with meat freshness. However, the CIELAB color space analysis of the RGB images confirmed that meat freshness could not be distinguished after the 7th day of storage, although meat remained edible at \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e \u0026lt; 10\u003csup\u003e7\u003c/sup\u003e (CFU/cm\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e). The correlation among \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e, storage date (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:d\\)\u003c/span\u003e\u003c/span\u003e), and FI was obtained by fitting the experimental data shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed and e using a relation defined as\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\:\\text{log}\\left({N}_{bac}\\right)=2.6\\sqrt{d+6.4}-3.4$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$\\:\\text{F}\\text{I}\\:=0.066\\left(\\text{log}\\left({N}_{bac}\\right)+2.5\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eUsing these functions, guidelines were added to the experimental data by applying a width of \u0026plusmn;\u0026thinsp;1.0 and \u0026plusmn;\u0026thinsp;0.1 in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed and e. According to Eq.\u0026nbsp;(\u003cspan refid=\"Equ3\" class=\"InternalRef\"\u003e3\u003c/span\u003e), FI\u0026thinsp;\u0026gt;\u0026thinsp;0.63 for the inedible state and is indicated as the pink shaded area in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ed. The preparation process and number of samples are stated explicitly in Methods.\u003c/p\u003e \u003cp\u003eThe FI of the meat specimens, one stored in a refrigerator and the other in a freezer, was monitored as a function of storage time to prove the representativeness of meat freshness at unknown storage or commercial distribution channel history. Predictably, the FI values of the sample stored in the refrigerator increased gradually to approximately 0.7, but those of the sample stored in the freezer remained at approximately 0.55, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef. At the first stage, the initial increase in the FI value for the frozen meat could be attributed to the structural changes in the meat specimen or an actual increase in \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e before freezing.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe color of meat specimens is particularly related to the oxidation of myoglobin. Meat is bright red, which looks fresh to the human eyes, owing to iron oxidation at the center of the heme ring in myoglobin molecules by attaching oxygen or water molecules. When the meat specimen is vacuum-packaged, its color turns dark brown, which looks stale to the human eye regardless of its freshness. The vacuum-packaged meat specimens are investigated as a function of storage time using the FI and \u003cem\u003ea\u003c/em\u003e*, \u003cem\u003eb\u003c/em\u003e* values from hyperspectral and RGB images, respectively. The color of the meat specimen was darker even on the 0th day, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, due to the desorption of oxygen molecules from myoglobin on the meat surface during vacuum packaging. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea shows an RGB image under white light captured by a digital camera, an RGB\u003csup\u003ehyper\u003c/sup\u003e image, and the FI map of a vacuum-packaged meat specimen on the 0th day. In Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb, a* and \u003cem\u003eb\u003c/em\u003e* for the vacuum-packaged meat on the 0th day were distributed in a range similar to that of inedible meats shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea, and they remained similar until the 12th day, as shown in Supplementary Fig.\u0026nbsp;4a. The values of \u003cem\u003ea\u003c/em\u003e*, \u003cem\u003eb\u003c/em\u003e* and FI were monitored as a function of storage time for both vacuum- and PE-packaged samples, and the results are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec and d, respectively. On the 0th day, the vacuum-packaged meat exhibited relatively lower average values of \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* when compared to those of the PE-packaged specimen. With prolonged storage time, the mean values of \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* of the PE-packaged meat tended to decrease, while they remained almost constant for the vacuum-packaged specimen, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec. From the 0th to the 12th day, the mean values of \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* changed slightly from 13.7 to 12.5 and from 7.6 to 4.7 for the vacuum-packaged meat, respectively. Furthermore, the color of meat could be influenced by various factors such as its freshness, pH, part of meat, and nutritional state. However, larger differences between the FI value of the PE- and vacuum-packaged specimens than \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* were observed. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e was monitored in the PE-packaged meat specimen stored together with the vacuum-packaged meat specimen (Supplementary Fig.\u0026nbsp;4b). \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e for the present batch increased rapidly when compared to that shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e because of different environmental conditions, including weather, preparation process, and distribution channel history. The representative fluorescence spectra on the 0th and 12th day are shown in Supplementary Fig.\u0026nbsp;4c. The PE-packaged meat exhibited a gradual increase in the FI value, which is proportional to that of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\text{log}\\left({N}_{bac}\\right)\\)\u003c/span\u003e\u003c/span\u003e, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ed due to exposure to air. However, the FI value of the vacuum-packaged meat was maintained approximately to the 12th day because the meat stayed fresh by preventing oxygen adsorption on its surface. The freshness of the vacuum packaged meat was verified through the hyperspectral image of the meat fluorescence and its FI, as shown in Supplementary Fig.\u0026nbsp;4d. The gray shaded area in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ed represent the guidelines that are calculated using Eqs.\u0026nbsp;(\u003cspan refid=\"Equ2\" class=\"InternalRef\"\u003e2\u003c/span\u003e) and (\u003cspan refid=\"Equ3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). The slight discrepancy between the experimental data (black dots) illustrated may have originated from the different relative ratios of fat and flesh contents in the meat specimens used. Since the characteristic band of fatty acids is located in a wavelength range similar to that of NADH \u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e, Eqs.\u0026nbsp;(\u003cspan refid=\"Equ2\" class=\"InternalRef\"\u003e2\u003c/span\u003e) and (\u003cspan refid=\"Equ3\" class=\"InternalRef\"\u003e3\u003c/span\u003e) can be modified to fit FI with a higher accuracy by excluding the areas of fat while calculating FI. Thus, the results suggest that meat freshness can be measured more clearly by using hyperspectral image sensors than by RGB image sensors. Expectedly, RGB imaging of fluorescence signal might also be considered to distinguish meat freshness. We simulated the RGB image of meat fluorescence by converting the hyperspectral image into RGB\u003csup\u003ehyper\u003c/sup\u003e images as a function of the storage time. We observed that the \u003cem\u003ea\u003c/em\u003e* and \u003cem\u003eb\u003c/em\u003e* values of the RGB\u003csup\u003ehyper\u003c/sup\u003e images did not change sensitively within the edible state (\u0026le;\u0026thinsp;7th day of storage), as shown in Supplementary Fig.\u0026nbsp;5. Thus, hyperspectral imaging is confirmed to be a powerful tool for discerning meat freshness against RGB images taken under white or 365 nm LED. Since the freshness stage of meat is not easily accessible to consumers, the proper analysis tool needs to be devised for precise freshness evaluation.\u003c/p\u003e \u003cp\u003eIn an experiment, four types of packaging materials were investigated to understand their influence on the fluorescence spectrum listed in Supplementary Table\u0026nbsp;1. The results are shown in Supplementary Fig.\u0026nbsp;6a. PE and PVC wrap or zipper bags produced unnoticeable fluorescence signals, while vacuum packaging material produced a non-negligible intensity of broad fluorescence at approximately 470 nm. However, the condition was magnified more than 20 times, where 5 s integration with two layers was performed as compared with the experimental condition of 400 ms to obtain fluorescence from the meat specimen. On packaging the meat in PE or PVC wrap and zipper bags, the shape of the fluorescence spectrum was maintained while its intensity decreased, as shown in Supplementary Fig.\u0026nbsp;6b. The overall shape of the fluorescence spectrum was independent of the packaging material. This result indicates that fluorescence spectroscopy can be applied in daily life to analyze meat freshness when commercial food packaging materials are used.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eHyperspectral Imaging and ML for Meat Freshness by Snapshot type HIS\u003c/h3\u003e\n\u003cp\u003eFurthermore, a 16-channel (CH) snapshot type HIS was fabricated as Fabry-Perot filters were formed periodically on a CMOS image sensor operating in the 380\u0026thinsp;~\u0026thinsp;840 nm range including a blank channel. To optimize transmission and each filter\u0026rsquo;s resonance wavelength, SiN films with variable thicknesses were stacked vertically along with Cu or Al reflectors at the top and bottom. The final characteristic transmission curves shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ea were obtained by multiplying the quantum efficiency (QE) of the CMOS image sensor and transmission of each filter. Despite relatively lower spatial and spectral resolution compared to line-scan type HIS, the snapshot type HIS offered competitive advantages in several aspects, such as cost effectiveness and efficiency of computing resources. Using this methodology, a smartphone installed with a hyperspectral camera can be used to capture both outer appearance and spectral information of meat at any place to determine meat freshness. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb shows the hyperspectral image of fluorescence from the meat specimen supported by a black Styrofoam tray, excited by 365 nm LEDs. The image was demosaiced into each CH, as shown on the right side of Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003eb, where the enlarged CH distribution of the small yellow square in the hyperspectral image is shown on the right-hand side. Hyperspectral images from the same meat specimen were taken as a function of storage time. For the 0th and 15th day, two representative fluorescence spectra from the fresh and rotten states of the meat specimen taken by the snapshot type HIS, respectively, are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003ec. Comparatively, a large enhancement at approximately 500 nm was observed on the 15th day as opposed to the 0th day. Thus, the results were consistent with those obtained from the line-scan type HIS, as illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. ML can be applied to reduce the dimensions of hyperspectral data and to extract features related to meat freshness from hyperspectral images. The reduction in data dimension can maximize the efficiency of computing resources, such as computing time and memory. Moreover, the risk of overfitting data resulting from a complicated analysis model can be reduced, and the dimensionality reduction can prove to be advantageous for the ease of data interpretation. Among various algorithms available for dimensionality reduction, principal component analysis (PCA) and linear discriminant analysis (LDA) are commonly used.\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e,\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e As LDA is suitable for supervised learning and classification performance\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e, LDA is opted for dimensionality reduction of hyperspectral data from 11 to 2 dimensions (indicated by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{1}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{2}\\)\u003c/span\u003e\u003c/span\u003e in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ea). Even though the sum of percentage of variance explained will be increased up to 1 with a greater number of dimensions, we choose 2 LDA components as the sum of percentage of variance explained is already 0.989 and fluorescence signal to determine freshness of meat is originated from two chemicals. Four long wavelength CHs and one blank CH were initially excluded out of the 16 CHs that did not convey information on the fluorescence signal from the meat specimen. The raw intensity profiles of the remaining 11 CHs as a function of wavelength were standardized by their mean value and standard deviation. Furthermore, dimensionality reduction was conducted by subtracting the channel-dependent constant (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\stackrel{-}{{x}_{j}}\\)\u003c/span\u003e\u003c/span\u003e) from value of each CH (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{x}_{j})\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:j=1,\\:\\dots\\:11\\)\u003c/span\u003e\u003c/span\u003e, followed by the multiplication of two coefficients (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{s}\\text{c}\\text{a}\\text{l}\\text{i}\\text{n}\\text{g}\\:\\text{f}\\text{a}\\text{c}\\text{t}\\text{o}\\text{r}}_{1}\\)\u003c/span\u003e\u003c/span\u003eand \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{s}\\text{c}\\text{a}\\text{l}\\text{i}\\text{n}\\text{g}\\:\\text{f}\\text{a}\\text{c}\\text{t}\\text{o}\\text{r}}_{2}\\)\u003c/span\u003e\u003c/span\u003e), which were determined using the LDA method. The data reduction process and coefficients are shown in Supplementary Fig.\u0026nbsp;7. Here,\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$$\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{i}=\\:\\sum\\:_{j=1}^{11}{\\text{s}\\text{c}\\text{a}\\text{l}\\text{i}\\text{n}\\text{g}\\:\\text{f}\\text{a}\\text{c}\\text{t}\\text{o}\\text{r}}_{j}\\bullet\\:\\left({x}_{j}-\\stackrel{-}{{x}_{j}}\\right),\\:\\:\\:i=1,\\:2$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{1}\\)\u003c/span\u003e \u003c/span\u003e was plotted against \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{2}\\)\u003c/span\u003e\u003c/span\u003e of the hyperspectral images of meat as a function of storage time, as illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ea. With longer durations of storage time, highly scattered data were gradually merged to the negative values of the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{1}\\)\u003c/span\u003e\u003c/span\u003e, where each dot was extracted from dimensionality reduction using LDA. Even on the 0th day, the negative values of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{1}\\)\u003c/span\u003e\u003c/span\u003e appeared to originate from the fat tissues whose fluorescence was similar to that of NADH. The decision or evaluation boundaries of FI produced by QDA are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003eb as contour plots.\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e Using information on FI, averaged over each hyperspectral image by line-scan type HIS as reference data, QDA was adopted to obtain decision boundaries of the FI for the two LDA components under supervised learning. By hyperspectral imaging of meat fluorescence by snap-shop type camera and extracting two components through LDA, the value of FI was estimated using the decision boundary contour plot shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003eb. RGB\u003csup\u003ehyper\u003c/sup\u003e images were constructed from snapshot hyperspectral images, as shown in the upper row of Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec. Gradually, the image turned bluish. The values of R, G, and B were calculated by averaging the intensities of the three channels at (630 nm, 670 nm, 690 nm), (520 nm, 540 nm, 575 nm), and (430 nm, 460 nm, 495 nm), respectively. The FI maps constructed from the hyperspectral images by the LDA and QDA are shown in the lower panel of Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ec. Thus, gradual changes in the FI values of the meat specimens were visualized as a function of storage time. Besides QDA, there exist various algorithms to determine decision boundaries, and it is possible to have different decision boundaries. Supplementary Fig.\u0026nbsp;8 shows decision boundaries and FI with other 3 machine learning algorithms, LDA, decision trees, Gaussian Naive Bayes. Even though decision boundaries with other algorithms are not identical, they shared similar characteristics, determining FI as lower/higher value (fresh/rotten state) with higher/lower value of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\text{L}\\text{D}\\text{A}\\:\\text{c}\\text{o}\\text{m}\\text{p}\\text{o}\\text{n}\\text{e}\\text{n}\\text{t}}_{1}\\)\u003c/span\u003e\u003c/span\u003e, which result in similar value of FI. There may not be an absolute solution, but we can choose proper algorithm, considering properties of data and computation resources.\u003c/p\u003e \u003cp\u003eFurthermore, ML analysis with snapshot type hyperspectral images was applied to vacuum-packaged meat for comparison with the PE wrap-packaged meat. The results are presented in Supplementary Fig.\u0026nbsp;9. For the vacuum-packaged meat, the RGBhyper images converted from the hyperspectral images and its FI map were compared on the 0th and the 28th day. The FI distribution up to the 28th day remained similar to the initial state, as shown in Supplementary Fig.\u0026nbsp;9a. The dependency of the FI values of PE- and vacuum-packaged meat specimens on storage time, obtained by line-scan type and snapshot type HIS, is shown in Supplementary Fig.\u0026nbsp;9b and 9c, respectively. Irrespective of the HIS type, i.e. either line-scan or snapshot type the time dependent behavior of FI was quite similar, both qualitatively and quantitatively. Machine learning algorithm is developed using hyperspectral images from 5 different storage times, shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003ea and Supplementary Fig.\u0026nbsp;9c. Hyperspectral images from the same meat sample at 6 different storage times have been used to confirm validity of the developed algorithm. PE wrap-packaged samples, which are not included in the training set, produce FI values that align with the overall trend as a function of storage time. Also, vacuum-packaged samples, which are also excluded during training, produce the reasonable results for FI by the algorithm as shown in Supplementary Fig.\u0026nbsp;9. This methodology could be applied to other kinds of muscle food for meat freshness sensing because it is based on fluorescence signal change from transition of chemical composition. This allows the combinations of HIS and ML to produce the reliable and reproducible results on meat freshness sensing.\u003c/p\u003e \u003cp\u003eFabry-Perot filters on CMOS image sensor can be extended to line-scan type as shown in Supplementary Fig.\u0026nbsp;10 to take advantage of its competitive advantages explained above. Line-scan type can be more suitable for large areal measurement locating at its close distance from sample surface since the imaging area by snapshot type is limited by its FoV. As a demonstration, three band pass filters in line shaped arrays (495 nm, 563 nm, 595 nm) are integrated on the CMOS image sensor to form a line-scan type HIS. The wavelength of three bands were selected for the maximum fluorescence intensity of NADH, porphyrin and a reference point to calculate freshness index as shown in Eq.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). Its detailed information is described in Methods and Supplementary Fig.\u0026nbsp;10. The same meat specimen as PE-wrapped one in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e was investigated as a function of storage day shown in Supplementary Fig.\u0026nbsp;10. These results well match also with those in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e measured by grating based line-scan type HIS and provides opportunities to applying HIS and ML based algorithm in various home appliances including refrigerator and smartphone in more efficient manner. Demonstration of line-scan type meat freshness sensor in the refrigerator can be found in supplementary movie.\u003c/p\u003e"},{"header":"CONCLUSION","content":"\u003cp\u003eIn this study, hyperspectral imaging technique has been demonstrated as a possible candidate of machine vision for consumer electronics such as smartphone and refrigerator. HIS was successfully applied to confirm meat freshness and to map freshness index by adopting ML. The FI value of the meat specimen was defined from the fluorescence spectrum and the value was compared with bacterial density. The features correlated with the freshness of meat were studied by line-scan type HIS incorporated with conventional gratings having high spatial and spectral resolution. Based on these fundamental studies, ML was applied to the hyperspectral images captured by the snapshot type HIS to identify meat freshness. LDA was adopted to efficiently reduce data dimensionality for extracting the features of meat freshness. Furthermore, QDA was applied to construct the decision boundaries of FI under supervised learning against LDA components. This work demonstrates that ambiguous physical quantities such as meat freshness can be substantialized by HIS and ML.\u003c/p\u003e \u003cp\u003eHSI based freshness sensing can be competitive compared with not only conventional destructive methods but also even e-noses and opto-noses, mimicking olfactory to detect food freshness in the aid of smartphone.\u003csup\u003e\u003cspan additionalcitationids=\"CR29\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e There are disadvantages on selectivity of sensing molecules and recycling of the sensors in case of e-noses and opto-noses. To make it more reliable, its measurement condition needs to be well defined since density of molecule floating in the ambient condition could change the result dramatically. Our method is advantageous in this sense since it is irrespective of chemical binding of sensing molecules to the sensor, and detects optical signals from finger prints regarding meat freshness. Moreover, our sensing scheme detects only the density of target molecule on the meat surface which enhance the applicability in the numerous electronic appliances.\u003c/p\u003e \u003cp\u003eAlso, there might be methods to increase the prediction accuracy applying additional information such as separating fluorescence signal of NADH from fat, excluding fat area with RGB image, and etc. This technique will thrive in everyday life through investigation in reflectance of specimen under ambient light with advantages without additional light source, collection optics and safety issue compared to Raman scattering and fluorescence-based measurement. Besides meat freshness sensing, the physically undefined indices such as freshness of other kind of foods, moistness of skin and so on might be evaluated through artificial intelligence assisted computational power alongside with HIS. Our findings can be applied to advanced machine vision for consumer electronics including refrigerator, smartphone and etc. which will open a new business model toward hyper-personalization and hyper-customization of human life.\u003c/p\u003e"},{"header":"EXPERIMENTAL AND ANALYSIS METHODS","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eHyperspectral imaging system and measurement condition\u003c/h2\u003e \u003cp\u003eGrating based Line-scan type: A commercial spectrometer composed of a grating and an image sensor from Ximea Corp. was installed in the line-scan type HIS. The distance (\u003cem\u003eh\u003c/em\u003e) between the meat specimen and the 365 nm LEDs was maintained at 2.5 cm. Using 28 LEDs, approximately 37 mW was uniformly applied over the meat specimen. A two-dimensional hyperspectral image was obtained by collecting the spectrum from the thin rectangular window along the x-axis with an exposure time of 400 ms at each step. The length in the perpendicular direction (\u003cem\u003el\u003c/em\u003e) against the thin rectangular window along the y-axis is determined by configuration of the system. The lateral resolution of the x-axis was strongly dependent on \u003cem\u003eh\u003c/em\u003e. The y-axis resolution could be varied by controlling the mechanical movement. The size of the images was 972\u0026times;55 pixels, while the spatial resolution was found to be 0.0988 and 0.408 mm/pixel along the x- and y-axes at \u003cem\u003eh\u003c/em\u003e\u0026thinsp;=\u0026thinsp;2.5 cm, respectively.\u003c/p\u003e \u003cp\u003eSnapshot type with Fabry-Perot filters: The distance between the meat specimen and hyperspectral image sensor was maintained around 50 cm, while distance between the meat specimen and 365 nm LEDs was 23 cm. A lens of F/1.2 and f\u0026thinsp;=\u0026thinsp;6 mm, purchased from Fujinon (DF6HA-1B), was used to image an area of approximately 104\u0026times;8.4 mm. The CMOS image sensor had 2608\u0026times;1960 pixels with a size of 1.4 \u0026micro;m per pixel, containing 10 bits of information. Each channel was composed of two pixels. Therefore, the spatial size of the data was 326\u0026times;245 after the demosaicing process. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows an image of a specimen with 140\u0026times;140 pixels. The measurement conditions were set to 1 s exposure time, 16 analog gain, and 1 digital gain.\u003c/p\u003e \u003cp\u003eLine-scan type with Fabry-Perot filters: The same model of CMOS sensor described in snapshot type is used. Arrays of three band pass filters (at 495, 563 and 595 nm with 5, 7 and 9 nm of full width of half maximum (FWHM), respectively) are integrated on the CMOS image sensor with line shape. Each filter covers 14 pixels in the lateral direction (19.6 \u0026micro;m in width). Data of 2 pixels at the edge of each filter are discarded for data analysis to avoid cross talk signals.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eImaging RGB by digital camera\u003c/h2\u003e \u003cp\u003eThe RGB images of the meat specimen were obtained in a small box using a digital camera (DSC-RX100, Sony Corp.). Simultaneously, a reference white plate (MINOLTA calibration plate) was placed right next to the meat specimen. A white LED was illuminated through a window placed 50 cm from the meat specimen.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003ePreparation of meat specimen\u003c/h3\u003e\n\u003cp\u003eBeef sirloin was purchased from a store in Suwon, Korea Federation of Livestock Cooperatives. A loaf was sliced into pieces of 5\u0026times;5\u0026times;1.5 cm. Each piece of meat specimen was placed on a black Styrofoam tray, which was sanitized with alcohol and packaged in a PE wrap. Five pieces of the meat specimen were prepared for each storage time. Two of the five pieces were checked with hyperspectral imaging at each storage time and consumed for \u0026#119873;\u0026#119887;\u0026#119886;\u0026#119888; measurement. Three additional pieces of the meat specimen packaged in a PE wrap were kept in the refrigerator and measured by hyperspectral imaging as a function of storage time. Additional pieces of the meat specimen were vacuum packaged.\u003c/p\u003e\n\u003ch3\u003eMeasuring bacterial density ( (CFU/unit)) and temperature\u003c/h3\u003e\n\u003cdiv class=\"Heading\"\u003eMeasuring bacterial density (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e (CFU/unit)) and temperature\u003c/div\u003e \u003cp\u003eGeneral bacteria were collected by rubbing both sides of the meat specimen surface with a swab (3M Pipette Swab) and given number of rubbing times. The rubbed swab was rinsed in 1 mg of phosphate-buffered saline (PBS), which was further diluted 10 times. Two sets of 1 ml of the diluted solution containing the general bacteria were dropped onto a petrifilm, which was dried at 35\u0026thinsp;\u0026plusmn;\u0026thinsp;1\u0026deg;C for 48\u0026thinsp;\u0026plusmn;\u0026thinsp;2 h. The number of red colonies was then counted. The final \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e value was obtained by multiplying the counted number by 10. As a reference, a clean PBS solution was compared with a PBS solution containing the general bacteria collected from the meat surface.\u003c/p\u003e \u003cp\u003eThe temperatures of the refrigerator (T\u003csup\u003eref\u003c/sup\u003e), freezer (T\u003csup\u003efre\u003c/sup\u003e), and national weather service (T\u003csup\u003enat\u003c/sup\u003e) were registered to understand the possible correlation with the increasing value of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{N}_{bac}\\)\u003c/span\u003e\u003c/span\u003e. The results shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea\u0026thinsp;~\u0026thinsp;e were measured on May 12\u0026thinsp;~\u0026thinsp;29, 2020, at T\u003csup\u003eref\u003c/sup\u003e = 2.2\u0026deg;C and T\u003csup\u003enat\u003c/sup\u003e =13.5\u0026thinsp;~\u0026thinsp;20.0\u0026deg;C, and those in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e were measured from September 9 to 21, 2020, at T\u003csup\u003eref\u003c/sup\u003e = 2.9℃ and T\u003csup\u003enat\u003c/sup\u003e =18.3\u0026thinsp;~\u0026thinsp;22.6℃. The results shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ef were measured from January 22 to February 10, 2021, at T\u003csup\u003eref\u003c/sup\u003e = 3.5℃, T\u003csup\u003efre\u003c/sup\u003e = \u0026minus;\u0026thinsp;4℃, and T\u003csup\u003enat\u003c/sup\u003e = \u0026minus;\u0026thinsp;8.1\u0026thinsp;~\u0026thinsp;7.9℃. The results shown in Figs.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e and \u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e were measured from May 14 to 29, 2021, at T\u003csup\u003enat\u003c/sup\u003e = 14.8\u0026thinsp;~\u0026thinsp;20.9℃ for the PE-wrapped specimen, while for the vacuum-packaged specimen, the results were measured from May 14 to June 11, 2021, at T\u003csup\u003enat\u003c/sup\u003e = 14.8\u0026thinsp;~\u0026thinsp;20.9℃.\u003c/p\u003e "},{"header":"Abbreviations","content":"\u003cp\u003eHIS hyperspectral imaging system; ML machine learning; FI fresh index\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eASSOCIATED CONTENT\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSupporting Information\u003c/strong\u003e.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u0026nbsp;The supporting information is available free of charge at XXX.\u003c/p\u003e\n\u003cul start=\"50\"\u003e\n \u003cli\u003eSchematic of grating based line-scan type HIS; Schematic of snapshot type HIS with Fabry-Perot filters; Fluorescence spectrum of the meat specimen, NAD and myoglobin; Freshness of vacuum-packaged meat specimen on the 12\u003csup\u003eth\u003c/sup\u003e day; Storage time dependence of RGB\u003csup\u003ehyper\u003c/sup\u003e; Influence of the packaging material on the acquisition of fluorescence spectrum; Dimensionality reduction from 11 to 2 dimensions in the hyperspectral data by LDA; Decision boundaries and FI depending on machine learning algorithms; \u0026nbsp;Storage time dependent FI of the PE- and vacuum-packaged meat specimens from snapshot type HIS; Line-scan type HIS with Fabry-Perot filters and storage time dependent FI of vacuum-packaged meat specimens; List of packaging materials and their chemical components (PDF)\u003c/li\u003e\n \u003cli\u003e\u0026nbsp;Demonstration of meat freshness sensor installed in Samsung Family Hub T9000 (Video file)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eAUTHOR CONTRIBUTIONS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eU.J.K., S.L., and H.S. conceived and designed the research. U.J.K. and S.K. conducted optical measurements. H.K. fabricated the 16CH filter arrays on CMOS image sensor and measured the corresponding transmissions. S.L. devised and carried out the machine algorithm. H.S. designed and fabricated the line-scan type HIS, suggested experimental procedures and methods. J.S.H. had valuable discussions on meat storage in refrigerators. U.J.K. and S.L. wrote the manuscript. All authors reviewed the manuscript.\u003c/p\u003e\n\u003cp\u003eThe manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eACKNOWLDEGEMENTS\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNational Research Foundation of Korea (NRF) grant NRF-2021R1F1A1062182 (PV, YP)\u003c/p\u003e\n\u003cp\u003eNational Research Foundation of Korea (NRF) grant NRF-020R1A6A1A03047771 (PV, YP)\u003c/p\u003e\n\u003cp\u003eKorea Institute for Advancement of Technology (KIAT) grant (PV, YP)\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eJain G, Paul J, Shrivastava A, Hyper-Personalization (2021) Co-Creation, Digital Clienteling and Transformation. J Bus Res 124:12\u0026ndash;23. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jbusres.2020.11.034\u003c/span\u003e\u003cspan address=\"10.1016/j.jbusres.2020.11.034\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJunmo Kim (2023) Digital Business Requirements in the Era of Hyper-Personalization. SamsungSDS July 28\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee S, Kim H, Kim G, Son H, Kim UJ (2024) Spectral Analysis on Color Detection Sharpness of Animal Vision toward Polychromatic Vision System. Adv Mater Technol. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/admt.202400671\u003c/span\u003e\u003cspan address=\"10.1002/admt.202400671\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJiang T, Li C, He Q, Peng ZK (2020) Randomized Resonant Metamaterials for Single-Sensor Identification of Elastic Vibrations. Nat Commun 11(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41467-020-15950-1\u003c/span\u003e\u003cspan address=\"10.1038/s41467-020-15950-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFeng C, Au WSA, Valaee S, Tan Z (2012) Received-Signal-Strength-Based Indoor Positioning Using Compressive Sensing. IEEE Trans Mob Comput 11(12):1983\u0026ndash;1993. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1109/TMC.2011.216\u003c/span\u003e\u003cspan address=\"10.1109/TMC.2011.216\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang X, Xie J, Li C, Xu R, Zhang Y, Liu S, Wang J (2018) MEMS-Based Super-Resolution Remote Sensing System Using Compressive Sensing. Opt Commun 426:410\u0026ndash;417. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.optcom.2018.05.046\u003c/span\u003e\u003cspan address=\"10.1016/j.optcom.2018.05.046\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShi Q, Zhang Z, He T, Sun Z, Wang B, Feng Y, Shan X, Salam B, Lee C (2020) Deep Learning Enabled Smart Mats as a Scalable Floor Monitoring System. Nat Commun 11(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41467-020-18471-z\u003c/span\u003e\u003cspan address=\"10.1038/s41467-020-18471-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGolestani N, Moghaddam M (2020) Human Activity Recognition Using Magnetic Induction-Based Motion Signals and Deep Recurrent Neural Networks. Nat Commun 11(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41467-020-15086-2\u003c/span\u003e\u003cspan address=\"10.1038/s41467-020-15086-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBallard Z, Brown C, Madni AM, Ozcan A (2021) Machine Learning and Computation-Enabled Intelligent Sensor Design. Nat Mach Intell 3(7):556\u0026ndash;565. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s42256-021-00360-9\u003c/span\u003e\u003cspan address=\"10.1038/s42256-021-00360-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee Y, Park J, Choe A, Shin YE, Kim J, Myoung J, Lee S, Lee Y, Kim YK, Yi SW, Nam J, Seo J, Ko H (2022) Flexible Pyroresistive Graphene Composites for Artificial Thermosensation Differentiating Materials and Solvent Types. ACS Nano 16(1):1208\u0026ndash;1219. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1021/acsnano.1c08993\u003c/span\u003e\u003cspan address=\"10.1021/acsnano.1c08993\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHadoux X, Hui F, Lim JKH, Masters CL, P\u0026eacute;bay A, Chevalier S, Ha J, Loi S, Fowler CJ, Rowe C, Villemagne VL, Taylor EN, Fluke C, Soucy JP, Lesage F, Sylvestre JP, Rosa-Neto P, Mathotaarachchi S, Gauthier S, Nasreddine ZS, Arbour JD, Rh\u0026eacute;aume MA, Beaulieu S, Dirani M, Nguyen CTO, Bui BV, Williamson R, Crowston JG, van Wijngaarden P (2019) Non-Invasive in Vivo Hyperspectral Imaging of the Retina for Potential Biomarker Use in Alzheimer\u0026rsquo;s Disease. Nat Commun 10(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41467-019-12242-1\u003c/span\u003e\u003cspan address=\"10.1038/s41467-019-12242-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoung HA, Ballard ZS, Wu J, Tseng DK, Teshome H, Zhang L, Horn EJ, Arnaboldi PM, Dattwyler RJ, Garner OB, Di Carlo D, Ozcan A (2020) Point-of-Care Serodiagnostic Test for Early-Stage Lyme Disease Using a Multiplexed Paper-Based Immunoassay and Machine Learning. ACS Nano 14(1):229\u0026ndash;240. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1021/acsnano.9b08151\u003c/span\u003e\u003cspan address=\"10.1021/acsnano.9b08151\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKim UJ, Lee S, Kim H, Roh Y, Han S, Kim H, Park Y, Kim S, Chung MJ, Son H, Choo H (2023) Drug Classification with a Spectral Barcode Obtained with a Smartphone Raman Spectrometer. Nat Commun 14(1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41467-023-40925-3\u003c/span\u003e\u003cspan address=\"10.1038/s41467-023-40925-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee YJ, Ko KS (2016) Effects of Extract of Lactic Acid Bacteria Culture Media on Quality Characteristics of Pork Loin and Antimicrobial Activity against Pathogenic Bacteria during Cold Storage. J Korean Soc Food Sci Nutr 45(10):1476\u0026ndash;1480. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3746/jkfn.2016.45.10.1476\u003c/span\u003e\u003cspan address=\"10.3746/jkfn.2016.45.10.1476\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSeol K-H, Kim KH, Kim YH, Youm KE, Lee M (2014) Effect of Temperature and Relative Humidity in Refrigerator on Quality Traits and Storage Characteristics of Pre-Packed Hanwoo Loin. Korean J Agricultural Sci 41(4):415\u0026ndash;424. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.7744/cnujas.2014.41.4.415\u003c/span\u003e\u003cspan address=\"10.7744/cnujas.2014.41.4.415\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAndr\u0026eacute;s S, Murray I, Navajas EA, Fisher AV, Lambe NR, B\u0026uuml;nger L (2007) Prediction of Sensory Characteristics of Lamb Meat Samples by near Infrared Reflectance Spectroscopy. Meat Sci 76(3):509\u0026ndash;516. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.meatsci.2007.01.011\u003c/span\u003e\u003cspan address=\"10.1016/j.meatsci.2007.01.011\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcManus C, Tanure CB, Peripolli V, Seixas L, Fischer V, Gabbi AM, Menegassi SRO, Stumpf MT, Kolling GJ, Dias E, Costa JBG (2016) Infrared Thermography in Animal Production: An Overview. Computers and Electronics in Agriculture. Elsevier B V April 1:10\u0026ndash;16. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.compag.2016.01.027\u003c/span\u003e\u003cspan address=\"10.1016/j.compag.2016.01.027\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKucha CT, Liu L, Ngadi MO (2018) Non-Destructive Spectroscopic Techniques and Multivariate Analysis for Assessment of Fat Quality in Pork and Pork Products: A Review. Sensors (Switzerland). MDPI AG Febr 1\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/s18020377\u003c/span\u003e\u003cspan address=\"10.3390/s18020377\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu B, Dahlberg K, Gao X, Smith J, Bailin J Rapid Measurement of Meat Spoilage Using Fluorescence Spectroscopy. In Imaging, Manipulation, and Analysis of Biomolecules, Cells, and, Tissues XV, SPIE (2017), ; Vol. 10068, p 1006820. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1117/12.2253526\u003c/span\u003e\u003cspan address=\"10.1117/12.2253526\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePu Y, Wang W, Alfano RR (2013) Optical Detection of Meat Spoilage Using Fluorescence Spectroscopy with Selective Excitation Wavelength. Appl Spectrosc 67(2):210\u0026ndash;213. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1366/12-06653\u003c/span\u003e\u003cspan address=\"10.1366/12-06653\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhuang Q, Peng Y, Yang D, Nie S, Guo Q, Wang Y, Zhao R (2022) UV-Fluorescence Imaging for Real-Time Non-Destructive Monitoring of Pork Freshness. Food Chem 396. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.foodchem.2022.133673\u003c/span\u003e\u003cspan address=\"10.1016/j.foodchem.2022.133673\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMinistry of Food and Drug Safety of Korea. KFDA. Food Code\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eA\u0026iuml;t-Kaddour A, Thomas A, Mardon J, Jacquot S, Ferlay A, Gruffat D (2016) Potential of Fluorescence Spectroscopy to Predict Fatty Acid Composition of Beef. Meat Sci 113:124\u0026ndash;131. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.meatsci.2015.11.020\u003c/span\u003e\u003cspan address=\"10.1016/j.meatsci.2015.11.020\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSu J, Yi D, Liu C, Guo L, Chen WH (2017) Dimension Reduction Aided Hyperspectral Image Classification with a Small-Sized Training Dataset: Experimental Comparisons. Sens (Switzerland) 17(12). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/s17122726\u003c/span\u003e\u003cspan address=\"10.3390/s17122726\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBeatriz PP, Garcia-Salgado; Volodymyr I, Ponomaryov; Sergiy NS (2020) Rogelio Reyes-Reyes. Efficient Dimension Reduction of Hyperspectral Images for Big Data Remote Sensing Applications. J Appl Remote Sens 14(3). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/adma.202002854\u003c/span\u003e\u003cspan address=\"10.1002/adma.202002854\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAyesha S, Hanif MK, Talib R (2020) Overview and Comparative Study of Dimensionality Reduction Techniques for High Dimensional Data. Inform Fusion 59:44\u0026ndash;58. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.inffus.2020.01.005\u003c/span\u003e\u003cspan address=\"10.1016/j.inffus.2020.01.005\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao B, Ulfarsson MO, Sveinsson JR, Chanussot J (2020) Unsupervised and Supervised Feature Extraction Methods for Hyperspectral Images Based on Mixtures of Factor Analyzers. Remote Sens (Basel) 12(7). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/rs12071179\u003c/span\u003e\u003cspan address=\"10.3390/rs12071179\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGuo L, Wang T, Wu Z, Wang J, Wang M, Cui Z, Ji S, Cai J, Xu C, Chen X (2020) Portable Food-Freshness Prediction Platform Based on Colorimetric Barcode Combinatorics and Deep Convolutional Neural Networks. Adv Mater 32(45). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/adma.202004805\u003c/span\u003e\u003cspan address=\"10.1002/adma.202004805\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAnisimov DS, Abramov AA, Gaidarzhi VP, Kaplun DS, Agina EV, Ponomarenko SA (2023) Food Freshness Measurements and Product Distinguishing by a Portable Electronic Nose Based on Organic Field-Effect Transistors. ACS Omega 8(5):4649\u0026ndash;4654. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1021/acsomega.2c06386\u003c/span\u003e\u003cspan address=\"10.1021/acsomega.2c06386\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIstif E, Mirzajani H, Dağ \u0026Ccedil;, Mirlou F, Ozuaciksoz EY, Cakır C, Koydemir HC, Yilgor I, Yilgor E, Beker L (2023) Miniaturized Wireless Sensor Enables Real-Time Monitoring of Food Spoilage. Nat Food 4(5):427\u0026ndash;436. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s43016-023-00750-9\u003c/span\u003e\u003cspan address=\"10.1038/s43016-023-00750-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Dongguk University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Hyperspectral Imaging, Freshness Sensing, Machine Learning, Advanced Machine Vision, Fluorescence Imaging","lastPublishedDoi":"10.21203/rs.3.rs-5551638/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5551638/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eImaging spectral information and analyzing its properties of materials have become intriguing for consumer electronics toward food inspection, beauty care and etc. Those sensory physical quantities are difficult to quantify. Hyperspectral cameras, which capture its figure and spectral information simultaneously, can be a good candidate for non-destructive remote sensing. In this study, with the aid of a hyperspectral imaging system (HIS) and machine learning (ML), meat freshness is converted into a measurable physical quantity, i.e., freshness index (FI). FI is defined from meat fluorescence, which has a strong correlation with bacterial density. Combined with ML techniques, hyperspectral data are processed more efficiently. By employing linear discriminant and quadratic component analyses, FI can be estimated from its decision boundary after hyperspectral data are obtained at an unknown freshness state. We demonstrate HIS grafted with ML performs as artificial eye and brain which is advanced machine vision for consumer electronics including refrigerators and smartphones. Advanced sensing versatility utilized by computational sensing systems allows hyper-personalization and hyper-customization of human life.\u003c/p\u003e","manuscriptTitle":"Machine Vision with CMOS based Hyperspectral Image Sensor Enables Meat Freshness Sensing","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-04 16:05:03","doi":"10.21203/rs.3.rs-5551638/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"980276bf-1eb6-40e1-b442-6c6ce29b1411","owner":[],"postedDate":"December 4th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":40961091,"name":"Photonics/optics"},{"id":40961092,"name":"Spectroscopy"}],"tags":[],"updatedAt":"2024-12-04T16:05:03+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-04 16:05:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5551638","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5551638","identity":"rs-5551638","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}