High-Q whispering gallery mode (WGM) resonators encapsulated in polydimethylsilozane (PDMS) for highly sensitive displacement detection
12320682 ยท 2025-06-03
Assignee
Inventors
- Jie Liao (St. Louis, MO, US)
- Lan Yang (St. Louis, MO)
- Abraham QAVI (St. Louis, MO, US)
- Maxwell ADOLPHSON (St. Louis, MO, US)
Cpc classification
International classification
Abstract
A displacement sensor including an optical whispering gallery mode (WGM) microresonator and a package encasing at least a portion of the WGM microresonator, the package comprising polydimethylsiloxane (PDMS). The WGM microresonator can be configured as a sensor and used in a displacement detection system that can detect displacement with high quality. Artificial intelligence can be implemented in the displacement detection system for improved sensing of different variables and/or pinpointing the location of perturbations.
Claims
1. A displacement sensor, comprising: an optical whispering gallery mode (WGM) microresonator; and a package encasing at least a portion of the WGM microresonator, the package comprising polydimethylsiloxane (PDMS), wherein the WGM microresonator is configured to capture information encoded in multimode spectra of the WGM microresonator for multiparameter sensing of perturbations and for generation of hyper-information barcodes, and wherein each mode of resonance of the WGM microresonator can sense different variables and/or pinpoint the location of perturbations.
2. The displacement sensor of claim 1, wherein the displacement sensor has a Q-factor of 10.sup.7 at 780 nm.
3. The displacement sensor of claim 1, wherein the displacement sensor has a detection limit of about 600 nm.
4. The displacement sensor of claim 1, wherein the package encases all of the WGM microresonator.
5. The displacement sensor of claim 1, wherein the package consists of PDMS only.
6. The displacement sensor of claim 1, wherein the WGM microresonator is configured as part of a sensor system, and the sensor system is configured to: generate the hyper-information barcodes using the WGM microresonator; analyze, via machine learning models, the hyper-information barcodes generated from the multimode spectra; and output from the machine learning models a multiparameter prediction.
7. The displacement sensor of claim 6, wherein the multiparameter sensing of perturbations includes position and amplitude of perturbations, and wherein the sensor system is further configured to simultaneously track both the position and amplitude of perturbations.
8. An optical whispering gallery mode (WGM) device comprising: a WGM microresonator; and a package encasing at least a portion of the WGM microresonator, the package comprising polydimethylsiloxane (PDMS), wherein the WGM microresonator is configured to capture information encoded in multimode spectra of the WGM microresonator for multiparameter sensing of perturbations and for generation of hyper-information barcodes, wherein each mode of resonance of the WGM microresonator can sense different variables and/or pinpoint the location of perturbations.
9. The WGM device of claim 8, wherein the WGM microresonator has a Q-factor of 10.sup.7 at 780 nm.
10. The WGM device of claim 8, wherein the WGM microresonator has a detection limit of about 600 nm.
11. The WGM device of claim 8, wherein the package encases all of the WGM microresonator.
12. The WGM device of claim 8, wherein the package comprises polydimethylsiloxane (PDMS).
13. A non-transitory computer-readable recording medium having computer executable instructions stored thereon, which when executed by a processor of a sensor system, cause the sensor system to: generate hyper-information barcodes using a multimode sensor, the multimode sensor configured to capture information encoded in multimode spectra of the multimode sensor for multiparameter sensing of perturbations; analyze, via machine learning models, the hyper-information barcodes generated from the multimode spectra; and output from the machine learning models a multiparameter prediction; wherein each mode of resonance of the multimode sensor can sense different variables and/or pinpoint the location of perturbations.
14. The non-transitory computer-readable recording medium of claim 13, wherein the instructions, when executed by the processor, further cause the sensor system to conduct real-time analysis and monitoring in dynamic and time-sensitive environments.
15. The non-transitory computer-readable recording medium of claim 13, wherein the multimode sensor comprises an optical whispering gallery mode (WGM) microresonator encased in a package comprising polydimethylsiloxane (PDMS).
16. The non-transitory computer-readable recording medium of claim 15, wherein the WGM microresonator has a Q-factor of 10.sup.7 at 780 nm.
17. The non-transitory computer-readable recording medium of claim 15, wherein the WGM microresonator has a detection limit of about 600 nm.
18. The non-transitory computer-readable recording medium of claim 15, wherein a package encases all of the WGM microresonator.
19. The non-transitory computer-readable recording medium of claim 18, wherein the package comprises polydimethylsiloxane (PDMS).
20. The non-transitory computer-readable recording medium of claim 13, wherein the instructions, when executed by the processor, further cause the sensor system to simultaneously track both the position and amplitude of perturbations.
Description
DESCRIPTION OF DRAWINGS
(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
(2) The accompanying drawings, which are incorporated in and form a part of the specification, illustrate the embodiments of the present disclosure and together with the description, serve to explain the principles of the disclosure. Those of skill in the art will understand that the drawings, described below, are for illustrative purposes only. The drawings are not intended to limit the scope of the present disclosure in any way.
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
(34)
(35)
(36)
(37)
(38)
(39)
(40)
(41)
(42)
(43)
(44)
(45)
(46)
(47)
(48)
(49)
(50)
(51)
(52)
(53)
(54)
(55)
(56)
(57)
(58)
(59) There are shown in the drawings arrangements that are presently discussed, it being understood, however, that the present embodiments are not limited to the precise arrangements and are instrumentalities shown. While multiple embodiments are disclosed, still other embodiments of the present disclosure will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative aspects of the disclosure. As will be realized, the invention is capable of modifications in various aspects, all without departing from the spirit and scope of the present disclosure. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.
DETAILED DESCRIPTION
(60) The packaging of WGM devices using non-toxic polydimethylsiloxane (PDMS) is disclosed herein. Q-factor as high as 10.sup.7 is achieved at the 780 nm band. The PDMS packaging technique not only endows the WGM device with better robustness and compactness but also enhances the humidity resistance significantly. By making use of the unique flexibility of PDMS, displacement detection with high sensitivity (e.g., of 0.1 pm/m) and a detection limit as low as 600 nm is achieved. The WGM devices disclosed herein exhibit such high sensitivity and low detection limit as well as long-term operational stability, all of which are crucial for potential applications in displacement sensing systems and medical diagnostics.
(61) Compared with conventional packaging techniques, which are costly, potentially toxic, and vulnerable to moisture, the PDMS packaging technique disclosed herein not only endows the WGM device with better robustness, stability, and compactness, but also enhances the humidity resistance significantly. Compared with all polymer WGM displacement sensors, the hybrid structure disclosed herein offers both high Q factors of silica microbubble resonators and high displacement sensitivity enhanced by the PDMS layer.
(62) The present disclosure details light confinement when the resonator is encapsulated within polymer materials with various refractive index contrasts, both theoretically and experimentally. The results disclosed herein indicate that materials with a higher refractive index (higher than that of low index polymers) can also provide significant optical confinement if the structure is properly engineered. In one aspect, the packaging of WGM microbubble resonators (MBRs) is carried out using polydimethylsiloxane (PDMS).
(63) Despite its high refractive index (1.4074 at 780 nm band), which may weaken the optical confinement when encapsulating a WGM resonator, PDMS bears the following key features that ensure its successful application for packaging: (1) PDMS has unique flexibility with a shear elastic modulus of 250 kPa, much lower than UV curable polymer (4.0 MPa)the flexible nature of PDMS makes it possible for the encapsulated WGM resonators to detect applied displacement by distributing deformation in a large volume; (2) PDMS forms a protective hydrophobic layer on the coating surface which can improve the structural/chemical stability of devices under humid conditions, and (3) PDMS has excellent thermal, mechanical, and chemical stability, non-toxicity, and ease of use for fabrication. Through the combination of these features, packaging WGM MBRs for displacement sensing is beneficial in reducing and/or solving the hygroscopicity, rigidity, and toxicity issues, as well as achieving improved detection limits.
(64) Additionally, accurate multiparameter sensing is crucial for capturing and analyzing complex interactions in diverse environments, leading to deeper, multi-faceted understandings and effective decision-making across various fields. Conventional multiparameter sensors relying on multiple sensors or microstructures, face limitations in further integration due to their complexity and their dependence on matrix methods (linear approximation) for signal analysis, which hinders their effectiveness in addressing real-world applications. To overcome these limitations, a new approach that integrates multimode sensing with machine learning algorithms in a single optical microresonator device is needed. The AI-empowered sensor disclosed herein leverages the hyper-information barcodes formed by the multimode sensing system, including resonances with distinct sensitivities and sensing hot spots. By extracting useful sensing information from the barcodes, the disclosed sensor achieves high-precision multiparameter sensing and perturbation tracking with excellent robustness against down-sampling and signal-to-noise ratio (SNR) reduction. The combination of multimode sensing and AI-driven data analysis position the disclosed sensing framework a promising tool for next-generation optical sensing and analysis systems.
(65) Optical microresonator sensors, based on the Whispering-gallery mode (WGM) resonance for signal enhancement, are a new sensing technology. Based on their unique working mechanism, WGM sensors have shown unprecedented sensitivity levels for the label-free optical detection of single biomolecule and ion, as well as physical parameters such as thermal radiation, optical absorption, magnetic field, displacement, and angular velocity. In addition, by making use of the collective pattern of the resonance spectrum, the accuracy and dynamic range can be significantly improved via the optical barcode technique. Furthermore, WGM resonators can support various WGMs in different wavelengths, mode numbers, polarizations, quality factors, and other properties. These diverse WGMs in one sensor can provide multiple sensing modalities with additional sensing information, akin to employing multiple sensors to detect a target parameter. With these capabilities, a multimode WGM sensor provides a simple solution to the complexity and linear-dependence challenges of multiparameter sensing.
(66) Disclosed herein is a hyper-information barcode technique for multiparameter sensing with only one single sensing element. The hyper-information barcode captures comprehensive data and multi-faceted information encoded in the multimode WGM spectrum of the sensor. Utilizing the distinct sensing channels provided by different WGMs, the sensor embedded under a micro-keyboard can accurately detect both the position and amplitude of perturbations applied. To discern patterns and relationships in raw sensor data, machine learning (ML) models are utilized for data analysis. These models are adept at analyzing both linear and nonlinear relationships, offering significant potential to overcome the limitations imposed by the conventional matrix method in existing multiparameter sensors. The performance of the models is further evaluated when dealing with sparse-sampled and noisy data. It is demonstrated to be a robust method for multiparameter sensing even in the presence of noise or down-sampling. Through continuous analysis of hyper-information barcodes, achieved simultaneous tracking of both the position and amplitude of perturbations was achieved. This dual tracking capability has enabled the application of a multi-factor authentication system that combines a password (a sequence of digital numbers) with the unique manner in which it is entered (subtle differences in pressing amplitude). This innovative approach enhances human-microrobot interaction, offering new possibilities in encryption and tactile sensing and other applications.
(67) In various aspects, the performance of WGM resonators may be assessed using any suitable existing analysis method without limitation.
(68)
(69)
(70) Plot 124 in
(71)
(72)
(73)
(74)
(75)
(76)
(77)
(78) In one aspect, database 706 includes displacement data 708 and algorithm data 710. Non-limiting examples of suitable displacement data 708 may include perturbations and position and amplitude of displacements. Non-limiting examples of suitable algorithm data 710 include any values of parameters defining the operation of the WGM resonator-based sensors, and displacement sensing systems. Additional non-limiting examples of suitable algorithm data 710 includes any algorithms and any values of parameters defining the algorithms associated with the disclosed method as described herein and or any displacement algorithms used to reconstruct or predict displacement as described herein.
(79) Computing device 702 also includes a number of components that perform specific tasks. In the example aspect, computing device 702 includes data storage device 712, displacement component 714, sensor component 716, and communication component 718. Data storage device 712 is configured to store data received or generated by computing device 702, such as any of the data stored in database 706 or any outputs of processes implemented by any component of computing device 702.
(80) Communication component 718 is configured to enable communications between computing device 702 and other devices (e.g., user computing device 612 and system 610, shown in
(81)
(82) Computing device 800 may also include at least one media output component 806 for presenting information to a user 808. Media output component 806 may be any component capable of conveying information to a user 808. In some aspects, media output component 806 may include an output adapter, such as a video adapter and/or an audio adapter. An output adapter may be operatively coupled to processor 802 and operatively coupled to an output device such as a display device (e.g., a liquid crystal display (LCD), organic light emitting diode (OLED) display, cathode ray tube (CRT), or electronic ink display) or an audio output device (e.g., a speaker or headphones). In some aspects, media output component 806 may be configured to present an interactive user interface (e.g., a web browser or client application) to user 808.
(83) In some aspects, computing device 800 may include an input device 810 for receiving input from user 808. Input device 810 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen), a camera, a gyroscope, an accelerometer, a position detector, and/or an audio input device. A single component such as a touch screen may function as both an output device of media output component 806 and input device 810.
(84) Computing device 800 may also include a communication interface 812, which may be communicatively coupled to a remote device. Communication interface 812 may include, for example, a wired or wireless network adapter or a wireless data transceiver for use with a mobile phone network (e.g., Global System for Mobile communications (GSM), 3G, 4G or Bluetooth) or other mobile data network (e.g., Worldwide Interoperability for Microwave Access (WIMAX)).
(85) Stored in memory 804 are, for example, computer-readable/-executable instructions for providing a user interface to user 808 via media output component 806 and, optionally, receiving and processing input from input device 810. A user interface may include, among other possibilities, a web browser and client application. Web browsers enable users 808 to display and interact with media and other information typically embedded on a web page or a website from a web server. A client application allows users 808 to interact with a server application associated with, for example, a vendor or business.
(86)
(87) Processor 902 may be operatively coupled to a communication interface 906 such that server system 900 may be capable of communicating with a remote device such as user computing device 612 (shown in
(88) Processor 902 may also be operatively coupled to a storage device 908. Storage device 908 may be any computer-operated hardware suitable for storing and/or retrieving data. In some aspects, storage device 908 may be integrated in server system 900. For example, server system 900 may include one or more hard disk drives as storage device 908. In other aspects, storage device 908 may be external to server system 900 and may be accessed by a plurality of server systems 900. For example, storage device 908 may include multiple storage units such as hard disks or solid state disks in a redundant array of inexpensive disks (RAID) configuration. Storage device 908 may include a storage area network (SAN) and/or a network attached storage (NAS) system.
(89) In some aspects, processor 902 may be operatively coupled to storage device 908 via a storage interface 910. Storage interface 910 may be any component capable of providing processor 902 with access to storage device 908. Storage interface 910 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing processor 902 with access to storage device 908.
(90) Memory 804 (shown in
(91)
(92) I. Theoretical Analysis and Numerical Simulations
(93) The structure of an MBR encapsulated in a packaging material is shown in
(94) One important optical property is optical absorption. To measure the absorptions of packaging materials (e.g., 106), solutions of low index UV curable polymer (e.g., MY Polymers, MY133) and PDMS (e.g., SYLGARD) are filled within square cuvettes of 10 mm in length. The absorption spectra are measured by a UV-visible spectrophotometer (e.g., Varian Cary 50 Bio). Before the absorption measurement, an empty square cuvette is measured as a reference sample. Each sample is measured for four facets and the absorption spectra are obtained from the average of the four measurements. When the solutions are cured, their absorption spectra are measured again using the same procedure. Due to the low optical absorption of PDMS, as shown in
(95) In addition to refractive index and optical absorption, the radial field distributions also depend on the geometrical features of MBRs, such as wall thickness and radius. A 2D simulation of WGMs of the PDMS packaged MBR (e.g., device 326) may be carried out by commercially available software (e.g., COMSOL Multiphysics (a commercial finite-element method software)). Device 326 (e.g., MBR 102 encapsulated in PDMS as packaging material 106) is considered as a rotationally axisymmetric dielectric structure. In the 780 nm wavelength band, the refractive indices of silica, low index UV curable polymer, and PDMS are 1.4537, 1.3240, and 1.4074, respectively. The radius (R) of the MBR (e.g., 102) is kept at 150 m, and the wall thickness (t) is set as 1.6 m, 3.9 m, and 6.3 m, respectively. The radial distributions of the electric field for various t are shown in
(96) To verify the simulation results, MBRs with various wall thicknesses were fabricated and their WGMs are tested.
(97) II. Device Performance
(98) Device performance parameters regarding stability, humidity resistance, and displacement detection are outlined below.
(99) A. Stability and Humidity Resistance
(100) The stability of packaged WGM devices is important for practical applications since changes in the environment may introduce noise and drift to the system. In general, the long-term deviation is mainly due to the ambient temperature drift, changes in relative ambient humidity (RH), and instability of the laser power. To show the stability improvement of the PDMS packaged device (e.g., 326), a comparison between its stability with another WGM device (e.g., 324) packaged with UV curable polymer can be conducted. Both devices (e.g., 324, 326) are placed together within a closed chamber (e.g., 310) and exposed to the same environmental condition. The temperature and the humidity are roughly controlled by an air conditioner (not shown). The temperature is not controlled precisely since the focus is on exploring the performance of both devices (e.g., 324, 326) under temperature drift.
(101) The stability of the two devices (e.g., 324, 326) under different operating laser power (e.g., of laser 302) can be tested first. In the 780 nm band, the optical absorption of PDMS is lower than that of UV curable polymer. Therefore, the PDMS packaged device (e.g., device 326) absorbs less energy and becomes more stable when operating at a high laser power. The temperature is around 18 C., and the humidity is around 50%.
(102)
(103) B. Displacement Detection
(104) With the protection of the flexible PDMS layer, displacement can be applied to the surface of the packaged device without breaking the sensor. Here, the capability of the present PDMS-based sensor for small displacement detection is demonstrated. As shown in the inset of
(105) The sensor (e.g., 412) disclosed herein exhibits good repeatability with a response time below 33 ms, which is shorter than the typical response time for piezo-resistive sensors, and the dynamic response to changes in displacement is measurable. The long-term drift (e.g., resonance shifts due to the humidity of the environment) would be ignorable in a small time scale.
(106) Further advantage can be taken of the high flexibility of PDMS that helps distribute stress induced by displacement across a large volume. As a result of large deformation volume, MBR can also detect displacement applied at other positions away from it. With reference to
(107) C. Conclusion
(108) A WGM resonator-fiber coupling system (e.g., 100) packaged in PDMS (e.g., 106) for displacement sensing demonstrates high flexibility, great resistance to moisture penetration, and non-toxicity, rendering PDMS a suitable material for packaging WGM resonators as displacement sensors. Packaged devices with Q-factors as high as 10.sup.7 at the 780 nm band are obtained. The optimized device exhibits a high sensitivity of 0.1 pm/um with a detection limit as low as 600 nm and demonstrated long-term operational stability, all of which are crucial for potential applications in displacement sensing systems and medical diagnostics. Finally, the disclosed PDMS-based sensor is capable of measuring displacement at different locations near the resonator. Compared with all polymer WGM sensors, the disclosed hybrid structure offers both high quality factors of silica microbubble resonators and high displacement sensitivity enhanced by the PDMS layer. The disclosed packaged device (e.g., 326) is portable, free from contamination, and capable of field sensing applications. When empowered by artificial intelligence (AI), the disclosed sensor can not only to detect the displacement with high sensitivity, but also pinpoint the location of the displacement with high accuracy, useful in a variety of fields and technologies, including robotics, human-machine interface, tactile object recognition, and minimally invasive surgery. Additional AI aspects are disclosed below.
(109) III. AI-empowered Hyper-information Barcodes for Multiparameter Sensing
(110) A. Results-Multimode Sensing as an Effective Integration of Multiple Sensors
(111) In the case of multimode sensing, the WGM sensor supports multiple high Q resonances with various mode numbers within a fine-scanning wavelength range. Attributed to their distinct modal profiles, material and geometry dispersion, as well as wavelength-dependent light-matter interaction, each of them can potentially be used to detect a different aspect of the local environment. The working principle is similar to an array of individual sensors, where each sensor is designed to sense a specific local environment or a particular environmental variable. The spatial arrangement of the individual sensors can cause each sensor to have a different sensing hotspotan area of maximum sensitivity where it can detect changes in the environment most effectively. This characteristic of sensor arrays can be leveraged for multiparameter sensing, where multiple variables are measured simultaneously.
(112)
(113)
(114)
(115)
(116)
(117)
(118)
(119)
(120)
(121)
(122)
(123) As illustrated in
(124) To demonstrate this concept, the microsensor can be embedded in a micro-keyboard with 9 digits, and each digit corresponds to one position on the micro-keyboard. When a click perturbation is applied, the effective refractive indexes of modes in the microsensor can be affected, leading to varying responses, depending on their sensing hotspots and coupling conditions. As shown in
(125) The optical microbubble resonator (MBR) is presented as an optimal platform for multimodal sensing. A first validation of its comprehensive responses of different modes to localized disturbances is conducted. As shown in
(126) B. Hyper-Information Barcode Sensing Framework
(127) To analyze the comprehensive response from multimode and multi-faceted information in the hyper-information barcodes, machine learning algorithms are explored to model the response of individual modes and extract the sensing information automatically. In this experiment, perturbations can be applied at different positions on the micro-keyboard, each spaced 20 m apart and representing one digit. The hyper-information barcodes are collected when the perturbation is applied at each position with a certain amplitude.
(128) Particularly, the barcode is represented as a feature vector I()=[I1, I2, . . . , Id] with a dimension of d. The individual feature value I corresponds to the intensity of a specific wavelength within the measured spectrum range. One objective is to determine the perturbation position p as well as perturbation amplitude d of the barcode data. The sensing task is divided into two separate problems: (1) predicting the discrete perturbation position is a classification problem, and predicting the perturbation amplitude is a regression problem. Different machine-learning models can be applied to capture the features and derive sensing information. Then features in the data and decisions made by models are visualized.
(129) To show how the hyper-information barcodes capture details about size and position, principal component analysis (PCA) can be applied, which is a statistical method to reduce a cases-by-variables data table to its principal components, to the barcode data. This helps us easily see how these barcodes group together based on their characteristics. As shown in
(130) C. Results and Model Performance
(131) Three different models are implemented in the classification task: random forest (RF), an ensemble learning algorithm that combines the predictions of multiple decision trees; support vector machine (SVM), a supervised learning algorithm that searches for the optimal hyperplane that best separates the data points into different classes; and artificial neural network (ANN), a machine learning algorithm that takes inspiration from the structure and function of biological neural networks, like those found in the human brain. After identifying the perturbation position, the spectrum can be used to estimate the amplitude of perturbation using linear regression. The details of these models are described herein.
(132) The evaluation results on the testing data are demonstrated in the following tables (Table 1 and Table 2). To adequately evaluate the model performance, multiple metrics like accuracy, macro recall, macro precision and macro F1 score for the classification task on position prediction can be employed, and also the use of the rooted metric of rooted mean squared error (RMSE) and R square (R.sup.2) for the regression task on amplitude prediction. More detailed description of evaluation metrics are provided herein. Table 1 lists accuracy and other information of various models:
(133) TABLE-US-00001 TABLE 1 The overall result of classification on perturbation prediction on various models and metrics. Macro Macro Model Accuracy Recall Precision RF 0.9088 0.9098 0.9105 SVM 0.7506 0.7603 0.7506 ANN 0.9712 0.9717 0.9713
(134) From Table 1, the ANN model achieved the best performance, the possible reason could be ANN provides a more complicated structure for capturing non-linear relations from data. While vanilla SVM only works if the data is linearly separable, leading to the worst performance.
(135) Further details of the classification result, and model architectures are provided herein. Model Details where the confusion matrices demonstrated the classification result of each position, indicating that the models are easily misclassify between the position 1 and 2. Table 2 lists results of linear regression and ANN (Regression):
(136) TABLE-US-00002 TABLE 2 The overall result of regression on perturbation prediction on various models and metrics Model RMSE R.sup.2 Linear Regression 0.1493 0.9999 ANN(Regression) 0.2572 0.9997
(137) As for the regression task on amplitude prediction, both linear regression and ANN could achieve extremely high performance on the testing set with R.sup.2 closing to 1. However, the linear regression model slightly outperforms the complicated neural network model.
(138)
(139) D. Assessing Model Performance with Sparse-sampled Data and Noise Levels.
(140) Assessment of model performance with sparse-sampled data and noise levels are described below.
(141) (i) Sparse-sampled Data
(142) In low-resource measurements or edge-sensing applications where fast analysis is crucial, a lower data sampling rate is often employed to reduce computational load, data storage requirements, and power consumption. However, it's important to note that a lower sampling rate might compromise the resolution or accuracy of the data collected, potentially impacting the effectiveness of the measurement.
(143) To simulate limited data acquisition rates, sparse-sampled data can be obtained by down-sampling the spectra in the disclosed database with down-sampling factors with the stride of 10, 20, 50, 100, 500, and 1000. The representative resonant spectra after down-sampling are shown in
(144) The down-sampling factor can have a significant impact on the accuracy of machine-learning models. When the down-sampling factor is too high, the loss of spectral features can result in a reduction of the amount of useful information available to the model, leading to a decrease in accuracy. As shown in
(145) (ii) Noises in Data
(146) To simulate noise in the measurements, Additive White Gaussian Noise (AWGN) to the spectra in the disclosed database at various signal-to-noise ratios (SNRs) of 100 dB, 60 dB, 50 dB, 40 dB, 30 dB, 20 dB, and 10 dB was applied. AWGN is a basic noise model used in information theory to mimic the effect of many random processes that occur in nature. A randomized sample with white noise can be generated for each spectral data. The representative resonant spectra after adding noise are shown in
(147) E. Continuous Tracking of Inputs on a Micro-Keyboard
(148) With the capability to robustly decipher the multiparameter from the hyper-information barcodes, the disclosed sensor can be employed to continuously track the inputs on the micro-keyboard. Two sets of input sequences including combinations of different perturbation positions and amplitudes may be applied. The continuously measured spectrum is transformed into barcodes and analyzed by the machine learning models. As shown in
(149) In security and encryption, this sensing capability transforms into a powerful tool for secure access systems. The sensor's ability to discern not just the correct password or code, but also the unique manner of its entry, incorporating elements such as pressure and speed, introduces an additional layer of security. This dual recognition system-identifying both the code and the specific input methodprovides a sophisticated approach to security, greatly enhancing protection against unauthorized access.
(150) The performance of machine learning models trained on spectra with different levels of noise can be affected. As the noise level increases, the accuracy of the model can decrease, particularly for small resonance dips or features that are close to the noise level. As shown in
(151) F. Conclusion.
(152) Disclosed herein is the concept of hyper-information barcode using a multimode sensor as an effective sensor array to capture comprehensive data and multi-faceted information for multiparameter sensing. Each mode of resonance can sense different variables or pinpoint the location of perturbation, thus performing the same functionality as individual sensors in a sensor array. Machine learning models are employed to analyze the hyper-information barcodes generated from multimode spectra, enabling multiparameter measurements in one sensing device with high accuracy. RF, SVM, ANN, and linear regression models can be employed to predict the position and amplitude of perturbation. In the blind test, the trained models achieved a prediction accuracy of over 90% for the position and an MSE of only 0.0223 m2 for the amplitude, with a high R2 value of 0.9999. The performance variance of the models in dealing with sparse-sampled data and noise levels can also be evaluated. Some of the models perform well even when dealing with sparse-sampled data and noise levels. Furthermore, the continuous tracking of varying perturbation positions and amplitudes applied on the sensor can be demonstrated. These inputs on the micro-keyboard can be predicted by the model with high accuracy, showing potential applications in human-microrobot interaction, tactile sensing, as well as security and encryption. The results demonstrate the potential of AI-empowered multimode sensing as a powerful tool and sensing framework for high-precision and multiparameter measurement.
(153) The high accuracy and robustness of the models in handling noisy and sparse-sampled data suggest that this approach could be applied in a wide range of practical applications, such as chemical and biological sensing, environmental monitoring, and structural health monitoring. The training time required for the models is remarkably short, with the longest time recorded at only 19.90 seconds. The average time to analyze a single barcode is 4.3 ms. This rapid analysis time allows for the potential of real-time analysis and monitoring, enabling the models to be used in dynamic and time-sensitive environments. Additionally, the low computational requirements of the models make them suitable for deployment on low-power devices. For instance, the trained algorithm can be implemented within an embedded AI system with remote control via a customized application/app (e.g., an iOS app), enabling real-time sensing and analysis, which further expands their potential applications.
(154) Overall, the synergy between multimode sensing and AI algorithms offers numerous advantages, including high-precision multiparameter sensing, automated analysis and component identification, and robustness against downsampling and SNR reduction, all in a single sensing device, with a simple and compact structure. This sensing framework could also work for other sensors supporting multimode. The vast amounts of data generated by multimode sensors can be efficiently analyzed by AI algorithms, which can recognize intricate patterns and correlations that could be difficult for humans to detect, or time-consuming, and resource-intensive. Additionally, the rapid training and analysis time as well as low computational requirement make the integration of multimode sensing and AI models more appealing for diverse sensing and monitoring applications. This promising combination has the potential to revolutionize multiple fields, enabling real-time, accurate, and reliable multiparameter sensing and monitoring, and opening up new possibilities for various applications.
(155) G. Methods
(156) (i) Training of ML Models
(157) Collecting a large amount of data is important in building a robust and accurate model. To ensure sufficient data for training and testing, 10,800 frames of spectra, with 1,200 spectra recorded at each position and amplitude were collected. Each spectrum in the datasets is labeled with 36 attributes that encoded the displacement amplitudes of the nine positions (1-9), with 6 attributes (10 m-60 m) for each position. The data is divided into a training set including 70% of the library and a testing set including 30%. The spectrum samples in these two datasets are selected randomly from the data library without overlap, ensuring that the trained model does not receive any information about the testing dataset. The training set is used to optimize the model parameters, while L2 regularizations are introduced to avoid overfitting. The resonant spectra in the training set are transformed into matrices as the input of the AI model, which is set up using, for example, Scikit-learn (Sklearn) over an Ubuntu 16.04 server with one Intel Core i7-8700K CPU @ 3.70 GHz12 (memory 15.5 GiB, without GPU acceleration). The AI model is subsequently trained to decompose the resonant spectrum of the mixture into resonant spectra of individual components and predict the respective position and amplitude of the perturbation. After training, the AI model is evaluated using a separate testing set, which is disjoint from the training set and is not allowed to be used in the training process. This approach allowed us to assess the model's ability to generalize to new data and estimate its accuracy in real-world scenarios. Algorithms and data of the various models disclosed herein may be stored, for example, in a database such as algorithm database 710 (shown in
(158) (ii) Hyperparameter Tuning
(159) To obtain the optimal hyperparameters of a model with high accuracy and fast computation without overfitting, grid search is employed to exhaustively search through a specified parameter grid. The best parameters for each model are chosen based on the highest validation accuracy and low mean fit time.
(160) (iii) Measurement of WGM Spectrum and Generation of Hyper-Information Barcodes
(161) To capture the spectrum, a tunable laser source within the 780 nm wavelength band is utilized to scan across multiple modes. The transmission spectrum is then captured by a photodetector. To optimize the light intensity, an optical attenuator, and a polarization controller is used to manage the light's polarization can be employed. By carefully adjusting the coupling position of the resonator, high-order modes can be effectively excited. The output from the photodetector is displayed on an oscilloscope for real-time observation and is also fed into a computer via a data acquisition card for detailed analysis. The transmission intensity in the spectrum is divided into 10000 pieces to form a one-dimensional array. Each element of the array corresponds to a rectangular area in the hyper-information barcode image, whose color is determined by the value of the element through a colormap. The colormap used here is the Parula colormap, which can maintain a smooth color gradient even when plotted in greyscale.
(162) H. Supporting Information
(163) Additional (e.g., supporting) information is provided below.
(164) (i) Sensing Framework Based on Machine Learning
(165) Our objective is to determine the perturbation position p as well as perturbation amplitude d of the barcode data, represented by p=f.sub.1(I) and d=f.sub.2(I), respectively. These parameters differ in nature: the perturbation position is a discrete variable and can take any one of the positions tested in the experiment, while the perturbation amplitude is a continuous variable that depends on the magnitude of the displacement applied. Therefore, the sensing task is divided into two separate problems: (1) determine the relationship between the barcode data and the discrete perturbation position as a classification problem, and the other is to determine the relationship between the barcode data and the continuous perturbation amplitude as a regression problem. Once these relationships are known, the perturbation position and its amplitude directly from the measured spectrum by p=f.sub.1(I) and d=f.sub.2 (I), respectively, can be determined. To achieve this, a large number of barcodes at various perturbations are first measured. This step serves as the calibration step for the multimode sensor. The collected dataset D, which contains N examples in the form of (I.sub.i, p.sub.i, d.sub.i), i=1 . . . . N, is employed to train the disclosed machine learning models and learn the relationships between the spectrum data I and the parameters p and d (f.sub.1 and f.sub.2). This underlying relationship is obtained via a parametric model in the form of y=g (I, w), where w is the weight vector. A loss function L is then defined to estimate the error or mismatch between the predicted output y and the true value of p (d). The learning algorithms aim to find the optimal parameter vector w that minimizes the loss function L and best fits the data by using optimization techniques. By minimizing the loss function L, the disclosed model can improve the accuracy and performance in predicting the output parameter p (d) for new data inputs.
(166) (ii) Model Details
(167) (1) PCA Analysis
(168) PCA functions as an orthogonal linear transformation that converts features into a new coordinate system, such that the largest variance by a scalar projection of the data comes to lie on the first coordinate (the first principal component), the second largest variance on the second coordinate (the second principal component), and so on. Given a dataset with matrix XR.sup.(nd), where n denotes the number of samples in the dataset and d denotes the dimensionality of the features, the covariance matrix X.sup.TX of the data can be computed and the eigenvalues and corresponding eigenvectors to establish the new coordinate can be calculated.
(169) However, calculating the covariance matrix and its eigenvalues could be computationally consuming. Thus, PCA can be associated with matrix factorization methods like the singular value decomposition (SVD) instead of computing the covariance matrix, the SVD of matrix X can be written as equation 1:
(170)
(171) Here is a rectangular diagonal matrix of R.sup.(nd) with entries being the positive number considered as singular value of X, and U, W are both matrices whose columns are orthogonal unit vectors while W is also the same as the eigenvectors of X.sup.TX. Then the transformation T=XW could map the data from the original space to a new coordinator system. To reduce the dimension, only the first L principal components (columns in W) could be kept, so that the transformed data T is truncated as equation 2:
(172)
(173) PCA analysis was implemented on both displacement prediction (regression) and position prediction (classification). The results revealed distinct decision boundaries for features extracted by the machine learning model, capturing the desired relationships across the samples. Specifically, one can take the output of the hidden layer j of the Artificial Neural Network (ANN) denoted in
(174) From
(175) (2) Random Forest Classifier
(176) In the disclosed model, the number of trees in the forest is 200 with a maximum depth of 6 for individual trees.
(177) (3) Support Vector Machine (SVM)
(178) Kernel functions in SVMs are used to transform the data features into a higher-dimensional space, where it becomes easier to separate the data into classes using a linear boundary. Different types of kernel functions have different characteristics and are suitable for different types of datasets. The choice of kernel function can greatly affect the accuracy and speed of the SVM. In this case, the WGM spectra contain multiple resonance dips and are therefore more suitable for nonlinear kernel functions. Specifically, using the radial basis function kernel (RBF kernel) in SVM results in a higher average accuracy of 91% compared to the sigmoid kernel (75%). The RBF kernel also has a shorter training time of 9.63 seconds, compared to the sigmoid kernel's training time of 16.74 seconds.
(179) (4) Artificial Neural Network (ANN)
(180) In the disclosed settings, a shallow ANN (with a single hidden layer) is constructed with 10 neurons in the hidden layer. The rectified linear unit (ReLU) function, f(x)=max (0, x), is used as the activation function for the hidden layer. The learning rate scheduled for weight updates is 0.1. The training time is 19.90 seconds. The results show an average accuracy of 96%, indicating that the ANN model is very effective for identifying the perturbation positions.
(181) I. Evaluation Metrics
(182) To evaluate the performance of classifying the perturbation position, predictions were made on the testing data and measured these prediction outcomes with various metrics: Accuracy: The proportion of correct predictions out of all predictions from the testing set. Macro Recall: The metric of recall is originally applied to binary classifications, which calculates the ratio of covered positive samples by the model:
(183)
(184) For classifications of multiple categories in the disclosed experiments, the macro recall defined as the mean value of recall in each class where the corresponding class is considered the positive label can be used.
(185) Macro Precision: Similarly, the metric of precision is originally applied for the binary classification, which calculates the ratio of correctly predicted positive samples by the model:
(186)
(187) For classifications of multiple categories in the disclosed experiments, the macro precision defined as the mean value of precision in each class where the corresponding class is considered the positive label can be used.
(188) To evaluate the regression performance on amplitude prediction, the rooted mean square error and R square score are used for evaluating the performance.
(189) Rooted Mean Square Error (RMSE): it is a measure of the differences between values predicted by a model and the values actually observed from the environment that is being modeled. It is a standard way to measure the error of a model in predicting quantitative data:
(190)
(191) R Squared (R.sup.2): it is a statistical measure of how close the data are to the fitted regression line. As R Squared is closer to 1, the model fits better.
(192)
(193) For all of the above-described embodiments and usages, any code and/or data or other information may be stored in a memory of the above-described system, and/or in a remote (e.g., cloud) storage system (e.g., in a dedicated database or other centralized storage mechanism). Embodiments of the invention may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. The computer-executable instructions may be organized into one or more computer-executable components or modules. Generally, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform particular tasks or implement particular abstract data types. Aspects of the invention may be implemented with any number and organization of such components or modules. For example, aspects of the invention are not limited to the specific computer-executable instructions or the specific components or modules illustrated in the figures and described herein. Other embodiments of the invention may include different computer-executable instructions or components having more or less functionality than illustrated and described herein. Aspects of the invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
(194) In operation, a computer executes computer-executable code/instructions embodied in one or more computer-executable components stored on one or more computer-readable media to implement aspects of the invention described and/or illustrated herein. Code can include application source code, application object code, configuration data, additional input events, application states, assertion statements, validation results, and/or any other type of data. The order of execution or performance of the operations in embodiments of the invention illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the invention.
(195) The raw and/or processed data and/or any related graphical or other representations of the data may be processed by the above-described computer system or the like and output for display on a display device such as a TV, monitor, mobile device (e.g., mobile phone or tablet) and the like such that a technician/practitioner/evaluator/therapist/user can view and/or manipulate the data (e.g., the data may be presented in a visual format for presenting certain aspects of the test results, for example as shown in the applicable above-noted figures). For example, a display monitor may be connected (e.g., wired or wirelessly) to the above-described computer system to provide a visual output on the computer system. The computer system may have an operating system with a graphical user interface capable of being used by a user to (i) input, view, execute and/or manipulate the above-described computer code and/or (ii) process the obtained sensor data and any related graphical representations of such data in the manners described above. The operating system may be capable of running software applications such as those described above (e.g., MatLab and the like) for carrying out the above-described techniques and also any necessary post-processing and/or outputting of the obtained sensor data for viewing, such as for viewing by a therapist that is treating/diagnosing a patient/test subject. Additional software for other code/data manipulations and/or for generating other visuals relating to the data may also be present on the computer system.
(196) In the present disclosure, all or part of the units or devices of any system and/or apparatus, and/or all or part of functional blocks in any block diagrams and flow charts may be executed by one or more electronic circuitries including a semiconductor device, a semiconductor integrated circuit (IC) (e.g., such as a processor), or a large-scale integration (LSI). The LSI or IC may be integrated into one chip and may be constituted through combination of two or more chips. For example, processor as used herein refers generally to any programmable system including systems and microcontrollers, reduced instruction set circuits (RISC), application specific integrated circuits (ASIC), programmable logic circuits (PLC), and any other circuit or processor capable of executing the functions described herein. The functional blocks other than a storage element may be integrated into one chip. The integrated circuitry that is called LSI or IC in the present disclosure is also called differently depending on the degree of integrations, and may be called a system LSI, VLSI (very large-scale integration), or ULSI (ultra large-scale integration). For an identical purpose, it is possible to use an FPGA (field programmable gate array) that is programmed after manufacture of the LSI, or a reconfigurable logic device that allows for reconfiguration of connections inside the LSI or setup of circuitry blocks inside the LSI. Furthermore, part or all of the functions or operations of units, devices or parts or all of devices can be executed by software processing (e.g., coding, algorithms, etc.). In this case, the software is recorded one or more non-transitory computer-readable recording media, such as one or more ROMs, RAMs (e.g., DRAM, SRAM), optical disks, hard disk drives, solid-state memory, servers, cloud storage, and so on and so forth, having stored thereon executable instructions which can be executed to carry out the desired processing functions and/or circuit operations. For example, when the software is executed by a processor, the software causes the processor and/or a peripheral device to execute a specific function within the software. The system/method/device of the present disclosure may include (i) one or more non-transitory computer-readable recording mediums that store the software, (ii) one or more processors (e.g., for executing the software or for providing other functionality), and (iii) a necessary hardware device (e.g., a hardware interface). Artificial intelligence in any and all types and formats may be utilized in any of the steps, techniques, protocols, analyses, and/or any other manipulation, generation, or other creation of data, results and/or any information described herein. This includes but is not limited to computer visions, machine learning, deep learning, neural networks, algorithms, and any data, models, and training needed for such. The above examples are example only, and thus are not intended to limit in any way the definitions and/or meanings of the terms.
(197) Data conduits and any other communication or data transfer as described herein may include wired or wireless connections. For example, a wired network connection (e.g., Ethernet or an optical fiber), a wireless communication means, such as radio frequency (RF), e.g., FM radio and/or digital audio broadcasting, WiFi (e.g., IEEE 802.11 standards), WIMAX, a short-range wireless communication channel such as BLUETOOTH, a cellular phone technology (e.g., GSM), a satellite communication link, and/or any other suitable communication means. Such data conduits, in particular wired versions, can also be referred to as a system bus.
(198) The embodiments were chosen and described in order to best explain the principles of the disclosure and its practical application to thereby enable others skilled in the art to best utilize the disclosure in various embodiments and with various modifications as are suited to the particular use contemplated. Aspects of the disclosed embodiments may be mixed to arrive at further embodiments within the scope of the invention.
(199) As various modifications could be made in the constructions and methods herein described and illustrated without departing from the scope of the disclosure, it is intended that all matter contained in the foregoing description or shown in the accompanying drawings shall be interpreted as illustrative rather than limiting. Thus, the breadth and scope of the present disclosure should not be limited by any of the above-described example embodiments, but should be defined only in accordance with the following claims appended hereto and their equivalents.
(200) Various aspects of the embodiments described above may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and is therefore not limited in its application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects describe in other embodiments.
(201) Use of ordinal terms such as first, second, third, etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.
(202) Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of including, comprising, having, containing, involving, and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.
(203) The word example is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc. described herein as example should therefore be understood to be an illustrative example and should not be understood to be a preferred or advantageous example unless otherwise indicated.
(204) It should be noted that, as used herein, the term couple is not limited to a direct mechanical, electrical, and/or communication connection between components, but may also include an indirect mechanical, electrical, and/or communication connection between multiple components.
(205) Having thus described several aspects of at least one embodiment, it is to be appreciated that various alternations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.