PERSONALIZED TONGUE-MACHINE-INTERFACE

Abstract

For exploiting novel use-cases, in particular sophisticated human-machine interaction, with an intraoral electronic tongue monitoring system designed to be worn by a user on the upper or lower jaw and featuring a support sheet bearing a number of intraoral sensors arranged in an array for recording tongue movement and/or tongue pressure, it is proposed that the system comprises at least one extraoral sensor located outside of the oral cavity delimited by the teeth when the system is in place, in particular such that extraoral and/or intraoral and/or interlabial movements of the tongue and/or lip pressure can be recorded with the system and/or such that the system may be used as an input device controlled through tongue movement using a human-machine-interface provided by the system.

Claims

1. An intraoral electronic tongue monitoring system, comprising a support sheet to be attached to a human upper or lower jaw in an oral cavity and an sensor array arranged on the support sheet, wherein the sensor array comprises at least one intraoral sensor and at least one extraoral sensor, in particular at least one interlabial sensor.

2. An intraoral electronic tongue monitoring system, comprising a support sheet to be attached to an upper or lower jaw in an oral cavity and an sensor array arranged on the support sheet, characterized in that the system comprises data acquisition electronics for acquiring a sensor signal form the sensor array and control electronics configured to generate control signals based on said acquired sensor signal, wherein the control signals are designed to control a human-machine-interface of the system.

3. The system according to claim 1, wherein the system comprises at least one actuator for providing feedback perceptible to the tongue and/or wherein the system, in particular the support sheet and the sensing film, show a C-shape with a central recess.

4. The system according to claim 3, wherein the system comprises an electronic driving unit configured to provide a drive signal to said at least one actuator, in particular for providing interactive feedback to a user of the system and/or in response to a signal received from the sensor array, preferably wherein the system comprises a, preferably wireless, communication module for bidirectional communication with an external electronic device and/or for enabling the interactive feedback through interaction with an external electronic device.

5. The system according to claim 1, wherein an intraoral portion of the system features a maximum material thickness, in particular with respect to a stack comprising the support sheet, a sensing film bearing the sensor array and a cover film, of less than 2.0 mm, preferably of less than 1.5 mm, most preferably of less than 1.0 mm, and/or wherein the support sheet is 3D-printed or thermoformed and/or wherein the support sheet shows a thickness of less than 1.0 mm, preferably of less than 0.5 mm, most preferable of less than 0.3 mm.

6. The system according to claim 1, wherein the sensors of the sensor array are capacitive sensors, preferably wherein each of the capacitive sensors is formed by a dielectric elastomer transducer, comprising a soft and reversibly compressible dielectric layer sandwiched between two conformable electrodes, in particular wherein said at least one actuator is formed by such a DET.

7. The system according to claim 1, wherein said support sheet includes a lead protruding from the oral cavity, preferably in the area of the incisors and/or with a width of less than the four upper incisors, for supporting said at least one extraoral sensor, in particular said at least one interlabial sensor.

8. System according to claim 1, wherein the sensor array comprises at least one extraoral sensor facing a buccal face or labial face of a tooth of the upper or lower jaw, preferably at the location of a canine teeth and/or of at least one posterior teeth, preferably a pre-molar, and/or at least one intraoral sensor facing a palatinal face of a tooth of the upper or lower jaw, preferably of a canine teeth and/or a posterior teeth.

9. The system according to claim 1, wherein the sensor array comprises one central sensor patch located at the tongue resting position, preferably and two buccal sensor patches facing left and right buccal side surfaces of teeth of the upper or lower jaw, preferably at the location of canine teeth and/or posterior teeth, most preferably and two palatinal sensor patches facing left and right palatinal side surfaces of teeth of the upper or lower jaw, preferably of posterior teeth, in particular, wherein each of said sensor patches features at least two, preferably at least three, separate sensors for spatially and/or temporally resolving tongue pressure as exerted on the respective sensor patch.

10. The system according to claim 1, wherein said sensor array is applied as a flexible sensing film to a surface of said support sheet, for example by gluing or deep drawing or thermoforming, in particular wherein said sensing film has a thickness of less than 500 μm, preferably less than 100 μm, and most preferably less than 50 μm and/or is stretchable.

11. Use of an intraoral electronic tongue monitoring system, in particular according to claim 1, for measuring a pressure exerted by the tongue on single teeth of the upper or lower jaw, in particular on premolar and/or molar teeth, and/or for measuring a pressure distribution exerted by the tongue on a support sheet of the system and/or on a sensing film of the system and/or for measuring interlabial pressure and/or for detecting extraoral movements of the tongue, in particular for tongue training.

12. Use of an intraoral electronic tongue monitoring system, in particular according to claim 1, as a human-machine-interface, in particular for controlling a software application, for example a video game, or an actuator through tongue movement, respectively, preferably wherein a tongue pressure distribution is spatially and/or temporally resolved by the system and read out for controlling said software/said actuator, most preferably, wherein feedback perceptible to the tongue is provided to a user by at least one actuator of the system.

13. Use of an intraoral electronic tongue monitoring system, in particular according to claim 1, as a training device for training of tongue movement and position and/or tongue muscle tonus, in particular wherein said system comprises at least one actuator for providing feedback perceptible to the tongue, or for treatment of snoring or sleep apnea or speech impediments.

14. A process for fabricating an intraoral electronic tongue monitoring system, in particular according to claim 1, comprising a support sheet to be attached to a human upper or lower jaw in an oral cavity and a sensor array arranged on the support sheet, characterized in that sensors of the sensor array are, at least partly, preferably fully, fabricated on a separate film, preferably wherein this film shows a thickness of less than 500 μm, most preferably of less than 100 μm, to form a sensing film, and said sensing film is attached to said support sheet, preferably wherein said film as well as the sensors formed on the film are stretched during application of the sensing film to the support sheet.

15. The process according to claim 14, wherein said support sheet is fabricated by casting or 3d-printing a 3D-model of an upper or lower jaw of an individual patient and by subsequent 3d-forming, in particular deepdrawing or thermo-forming, of the support sheet using said 3D-model, preferably wherein said 3D-model includes a foundation for replicating a lead designed for bearing extraoral, in particular interlabial, sensors of the system, or by acquiring 3D-data of a topology of an upper or lower jaw of an individual patient, in particular using an intraoral optical 3D-scanner, and by subsequent 3D-printing of the support sheet as a negative of said topology, based on said 3D-data, such that the support sheet accurately matches said topology.

Description

[0117] With reference to the accompanying drawings, where features with corresponding technical function are referenced with same numerals even when these features differ in shape or design:

[0118] FIG. 1 shows an upper jaw of an individual patient,

[0119] FIG. 2 shows a 3D-model of the jaw of FIG. 1 with an additional foundation,

[0120] FIG. 3 illustrates a support sheet of a system according to the invention obtained by thermoforming using the 3D-model of FIG. 2,

[0121] FIG. 4 illustrates the attachment of a sensing film onto the support sheet of FIG. 3 in a perspective view,

[0122] FIG. 5 illustrates in a top view the locations of single sensor patches of the sensing film of FIG. 4 on the support sheet of FIG. 3,

[0123] FIG. 6 illustrates the attachment of a cover film onto the stack of support sheet and attached sensing film shown in FIG. 4,

[0124] FIG. 7 shows the resulting stack of support sheet and sensing film with attached cover film and

[0125] FIG. 8 shows the final intraoral electronic tongue monitoring system after a central recess has been cut out from the support sheet, sensing film and cover film to result in a C-shaped device.

[0126] FIG. 8 shows an intraoral electronic tongue monitoring system 1 according to the invention, comprising a support sheet 2 to be attached to the human upper jaw 36 visible in FIG. 1. The system 1 is intended to be worn by a user inside of the oral cavity 3 delimited by the teeth 19 of the jaw 36 (cf. FIG. 1). In other words, the lower side of the support sheet 2 visible in FIGS. 8 and 3 is customized individually to the topology 32 of the upper jaw 36 of the user to allow reversible and repeated attachment of the support sheet 2 to the teeth 36 of the upper jaw 36. When the system 1 shown in FIG. 8 is in place, i.e. attached to the upper jaw 36, the upper side visible in FIG. 8 is directed towards the lower jaw and the tongue.

[0127] As is visible in FIGS. 4 and 8, a sensor array 4 consisting of a multitude of sensors 17 is arranged on the support sheet for recording movements of the tongue as well as pressure exerted by the tongue onto the support sheet 2, when the system 1 is in place on the upper jaw 36. As indicated in FIG. 8, the system 1 offers both intraoral sensors 5 as well as extraoral sensors 6 for recording tongue movement and/or pressure, i.e. tongue movement can also be detected outside of the oral cavity 3 delimited by the teeth 19.

[0128] The construction of the system 1 is best understood by understanding the fabrication process first:

[0129] To start with the physiology, FIG. 1 illustrates the position of single teeth 19 on an upper jaw 36 of a user of the system 1. Visible are incisors 37, canine teeth 25, pre-molars 22 as well as molar teeth 38. The C-row of teeth 19 thus delimits the oral cavity 3, in which the tongue is located, with a natural tongue resting position 27 on the upper jaw 36.

[0130] Indicated in FIG. 1 is also that the teeth 19 have outer surfaces, namely buccal faces 23 (cf. the molar teeth 38 and the pre-molars 22) facing the cheeks, and labial faces 24 (cf. the incisors 37 and canines 25) facing the lips. On the inner sides of the teeth 19, palatinal faces 20 of the teeth 19 are oriented towards the palate and the tongue.

[0131] As a first step in the fabrication of the system 1, 3D-data of the topology 32 of the upper jaw 36 of the user shown in FIG. 1 are acquired using an intraoral optical 3D-scanner. Based on these data, a 3D-model 30 resembling closely the upper jaw 36 is obtained by 3D-printing. Alternatively, such a 3D-model 30 can also be obtained by a casting process from the upper jaw 36 shown in FIG. 1.

[0132] As is visible in FIG. 2, the 3D-model 30 includes a foundation 31 for replicating a lead 18 designed for bearing the extralabial sensors 43 of the system 1, as is visible in FIG. 8.

[0133] As a next step, a support sheet 2 out of a thermoplastic material is thermo-formed from the 3D-model 30 by deepdrawing. As a result, the support sheet 2 shown in FIG. 3 is obtained featuring, the desired lead 18.

[0134] As the support sheet 2 is very thin, its upper surface (with respect to FIG. 3), facing the tongue when the system 1 is in place on the upper jaw 36, offers a topology 32″, which closely resembles the topology 32′ of the 3D-model 30, which itself is an accurate copy of the physiologic topology 32 of the upper jaw 36 of the user.

[0135] Alternatively, the support sheet 2 may be directly obtained from the 3D-data of the upper jaw 36 recorded with the intraoral optical 3D-scanner by 3D-printing the support sheet 2 as a negative of the topology 32, based on said 3D-data. In this case, a similar accurate copy of the topology 32 can be obtained on the support sheet 2.

[0136] In another step, the multitude of sensors 17 are micro-fabricated as a sensor array 4 by batch processes, using standard lithography tools, on an elastic and stretchable thin film. As a result, an elastic and stretchable sensing film 16 is obtained comprising the thin film and the sensor array 4.

[0137] The sensing film 16 is next cut into the five-finger-shape visible in FIGS. 4 and 5, with a protruding portion matching the shape of the lead 18 of the support sheet 2.

[0138] The pre-cut sensing film 16 is next attached onto the thermo-formed support sheet 2 using a thin glue layer. The sensing film 16 is attached on the side of the support sheet 2 facing the tongue when the system 1 is in place. During this process, the sensing film 16 may be stretched to conformally adapt to the complex topology 32″ of the support sheet 2. The result is visible in FIG. 5, illustrating the relative position of the sensing film 16 on the support sheet 2. Importantly, the sensing film 16 extends onto an extraoral portion 14 of the support sheet 2, in particular onto the lead 18 (cf. FIGS. 5 and 6).

[0139] As an optional step, illustrated in FIGS. 6 and 7, a cover film 42 is next attached onto the stack 15 comprising the support sheet 2 with attached sensing film 16 for protection of the delicate sensors 17. The resulting structure consisting of a stack of support sheet 2, sensing film 16, and attached cover film 42 is illustrated in FIG. 7. The cover film 42 also covers the lead 18.

[0140] As a final step, a recess 34, indicated by dashed lines in FIG. 8, is cut out of the aforementioned stack. In conclusion, the final system 1 shown in FIG. 8, in particular its support sheet 2, shows a C-shape resembling the C-arc of the teeth 29 visible in FIG. 1. Due to the recess 34, a central and rear part of the palate is left free for natural tongue contact, when the system 1 is worn by a user on the upper jaw. In applications, where the system 1 is designed to be worn on the lower jaw by a user, the recess 34 may be designed such that enough free space is left for natural movement of the tongue.

[0141] Referring now to FIG. 5, displaying a top view on the stack 15 consisting of the support sheet 2 and attached sensing film 16 bearing the sensor array 4, the two dashed lines indicate the interlabial space which is delimited by the lips. The intraoral space (i.e. the oral cavity 3) is located above the upper of the two dashed lines, and the extra-oral space as well as the extraoral portion 14 of the system 1 is located below the upper of the two dashed lines, in particular below the lower of the two dashed lines.

[0142] The intraoral portion 13 of the system 1 shows a maximum material thickness with respect to the stack consisting of support sheet 2, sensing film 16 and cover film 42 of less than 1 mm and can thus be worn comfortably by a user over longer times. The support sheet 2 shows a thickness of less than 0.5 mm, and the sensing film 16 and cover film 42 of less than 0.1 mm, respectively.

[0143] As is visible in FIG. 5, the sensor array 4 comprises several intraoral sensors 5, which are positioned in the oral cavity 3 delimited by the teeth (cf. FIG. 1), when the system 1 is worn by the user. Moreover, the sensor array 4 comprises a multitude of extraoral sensors 6, namely [0144] several extraoral sensors 6 positioned on outer surfaces of the teeth 19 (cf. the buccal sensor patches 28), [0145] several interlabial sensors 7 positioned in the interlabial space delimited by upper a lower lib, and [0146] several extralabial sensors 43 positioned on the lead 18 and hence outside of the oral cavity 3 and outside of the cavum oris delimited by the lips.

[0147] In greater detail, sensor array 4 features a central sensor patch 26 positioned centrally on the palatum durum. With this patch, movement of the tongue to the tongue resting position (cf. FIG. 4) can be detected.

[0148] In addition, sensor array 4 offers two palatinal sensor patches 29, detecting movement of the tongue along inner/palatinal side surfaces of the teeth 19, in particular on palatinal faces of the canine teeth 25 and the first premolars 22.

[0149] Sensor array 4 features also two buccal sensor patches 28 facing left and right buccal side surfaces of the canine teeth 25 and first pre-molar teeth 22 (cf. FIG. 1 and FIG. 5) and thus reaching out of the oral cavity into the extraoral space. With these patches 28, movement of the tongue in the extraoral space can be detected, in particular along the outer sides of the teeth 19 of the upper jaw 36, i.e. in the vestibule of mouth (the space between the lips or cheeks and teeth).

[0150] Of course, other sensor positions within the intraoral or extraoral space can be realized. For example, one design according to the invention suggests to use sensors facing palatinal faces of anterior teeth and/or labial faces of anterior teeth.

[0151] In addition to the extraoral sensors 6 of the two 28 buccal sensor patches 28, the system 1 also offers three additional extraoral sensor patches 41, namely one patch featuring several interlabial sensors 7 positioned in the interlabial space 35, and two patches each featuring extralabial sensors 43, located outside of the cavum oris delimited by the lips. The latter three patches are all located on the lead 18, as is visible in FIGS. 4 and 5.

[0152] Each of the eight patches 26, 29, 28, 41 bears a minimum of three separate sensors 17 for spatially resolving tongue pressure on each of the patches and/or tongue movement over each patch.

[0153] Each of these sensors 17 is formed as a capacitive sensor, namely as a dielectric elastomer transducer (DET), and offers a very high sensitivity for spatially and temporally resolving tongue pressure exerted on the respective sensor patch 26, 29, 28, 41.

[0154] As indicated by the dashed line in FIG. 4, the system 1 also features an electromechanical interface 40 in the form of a socket, which can be electrically connected to a cable for measuring the sensor signal of the sensor array 4. The electromechanical interface 40 is connected to electrical connection lines 39 on the lead 18, which route sensor signals from the sensor array 4 to the electromechanical interface 40.

[0155] The central sensor patch 26 is equipped with an electromechanical actuator 10 for providing haptic feedback to the user in the form of vibrations, which are perceptible with the tongue. The actuator 10 is actually realized by one of the DET-sensors 17 of the central patch 26, as DETs can be used both as sensors and actuators.

[0156] As indicated in FIG. 4, the system 1 further comprises data acquisition electronics 8 for acquiring a sensor signal from the sensor array 4, control electronics 9 for generating control signals based on the acquired sensor signal, an electronic driving unit 11 for driving the actuator 10, and a communication module 12 for wireless communication with an external electronic device in the form of a tablet. These electronic components are all realized as miniaturized and low-weight ICs and attached to the lower side of the lead 18 (not visible in the figures) and thus placed outside of the oral cavity 3, when the system 1 is attached to the upper jaw 36.

[0157] The control signals are designed to control a human-machine-interface of the system 1. In other words, a user may generate said sensor signal by tongue movement and/or tongue pressure and thereby control signals for controlling a software application running on the tablet though the human-machine-interface. For this purpose, the system 1 spatially and temporally resolves tongue pressure distributions on the various sensor patches 26, 28, 29, 41 and provides a corresponding sensor signal. The pressure distributions are then read out by the system 1 for generating the control signals designed for controlling the software application on the tablet. These operations are performed by the control electronics 9, which feature also signal amplification circuitry, AD- and DA-converter, as well as logical circuits providing some intelligence to the system 1.

[0158] The tablet on the other hand can communicate wirelessly via the communication module 12 with the system 1 and send a feedback signal to the system 1. This feedback signal is then used to generate a driving signal using the electronic driving unit 11 to generate a haptic feedback to the user through the actuator 10.

[0159] By this approach, sophisticated user-machine-interaction—in both directions—can be accomplished, i.e. the user can control the software application on the table via his tongue and he can also sense feedback sent from the tablet to the system using his tongue.

[0160] Based on this approach, it is possible for example to train the muscle of the tongue by letting the user perform a video game on the tablet using the system 1 as an input device for generating control signals to be send wirelessly via the communication module from the system 1 to the tablet. For this purpose, it is sufficient if the tongue touches the sensor array 4 to generate a sensor signal.

[0161] As the sensor array 4 offers various sensors at different positions, in particular in the interlabial and extraoral space, various movements of the tongue can be differentiated. For example, it is possible to detect when the user is striking with this tongue along the outer surfaces of his teeth 19 and thus over the buccal sensor patches 28.

[0162] The extralabial sensors 43 on the lead 18 offer the possibility of training specific tongue movements, in which the user is touching the extraoral sensor patches 41 on the lead 18 with the tip of his tongue. Moreover, movements of the tongue in the extralabial space, i.e. outside of the cavum oris delimited by the lips, can be detected with the extralabial sensors 43.

[0163] In conclusion, the system 1 can be used as a training device for training of tongue movement and position and/or tongue muscle tonus. This functionality is highly attractive for treatment of snoring, sleep apnea or speech impediments.

[0164] Extraoral movements of the tongue in the vestibule of mouth, i.e. in the space between the lips or cheeks and teeth 19, can be detected with the buccal sensor patches 28 of the system 1.

[0165] Using the interlabial sensors 7, the pressure exerted by the lips can be resolved, in addition to the tongue pressure recorded by the other intraoral sensors 5 and extraoral sensors 6 of the sensor array 4. Such measurements are particularly interesting for logopedic tests.

[0166] As another use, the sensor array 4 can be used to measure the pressure that the tongue exerts on single teeth, for example on the pre-molars 22 using the buccal and palatinal sensor patches 28 and 29.

[0167] In summary, the invention aims at exploiting novel use-cases, in particular sophisticated human-machine interaction, with an intraoral electronic tongue monitoring system 1 designed to be worn by a user on the upper or lower jaw and featuring a support sheet 2 bearing a number of intraoral sensors 17 arranged in an array 4 for recording tongue movement and/or tongue pressure. To achieve this goal, it is proposed that the system 1 comprises at least one extraoral sensor 6 located outside of the oral cavity 3 delimited by the teeth 19 when the system 1 is in place, in particular such that extraoral and/or intraoral and/or interlabial movements of the tongue and/or lip pressure can be recorded with the system 1. Moreover, it is proposed to use the system 1 as an input device controlled through tongue movement using a human-machine-interface provided by the system 1.

LIST OF REFERENCE NUMERALS

[0168] 1 intraoral electronic tongue monitoring system [0169] 2 support sheet [0170] 3 oral cavity [0171] 4 sensor array [0172] 5 intraoral sensor [0173] 6 extraoral sensor [0174] 7 interlabial sensor [0175] 8 data acquisition electronics [0176] 9 control electronics [0177] 10 actuator [0178] 11 electronic driving unit [0179] 12 communication module [0180] 13 intraoral portion (of 1) [0181] 14 extraoral portion (of 1) [0182] 15 stack (of 2 and 16) [0183] 16 sensing film [0184] 17 sensor (of 4) [0185] 18 lead [0186] 19 tooth [0187] 20 palatinal face (of 19) [0188] 21 posterior tooth [0189] 22 pre-molar [0190] 23 buccal face [0191] 24 labial face [0192] 25 canine tooth [0193] 26 central sensor patch [0194] 27 tongue resting position [0195] 28 buccal sensor patch [0196] 29 palatinal sensor patch [0197] 30 3d-model (of upper/lower jaw) [0198] 31 foundation (for replication of 18) [0199] 32 topology (of upper/lower jaw) [0200] 33 C-shape [0201] 34 recess [0202] 35 interlabial space [0203] 36 jaw [0204] 37 incisor [0205] 38 molar tooth [0206] 39 electrical connection line [0207] 40 electromechanical interface [0208] 41 extraoral sensor patch [0209] 42 cover film [0210] 43 extralabial sensor