Imaging device and contact lens
11048148 · 2021-06-29
Assignee
Inventors
- Matthew John Lawrenson (Bussigny, CH)
- Jacob STRÖM (Stockholm, SE)
- Lars Andersson (Solna, SE)
- Till Burkert (Huddinge, SE)
Cpc classification
G03B13/20
PHYSICS
G02C7/049
PHYSICS
International classification
Abstract
An imaging device comprising a first camera with a controllable focus is provided. The imaging device is operative to detect that a user of the imaging device intends to capture an image of an object using the first camera, control the focus of the first camera to assume a target focal length, and capture the image. The target focal length is derived based on a measured accommodation of an eye lens of the user. Further, a contact lens comprising an eye-accommodation detector and a communications module is provided. The contact lens is operative to measure an accommodation of an eye lens to which the contact lens is attached, and transmit, to an imaging device comprising a first camera with a controllable focus, information pertaining to the measured accommodation of the eye lens and/or a focal length of the eye lens corresponding to the measured accommodation.
Claims
1. An imaging device comprising: a first camera with a controllable focus, the imaging device being operative to: detect that a user of the imaging device intends to capture an image of an object using the first camera, control the focus of the first camera to assume a target focal length which is derived based on a measured accommodation of an eye lens of the user, and capture the image.
2. The imaging device according to claim 1, being operative to detect that the user intends to capture an image of an object by any one, or a combination, of: receiving an instruction from the user, detecting that a timer has expired, detecting that the accommodation of the eye lens is substantially stable, and detecting that a gaze of the user is substantially stable.
3. The imaging device according to claim 1, further comprising a communications module, the imaging device being further operative to receive, via the communications module, information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and the target focal length of the first camera.
4. The imaging device according to claim 3, wherein the received information pertains to at least one of: the measured accommodation of the eye lens and a focal length of the eye lens corresponding to the measured accommodation, the imaging device being further operative to derive, from the received information, the target focal length of the first camera.
5. The imaging device according to claim 1, being further operative to: measure the accommodation of the eye lens, and derive the target focal length of the first camera based on the measured accommodation of the eye lens.
6. The imaging device according to claim 5, further comprising: a light source configured to emit structured light, and a second camera configured to capture an image a fundus of the eye, the imaging device being operative to measure the accommodation of the eye lens by: identifying, by image processing the captured image, a reflection of the structured light by the fundus, and comparing a structure of the identified reflection with a structure of the emitted structured light.
7. A contact lens comprising: an eye-accommodation detector, and a communications module, the contact lens being operative to: measure an accommodation of an eye lens to which the contact lens is attached, and transmit, to an imaging device comprising a first camera with a controllable focus, information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and a target focal length of the first camera.
8. The contact lens according to claim 7, the eye-accommodation detector comprising one or more pairs of electrodes arranged to measure an electrical impedance of a ciliary muscle of the eye, the contact lens being operative to measure the accommodation of the eye lens by measuring the electrical impedance of the ciliary muscle of the eye.
9. The contact lens according to claim 7, the eye-accommodation detector comprising one or more shape sensors, the contact lens being operative to measure the accommodation of the eye lens by measuring a change in curvature of the eye lens.
10. The contact lens according to claim 7, being further operative to: derive, from the measured accommodation of the eye lens, at least one of: the focal length of the eye lens and the target focal length of the first camera.
11. A method of an imaging device comprising a first camera with a controllable focus, the method comprising: detecting that a user of the imaging device intends to capture an image of an object using the first camera, controlling the focus of the first camera to assume a target focal length which is derived based on a measured accommodation of the eye lens of the user, and capturing the image.
12. The method according to claim 11, wherein the detecting that the user intends to capture an image of an object comprises any one, or a combination, of: receiving an instruction from the user, detecting that a timer has expired, detecting that the accommodation of the eye lens is substantially stable, and detecting that a gaze of the user is substantially stable.
13. The method according to claim 11, further comprising receiving information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and the target focal length of the first camera.
14. The method according to claim 13, wherein the received information pertains to at least one of: the measured accommodation of the eye lens and a focal length of the eye lens corresponding to the measured accommodation, the method further comprising deriving, from the received information, the target focal length of the first camera.
15. The method according to claim 11, further comprising: measuring the accommodation of the eye lens, and deriving the target focal length of the first camera based on the measured accommodation of the eye lens.
16. The method according to claim 15, wherein the measuring the accommodation of the eye lens comprises: emitting structured light from a light source comprised in the imaging device, capturing an image of a fundus of the eye, using a second camera comprised in the imaging device, identifying, by image processing the captured image, a reflection of the structured light by the fundus, and comparing of a structure of the identified reflection with a structure of the emitted structured light.
17. A method of a contact lens comprising an eye-accommodation detector and a communications module, the method comprising: measuring an accommodation of an eye lens to which the contact lens is attached, and transmitting, to an imaging device comprising a first camera with a controllable focus, information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and a target focal length of the first camera.
18. The method according to claim 17, wherein the measuring an accommodation of a lens of an eye to which the contact lens is attached comprises measuring an electrical impedance of a ciliary muscle of the eye, using two or more electrodes comprised in the eye-accommodation detector.
19. The method according to claim 17, wherein the measuring an accommodation of a lens of an eye to which the contact lens is attached comprises measuring a change in curvature of the eye lens, using one or more shape sensors comprised in the eye-accommodation detector.
20. The method according to claim 17, further comprising: deriving, from the measured accommodation of the eye lens, at least one of: the focal length of the eye lens and the target focal length of the first camera.
21. A non-transitory computer-readable storage medium comprising a computer program product including instructions to cause at least one processor to: detect that a user of an imaging device intends to capture an image of an object using a first camera, control focus of the first camera to assume a target focal length, which is derived based on a measured accommodation of an eye lens of a user, and capture the image.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10) All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.
DETAILED DESCRIPTION
(11) The invention will now be described more fully herein after with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
(12) In
(13) In
(14) In
(15) Further with reference to
(16) Advantageously, by adjusting the target focal length f.sub.c of first camera 101 to a focal length which is commensurate with the current focal length f.sub.e of the eye lens, user 110 of imaging device 100/200 may capture an image of object 120 by simply gazing at object 120 to set the focus of first camera 101 to be commensurate with the current focal length f.sub.e of the eye lens.
(17) There are different ways of detecting that user 110 intends to capture an image of object 120. For instance, imaging device 100/200 may be operative to receive an instruction from user 110. This may, e.g., be a spoken instruction uttered by user 110 and received by a microphone comprised in imaging device 100/200 (not illustrated in
(18) Alternatively, imaging device 100/200 may be operative to detect that a timer has expired. Self-timers are well known in the field of cameras and photography, and may be set to a value which allows user 110 to gaze at object 120 and let his/her eye lens accommodate before an image is captured.
(19) As a further alternative, imaging device 100/200 may be operative to detect that user 110 intends to capture an image of object 120 by detecting that the accommodation of the eye lens of eye 111 is substantially stable, i.e., that user 110 has stopped switching gaze. This may, e.g., be achieved by monitoring the measured accommodation of the eye lens over time and determining that the accommodation of the eye lens is substantially stable if a variation of the measured eye accommodation is below a certain threshold value during a fixed period of time, e.g., half a second or one second. The threshold value may, e.g., be set to a value between 1 and 10%. The threshold value may either be configured by user 110, by a manufacturer of imaging device 100/200, or by a provider of a photo app, or the like, which is executed by processing means 103 comprised in imaging device 100/200.
(20) As yet a further alternative, imaging device 100/200 may be operative to detect that user 110 intends to capture an image of object 120 by detecting that a gaze of user 110 is substantially stable, i.e., that a variation of direction of gaze 131 over time is below a certain threshold value during a fixed period of time, e.g., half a second or one second. The threshold value may, e.g., be set to a value between 1 and 10%. The threshold value may either be configured by user 110, by a manufacturer of imaging device 100/200, or by a provider of a photo app, or the like, which is executed by processing means 103 comprised in imaging device 100/200. The direction of gaze of user 100 may, e.g., be monitored by utilizing front-facing camera 102 of imaging device 100, or the second camera comprised in imaging device 200. More specifically, by analyzing a sequence of images capturing the eye or eyes of user 110, direction of gaze 131 may be determined by tracking the corneal reflection (the first Purkinje image) and the center of the pupil over time.
(21) It will also be appreciated that imaging device 100/200 may be operative to detect that user 110 intends to capture an image of object 120 based on any combination of the above alternatives. For instance, imaging device 100/200 may be operative to start a self-timer with a duration of, e.g., one second, in response to receiving a user instruction, and to capture an image when the self-timer has expired. In this way, user 110 can initiate capturing of an image by, e.g., pressing a button, and then gazing at object 120 before the accommodation of the eye lens of eye 111 is measured and an image is captured.
(22) Imaging device 100/200 may by operative to control the focus of first camera 101 to assume the target focal length f.sub.c based on an accommodation of the eye lens which is measured by a separate device, e.g., an embodiment 300 of the contact lens according to the second aspect of the invention, which is described in further detail below, with reference to
(23) More specifically, imaging device 100/200 may comprise a communications module 104 which is operative to effect wireless communications through a Wireless Local Arena Network (WLAN)/Wi-Fi network, Bluetooth, ZigBee, or any other short-range communications technology. Alternatively, or additionally, communications module 104 may further be operative to effect wireless communications with a Radio Access Network (RAN) based on a cellular telecommunications technique such as the Global System for Mobile communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), or any 5G standard, e.g., Next Generation (NG) and New Radio (NR). As yet a further alternative, communications module 104 may be operative to effect wireless communications using light, e.g., Visible Coded Light (VLC), or IR light.
(24) Imaging device 100/200 is further operative to receive, via communications module 104, information pertaining to the measured accommodation of the eye lens, the focal length f.sub.e of the eye lens which corresponds to the measured accommodation, or the target focal length f.sub.c of first camera 101. The information may, e.g., be comprised in a message, e.g., as an information element contained in a Constrained Application Protocol (CoAP) or HyperText Transfer Protocol (HTTP) message, or encoded in a signal using amplitude, frequency, or phase modulation. The information may be received from any other device which is capable of measuring the eye accommodation of the eye lens, in particular from contact lens 300 which is attached to eye 111 (see
(25) The information pertaining to the measured accommodation of the eye lens may, e.g., comprise a value representing the measured accommodation of the eye lens, which is typically expressed in units of optical power, i.e., Diopter or m.sup.−1. The measured eye accommodation may either be expressed in absolute values or as difference between the measured eye accommodation and the far point (which is characterized by no eye accommodation, i.e., a relaxed ciliary muscles) or the near point (which is characterized by maximum accommodation), respectively. Alternatively, information pertaining to the focal length f.sub.e of the eye lens may be received, e.g., in units of lengths, i.e., m. It will also be appreciated that the information pertaining to the measured accommodation of the eye lens may alternatively comprise any measure derived from the measured accommodation of the eye lens or the corresponding focal length f.sub.e. For instance, the received information may comprise the target focal length f.sub.c of first camera 101, thereby accounting for a difference in focal lengths of the eye lens, f.sub.e, and that of first camera 101, f.sub.c, when focusing on the same object 120. The difference in focal length may be approximated by an estimated distance between eye 111 and first camera 101.
(26) Further optionally, imaging device 100/200 may be operative to derive the target focal length f.sub.c of first camera 101 from the received information, in particular from information pertaining to the measured accommodation of the eye lens or the focal length f.sub.e of the eye lens corresponding to the measured accommodation. For instance, this may be achieved by utilizing a look-up table maintained by imaging device 100/200 which stores values for the target focal length f.sub.c and corresponding values of the measured accommodation of the eye lens. The look-up table may, e.g., be populated by using a calibration procedure, as is described further below. Alternatively, a mathematical function may be utilized, based on which the target focal length f.sub.c may be calculated for a value of the measured accommodation of the eye lens. For instance, the accommodation of the human eye may be modelled by as a thick-lens problem. If a calibration procedure is employed, a mathematical function which best represents the observed relation between the focal length of the eye lens, f.sub.e, and the eye accommodation may be utilized and fitted to the measured values. Subsequently, the mathematical function may be used, with its fitted parameters, to calculate the focal length of the eye lens, f.sub.e, for a measured value of the accommodation of the eye lens. If the focal length f.sub.e of the eye lens is received, the target focal length f.sub.c of first camera 101 may be calculated by adding a value accounting for a difference in focal lengths of the eye lens, f.sub.e, and that of first camera 101, f.sub.c, when focusing on the same object 120. The difference in focal length may be approximated by an estimated distance between eye 111 and first camera 101.
(27) In the following, embodiments of the contact lens 300 in accordance with the second aspect of the invention are described with reference to
(28) Contact lenses 310 and 320 (in the following referred to as contact lens 310/320) comprise an eye-accommodation detector 313 and 323, respectively, and a communications module 314, and are operative to measure an accommodation of an eye lens to which contact lens 310/320 is attached, and to transmit information pertaining to the measured accommodation of the eye lens, a focal length f.sub.e of the eye lens corresponding to the measured accommodation, or a target focal length f.sub.c of a first camera. The information is transmitted to an imaging device comprising a first camera with a controllable focus, such as imaging device 100/200 described hereinbefore. The transmitted information may, e.g., be comprised in a message, e.g., as an information element contained in a CoAP or HTTP message, or encoded in a signal using amplitude, frequency, or phase modulation.
(29) A first embodiment 310 of contact lens 300 may be based on shape sensors. More specifically, eye-accommodation detector 313 comprises one or more shape sensors 311 and 312 which are arranged to measure a change in curvature of the eye lens to which contact lens 310 is attached. For instance, as is illustrated in
(30) Further with reference to
(31) Eye-accommodation modules 313 and 323, and communications module 314, as well as any additional modules, may be implemented by any kind of electronic circuitry, e.g., any one, or a combination of, analogue electronic circuitry, digital electronic circuitry, polymer electronics, and processing means executing a suitable computer program, i.e., software. It will also be appreciated that the electrical power which is required for operating an embodiment of contact lens 310/320, or charging a battery comprised in contact lens 310/320, may, e.g., be harvested from light (see, e.g., U.S. Pat. No. 9,158,133 B1) or by means of wireless inductive charging. As an alternative to receiving information pertaining to a measured accommodation of the eye lens of user 110 from a separate device, such as contact lens 300, imaging device 100/200 may alternatively be operative to measure the accommodation of the eye lens, and to derive the target focal length f.sub.c of first camera 101 based on the measured accommodation of the eye lens. This may, e.g., be achieved by using structured light which is reflected from the fundus of eye 111 (“Dynamic Measurement of Accommodation and Pupil Size Using the Portable Grand Seiko FR-5000 Autorefractor”, by J. S. Wolffsohn, K. Ukai, and B. Gilmartin, Optometry and Vision Science, vol. 83, pages 306-310, American Academy of Optometry, 2006), as is described in the following with reference to
(32) An embodiment of the imaging device relying on structured light is described with reference to imaging device 200 illustrated in
(33) With reference to
(34) Further with reference to
(35) To this end, an embodiment 200 of the imaging device comprising light source 401 and second camera 402 is further operative to identify reflection 411 and 412 of the structured light by the fundus, by image processing an image captured by second camera 402, and to compare a structure of identified reflections 411 and 412 with a structure of the emitted structured light. As is illustrated in
(36) Embodiments of the invention may rely on a calibration procedure for establishing a relation between a measured eye accommodation of an eye lens of user 110, which may, e.g., be measured by utilizing shape sensors, impedance cyclography, or structured light, and a current focal length f.sub.e of the eye lens corresponding to the measured eye accommodation. For instance, an embodiment of imaging device 100/200 may be operative, during a calibration phase, to measure the eye accommodation, or to receive information pertaining to the measured accommodation of the eye lens and/or a focal length of the eye lens corresponding to the measured accommodation, when user 110 is gazing at object 120 at a known distance (which corresponds to the focal lens of the accommodated eye lens, f.sub.e). Imaging device 100/200 is further operative to store information pertaining to the eye accommodation of user 110 and the known distance. Preferably, the user's eye accommodation is measured, or information is received, for a number of different distances, i.e., eye accommodations and corresponding focal lengths f.sub.e. This may, e.g., be achieved by instructing the user to gaze at different objects which are located at different, known distances from the user. The embodiment of the imaging device is further operative to store the measured eye accommodations and the corresponding known distances for later use. The measured eye accommodations and the corresponding known distances may, e.g., be stored associatively, in a list or database. Alternatively, an equation describing the relation between accommodation of the eye lens and its focal length may be fitted to the pairs of values, each pair comprising measured eye accommodation and known distance, so as to derive a set of parameters which are used to define the equation.
(37) Subsequently, during normal use, the focal length f.sub.e of the eye lens may be derived based on the measured eye accommodation, either by looking up the focal length in a list or database, or by calculating the focal length using the fitted equation. The obtained focal length f.sub.e of the eye lens may be used directly as target focal length f.sub.c for first camera 101, thereby neglecting the difference in distance between f.sub.e and f.sub.c. Alternatively, the obtained focal length f.sub.e of the eye lens may be corrected to account for the difference, e.g., by adding a value accounting for a difference in focal lengths of the eye lens, f.sub.e, and that of first camera 101, f.sub.c, when focusing on the same object 120. The difference in focal length may be approximated by an estimated distance between eye 111 and first camera 101.
(38) During the calibration procedure, the measured eye accommodation may e.g., be stored, or used for fitting, in units of Diopters. Alternatively, rather than using the measured eye accommodation, embodiments of the imaging device may utilize the measured curvature of the eye lens, the measured electrical impedance of the ciliary muscle, or measurements of structured light reflected off the fundus, e.g., one or both of d.sub.v and d.sub.h.
(39) In the following, embodiments of processing means 103 comprised in embodiments of the imaging device, such as imaging devices 100 and 200, are described with reference to
(40) A first embodiment 500 of processing means 103 is shown in
(41) An alternative embodiment 600 of processing means 103 is illustrated in
(42) In particular, user-interface module 602 is configured to detect that a user of imaging device 100/200 intends to capture an image of an object using first camera 101. Camera module 604 is configured to control the focus of first camera 101 to assume a target focal length which is derived based on a measured accommodation of an eye lens of the user and capture the image. The target focal length is acquired from eye-accommodation module 603.
(43) For instance, user-interface module 602 may be configured to detect that the user intends to capture an image of an object by any one, or a combination, of: receiving an instruction from the user, detecting that a timer has expired, detecting that the accommodation of the eye lens is substantially stable, and detecting that a gaze of the user is substantially stable.
(44) Eye-accommodation module 603 may be configured to receive information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and the target focal length of first camera 101. The information is received via communications module 104. Optionally, if the received information pertains to the measured accommodation of the eye lens and/or a focal length of the eye lens corresponding to the measured accommodation, eye-accommodation module 603 may be configured to derive, from the received information, the target focal length of first camera 101.
(45) Alternatively, eye-accommodation module 603 may further be configured to measure the accommodation of the eye lens, and to derive the target focal length of first camera 101 based on the measured accommodation of the eye lens. For instance, if imaging device 100/200 comprises a light source 401 configured to emit structured light and a second camera 402 configured to capture an image a fundus of the eye, eye-accommodation module 603 may be configured to identify a reflection of the structured light by the fundus, by image processing the captured image, and to compare a structure of the identified reflection with a structure of the emitted structured light.
(46) Interfaces 501 and 601, and modules 602-604, as well as any additional modules comprised in processing means 600, may be implemented by any kind of electronic circuitry, e.g., any one, or a combination of, analogue electronic circuitry, digital electronic circuitry, and processing means executing a suitable computer program, i.e., software
(47) In the following, embodiments 700 of the method of an imaging device are described with reference to
(48) Method 700 comprises detecting 701 that a user of the imaging device intends to capture an image of an object using the first camera, controlling 705 the focus of the first camera to assume a target focal length which is derived based on a measured accommodation of the eye lens of the user, and capturing 706 the image.
(49) For instance, detecting 701 that the user intends to capture an image of an object may comprise any one, or a combination, of: receiving an instruction from the user, detecting that a timer has expired, detecting that the accommodation of the eye lens is substantially stable, and detecting that a gaze of the user is substantially stable.
(50) Method 700 may further comprise receiving 702 from an eye-accommodation detector information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and the target focal length of first camera 101. Optionally, if the received information pertains to the measured accommodation of the eye lens and/or a focal length of the eye lens corresponding to the measured accommodation, method 700 may further comprise deriving 704, from the received information, the target focal length of the first camera.
(51) Alternatively, method 700 may further comprise measuring 703 the accommodation of the eye lens, and deriving 704 the target focal length of the first camera based on the measured accommodation of the eye lens. For instance, measuring 703 the accommodation of the eye lens may comprise emitting structured light from a light source comprised in the imaging device, capturing an image of a fundus of the eye, using a second camera comprised in the imaging device, identifying a reflection of the structured light by the fundus, by image processing the captured image, and comparing a structure of the identified reflection with a structure of the emitted structured light.
(52) It will be appreciated that method 700 may comprise additional, or modified, steps in accordance with what is described throughout this disclosure. An embodiment of method 700 may be implemented as software, such as computer program 504, to be executed by a processing unit comprised in the imaging device, whereby the imaging device becomes operative to perform in accordance with embodiments of the invention described herein.
(53) In the following, embodiments 800 of the method of a contact lens are described with reference to
(54) Method 800 comprises measuring 801 an accommodation of an eye lens to which the contact lens is attached, and transmitting 803, to an imaging device comprising a first camera with a controllable focus, information pertaining to at least one of: the measured accommodation of the eye lens, a focal length of the eye lens corresponding to the measured accommodation, and a target focal length of the first camera.
(55) For instance, measuring 801 an accommodation of a lens of an eye to which the contact lens is attached may comprise measuring an electrical impedance of a ciliary muscle of the eye, using two or more electrodes comprised in the eye-accommodation detector. Alternatively, measuring 801 an accommodation of a lens of an eye to which the contact lens is attached may comprise measuring a change in curvature of the eye lens, using one or more shape sensors comprised in the eye-accommodation detector.
(56) Optionally, method 800 may further comprise deriving 802, from the measured accommodation of the eye lens, at least one of the focal length of the eye lens and the target focal length of the first camera.
(57) It will be appreciated that method 800 may comprise additional, or modified, steps in accordance with what is described throughout this disclosure. An embodiment of method 800 may be implemented as software, such as a computer program, to be executed by a processing unit comprised in the contact lens, whereby the contact lens becomes operative to perform in accordance with embodiments of the invention described herein.
(58) The person skilled in the art realizes that the invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.