DEVICE FOR PRODUCING AN IMAGE OF DEPTH-DEPENDENT MORPHOLOGICAL STRUCTURES OF SKIN LESIONS

Abstract

The present invention describes a device (10) for producing a plurality of images for assisting in characterizing skin lesions, the device comprising a capturing unit (1), in particular a video dermatoscope, for selectively illuminating and capturing recordings of an area of skin (11) presenting a skin lesion (12), the capturing unit (1) comprising magnifying optics (2) providing at least 160×, preferably at least 240×, optical magnification of the area of skin (11) for capturing morphological structures of the skin lesion (12); a focusing unit (3) associated with the magnifying optics (2) and configured to place a focal plane (F) of the magnifying optics (2) in a plurality of different depth planes (t1, . . . , tn) below a skin surface of the area of skin (11) starting from the skin surface (t0), preferably in steps; a control unit (4) configured to control the focusing unit (3) and/or the capturing unit (1) and preferably controlling them in such a manner that the capturing unit (1) captures a recording when the focal plane (F) is located in a respective depth plane (t1, . . . , tn); and a processing and output unit (5) for image production based on the image information provided by the capturing unit (1).

Claims

1. A device (10) for producing a plurality of images for assisting in characterizing skin lesions, the device comprising: a capturing unit (1) for selectively illuminating and capturing recordings of an area of skin (11) presenting a skin lesion (12), the capturing unit (1) comprising magnifying optics (2) providing at least 160× optical magnification of the area of skin (11) for capturing morphological structures of the skin lesion (12); a focusing unit (3) associated with the magnifying optics (2) and configured to place a focal plane (F) of the magnifying optics (2) in a plurality of different depth planes (t1, . . . , tn) below a skin surface (t0) of the area of skin (11) starting from the skin surface; a control unit (4) configured to control the focusing unit (3) or the capturing unit (1) and in such a manner that the capturing unit (1) captures a recording when the focal plane (F) is located in a respective depth plane (t1, . . . , tn); and a processing and output unit (5) for image production based on the image information provided by the capturing unit (1).

2. The device according to claim 1, wherein the magnifying optics (2) allows 250× to 400× optical magnification of the area of skin (11).

3. The device according to claim 1, wherein the focusing unit (3) is configured to automatically or manually place the focal plane (F) in a depth plane (t1, . . . , tn) of 1 to 50 μm below the skin surface (t0).

4. The device according to claim 1, wherein the focusing unit (3) is configured to automatically or manually vary the focal plane (F) in steps of 1 to 7 μm.

5. The device according to claim 1, wherein the control unit (4) is configured to control the focusing unit (3) or the capturing unit (1) in such a manner that a plurality of predefined depth planes (t1, . . . , tn) are focused one after the other in one sequence and at least one recording is captured in each depth plane.

6. The device according to claim 1, wherein the magnifying optics (2) and the associated focusing unit (3) are integrated in an exchangeable front-end module (1b) for a camera module (1a) of a capturing unit (1).

7. The device according to claim 1, wherein the capturing unit (1) comprises illuminating means (6), preferably a plurality of LED light sources, for reflected-light illumination (6a) or direct coupling in of light (6b) into the area of skin (2), the illuminating means (6) being integrated in or connectable to the capturing unit (1).

8. The device according to claim 1, wherein the device (10) comprises a positioning unit (7) configured to selectively project a marking onto the area of skin.

9. The device according to claim 1, wherein the processing and output unit (5) is configured to display a respective detail image (13) of morphological structures of the skin lesion (12) in a respective depth plane (t1, . . . , tn) as a live video image or as an individual image.

10. The device according to claim 1, wherein the processing and output unit (5) is configured to display a respective detail image (13) of morphological structures of the skin lesion (12) in a respective depth plane (t1, . . . , tn) as a live video image or as an individual image in combination with a captured reference image (14) of the skin lesion (12).

11. The device according to claim 10, wherein the processing and output unit (5) is configured to display an indicator (14a) for the respective current position of the displayed detail image (13) of the skin lesion (12) in the reference image (14).

12. The device according to claim 9, wherein the processing and output unit (5) is configured to display a clinical overview image (16) of a patient (P) in combination with the respective detail image (13) of the skin lesion (12) or a captured reference image (14) of the skin lesion (12), the clinical overview image (16) containing an indicator (16a) for the respective position of the displayed skin lesion (12) in the clinical overview image (16).

13. The device according to claim 1, wherein the device comprises a processor and memory unit (8) configured to pre-characterize or assess the captured detail image (13) of morphological structures of the area of skin, and to output an associated information based thereon by means of the processing and output unit (5).

14. A method for producing a plurality of images for assisting in characterizing skin lesions, using a device (10) according to claim 1, the method comprising the following steps: illuminating an area of skin (11) to be examined by means of illuminating means (6); selectively focusing an optical capturing unit (1) with associated magnifying optics (2) on a plurality of different depth planes (t1, . . . , tn) below a skin surface (H) stating from the skin surface (H), the magnifying optics (2) allowing at least 160× optical magnification of the area of skin (11) for capturing morphological structures of a skin lesion (12) in the area of skin (11); capturing at least one recording of the skin lesion (12) in a respective depth plane (t1, . . . , tn) by means of a capturing unit (1); producing at least one detail image (13) of morphological structures of the skin lesion (12) based on the image information provided by the capturing unit (1).

15. The method according to claim 14, wherein the focusing into the respective depth planes takes place automatically and in steps at predefined intervals.

16. The method according to claim 14, wherein predefined depth planes (t1, . . . , tn) are focused one after the other in one sequence and at least one recording is captured and stored after the respective focusing.

17. The method according to claim 14, wherein a respective detail image (13) is produced or output as a live image in real time by means of a processing and output unit (5).

18. The method according to claim 14, the method comprising the following further steps: capturing a skin lesion (12) to be examined; and displaying a reference image (14) of the skin lesion (12) in combination with an associated detail image (13) of the skin lesion (12) on a display (5a) of a processing and output unit (5).

19. The method according to claim 18, the method comprising the further step of displaying an indicator (14a) for marking a current position of the displayed detail image (13) in the skin lesion (12).

Description

[0043] Other advantages, features and details of the invention are apparent from the following description of preferred examples of configurations and from the drawings.

[0044] FIG. 1 is a schematic illustration of a preferred embodiment of the device according to the invention;

[0045] FIG. 2 is a schematic illustration of the magnifying optics and the associated focusing unit;

[0046] FIG. 3 is a schematic side view of a preferred embodiment of the capturing unit as a camera module with an exchangeable front-end module;

[0047] FIG. 4 is a preferred depiction of a situation in which the image of a skin lesion is output according to the invention with associated information.

[0048] FIG. 1 shows a preferred embodiment of device 10 according to the invention. Device 10 comprises a capturing unit 1, which is configured to capture a recording or an image, in particular an optically highly magnified detail recording of morphological structures 13a of a skin lesion 12 in an area of skin 11 of a patient P (see FIG. 4). Capturing unit 1 preferably provides digital image information or a signal representing the latter based on the captured recording. Capturing unit 1 preferably comprises a known video dermatoscope. Capturing unit 1 can preferably be selectively operated in a video mode or an individual image mode. Capturing unit 1 has a magnification option 2 and a focusing unit 3 associated with the latter and is preferably configured as a mobile handheld device.

[0049] Device 10 can optionally comprise an additional capturing unit 1′. The latter can preferably comprise a high-resolution digital image or video camera and is configured to capture a reference image 14 of skin lesion 12 and/or a clinical overview image 16 of patient P.

[0050] Device 10 comprises a processing and output unit 5, which is connected to capturing unit 1, 1′. Processing and output unit 5 is configured to electronically process the provided image information or image data and to output corresponding images. For this purpose, processing and output unit 5 comprises at least one output unit in the form of a display 5a. The output unit can additionally be configured to provide acoustic signals or warning messages.

[0051] A control unit 4 of device 10 for controlling focusing unit 3 and/or capturing unit 1 can be integrated in processing and output unit 5 or be provided separately. Moreover, processing and output unit 5 comprises a processor and memory unit 8. The latter is preferably configured to store and execute a software program for controlling the device. Processing and output unit 5 preferably has a user interface 9a for controlling device 10 and/or a software executed thereon. User interface 9a can comprise known input means, such as a keyboard, a mouse, and/or a touchscreen.

[0052] In a preferred embodiment, processor and memory unit 8 of processing and output unit 5 comprises an artificial neural network, which is preferably stored thereon and configured to identify and/or classify morphological structures of skin lesions. The artificial neural network can access data stored in memory unit 8 and/or can access an external server or an external memory unit 30. The latter can be connected to processor and memory unit 8 via a communication interface 9b of processing and output unit 5. Communication interface 9b can additionally be configured to connect capturing unit 1, 1′ to processing and output unit 5. Communication interface 9b can enable wireless and/or cable communication with capturing unit 1, 1′ and/or the external server or an external memory unit 30.

[0053] Moreover, device 10 can comprise a positioning unit 7, which is configured to selectively project a marking, in particular a dot or cross-line laser, onto area of skin 11.

[0054] FIG. 2 shows a schematic illustration of magnifying optics 2 and associated focusing unit 3. Magnifying optics 2 is configured as a front-end module 1b for a camera module 1a of capturing unit 1 and can preferably be selectively connected to camera module 1a. Magnifying optics 2 is configured to provide at least 160×, more preferably at least 240×, optical magnification of area of skin 11 or of skin lesion 12 so that morphological structures 13a (see FIG. 4) can be captured in different depth planes of skin lesion 12. The magnifying optics preferably comprises at least two lenses 2a, 2b, which are adjustable relative to each other, and/or a lens module adjustable relative to camera module 1a.

[0055] Focusing unit 3 associated with magnifying optics 2 is preferably configured to vary a distance between lenses 2a, 2b of optics 2 and/or a distance between a lens module and camera module 1a of capturing unit 1. To this end, focusing unit 3 preferably comprises adjusting means in the form of an electric motor and/or a micro-actuator. The adjusting means can be controlled manually by an operator and/or automatically, in particular by means of associated control unit 4 and/or associated processing and output unit 5. As shown in FIG. 2, focusing unit 3 is configured not only to focus skin surface H of a skin lesion 12 to be examined, skin surface H corresponding to a zero depth plane t.sub.0, but also to selectively focus a plurality of different depth planes t.sub.1, . . . , t.sub.n below skin surface H.

[0056] Focusing unit 3 is preferably configured to preferably automatically focus skin surface H as a first step in the course of an initialization once capturing unit 1 has been positioned on skin lesion 12. Focal plane F of magnifying optics 2 is placed in corresponding zero depth plane t.sub.0 (see illustration on the left in FIG. 2). Subsequently, focal plane F can be adjusted to depth planes t.sub.1, . . . , t.sub.n located below skin surface H manually by a user and/or automatically by control unit 4 and/or by processing and output unit 5. In the illustration on the right in FIG. 2, for example, focal plane F is placed in a first depth plane t.sub.1 below skin surface H, i.e., said first depth plane t.sub.1 is focused.

[0057] Predefined depth planes t.sub.1, . . . , t.sub.n can be selectively focused individually or one after the other in steps in a predefined sequence, wherein respective recordings or images can be captured by capturing unit 1 in each of depth planes t.sub.1, . . . , tn.

[0058] Focusing unit 3 is preferably configured to selectively place focal plane F in a depth plane of 1 μm to 50 μm, more preferably 5 μm to 50 μm, below the skin surface. Respective focal plane F can be automatically or manually varied in steps of 1 μm to 7 μm, more preferably in 2.5 μm to 5 μm.

[0059] FIG. 3 shows a schematic side view of a preferred embodiment of capturing unit 1, which comprises camera module 1a with an exchangeable front-end module 1b. Capturing unit 1 preferably comprises illuminating means 6, preferably a plurality LED light sources or LED chips in particular emitting white light, which are integrated in or connectable to capturing unit 1. Illuminating means 6 preferably comprise illuminating means 6b disposed at the end of, i.e., distally on, capturing unit 1 for coupling light directly into the area of skin 11 to be examined. They are preferably in direct contact with skin surface H during the examination of area of skin 11. In order to reduce the refraction of light at the skin surface, an immersion gel can be applied, on which capturing unit 1 is then placed. Optionally, capturing unit 1 additionally comprises reflected-light illumination means 6a, which are disposed at a predefined distance to skin surface H. Furthermore, camera module 1a comprises a sensor system 1c for capturing the recordings or image information during the examination of illuminated lesion 12. Sensor system 1c preferably comprises at least one RGB sensor. Optionally, the sensor system can comprise other sensors for capturing image information during the examination of skin lesion 12 in the different depth planes t.sub.1, . . . , t.sub.n, in particular an IR sensor, a multi-spectral sensor or a hyperspectral sensor.

[0060] FIG. 4 shows a preferred illustration of a situation in which the image of a skin lesion 12 is output according to the invention together with associated information by means of a display 5a of processing and output unit 5. The latter constitutes a graphical interface 20 on which at least one detail image 13 of morphological structures 13a, in particular cellular structures and/or capillaries, in a predefined depth plane of skin lesion 12 are displayed for analysis by a treating physician. Detail image 13 can be displayed as a live video image or as an individual image.

[0061] Graphical interface 20 is preferably provided by a control software of processing and output unit 5. Graphical interface 20 preferably comprises patient information 18, such as a name and an age of the person to be examined. Moreover, graphical interface 20 comprises control elements 17, in particular elements for display or recording control 17a and for selectively controlling a magnification by magnifying optics 17b.

[0062] In addition to detail image 13, graphical interface 20 preferably comprises a field 21 for displaying a pre-characterization and/or an assessment of currently captured lesion 12. In particular, a risk class, a preferably numerical risk value, and/or an identified skin lesions class can be displayed. They can be calculated or determined based on the evaluation of the captured recordings by means of artificial intelligence as described above.

[0063] Graphical interface 20 preferably comprises a control element 15, which is associated with detail image 13 and serves to display and/or adapt respective depth plane t.sub.1, . . . , t.sub.n. In the latter case, a user can change the focused depth plane of capturing unit 1 selectively and preferably in predefined steps by shifting respective control element 15. Control element 15 has a scale with predefined steps for changing the focus in the unit “μm”, preferably from +20 μm to −50 μm. Moreover, control element 15 can comprise an adjusting function for automatic (“auto”) or manual (“fixed”) focusing.

[0064] Furthermore, graphical interface 20 comprises a reference image 14 of skin lesion 12, which has a significantly lower optical magnification and in which skin lesion 12 is depicted completely. Reference image 14 preferably contains an indicator 14a, which shows the current position of detail image 13 in depicted skin lesion 12. A change in position of capturing unit 1 on skin lesion 12 is preferably displayed in reference image 14 in real time.

[0065] Moreover, graphical interface 20 can comprise another display of a reference image 14′. The latter can correspond to reference image 14 mentioned above but be displayed in such a manner that it can be changed by the user in terms of color rendition and/or size, for example. For this purpose, other reference image 14′ can comprise associated image information 19, such as a magnification value.

[0066] Additionally, graphical interface 20 preferably comprises a clinical overview image 16 of patient P. The overview image preferably contains an indicator 16a, which shows the respective position of displayed skin lesion 12.

REFERENCE SIGNS

[0067] 1 capturing unit [0068] 1a camera module [0069] 1b front-end module [0070] 1c camera sensor system [0071] 2 magnifying optics [0072] 3 focusing unit [0073] 4 control unit [0074] processing and output unit [0075] 5a display [0076] 6 illuminating means [0077] 6a reflected-light illumination means [0078] 6b means for directly coupling in light [0079] 7 positioning unit [0080] 8 processor and memory unit (computing means) [0081] 9a user interface [0082] 9b communication interface [0083] 10 device [0084] 11 area of skin [0085] 12 lesion [0086] 13 detail image of morphological structures [0087] 13a morphological structures (cellular structures, capillaries) [0088] 14 reference overview image (overview image) [0089] 14a detail image position determination indicator [0090] 15 depth plane control element [0091] 16 clinical overview image [0092] 16a lesion position determination indicator [0093] 17 navigation and/or control elements [0094] 17a display/recording controller [0095] 17b magnification controller [0096] 18 patient information [0097] 19 image information [0098] 20 graphical interface [0099] 21 output of risk value and/or identified skin lesion class [0100] 30 external server [0101] t.sub.1, . . . , t.sub.n depth planes [0102] t.sub.0 zero depth plane (skin surface) [0103] F focal plane [0104] H skin surface [0105] P patient