EVALUATION METHOD FOR HAIR TRANSPLANT PROCESS USING IMAGE PROCESSING AND ROBOTIC TECHNOLOGIES AND SYSTEM THEREOF

Abstract

A method and system based on the image processing technology and robotic technologies are used for planning and evaluating the processes of the hair transplant operations. This method allows a rapid detection of the follicle, the hair number in each follicle and the hair thickness on the images obtained by the scanning of the scalp of the patient before the operation through the image processing technology. Considering these data, Coverage Value calculation is automatically carried out; maximum hair number to be harvested in each area is automatically analyzed and calculated. As a result, more reliable and healthy results relative to the present operations are obtained for evaluating the whole operational process through the systematic data, and a system facilitating the method is applied.

Claims

1. A system for planning and evaluating a process of a hair transplant operation, wherein the system is based on an image processing technology and a robotic technology, and the system comprises an operational unit, a user interface-screen element facilitating a control of the operational unit, at least one camera detecting an image in the system, at least one lighting element capable of emitting light with stable or different wavelengths, and a robot controller providing mobility for a robotic system.

2. The system according to claim 1, further comprising telecentric, bi-telecentric or similar lenses in accordance with a function of the camera systems.

3. The system according to claim 1, wherein the operational unit comprises one of a computer element capable of functioning separately or in an embedded position, stationary-portable computers or embedded cards carrying MCU (Micro Controller), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or GPU (Graphic Processing Unit).

4. A method for planning and evaluating a process of a hair transplant operation, wherein the method is based on an image processing technology and a robotic technology and the method comprises the following stages: an initial stage; an image-taking stage before the hair transplant operation; a calculation stage; a transplant stage; an image-taking stage after the hair transplant operation; and a calculation and reporting stage.

5. The method according to claim 4, wherein the initial stage comprises the steps of positioning a person to be hair transplanted in a determined area for the hair transplant operation to be performed by using a robotic system, activating a command through a user interface by an operator and then switching the robotic system to a suitable position for initiating a scanning process of a head of the person via sending required commands to a robot controller by an operational unit.

6. The method according to claim 5, wherein the initial stage further comprises the steps of sending the required commands to cameras by the operational unit on the robotic system and preparing the cameras to take images.

7. The method according to claim 4, wherein the image-taking stage before the hair transplant operation comprises the steps of moving a camera capable of calculating a depth through a robotic system and creating a three-dimensional image of a head of a person.

8. The method according to claim 4, wherein the image-taking stage before the hair transplant operation comprises the steps of performing a planned route using three dimensional data obtained through an operational unit, bringing visible and/or IR cameras to appropriate points around a head by a help of a robotic system along the planned route, and taking images having partial overlaps of each other.

9. The method according to claim 4, wherein the calculation stage comprises the steps of calculating a follicle number, a hair number at each follicle, and a hair thickness by using an image processing technology over images taken at the image-taking stage before the hair transplant operation, and meanwhile, calculating fields of various areas marked on a head.

10. The method according to claim 4, wherein the calculation stage comprises the steps of calculating a Coverage Value for a whole of a head or for definite donor sites of the head after basic calculations and calculating a regional donor capacity.

11. The method according to claim 4, wherein the transplant stage comprises transplanting hair grafts harvested from donor sites to parts with sparse hair on a head in a sterile environment by evaluating analysis results obtained in the calculation stage.

12. The method according to claim 4, wherein the image taking stage after the hair transplant operation comprises the steps of taking images of a head using a pre-calculated route or a route created after a new three dimensional scanning, wherein the image taking stage after the hair transplant operation is performed by using previous three-dimensional model information of the head in a similar way to the image-taking stage before the hair transplant operation with or without performing an additional three-dimensional scanning for the head.

13. The method according to claim 4, wherein the calculation and reporting stage comprises the steps of detecting a follicle number harvested from a donor site and a transplanted follicle number over three-dimensional images by image processing technologies, calculating a number of hair taken from the donor site, and a number of transplanted hair by the operational unit, and reporting results of the calculating to inform a doctor and a person having the hair transplant operation.

14. The system according to claim 2, wherein the operational unit comprises one of a computer element capable of functioning separately or in an embedded position, stationary-portable computers or embedded cards carrying MCU (Micro Controller), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or GPU (Graphic Processing Unit).

15. The method according to claim 4, wherein the initial stage comprises the steps of sending required commands to cameras by an operational unit on a robotic system and preparing the cameras to take images.

16. The method according to claim 6, wherein the image-taking stage before the hair transplant operation comprises the steps of moving a camera capable of calculating a depth through the robotic system and creating a three-dimensional image of the head of the person.

17. The method according to claim 7, wherein the image-taking stage before the hair transplant operation comprises the steps of performing a planned route-using three-dimensional data obtained through an operational unit, bringing visible and/or IR cameras to appropriate points around the head by a help of the robotic system along the planned route, and taking images having partial overlaps of each other.

18. The method according to claim 8, wherein the calculation stage comprises the steps of calculating a follicle number, a hair number at each follicle, and a hair thickness by using an image processing technology over images taken at the image-taking stage before the hair transplant operation, and meanwhile, calculating fields of various areas marked on a head.

19. The method according to claim 9, wherein the calculation stage comprises the steps of calculating a Coverage Value for a whole of the head or for definite donor sites of the head after basic calculations and calculating a regional donor capacity.

20. The method according to claim 10, wherein the transplant stage comprises transplanting hair grafts harvested from donor sites to parts with sparse hair on the head in a sterile environment by evaluating analysis results obtained in the calculation stage.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG. 1 is a representative view showing the components of the system of the invention.

[0025] FIG. 2 is a schematic view showing the components of the system of the invention and the connection therebetween.

[0026] FIG. 3A is a representative view showing the donor site and hair graft determined on the head before the operation by using the system of the invention.

[0027] FIG. 3B is a detailed view showing the hair graft to be harvested from the donor site determined by using the system of the invention and the structure thereof.

[0028] FIG. 3C is a representative view showing the results of the treatment on the head after the operation by using the system of the invention.

[0029] FIG. 4A is a schematic view showing the area, the image of which is taken by the system of the invention and on which the transplant will be performed.

[0030] FIG. 4B is a schematic view showing the position of the donor site determined by the system of the invention on the head.

[0031] FIG. 5 is a flow chart showing the process steps in the system of the invention.

STAGE NAMES, REFERENCE NUMERALS OF THE SECTIONS AND PARTS TO SERVE FOR DESCRIBING THE INVENTION

[0032] 1Operational unit [0033] 1aUser interface-screen [0034] 2Camera [0035] 3Lenses [0036] 4Lighting element [0037] 5Robotic systems [0038] 6Robot controller [0039] 7Donor site [0040] 7aHair graft

[0041] Method Stages; [0042] 110: Initial stage [0043] 120: Image-taking stage before the operation [0044] 130: Calculation stage [0045] 140: Transplant stage [0046] 150: Image-taking stage after the operation [0047] 160: Calculation and reporting stage

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The invention comprises developing a method using the image processing technology in cooperation with the robotic technologies in order to plan and evaluate the process in the hair transplant operations and realizing said evaluation method by a novel technological system. The invention comprises a method constituted to obtain more reliable and accurate results for the hair transplant operations being a common practice in the medical field through a different manner from the conventional operations and being based on the analysis of the situation before and after the treatment and the comparison therebetween, and also the image processing technologies and robotic systems allowing applying said operations.

[0049] FIG. 1 is a representative view showing the components of the system of the invention. FIG. 2 is a schematic view showing the components of the system of the invention. FIG. 3A is a representative view showing the donor site determined by use of the system of the invention, the hair graft harvested therefrom, and the situation of the head before the hair transplant. FIG. 3B is a detailed view showing the hair graft to be harvested from the donor site and the structure thereof. FIG. 3C is a representative view showing the situation of the head after the hair transplant. FIG. 4A and FIG. 4B are the representative view showing the area on which the hair transplant is performed and the donor site. FIG. 5 is a flow chart showing the process steps in the system of the invention.

[0050] The system comprised by the method of the invention generally comprises an operational unit (1), the camera (2) systems in connection with said operational unit (1) and the robotic systems (5) controlled by the operational unit (1), also the lighting elements (4) being capable of emitting light in different wavelengths in order to capture the images recorded by the camera (2) systems more effectively, and the robot controller (6) enabling the motion of the whole system.

[0051] In the operational unit (1) section arranged on the system of the invention, a computer element which is capable of working alone or in an embedded manner, for example stationary/portable computers or embedded cards carrying MCU (Micro Controller), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or GPU (Graphic Processing Unit) may be used. In said system, the cameras (2) detecting the image, being visible, detecting the infrared (IRInfrared Radiation) lights, having depth sensibility (Stereo or Time-of-flight etc.) and the lenses (3) being appropriate for the function of said camera (2) systems are used. The telecentric or bi-telecentric lenses (3) may be used in said system as an alternative to the conventional lenses especially for thickness measurement and counting.

[0052] The robotic system (5) moving the lighting elements (4) and the cameras (2) used in the system of the invention are responsible for taking the cameras (2) to the preferred position. The positions of the cameras (2) are calculated by the operational unit (1) through the program located therein. The required commands are transferred to the robotic system (5) by the robot controller (6) after said calculation process. After taking the images obtained by said cameras (2), counting, analysis, evaluation, and reporting steps are performed by the operational unit (1) again. Therefore, a separate calculation unit may be used in the system.

[0053] In the system of the invention, the cameras (2) capable of measuring the depth and/or calculation scan the head of the person to be treated with hair transplant by means of the robotic system (5) and a three-dimensional (3D) model of the head is outlined in a digital environment. In order to screen the head in detail over the outlined three-dimensional model, a route is determined to follow, the robotic system (5) is moved along said route, and it takes the images of a standard visible area and/or definite areas of the head by means of the IR cameras (2). Said images are taken during the subsequent shots such that the images will be overlapping. The overlap degree of said images taken is calculated by using the image processing techniques. The data of said overlap degree are used to eliminate the counting in the overlapping areas both while creating a combined image from the images taken separately and realizing the counting processes on the separate images.

[0054] The follicle number on various areas of the head, hair number in each follicle, and the areas of the marked fields on the head are calculated over the images taken before the operation. Coverage Value calculation is carried out for each area by using these data and the total donor capacity of the patient, the maximum follicle, i.e. hair graft (7a) number to be harvested from the donor sites (7) are determined. These calculations are carried out by the image processing algorithms functioning on the operational unit (1). While these calculations may be carried out in parallel with the image-taking process, it may also be made after the whole image taking process.

[0055] After the operation, the number of the follicles harvested from the head, i.e. Hair graft (7a) is determined from the new image taken by the camera (2) being capable of measuring/calculating the predetermined route or depth, along the route to be determined. These calculations are carried out by the image processing algorithms functioning on the operational unit (1). While these calculations may be carried out in parallel with the image-taking process, it may also be made after the whole image taking process. The quantitative evaluation of the operation is possible by means of these calculations.

[0056] The method having the image processing technology and the robotic technologies used for the hair transplant planning and evaluation operations, considering it as a process flow, mainly comprises the following stages: [0057] Initial stage (110) [0058] Image-taking stage before the operation (120) [0059] Calculation stage (130) [0060] Transplant stage (140) [0061] Image-taking stage after the operation (150) [0062] Calculation and reporting stage (160)

[0063] Initial stage (110): At the initial stage, the person is firstly positioned in front of the developed robotic system (5) in the determined area and position in order to enable the planning of the whole process and then carrying out the analyses. The operator activates the related command through the user interface (1a) in order to actuate the system and the operational unit (1) sends the required commands to the robot controller (6) and switches the robotic system (5) to the initial position.

[0064] After the first stage, the operational unit (1) sends the required command to the cameras (2) and enable the cameras (2) to be ready for taking images. The operator may control the system directly through the operational unit (1) or through a user interface (screen) (1a) connected to the operational unit (1).

[0065] Image-taking stage before the operation (120): This stage consists of two separate process steps. In the first step, a three-dimensional image is created on the interface by moving a camera (2) which is capable of calculating the depth around the head of the person by means of the robotic system (5). After this scanning process, the route is planned by the operational unit (1) over said three-dimensional data. In the second step, the visible/IR cameras (2) are brought to the appropriate points around the head through the robotic system (5) over the planned route and the images are taken which have partial overlaps between each other.

[0066] Calculation stage (130): In this stage, the follicle number, the hair number in each follicle and the hair thickness are calculated through the image processing technology by using the images taken at the previous stage. During this process, the fields of various areas marked on the head are also calculated. After these basic calculations, the Coverage Value calculation is carried out for the whole head and for the certain donor sites (7) on the head and the regional donor capacity is calculated.

[0067] Transplant stage (140): The results of the analysis is evaluated which has carried out by using the method of the invention and the system of said method at the previous stage and the hair grafts (7a) taken from the donor sites (7) by the doctors are transplanted to the areas where there are hair sparseness.

[0068] Image-taking stage after the operation (150): The image-taking stage after the operation is performed in a similar way to the image-taking step before the operation (120). Also the previous three dimensional model information of the person may be used without doing an additional three dimensional scanning for the same person or the images are taken through a pre-calculated route or through a route created after a new scanning.

[0069] Calculation and reporting stage (160): At this stage, the number of follicles taken from the donor sites (7) and the transplanted follicle number is determined by the image processing techniques over the images taken at the previous stage. Thus, the operational unit (1) calculates how many hairs are taken from an area of the head and how many of them can be transplanted.

[0070] These calculations are then reported so as to inform the doctor and the person who had the operation.

INDUSTRIAL APPLICABILITY OF THE INVENTION

[0071] The technology and the technical system used in the present invention is developed to count and analyze considerably small objects on a big non-planar area by the image processing techniques. In various counting processes in the industry, there are pick-and-place applications intended for detecting and changing the location of the objects with greater sizes. The invention presented in this patent will enable these processes to be more effectively carried out for the objects with relatively smaller sizes and provide new technological developments.