SYSTEM AND METHOD FOR IMPROVING HARDWARE USAGE IN CONTROL SERVER USING ARTIFICIAL INTELLIGENCE IMAGE PROCESSING

Abstract

A system for improving hardware usage in a control server using artificial intelligence image processing, including: an image input unit receiving multiple images, temporarily storing input original images, and transmitting image data into one hardware component; an image data pre-processing unit converting multiple images into images with smaller resolution to process the multiple images according to the image resolution of a single image and concatenating the converted images into a single image; an artificial intelligence task performance unit retrieving a data learning model for an object to be recognized and performing an object recognition artificial intelligence task using the data for which pre-processing of image data has been applied; and a resultant image output unit checking boundary coordinates along which images are concatenated according to the number of input images and calculating the center coordinates of each recognized object in the vicinity of the image boundaries.

Claims

1. A system for improving hardware usage in a control server using artificial intelligence image processing, the system comprising: an image input unit receiving multiple images, temporarily storing input original images, and transmitting image data into one hardware component; an image data pre-processing unit converting multiple images into images with smaller resolution to process the multiple images according to the image resolution of a single image and concatenating the converted images into a single image; an artificial intelligence task performance unit retrieving a data learning model for an object to be recognized and performing an object recognition artificial intelligence task using the data for which pre-processing of image data has been applied; and a resultant image output unit checking boundary coordinates along which images are concatenated according to the number of input images and calculating the center coordinates of each recognized object in the vicinity of the image boundaries, checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object and performing calculation of coordinate generation in proportion to the original image size through the detected coordinates of the object, and converting an object detection area in proportion to the original image size and outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

2. The system of claim 1, wherein the image data pre-processing unit includes an image size conversion unit converting multiple images into images with a smaller size to be processed according to the resolution of a single image and a single image generation unit concatenating the converted images at the same size into a single image.

3. The system of claim 1, wherein the resultant image output unit includes a boundary coordinate checking unit checking boundary coordinates along which images are concatenated according to the number of input images, a center coordinate detection calculation unit calculating detection of center coordinates of each recognized object in the vicinity of image boundaries, a length ratio checking unit checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object, and a boundary coordinate distance comparison unit comparing a distance from the center point of an object to the image boundary with a half horizontal length of the object.

4. The system of claim 3, wherein the boundary coordinate distance comparison unit compares the distance from an object's center position to the image boundary with a half horizontal length of the object and excludes the corresponding object if the distance between the center point and the boundary coordinates is smaller than a reference value.

5. The system of claim 3, wherein the boundary coordinate distance comparison unit compares the distance from an object's center position to the image boundary with a half horizontal length of the object and limits a detected size of the corresponding object up to the image boundary if the distance between the center point and the boundary coordinates is larger than a reference value.

6. The system of claim 3, wherein the resultant image output unit further includes: a coordinate generation calculation unit retrieving a temporarily stored original image by reflecting a comparison result of the boundary coordinate distance comparison unit and calculating generation of coordinates in proportion to the original image size through the object's detected coordinates, an object detection area conversion unit converting an object detection area in proportion to the original image size, and a detection area output unit outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

7. A method for improving hardware usage in a control server using artificial intelligence image processing, the method comprising: inputting images by receiving multiple images, temporarily storing input original images, and transmitting image data into one hardware component; pre-processing image data by converting multiple images into images with smaller resolution to process the multiple images according to the image resolution of a single image and concatenating the converted images into a single image; performing an artificial intelligence task by retrieving a data learning model for an object to be recognized and performing an object recognition artificial intelligence task using the data for which pre-processing of image data has been applied; and outputting a resultant image by checking boundary coordinates along which images are concatenated according to the number of input images and calculating the center coordinates of each recognized object in the vicinity of image boundaries, checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object and performing calculation of coordinate generation in proportion to the original image size through the detected coordinates of the object, and converting an object detection area in proportion to the original image size and outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

8. The method of claim 7, wherein the pre-processing image data includes converting an image size converting multiple images into images with a smaller size to be processed according to the resolution of a single image and generating a single image concatenating the converted images at the same size into a single image.

9. The method of claim 7, wherein the outputting the resultant image includes checking boundary coordinates checking boundary coordinates along which images are concatenated according to the number of input images, calculating detection of center coordinates calculating detection of center coordinates of each recognized object in the vicinity of image boundaries, checking a length ratio checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object, and comparing a boundary coordinate distance comparing a distance from the center point of an object to the image boundary with a half horizontal length of the object.

10. The method of claim 9, wherein the comparing the boundary coordinate distance compares the distance from an object's center position to the image boundary with a half horizontal length of the object and excludes the corresponding object if the distance between the center point and the boundary coordinates is smaller than a reference value.

11. The method of claim 9, wherein the comparing the boundary coordinate distance compares the distance from an object's center position to the image boundary with a half horizontal length of the object and limits a detected size of the corresponding object up to the image boundary if the distance between the center point and the boundary coordinates is larger than a reference value.

12. The method of claim 9, wherein the outputting the resultant image further includes calculating generation of coordinates by retrieving a temporarily stored original image by reflecting a comparison result of the boundary coordinate distance comparison unit and calculating generation of coordinates in proportion to the original image size through the object's detected coordinates, converting an object detection area converting an object detection area in proportion to the original image size, and outputting a detection area outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

13. The method of claim 7, wherein the outputting the resultant image uses the following pseudo equation for checking whether the position of an object detected in an image obtained by concatenating N images corresponds to a valid object actually existing in adjacent images, $\mathrm{Object}\mathrm{in}a\mathrm{image}N=\{\begin{array}{cc}1& \left(O.C.W-\mathrm{Image}N\mathrm{start}\mathrm{width}>1/4O.C.W\right)\&\&\left(O.C.H-\mathrm{Image}N\mathrm{start}\mathrm{height}>1/4O.C.H\right)\\ 0& \mathrm{New}\mathrm{objects}\mathrm{Recognized}\mathrm{at}\mathrm{Boundaries}\end{array}$ where the equation corresponding to 1 indicates that the corresponding detected image object belongs to the N-th image, and the equation corresponding to 0 indicates that the corresponding detected image object does not belong to the area.

14. The method of claim 13, wherein, in the equation corresponding to 1, O.C.W stands for Object Center Width and represents the horizontal coordinate of a detected object's center when the top-left coordinates of the object are set to 0, and Image N start width represents the horizontal distance from the boundary of an adjacent image to the center of the detected object, on the other hand, O.C.H stands for Object Center Height and represents the vertical coordinate of a detected object's center when the top-left coordinates of the object are set to 0, and Image N start height represents the vertical distance from the boundary of the adjacent image to the center of the detected object.

15. The method of claim 13, wherein, in the case of Image N start height, images concatenated in the horizontal direction unconditionally show the value of 1, and images concatenated in the vertical direction are used to check which part of an upper and a lower image contains an object, on the other hand, in the case of Image N start width, images concatenated in the vertical direction unconditionally show the value of 1, images concatenated in the horizontal direction are used to check which part of a left and a right image contains the object, and any object not belonging to the case of the pseudo equation corresponding to 1 is classified as false.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG. 1 shows the structure of a system for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure.

[0059] FIG. 2 shows a detailed structure of an image data pre-processing unit according to the present disclosure.

[0060] FIG. 3 shows a detailed structure of a resultant image output unit according to the present disclosure.

[0061] FIGS. 4A and 4B are flow charts illustrating a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure.

[0062] FIGS. 5A to 5D illustrate an example in which a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure is applied.

[0063] FIG. 6 illustrates a pseudo-equation in a question sentence for evaluating boundary images.

DETAILED DESCRIPTION

[0064] In what follows, preferred embodiments of a system and a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure will be described in detail.

[0065] The characteristics and advantages of a system and a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure will be clearly understood through detailed descriptions on the respective embodiments below.

[0066] FIG. 1 shows the structure of a system for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure.

[0067] A system and a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure enable a control server using artificial intelligence image processing to efficiently process a plurality of artificial intelligence processes that are inefficient or may not be performed.

[0068] To this purpose, the present disclosure may include a structure that processes a plurality of image data by compressing and concatenating a plurality of input data in consideration of the data input size to handle multiple data simultaneously and resolves a difficulty in an image processing task performed in real-time.

[0069] The present disclosure may include a structure that processes multiple data using one hardware component by designing an optimal task processed according to the size of input image data by including a process which receives images, compresses image resolution appropriately to be suitable for the number of input cameras, performs deep learning, and crops out the input images subsequently before executing an artificial intelligence model.

[0070] A system and a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure do not alter a training model for deep learning.

[0071] Modifying a deep learning model may directly help improve the performance. However, it will take a long time to validate the modification and run a modified process, which will eventually affect hardware usage.

[0072] It is so because, although the model depth for deep learning may improve the accuracy, a deeper model depth is the opposite of making a lightweight system.

[0073] As shown in FIG. 1, a system for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure comprises an image input unit 10 receiving multiple images, temporarily storing input original images, and transmitting image data into one hardware component (server); an image data pre-processing unit 20 converting multiple images into images with smaller resolution to process the multiple images according to the image resolution of a single image and concatenating the converted images, namely, at the same size, into a single image; an artificial intelligence task performance unit 30 retrieving a data learning model for an object to be recognized and performing an object recognition artificial intelligence task using the data for which pre-processing of image data has been applied; and a resultant image output unit 40 checking boundary coordinates along which images are concatenated according to the number of input images and calculating the center coordinates of each recognized object in the vicinity of the image boundaries, checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object and performing calculation of coordinate generation in proportion to the original image size through the detected coordinates of the object, and converting an object detection area in proportion to the original image size and outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

[0074] A detailed structure of the image data pre-processing unit 20 according to the present disclosure is described below.

[0075] FIG. 2 shows a detailed structure of an image data pre-processing unit according to the present disclosure.

[0076] The image data pre-processing unit 20 includes an image size conversion unit 21 converting multiple images into images with a smaller size to be processed according to the resolution of a single image and a single image generation unit 22 concatenating the converted images at the same size into a single image.

[0077] A detailed structure of a resultant image output unit 40 according to the present disclosure is described below.

[0078] FIG. 3 shows a detailed structure of a resultant image output unit according to the present disclosure.

[0079] As shown in FIG. 3, a resultant image output unit 40 according to the present disclosure includes a boundary coordinate checking unit 41 checking boundary coordinates along which images are concatenated according to the number of input images, a center coordinate detection calculation unit 42 calculating detection of center coordinates of each recognized object in the vicinity of the image boundaries, a length ratio checking unit 43 checking the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object, and a boundary coordinate distance comparison unit 44 comparing a distance from the center point of an object to the image boundary with a half horizontal length of the object.

[0080] Here, the boundary coordinate distance comparison unit 44 compares the distance from an object's center position to the image boundary with a half horizontal length of the object and excludes the corresponding object if the distance between the center point and the boundary coordinates is smaller than a reference value.

[0081] And the boundary coordinate distance comparison unit 44 compares the distance from an object's center position to the image boundary with a half horizontal length of the object and limits a detected size of the corresponding object up to the image boundary if the distance between the center point and the boundary coordinates is larger than a reference value.

[0082] And the resultant image output unit 40 according to the present disclosure further includes a coordinate generation calculation unit 45 retrieving a temporarily stored original image by reflecting a comparison result of the boundary coordinate distance comparison unit 44 and calculating generation of coordinates in proportion to the original image size through the object's detected coordinates, an object detection area conversion unit 46 converting an object detection area in proportion to the original image size, and a detection area output unit 47 outputting a resultant image by displaying the corresponding coordinates and detected area on each original image.

[0083] A method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure is described in detail as follows.

[0084] FIGS. 4A and 4B are flow charts illustrating a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure. First, the method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure temporarily stores the original input sources when multiple images are received and transmits the image data to one hardware component (server) S401.

[0085] Next, the method converts multiple images into images with smaller resolution to process the multiple images according to the image resolution of a single image S402 and concatenates the converted images, namely, at the same size, into a single image S403.

[0086] And the method retrieves a data learning model for an object to be recognized S404 and performs an object recognition artificial intelligence task S405.

[0087] And the method performs the step of outputting a resultant image S406 to S414 as follows.

[0088] First, the method checks boundary coordinates along which images are concatenated according to the number of input images S406 and calculates the center coordinates of each recognized object in the vicinity of the image boundaries S407.

[0089] And the method checks the distances from the center point of each object to the image boundaries and the length ratio of horizontal ends of the object S408.

[0090] Next, the method compares the distance from an object's center position to the image boundary with a half horizontal length of the object S409 and excludes the corresponding object if the distance between the center point and the boundary coordinates is smaller than a reference value S410 while the method limits a detected size of the corresponding object up to the image boundary if the distance between the center point and the boundary coordinates is larger than the reference value S411.

[0091] And the method performs calculation of coordinate generation in proportion to the original image size through the detected coordinates of the object S412.

[0092] Next, the method converts the object detection area in proportion to the original image size S413 and outputs a resultant image by displaying the corresponding coordinates and detected area on each original image S414.

[0093] FIGS. 5A to 5D illustrate an example in which a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure is applied.

[0094] FIG. 5A shows an example of the overall process of a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure, where related processes are applied before and after performing a deep learning model.

[0095] Before performing the model, images are received, and their resolution is reduced appropriately according to the number of inputs cameras. After that, deep learning is performed, and the input images are cropped out subsequently.

[0096] FIG. 5B shows an example of an image data pre-processing process, which shows an image obtained by synthesizing four images.

[0097] The original camera images processed in the server are downsized appropriately according to the number of input camera images into a single image.

[0098] When HD resolution is employed, and the input image is compressed to 640*360, which is half of 1280*720, a total of 4 camera images may be processed simultaneously.

[0099] Since a compressed image is used for deep learning, the original image is not modified.

[0100] FIG. 5C shows an example of outputting a resultant image, which requires two processes after deep learning is performed.

[0101] Since there is no boundary when four images are concatenated into a single image, deep learning is performed on the single image. It means that an object may be wrongly recognized at the four image boundaries.

[0102] FIG. 6 illustrates a pseudo-equation in a question sentence for evaluating boundary images.

[0103] Eq. 1 shows a pseudo-equation in a question sentence for evaluating boundary images.

[00002]$\begin{array}{cc}\mathrm{Object}\mathrm{in}a\mathrm{image}N=\{\begin{array}{cc}1& \left(O.C.W-\mathrm{Image}N\mathrm{start}\mathrm{width}>1/4O.C.W\right)\&\&\left(O.C.H-\mathrm{Image}N\mathrm{start}\mathrm{height}>1/4O.C.H\right)\\ 0& \mathrm{New}\mathrm{objects}\mathrm{Recognized}\mathrm{at}\mathrm{Boundaries}\end{array}& \left[\mathrm{Eq}.1\right]\end{array}$

[0104] Eq. 1 is a pseudo-equation that checks whether an object detected in the image obtained by concatenating N images is a valid object actually present in a nearby image, where the equation corresponding to 1 indicates that the image belongs to the Nth image while 0 indicates that the corresponding detected image does not belong to the region.

[0105] In the equation corresponding to 1, O.C.W stands for Object Center Width and represents the horizontal coordinate of a detected object's center when the top-left coordinates of the object are set to 0.

[0106] Image N start width represents the horizontal distance from the boundary of an adjacent image to the center of the detected object. On the other hand, O.C.H represents the vertical coordinate of a detected object's center when the top-left coordinates of the object are set to 0, and Image N start height represents the vertical distance from the boundary of the adjacent image to the center of the detected object.

[0107] In the case of Image N start height, images concatenated in the horizontal direction unconditionally show the value of 1, and images concatenated in the vertical direction are used to check which part of an upper and a lower image contains the object.

[0108] On the other hand, in the case of Image N start width, images concatenated in the vertical direction unconditionally show the value of 1, and images concatenated in the horizontal direction are used to check which part of a left and a right image contains the object.

[0109] Any object not belonging to the case of the pseudo equation's value of 1 in Eq. 1 is classified as false (not belonging to the corresponding image).

[0110] Therefore, the present disclosure first performs the algorithm that evaluates which image contains a recognized object or whether the recognized object is regarded as a new object.

[0111] Eq. 1 is an equation that evaluates whether an object recognized in the vicinity of the image boundary is a valid object using the center point of the recognized object.

[0112] After evaluation, a correction process is performed to insert the reduced positions and recognized size for each camera image area into the original image size. After the correction, a resultant image is obtained by inserting recognized object information into the original, unmodified image.

[0113] FIG. 5D shows an example of a result obtained by applying a method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure.

[0114] The test of FIG. 5D has been performed using two cameras.

[0115] In general, when a deep learning function is performed, a process is assigned and performed according to an input value. Given that one hardware component and two input data, if parallel processing is not employed, the execution speed is cut down to half even if the processes are executed simultaneously. Or the processes are not performed since they are already occupied.

[0116] In the case of a deep learning model used in the test of FIG. 5D according to the present disclosure, the process is designed to be performed at a maximum speed of 30 fps and not to show a significant change in the processing speed even for two input data. If a white area with a width of 10 pixels is applied between the two images when they are concatenated during the initial stage of the process, misrecognition may be further reduced.

[0117] The system and the method for improving hardware usage in a control server using artificial intelligence image processing according to the present disclosure process multiple data using one hardware component by designing an optimal task processed according to the size of input image data by including a process which receives images, compresses image resolution appropriately to be suitable for the number of input cameras, performs deep learning, and crops out the input images subsequently before executing an artificial intelligence model.

[0118] As described above, it should be understood that the present disclosure may be implemented in various other modified forms without departing from the inherent characteristics of the present disclosure.

[0119] In this respect, the specific embodiments should be considered in a descriptive point of view rather than restrictive point of view. The technical scope of the present disclosure should be judged by the appended claims rather than the descriptions given above, and all of the discrepancies which may be found within the range equivalent to the technical scope of the present disclosure should be interpreted to belong thereto.

DETAILED DESCRIPTION OF MAIN ELEMENTS

[0120] 10: Image input unit [0121] 20: Image data pre-processing unit [0122] 30: Artificial intelligence task performance unit [0123] 40: Resultant image output unit