DEVICE AND METHOD FOR PROCESSING HIGH-DEFINITION 360-DEGREE VR IMAGE

Abstract

Disclosed is an apparatus and method of providing a high quality 360-degree VR image. A method of decoding a 360-degree VR image according to the present disclosure includes: receiving a bit stream including 360-degree VR image information; decoding information related to a 360-degree VR service from the bitstream; detecting a region of interest based on the information related to the 360-degree VR service; and providing to a user a 360-degree VR image for the region of interest.

Claims

1. A method of decoding a 360-degree image, the method comprising: receiving a bitstream including 360-degree virtual reality (VR) image information; decoding information related to a 360-degree VR service from the bitstream; detecting a region of interest based on the information related to the 360-degree VR service; and providing to a user a 360-degree VR image for the region of interest.

2. The method of claim 1, wherein the 360-degree VR image information includes information related to an identical image frame having a plurality of resolutions.

3. The method of claim 1, wherein the region of interest is obtained by motion information of the user, and by a user input.

4. The method of claim 1, wherein the 360-degree VR image information includes at least one of information related to a image frame having a first resolution, and information of the image frame having a second resolution higher than the first resolution, wherein the image frame having the second resolution is reconstructed based on the information related to the 360-degree VR service.

5. The method of claim 4, wherein the detecting of the region of interest includes: decoding division information of the image frame having the second resolution from the information related to the 360-degree VR service; and detecting the region of interest based on the division information.

6. The method of claim 5, wherein the providing of the 360-degree VR image includes: performing mapping the image frame having the second resolution and the detected region of interest in the image frame having the first resolution; and providing to the user a result image of the mapping.

7. The method of claim 1, wherein the information related to the 360-degree VR service includes at least one of dimensional information, projection type information, size information, resolution information, and division information of the 360-degree VR image.

8. An apparatus for decoding a 360-degree image, wherein the apparatus receives a bitstream including 360-degree VR image information, decodes information related to a 360-degree VR service from the bitstream, detects a region of interest based on the information related to the 360-degree VR service, and provides to a user a 360-degree VR image for the region of interest.

9. The apparatus of claim 8, wherein the 360-degree VR image information includes information of an identical image frame having a plurality of resolutions.

10. The apparatus of claim 8, wherein the region of interest is obtained by motion information of the user, and by a user input.

11. The apparatus of claim 8, wherein the 360-degree VR image information includes at least one of information related to an image frame having a first resolution, and information of the image frame having a second resolution higher than the first resolution, wherein the image frame having the second resolution is reconstructed based on the information related to the 360-degree VR service.

12. The apparatus of claim 11, wherein the apparatus decodes division information of the image frame having the second resolution from the information related to the 360-degree VR service, and detects the region of interest based on the division information.

13. The apparatus of claim 12, wherein the apparatus performs mapping for the image frame having the second resolution and the detected region of interest in the image frame having the first resolution, and provides to the user a result image of the mapping.

14. The apparatus of claim 8, wherein the information related to the 360-degree VR service includes at least one of dimensional information, projection type information, size information, resolution information, and division information of the 360-degree VR image.

15. A method of encoding a 360-degree image, the method comprising: receiving 360-degree VR image information; recognizing a region of interest of a user; and encoding information related to a 360-degree VR service, wherein the information related to the 360-degree VR service includes at least one of division information of the 360-degree VR image and information of the region of interest.

16. The method of claim 15, wherein the 360-degree VR image information includes at least one of information of an image frame having a first resolution, and information of the image frame having a second resolution higher than the first resolution, and the division information is information of sub-regions obtained by diving the image frame having the second resolution to have a predetermined size.

17. The method of claim 15, wherein the region of interest is obtained by motion information of the user, and by a user input.

18. An apparatus for encoding a 360-degree image, wherein the apparatus receives 360-degree VR image information, recognizes a region of interest of a user, encodes information related to a 360-degree VR service, wherein the information related to the 360-degree VR service includes at least one of division information of the 360-degree VR image, and information of the region of interest.

19. The apparatus of claim 18, wherein the 360-degree VR image information includes at least one of information of an image frame having a first resolution, and information of the image frame having a second resolution higher than the first resolution, and the division information is information of sub-regions obtained by diving the image frame having the second resolution to have a predetermined size.

20. The apparatus of claim 18, wherein the region of interest is obtained by motion information of the user, and by a user input.

Description

DESCRIPTION OF DRAWINGS

[0015] FIG. 1 is a view showing a scenario of providing a high quality 360-degree VR media service by using a broadcasting network according to an embodiment of the present disclosure.

[0016] FIG. 2 is a view showing a structure of a 360-degree VR image stream transmitted through the broadcasting network according to an embodiment of the present disclosure.

[0017] FIG. 3 is a view showing a signaling structure for 360-degree VR broadcasting service according to an embodiment of the present disclosure.

[0018] FIG. 4 is a view showing a configuration of a high quality 360-degree VR image processing apparatus according to an embodiment of the present disclosure.

[0019] FIG. 5 is a view showing a process of providing a high quality 360-degree image of a region of interest (RoI) according to an embodiment of the present disclosure.

[0020] FIG. 6 is a view showing a process of providing a high quality 360-degree image of a region of interest (RoI) according to another embodiment of the present disclosure.

[0021] FIG. 7 is a view showing a process of providing a high quality 360-degree VR image of a region of interest of a user in a low quality 360-degree VR image according to an embodiment of the present disclosure.

BEST MODE

[0022] According to an aspect of the present disclosure, there may be provided a method of decoding a 360-degree image, the method including: receiving a bitstream including 360-degree virtual reality (VR) image information; decoding information related to a 360-degree VR service from the bitstream; detecting a region of interest based on the information related to the 360-degree VR service; and providing a 360-degree VR image for the region of interest to a user.

[0023] According to another aspect of the present disclosure, there may be provided an apparatus for decoding a 360-degree image, wherein the apparatus receives a bitstream including 360-degree VR image information, decodes information related to a 360-degree VR service from the bitstream, detects a region of interest based on the information related to the 360-degree VR service, and provides to a user a 360-degree VR image for the region of interest.

[0024] According to another aspect of the present disclosure, there may be provided a method of encoding a 360-degree image, the method including: receiving 360-degree VR image information; recognizing a region of interest of a user; and encoding information related to a 360-degree VR service, wherein the information related to the 360-degree VR service includes at least one of division information of the 360-degree VR image and information of the region of interest.

[0025] According to another aspect of the present disclosure, there may be provided an apparatus for encoding a 360-degree image, wherein the apparatus receives 360-degree VR image information, recognizes a region of interest of a user, encodes information related to a 360-degree VR service, wherein the information related to the 360-degree VR service includes at least one of division information of the 360-degree VR image, and information of the region of interest.

Mode for Invention

[0026] The present invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the present invention are shown. This present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the present invention to those skilled in the art. The similar reference numerals refer to the same or similar functions in various aspects. In the drawings, the shapes and sizes of elements may be exaggerated for clarity. In the following detailed description, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It is to be understood that the various embodiments of the invention, although different, are not necessarily mutually exclusive. For example, a certain feature, structure, or characteristic described herein in connection with one embodiment may be implemented within other embodiments without departing from the spirit and scope of the invention. In addition, it is to be understood that the location or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled.

[0027] It will be understood that, although the terms including ordinal numbers such as first, second, etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a second element could be termed a first element without departing from the teachings of the present inventive concept, and similarly a first element could be also termed a second element. The term and/or includes any and all combination of one or more of the associated items listed.

[0028] When an element is referred to as being connected to or coupled with another element, it can not only be directly connected or coupled to the other element, but also it can be understood that intervening elements may be present. In contrast, when an element is referred to as being directly connected to or directly coupled with another element, there are no intervening elements present.

[0029] The components as used herein may be independently shown to represent their respective distinct features, but this does not mean that each component should be configured as a separate hardware or software unit. In other words, the components are shown separately from each other for ease of description. At least two of the components may be combined to 5 configure a single component, or each component may be split into a plurality of components to perform a function. Such combination or separation also belongs to the scope of the present invention without departing from the gist of the present invention.

[0030] Terms used in the application are merely used to describe particular embodiments and are not intended to limit the present disclosure. A singular expression includes a plural expression unless the context clearly indicates otherwise. In the application, terms such as include or have are should be understood as designating that features, number, steps, operations, elements, parts, or combinations thereof exist and not as precluding the existence of or the possibility of adding one or more other features, numbers, steps, operations, elements, parts, or combinations thereof in advance. That is, in the present invention, the contents describing the specific configuration as including does not exclude the configuration other than the configuration thereof, and the additional configurations may be included within the scope of the practice of the invention or the technical scope of the invention.

[0031] Some elements may not serve as necessary elements to perform an essential function in the present invention, but may serve as selective elements to improve performance. The present invention may be embodied by including only necessary elements to implement the spirit of the present invention excluding elements used to improve performance, and a structure including only necessary elements excluding selective elements used to improve performance is also included in the scope of the present invention.

[0032] Hereinbelow, reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying. In the detailed description of the preferred embodiments of the disclosure, however, detailed depictions of well known related functions and configurations may be omitted so as not to obscure the art of the present disclosure with superfluous detail. Also, the same or similar reference numerals are used throughout the different drawings to indicate similar functions or operations.

[0033] FIG. 1 is a view showing a scenario of providing a high quality 360-degree VR media service by using a broadcasting network according to an embodiment of the present disclosure.

[0034] A first scenario is a scenario for simultaneously providing 360-degree VR image services of low quality and high quality based on an image obtained by using a 360-degree VR camera. Herein, high quality may mean a base resolution of media obtained from a 360-degree VR obtaining apparatus. A low quality 360-degree VR image 110 may mean an image obtained by downscaling a high quality 360-degree VR image obtained by the 360-degree VR camera to have a predetermined size. For example, when the high quality 360-degree VR image has a resolution of 16K8K, the low quality 360-degree VR image 110 may have a resolution of 4K2K. The resolution of 4K2K is an example based on a resolution of a current commercial UHD TV and a head mounted display (HMD). The resolution of the low quality 360-degree VR image 110 is not limited thereto, and may vary randomly. Meanwhile, the low quality 360-degree VR image 110 may be provided for compatibility with a conventional receiver in a broadcasting environment. In addition, the low quality 360-degree VR image 110 may be used for minimizing an MTP latency according to user motion or user selection.

[0035] A high quality 360-degree VR image 120 of the present disclosure may mean a high quality image provided for a region of interest (RoI). The region of interest may be recognized by user motion or user selection. For example, the user selection may mean a user input by using an external device, but it is not limited thereto. Predetermined motion of the user for determining the region of interest may be included. The high quality 360-degree VR image 120 may be divided into sub-regions having an arbitrary size, and each sub-region may be independently encoded. Referring to FIG. 1, the high quality 360-degree VR image 120 may be divided into sub-regions based on tiles. For example, the user may watch a low quality 360-degree VR image, and provided with a high quality 360-degree VR image for a region of interest of the user, and thus the user may watch a 360-degree VR image with clearer and better image quality for the region of interest.

[0036] A second scenario is a scenario of providing of a high quality 360-degree VR image service 130 under a broadcasting environment based on the high quality 360-degree VR image 130 that is divided into sub-regions having a predetermined size and without providing a low quality 360-degree VR image. For example, the receiver may play an initial view region 132 that is played among sub-regions by performing signaling, and when user motion or user selection is input, play the corresponding sub-region in response to the input. For example, the predetermined size of the sub-region may be determined to have a constant resolution such as FHD, 4K, etc. In addition, the predetermined size may vary according to a resolution supported by a TV, or a mobile device such as HMD.

[0037] FIG. 2 is a view showing a structure of a 360-degree VR image stream transmitted through the broadcasting network according to an embodiment of the present disclosure.

[0038] A low quality 360-degree VR image 210 may be encoded by an HEVC encoder 212 and transmitted by being distinguished by one of elementary stream (ES)_ID, Asset_ID or AdaptationSet ID. Alternatively, the low quality 360-degree VR image 210 may be transmitted in unique identifier information of a stream 214 that is separately encoded. The stream may mean a bitstream. ES_ID, Asset_ID, and AdaptationSet_ID may be respectively used as encoding stream unique identifier information to distinguish an encoding stream in MPEG-2 Systems, MPEG-MMT, and MPEG-DASH standard. For example, a low quality 360-degree VR image 210 may have a resolution of 4K2K, but it is not limited thereto. The resolution may vary according to a terminal providing the image.

[0039] Meanwhile, a high quality 360-degree VR image 220 may be divided into sub-regions having a constant size. The sub-regions may be respectively encoded by an HEVC encoder 222, and transmitted by being distinguished by one of ES ID 224, Asset ID or Adaptation_set ID by sub-regions. Herein, a number of streams of the high quality 360-degree VR image 220 may vary according to a number of sub-regions. For example, the stream of the high quality 360-degree VR image 220 may have a resolution of 4K2K, but it is not limited thereto. The resolution may vary according to a terminal providing the image. In addition, a low quality 360-degree VR image stream may be transmitted through a channel identical to a channel through which the sub-region stream of the high quality 360-degree VR image 220 is transmitted, or may be transmitted through an additional independent channel 216 for maintaining compatibility with a conventional receiver. For example, the additional independent channel may be an IP network, or an additional channel within the broadcasting network. In addition, the sub-region stream of the high quality 360-degree VR image 220 may be transmitted through one broadcasting channel by using a channel combination or may be transmitted through an IP network 226.

[0040] FIG. 3 is a view showing a signaling structure for 360-degree VR broadcasting service according to an embodiment of the present disclosure.

[0041] Referring to FIG. 3, a signaling structure for a 360-degree VR broadcasting service may include signaling information that is additionally required for distinguishing 360-degree VR image streams of low quality and high quality or for a broadcasting service.

[0042] According to an embodiment, a first signaling structure 310 is a signaling structure for distinguishing a high quality 360-degree VR broadcasting service, and may be information signaled by being applied to a program or to a service level.

[0043] VR_contents_ID may mean an identifier for distinguishing various VR images transmitted from a program or channel. In addition, VR_contents_ID may be used as unique identifier information for distinguishing VR content that is variably transmitted. In addition, VR_contents_ID may be used as information representing an actual VR content configuration. For example, the actual VR content configuration may be 2D or 3D dimensional information. VR_projectionType may mean various VR projection types. In addition, VR_projectionType may be used for remapping or restoring an image received in the receiver to a 360-degree VR image. Total_width and Total_height may respectively mean the total horizontal and vertical sizes of a high quality 360-degree VR image. Number_ROI may mean a total number of sub-regions.

[0044] In addition, according to an embodiment, a second signaling structure 320 may mean an identifier for distinguishing a low quality 360-degree VR image, and mean information signaled by being applied to an encoding stream level of each sub-region obtained by dividing a high quality 360-degree VR image stream by a predetermined size. 360-degree VR_totalview_present may be used for determining whether or not a low quality 360-degree VR image stream is present. Viewport_ID may mean an identifier for distinguishing each sub-region. For example, Viewport_ID may be used for decoding or playing a corresponding sub-region according to user motion or user selection. Initial_viewport_flag may be used for identifying a sub-region that is played first in a TV or in a terminal such as HDM when a service is provided based on sub-regions without providing a low quality 360-degree VR image as the second scenario described in FIG. 1. RoI_x_startposition and RoI_x_endposition may respectively mean horizontal starting coordinates and horizontal ending coordinates for each sub-region. RoI_y_startposition and RoI_y_endposition may respectively mean vertical starting coordinates and vertical ending coordinates for each sub-region. A position of a sub-region within the entire VR image may be known based on the horizontal/vertical starting coordinates and the horizontal/vertical ending coordinates.

[0045] Meanwhile, signaling information constituting the first signaling structure 310 or the second signaling structure 320 or both may be used by being transformed into various forms, in addition to the above structure, such as metadata, XML, etc.

[0046] FIG. 4 is a view showing a configuration of a high quality 360-degree VR image processing apparatus according to an embodiment of the present disclosure.

[0047] A high quality 360-degree VR image processing apparatus may include a 360-degree VR image encoding apparatus or decoding apparatus or both. In addition, a 360-degree VR image may be transmitted through a broadcasting network or through two types of networks including a broadcasting network and an IP network.

[0048] Referring to FIG. 4, a high quality 360-degree image processing apparatus 400 may include at least one of a receiving module 410, a buffer module 420, a viewport control module 430, a decoder module 440, an RoI mapping module 450, and a 360 VR mapping module 460. However, the figure shows only some of the components necessary for describing the present embodiment, constituent elements of the high quality 360-degree image processing apparatus 400 are not limited to the above example.

[0049] The receiving module 410 may receive a stream transmitted based on various standard regulations, and perform analysis of signaling information described in FIG. 3 and control information related to provide a 360-degree VR service. In addition, the receiving module 410 may separate a 360-degree VR stream based on a low quality or a sub-region or both, and transfer the separated 360-degree VR stream to a corresponding buffer. The buffer module 420 may perform buffering for the stream transmitted from the receiving module 410. In addition, the buffer module 420 may perform synchronization based on a decoding time stamp (DTS) or a presentation time stamp (PTS) of each stream. For example, when providing a 360-degree VR service, playing a scene synchronized according to user motion is required, and the buffer module 420 may perform synchronization by frames based on a DTS or a PTS. Herein, a low quality 360-degree VR stream may be transmitted to a corresponding device in an IP stream under an IP environment for providing compatibility with a conventional terminal such as HMD.

[0050] The viewport control module 430 may extract a corresponding stream according to user motion or user selection based on a TV or a terminal such as HMD. In addition, the viewport control module 430 may transmit the extracted stream to the decoder module 440. For example, the viewport control module 430 may transmit a low quality 360-degree VR image stream and a sub-region stream extracted by the viewport control module 430 to the decoder module 440. Herein, user motion information, user selection information, or regional information according to user motion or selection may be received under an IP environment. A sub-region stream may be mapped based on a low quality 360-degree VR stream, and may be used for providing a high quality image for a region that the user watches later. Meanwhile, the buffer module 420 may perform frame synchronization for sub-region streams that are not extracted by the viewport control module 430 based on a DTS or a PTS or both. In addition, the buffer module 420 may continuously drop a corresponding packet by performing synchronization. Decoding of a low quality 360-degree VR image stream may be continuously performed, while decoding of a sub-region stream may be dependent on a region of interest of the user. As the buffer module 420 performs synchronization for the entire stream, the high quality 360-degree VR image processing apparatus may provide a region of interest of high quality by minimizing an MTP latency generated according to user motion through an actual terminal such as HMD. In other words, the high quality 360-degree VR image processing apparatus may play a region of interest based on a low quality 360-degree VR image stream when the region of interest has been changed, and perform decoding of a sub-region stream corresponding to the selection of the region of interest.

[0051] The RoI mapping module 450 may re-adjust a sub-region stream extracted by the viewport control module 430 to a stream suitable for a resolution supported by a terminal such as HMD. The 360 VR mapping module 460 may perform mapping for a sub-region stream for a region of interest based on user motion or user selection and a low quality 360-degree VR image stream. In addition, the 360 VR mapping module 460 may transmit a mapping result 470 to a TV or a terminal such as HMD 470.

[0052] FIG. 5 is a view showing a process of providing a high quality 360-degree image for a region of interest (RoI) according to an embodiment of the present disclosure.

[0053] The high quality 360-degree VR image processing apparatus of the present disclosure may decode a sub-region stream included in a region of interest 512, and transfer the decoded stream to the 360 VR mapping module. The region of interest 512 may be detected by user motion or user selection. Referring to FIG. 5, a resolution of the extracted sub-region stream may have a horizontal resolution of 4K. For example, when a terminal such as HMD supports a horizontal resolution of 4K for a region of interest, a resolution of the extracted sub-region stream is identical to a resolution supported for a region of interest in a terminal such as HMD, and thus the high quality 360-degree VR image processing apparatus may transfer the extracted sub-region stream to the 360 VR mapping module of FIG. 4 without performing an additional process.

[0054] FIG. 6 is a view showing a process of providing a high quality 360-degree image for a region of interest (RoI) according to another embodiment of the present disclosure.

[0055] The high quality 360-degree VR image processing apparatus may decode a sub-region stream included in a region of interest 612 for the region of interest 612, and transfer the decoded stream to the 360 VR mapping module. The region of interest 612 may be determined by user motion or user selection. In FIG. 6, as FIG. 5, a terminal such as HMD supports a horizontal resolution of 4K for a region of interest. Referring to FIG. 6, different to FIG. 5, the region of interest 612 is positioned on nine sub-region streams, that is, on a sub-region stream having a horizontal resolution of 4K or more, and thus a process of re-adjusting a region is required. For example, when the region of interest 612 is selected as FIG. 6, the total of nine sub-region streams included in the region of interest 612 may be all decoded, but re-adjusting a size of a region that is decoded to be played in a terminal such as HMD may be required. In other words, the high quality 360-degree VR image processing apparatus may continuously decode the total of nine sub-region streams positioned at the region of interest 612. In addition, the high quality 360-degree VR image processing apparatus may detect a region of interest based on a center of a region corresponding to a region of interest 622 in the decoded partial image 620. For example, the high quality 360-degree VR image processing apparatus may set center coordinates 624 of a region corresponding to region of interest 622 in the decoded partial image 620 as (0,0), and extract a region corresponding to 2K to the left, 2K to the right, 1K to the upper side, and 1K to the low side, in a total of 4K by using the center coordinates. The high quality 360-degree VR image processing apparatus may transmit the finally extracted region of interest 630 to the 360 VR mapping module. Meanwhile, when user motion is performed within the decoded nine sub-regions, the high quality 360-degree VR image processing apparatus may extract a region of interest when performing re-adjusting the sub-region within the decoded nine images.

[0056] FIG. 7 is a view showing a process of providing a high quality 360-degree VR image for a region of interest of a user in a low quality 360-degree VR image according to an embodiment of the present disclosure.

[0057] Referring to FIG. 7, a user may watch a low quality 360-degree VR image 710, and a region of interest 712 may be determined according to user motion or user selection. The high quality 360-degree VR image processing apparatus may perform mapping between the determined region of interest 712 and a region of interest 722 that is re-adjusted on a high quality sub-region stream. Meanwhile, according to a relation between the region of interest 712 and a sub-region of the high quality image, each process described in FIG. 5 or FIG. 6 may be performed. In addition, the high quality 360-degree VR image processing apparatus may transmit a result image obtained by performing mapping for the high quality re-adjusted region of interest 722 to a TV or a terminal such as HMD.

[0058] According to the present disclosure, in a broadcasting service environment, a view according to viewer motion or selected by the viewer is provided in a resolution of 4K or more, and a high quality 360-degree VR image processing apparatus capable of minimizing an MTP latency through an in-house TV or a mobile device such as HMD may be provided.

[0059] In addition, according to the present disclosure, a receiving apparatus and method based on a region of interest selected by a user and which provides signaling information for a 360-degree VR broadcasting service may be provided.

[0060] In addition, according to the present disclosure, an apparatus and method of minimizing an MTP latency while proving a region of interest of high quality according to user motion or user selection for high quality 360-degree VR media under a broadcasting network environment may be provided.

[0061] In addition, according to the present disclosure, an apparatus and method of providing a high quality 360-degree VR broadcasting service to a viewer by using various broadcasting networks such as ground TV, cable TV, etc. may be provided

[0062] The above embodiments may be performed in a high quality 360-degree VR image encoding apparatus and a high quality 360-degree VR image decoding apparatus in the same method.

[0063] Orders applying the above embodiments may vary in a high quality 360-degree VR image encoding apparatus and in a high quality 360-degree VR image decoding apparatus. Alternatively, the order applying the above embodiments may be identical in a high quality 360-degree VR image encoding apparatus and in a high quality 360-degree VR image decoding apparatus.

[0064] The above embodiments may be performed in a high quality 360-degree VR image encoding apparatus and in a high quality 360-degree VR image decoding apparatus in the same method.

[0065] For luma and chroma signals, the above may be respectively performed, and the embodiment may be identically performed for the luma and chroma signals.

[0066] In the above-described embodiments, the methods are described based on the flowcharts with a series of steps or units, but the present invention is not limited to the order of the steps, and rather, some steps may be performed simultaneously or in different order with other steps. It should be appreciated by one of ordinary skill in the art that the steps in the flowcharts do not exclude each other and that other steps may be added to the flowcharts or some of the steps may be deleted from the flowcharts without influencing the scope of the present invention.

[0067] Further, the above-described embodiments include various aspects of examples. Although all possible combinations to represent various aspects cannot be described, it may be appreciated by those skilled in the art that any other combination may be possible. Accordingly, the present invention includes all other changes, modifications, and variations belonging to the following claims.

[0068] The embodiments of the present invention can be implemented in a form of executable program command through a variety of computer means recordable to computer readable media. The computer readable media may include solely or in combination, program commands, data files and data structures. The program commands recorded to the media may be components specially designed for the present invention or may be usable to a skilled person in a field of computer software. Computer readable recording media includes magnetic media such as hard disk, floppy disk, magnetic tape, optical media such as CD-ROM and DVD, magneto-optical media such as optical disk and hardware devices such as ROM, RAM and flash memory specially designed to store and carry out programs. Program commands include not only a machine language code made by a complier but also a high level code that can be used by an interpreter etc., which is executed by a computer. The aforementioned hardware device can work as more than a software module to perform the action of the present invention and they can do the same in the opposite case.

[0069] While the invention has been shown and described with respect to the preferred embodiments, it will be understood by those skilled in the art that various changes and modification may be made without departing from the spirit and scope of the invention as defined in the following claims.

[0070] Accordingly, the thought of the present invention must not be confined to the explained embodiments, and the following patent claims as well as everything including variations equal or equivalent to the patent claims pertain to the category of the thought of the present invention.