Multi-Viewpoint Video Capturing Device

Abstract

An imaging lighting instrument for an operation is disposed in a space between an operator's head and a patient, thereby configuring a multi-viewpoint video capturing device for a surgical operation in which blocking of a light by the operator's head or body and appearance of the head in a video are avoided. The imaging lighting instrument includes a plurality of cameras and a plurality of lights attached to a hollow ring-shaped or arc-shaped housing made of a wire member of a finite length, which is devised so as not to interfere with the visual field or work of the operator. Furthermore, the multi-viewpoint video capturing device is configured to have a function of estimating the context of the plurality of cameras and make it possible to perform operation support/recording of a direct-view surgical operation by using a multi-viewpoint video by adding a video information processing function for selecting a camera video in which an operators' hand or surgical instruments less appear in an image.

Claims

1. A multi-viewpoint video capturing device comprising: an imaging instrument including a plurality of cameras attached to a circular ring-shaped or arc-shaped housing made of a wire member of a finite length and aimed toward a subject being a target for work performed by an operator; a fixing instrument configured such that the imaging instrument is disposed at a position between the operator's head and the subject; a camera calibration processing unit configured to estimate camera parameters including positions, postures, and focal lengths of the plurality of cameras attached to the imaging instrument, based on imaging information of the subject imaged by the plurality of cameras; and a presentation viewpoint gaze point determination processing unit configured to detect a gaze point, which is a region of the subject perceived by the operator, in multi-viewpoint image data, estimate three-dimensional coordinate information of the gaze point with reference to the camera parameters, and select one or a plurality of cameras that will perform imaging from a direction suitable for observation by the operator or an observer outside an operation field based on the three-dimensional coordinate information.

2. (canceled)

3. The multi-viewpoint video capturing device according to claim 1, wherein the imaging instrument includes a plurality of lights, and the plurality of lights are disposed between the plurality of cameras attached to the housing.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0043] FIG. 1 is a schematic view illustrating the state of use of a multi-viewpoint video capturing device according to an embodiment of the present invention, and is a diagram illustrating the state of use of an imaging lighting instrument when viewed from above.

[0044] FIG. 2 is a transparent side view, a bottom view, and a transparent perspective view of the imaging lighting instrument.

[0045] FIG. 3 is a simulation diagram of the state of use of a hollow ring type multi-viewpoint video capturing device in an operating room.

[0046] FIG. 4 is six diagrams illustrating the hollow ring type multi-viewpoint video capturing device.

[0047] FIG. 5 is a simulation diagram viewed from a surgeon's viewpoint when the hollow ring type multi-viewpoint video capturing device is used.

[0048] FIG. 6 is a diagram illustrating an example of a variation in installation (a polygonal shape, an arc shape) of the multi-viewpoint video capturing device.

[0049] FIG. 7 is a diagram illustrating an example of a variation in installation (a linear shape, a V-shape) of the multi-viewpoint video capturing device.

[0050] FIG. 8 is a diagram illustrating an example of a variation in installation (an arch type, a combined type) of the multi-viewpoint video capturing device.

[0051] FIG. 9 is a diagram illustrating an example of a variation having no lighting and installation of an imaging apparatus.

[0052] FIG. 10 is a diagram illustrating an example of a variation having no video capturing device and a lighting apparatus.

[0053] FIG. 11 is a block diagram illustrating a hardware configuration of the multi-viewpoint video capturing device according to the embodiment of the present invention.

[0054] FIG. 12 is a block diagram illustrating a software function of a video lighting processing device.

[0055] FIG. 13 is a flowchart illustrating a flow of processing of a presentation viewpoint gaze point determination processing unit.

[0056] FIG. 14 is a schematic view illustrating the state of use of a multi-viewpoint video capturing device according to a first modification example of the present invention.

[0057] FIG. 15 is a schematic view illustrating the state of use of a multi-viewpoint video capturing device according to a second modification example of the present invention.

[0058] FIG. 16 is a schematic view illustrating an LED light and a camera of an imaging lighting instrument according to a third modification example of the present invention.

[0059] FIG. 17 is a schematic view illustrating an LED light and a camera of an imaging lighting instrument, is a time chart illustrating light emission timings of a visible LED and an infrared LED, and is a block diagram illustrating a software function of an infrared video acquisition unit provided in an input/output control unit of a video lighting processing device according to a fourth modification example of the present invention.

[0060] FIG. 18 is a schematic view illustrating a variation of a suspension arm according to a fifth modification example of the present invention.

DESCRIPTION OF EMBODIMENTS

Multi-Viewpoint Video Capturing Device

[0061] FIG. 1A is a schematic view illustrating the state of use of a multi-viewpoint video capturing device 101 according to an embodiment of the present invention, and FIG. 1B is a diagram illustrating the state of use of an imaging lighting instrument 102.

[0062] As illustrated in FIG. 1A, the multi-viewpoint video capturing device 101 includes the imaging lighting instrument 102 and a video lighting processing device 103 connected to the imaging lighting instrument 102. Further, a large-screen external monitor 104 is connected to the video lighting processing device 103.

[0063] A patient 106 under anesthesia and with an incision in the abdomen is laid on an operating table 105. A surgeon 107a performs an operation on the patient 106, and a surgeon 107b assists the surgeon 107a. That is, the surgeon 107a is a surgeon performing the operation, and the surgeon 107b is an assistant. Hereinafter, the surgeon 107a and the assistant 107b are collectively described as a “surgeon 107”.

[0064] The horizontal width of the operating table 105 is approximately 50 cm. The dissected abdomen, that is, an operation field 106a, has an opening of approximately 20 cm on the short side and a depth of approximately 10 cm.

[0065] As illustrated in FIGS. 1A and 1B, the hollow ring-shaped imaging lighting instrument 102 is installed at a position approximately 40 cm apart from the center of the operation field 106a in an upward direction. In the imaging lighting instrument 102, a plurality of cameras (see FIG. 2B) and a plurality of lights (see FIG. 2B) are installed toward the operation field 106a in a circular or elliptical ring-shaped housing in the example of FIG. 1. As a result, the cameras and lights do not interfere with the fields of vision of the surgeons 107a and 107b when visually recognizing the operation field 106a.

[0066] The surgeon 107 peeps at the operation field 106a through a hollow portion of the imaging lighting instrument 102 and inserts his or her hand into a space of approximately 40 cm below the imaging lighting instrument 102 to perform operation work. In order to secure a space for performing such operation work, the imaging lighting instrument 102 is suspended from a structure (for example, a ceiling) above the operation field by, for example, a fixing instrument 102b and is disposed between the head of the surgeon and the operation field 106a.

[0067] The surgeon 107 can visually recognize the operation field 106a to peep at the operation field 106a through the hollow portion of the imaging lighting instrument 102, but can also perform an operation and operation assistance using a space between the imaging lighting instrument 102 and the operation field 106a while referring to a video of the operation field 106a displayed on the external monitor 104.

[0068] In a direct-view operation of the related art, an area 60 cm directly above an operation part is widely recognized in the field of surgical medical treatment as a sacred area that cannot be accessed, and it was an area in which there was hesitancy in developing and disposing new medical equipment. That is, operation monitoring in a direct-view operation has been performed entirely by viewing the operation field 106a with the naked eye of the surgeons 107a and 107b performing an operation, and thus the arrangement of any object at the area 60 cm directly above the operation part has been recognized as interference with the field of vision of the surgeon 107.

[0069] The imaging lighting instrument 102 used in the multi-viewpoint video capturing device 101 according to the embodiment of the present invention is an instrument disposed within the area 60 cm directly above the operation part which is a taboo region. However, the surgeon 107 can visually recognize an operation field to peep at the ring-shaped device installed in the hollow, while the surgeon 107 can perform operation work without a delay because a sufficient working space is secured between the present device and an affected area. In addition, it is possible to continue capturing a clear operation field image under a lighting environment which is not blocked by the head and shoulder of the surgeon 107.

[0070] A large number of cameras provided ion the imaging lighting instrument 102 are disposed surrounding the operation field 106a. Thus, it is possible to image the operation field 106a from various directions during an operation.

[0071] In the related art, in order to image the operation field 106a from various directions, the cameras are moved in necessary directions each time. In the case of the imaging lighting instrument 102 that constitutes a portion of the multi-viewpoint video capturing device 101 according to the embodiment of the present invention, a function of physically moving the individual cameras is not required. By electronically switching between videos obtained by the plurality of cameras previously disposed surrounding the operation field 106a as necessary, it is possible to easily acquire a video in an imaging direction required by the surgeon 107.

[0072] A video captured by the camera of the imaging lighting instrument 102 can be viewed on the external monitor 104 installed in the vicinity of the operating table 105. The surgeon 107 can view the operation field 106a with the naked eye and also check the operation field 106a on the large-screen external monitor 104.

[0073] Further, as indicated by the name, the multi-viewpoint video capturing device 101 can form a video from a free viewpoint based on videos obtained from the plurality of cameras. Thus, the operation field 106a which is a subject can be viewed in a three-dimensional manner by switching the cameras to move viewpoints.

Overall Image of Imaging Lighting Instrument 102

[0074] FIG. 2A is a transparent side view of the imaging lighting instrument 102, FIG. 2B is a bottom view of the imaging lighting instrument 102 when viewed from the bottom, and FIG. 2C is a transparent perspective view of the imaging lighting instrument 102.

[0075] The imaging lighting instrument 102 includes a housing 102a having a hollow circle or elliptical ring shape and the fixing instrument 102b. The fixing instrument 102b is provided in the imaging lighting instrument 102 in order to dispose and fix the housing 102a of the imaging lighting instrument 102 at a position between the operation field 106a, which is a subject, and the head of the surgeon 107 which is an operator.

[0076] As illustrated in FIGS. 2A, 2B, and 2C, the circular housing 102a has a size of, for example, 30 to 40 cm. Since the housing 102a has a hollow shape, the surgeon 107 can directly view the operation field 106a with the naked eye through the hollow portion of the housing 102a.

[0077] The thickness of the housing 102a is preferably as thin as possible so as not to interfere with the field of vision of the surgeon 107. Since an imaging element has been miniaturized in recent years, the housing 102a may be formed of a thin pipe, and the imaging element may be accommodated in the pipe. In addition, an imaging element of, for example, approximately 1 cm may be fixed to the outside of a pipe and fastened. In this configuration, the pipe has such a thickness that cannot accommodate the imaging element.

[0078] That is, the surgeon 107 can also view the operation field 106a through the external monitor 104, in addition to directly viewing the operation field 106a with the naked eye in a direct-view operation. The existence of a plurality of means for observing the operation field 106a leads to a psychological sense of relief for the surgeon 107 and contributes to an improvement in safety.

[0079] In order not to interfere with the field of vision of the surgeon 107, the ring-shaped housing 102a is preferably as thin as possible, and it is essential that the fixing instrument 102b does not intervene between the imaging lighting instrument 102 and the operation field 106a so as not to interfere with an operation and operation assistance.

[0080] As illustrated in FIGS. 2A and 2B, for example, 36 cameras 201 and 36 lights 202 are alternately disposed in the housing 102a. The cameras 201 and the lights 202 are disposed toward the operation field 106a as illustrated in FIG. 2C.

[0081] In a case where the shape of the housing 102a is a perfect circular shape, and the number of cameras 201 is 36, each of the cameras 201 captures a video so as to surround the operation field 106a within a range of 10 degrees.

[0082] That is, the cameras 201 are disposed in the housing 102a at predetermined intervals, and all of the cameras 201 are disposed toward the operation field 106a. For this reason, the cameras 201 are disposed in the housing 102a so as to have different imaging angles with respect to the operation field 106a which is a subject.

[0083] However, when the number of cameras 201 is large, the cameras 201 are disposed side by side without gaps between the cameras 201. Even in such a case, the cameras 201 are disposed in the housing 102a with different imaging angles with respect to the operation field 106a which is a subject.

[0084] As described above, the imaging lighting instrument 102 is disposed within the area 60 cm directly above the operation field 106a which is a subject. Thus, the cameras 201 disposed in the housing 102a are at distances extremely close to the subject, which leads to a state where the imaging angles of all of the cameras 201 are necessarily different from each other.

[0085] Although it is only required that the number of cameras 201 and the number of lights 202 are determined arbitrarily, it is preferable that 36 or more cameras 201 be ideally disposed to obtain a high-resolution multi-viewpoint video. When the angle between the cameras 201 is approximately 10 degrees, the surgeon 107, who is viewing a video captured by the camera 201, can get a view by smoothly switching between videos. When the angle between the cameras 201 is approximately 15 degrees or more, the surgeon 107, who is viewing a video captured by the camera 201, can clearly switch between videos but may feel uncomfortable when switching between videos is performed.

[0086] As the camera 201, it is desirable to use an ultra-small imaging element capable of capturing a high-definition color video, such as an imaging element that has been used in, for example, smartphones in recent years. Each camera 201 includes an interface such as a USB interface, and is connected to the video lighting processing device 103 illustrated in FIG. 1 via a USB hub 203.

[0087] As a light source used in the light 202, it is desirable to use, for example, a high-brightness LED that emits three primary colors with high color reproducibility. In general, LEDs have directivity, and thus it is desirable to appropriately disperse light toward the operation field 106a which is a subject by sealing the housing 102a with an acrylic plate or vinyl chloride plate that has been made opaque by smoke processing, sandblasting, or the like.

[0088] Note that the light source used in the light 202 is not limited to an LED, and for example, a krypton sphere may be adopted, or an LED and a krypton sphere may be mixed in order to improve color reproducibility.

[0089] FIG. 3 is a simulation diagram illustrating a state where the ring-shaped imaging lighting instrument 102, which is the basic type, is used in the operating room. The surgeon 107 observes the operation field by peeping at the operation field through a ring formed by the housing 102a of the imaging lighting instrument 102, and performs an operation using a working space of approximately 40 cm between the present apparatus and the operation field.

[0090] FIG. 4A is a top view of the imaging lighting instrument 102, FIG. 4B is a bottom view of the imaging lighting instrument 102, FIG. 4C is a front view of the imaging lighting instrument 102, FIG. 4D is a rear view of the imaging lighting instrument 102, and FIGS. 4E and 4F are side views of the imaging lighting instrument 102.

[0091] As is apparent from FIGS. 4A, 4B, 4D, 4E and 4F, the fixing instrument 102b is provided with a handle 401 for the surgeon 107 to dispose the imaging lighting instrument 102 at an appropriate position.

[0092] FIG. 5 is a diagram illustrating a simulation of the operation field when viewed from the viewpoint of the surgeon 107. Although the present device is partially visible on the edge of the field of vision, operation work can be performed comfortably enough because there are few parts blocking the operation field.

[0093] FIGS. 6, 7 and 8 illustrate modification mode examples of various shapes of the present device.

[0094] FIG. 6A is a simulation diagram in an operating room using an imaging lighting instrument 601 according to a modification mode example of a first shape, and FIG. 6B is a top view of the imaging lighting instrument 601 according to the modification mode example of the first shape.

[0095] The imaging lighting instrument 601 includes a housing 601a having a rectangular shape and a fixing instrument 601b. The fixing instrument 601b is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0096] FIG. 6C is a simulation diagram in an operating room using an imaging lighting instrument 602 according to a modification mode example of a second shape, and FIG. 6D is a top view of the imaging lighting instrument 602 according to the modification mode example of the second shape.

[0097] The imaging lighting instrument 602 includes a first housing 602a having an arc shape, a first fixing instrument 602b, a second housing 602c having an arc shape, and a second fixing instrument 602d. Each of the first fixing instrument 602b and the second fixing instrument 602d is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0098] FIG. 7A is a simulation diagram in an operating room using an imaging lighting instrument 701 according to a modification mode example of a third shape, and FIG. 7B is a top view of the imaging lighting instrument 701 according to the modification mode example of the third shape.

[0099] The imaging lighting instrument 701 includes a first housing 701a having a linear shape, a first fixing instrument 701b, a second housing 701c having a linear shape, and a second fixing instrument 701d. Each of the first fixing instrument 701b and the second fixing instrument 701d is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0100] FIG. 7C is a simulation diagram in an operating room using an imaging lighting instrument 702 according to a modification mode example of a fourth shape, and FIG. 7D is a top view of the imaging lighting instrument 702 according to the modification mode example of the fourth shape.

[0101] The imaging lighting instrument 702 includes a housing 702a having a V-shape and a fixing instrument 702b. The fixing instrument 702b is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0102] FIG. 8A is a simulation diagram in an operating room using an imaging lighting instrument 801 according to a modification mode example of a fifth shape, FIG. 8B is a top view of the imaging lighting instrument 801 according to the modification mode example of the fifth shape, and FIG. 8C is a diagram illustrating the imaging lighting instrument 801 according to the modification mode example of the fifth shape when viewed in the horizontal direction.

[0103] The imaging lighting instrument 801 includes a housing 801a having an arc shape in the vertical direction, and a fixing instrument 801b. The fixing instrument 801b is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0104] FIG. 8D is a simulation diagram in an operating room using an imaging lighting instrument 802 according to a modification mode example of a sixth shape, FIG. 8E is a top view of the imaging lighting instrument 802 according to the modification mode example of the sixth shape, and FIG. 8F is a diagram illustrating the imaging lighting instrument 802 according to the modification mode example of the sixth shape when viewed in the horizontal direction.

[0105] The imaging lighting instrument 802 includes a housing 802a and a fixing instrument 802b, the housing having an arc portion in the vertical direction and further having arc ends at both ends of the arc portion. The fixing instrument 801b is provided with a handle 401 in the same manner as the imaging lighting instrument 102 described above.

[0106] Also, in a polygonal shape represented by a square illustrated in FIGS. 6A and 6B, a sufficiently large number of combinations of cameras and lights can be disposed in the same manner as a ring shape which is a basic shape, and thus a multi-viewpoint video can be created.

[0107] A C-type×2 in FIGS. 6C and 6D and an I-type×2 in FIGS. 7A and 7B are devised to be able to secure a better field of vision and workability by eliminating a device in front of an operator and an assistant in order to further reduce an obstruction of a field of vision by using a 360-degrees ring type. On the other hand, it is undeniable that the quality of a multi-viewpoint video is likely to deteriorate because the number of cameras and lights that can be disposed is reduced in this format as a compensation for an improvement in a field of vision.

[0108] FIGS. 7C and 7D illustrate a V-type device disposed on one side. While the fields of vision and workability of an operator and an assistant are excellent, it is assumed that the quality of a multi-viewpoint video device will deteriorate because the number of combinations of cameras and lights is halved.

[0109] On the other hand, FIGS. 8A, 8B, and 8C illustrate modification mode examples in which a plurality of cameras and lights are vertically disposed in an arch shape in a narrow range between an operator and an assistant. It is possible to dispose a large number of cameras and lights in the vertical direction while maintaining the fields of vision and workability of the operator and the assistant.

[0110] Modification mode examples in FIGS. 8D, 8E and 8F show a combination of the C-type×2 illustrated in FIG. 6D and the arch type illustrated in FIGS. 8B and 8C. It is possible to maintain good fields of vision and workability of the operator and the assistant and dispose a sufficiently large number of cameras and lights in order to constitute a multi-viewpoint field-of-vision video.

[0111] FIG. 9A is a top view of an imaging instrument 901 as a variation having no lighting, and FIG. 9B is a schematic view illustrating the state of use of the imaging instrument 901.

[0112] The imaging instrument 901 is configured such that the lights 202 are removed from the imaging lighting instrument 102, and only the plurality of cameras 201 are incorporated into a housing 901a.

[0113] The imaging lighting instrument 102 according to the embodiment of the present invention has a basic structure in which both a plurality of cameras and a plurality of lights are alternately disposed. However, as illustrated in FIGS. 9A and 9B, it is also assumed that the imaging lighting instrument 102 is designed and used as the imaging instrument 901 using a plurality of cameras in a variation having no lighting. In this case, a shadowless light installed in an existing operating room can be diverted, and thus it is advantageous in terms of cost when equipment is introduced.

[0114] FIG. 10A is a top view of a lighting instrument 1001 as a variation having no camera, and FIG. 10B is a schematic view illustrating the state of use of the lighting instrument 1001.

[0115] The lighting instrument 1001 is configured such that the cameras 201 are removed from the imaging lighting instrument 102, and only the plurality of lights 202 are incorporated into a housing 1001a.

[0116] The imaging lighting instrument 102 according to the embodiment of the present invention has a basic structure in which both a plurality of cameras and a plurality of lights are, for example, alternately disposed. However, as illustrated in FIGS. 10A and 10B, a variation having only lights is also assumed. In the related art, light from a shadowless lamp installed above the head of the surgeon 107 is blocked by the head or body of the surgeon 107, and it is necessary to frequently adjust the position of the lights. However, the present device installed between the head of a surgeon and a patient is never blocked by the head or body of the surgeon and serves as a completely shadowless lighting device.

[0117] The imaging lighting instrument 102 of the present invention is formed in various shapes such as a shape of a straight line having a finite length or a shape of a curve having a finite length as illustrated in FIGS. 3, 4, 5, 6, 7, and 8 in order to minimize the interference with an operator's line of sight. The housing 102a is formed as thin as possible in accordance with the sizes of the plurality of cameras 201 and lights 202 accommodated in the housing 102a. As a result, the thickness of the housing 102a does not interfere with the operator.

[0118] Here, the curve is a concept including a straight line in the world of mathematics, and the straight line is a special example of a curve. Thus, it can be said that the imaging lighting instrument 102 in the embodiment of the present invention is formed by a housing made of a wire member of a finite length, including the linear imaging lighting instrument 102. That is, the shape of the imaging lighting instrument 102 of the present invention is not limited to those illustrated in FIGS. 3 4, 5, 6, 7, and 8, and includes all housing shapes for the purpose of not interfering with the field of vision and work of an operator.

[0119] The plurality of cameras and the plurality of lights accommodated in the housing made of a wire member of a finite length are all disposed toward the operation field 106a which is a subject.

[0120] As described above, the imaging lighting instrument 102 used in the multi-viewpoint video capturing device 101 of the present invention can have various shapes.

[0121] The housing 102a is preferably formed of a lightweight and highly rigid material such as an aluminum alloy, polycarbonate, or carbon. However, a configuration in which the cameras 201 and the lights 202 may be attached to a flexible pipe made of stainless steel or the like used in a water pipe or the like so as to be freely deformable in accordance with the form of the operation field 106a may be adopted. The housing can also be deformed into various shapes described above as long as it is a material such as a flexible pipe that can be freely deformed.

Multi-Viewpoint Video Capturing Device 101: Hardware Configuration

[0122] FIG. 11 is a block diagram illustrating a hardware configuration of the multi-viewpoint video capturing device 101 according to the embodiment of the present invention.

[0123] As illustrated in FIG. 11, the multi-viewpoint video capturing device 101 includes a camera array unit 1101, a lighting array unit 1102, and a video lighting processing device 103 connected thereto.

[0124] The camera array unit 1101 and the lighting array unit 1102 are built in the imaging lighting instrument 102.

[0125] Further, the large-screen external monitor 104 is connected to the video lighting processing device 103 through a network 1103.

[0126] The camera array unit 1101 is an assembly of the plurality of cameras 201 incorporated in the imaging lighting instrument 102 described with reference to FIGS. 1 and 2. The cameras 201 are connected to a serial interface 1128 such as a USB interface included in the video lighting processing device 103.

[0127] The lighting array unit 1102 is an assembly of a plurality of lights 202 incorporated in the imaging lighting instrument 102 described with reference to FIGS. 1 and 2. In the lights 202, the emission of the light 202 designated by the video lighting processing device 103 is selectively controlled through a multiplexer 1105 controlled by a microcomputer 1104. In addition, light emission and driving are performed by a current flowing from a power supply voltage node through a current limiting resistor R1106 and the multiplexer 1105. Note that, in the switching control of the multiplexer 1105, it is possible not only to select one light, but also to simultaneously select a plurality of lights including simultaneous turn-on of all of the lights as necessary.

[0128] The microcomputer 1104 is connected between the multiplexer 1105 and the video lighting processing device 103, and the microcomputer 1104 controls the multiplexer 1105 in response to a command of the video lighting processing device 103. The microcomputer 1104 is connected to the serial interface 1128 such as a USB interface included in the video lighting processing device 103.

[0129] The microcomputer 1104 includes a CPU 1108, a ROM 1109, and a RAM 1110 which are connected to a bus 1107, and a serial interface 1111 (abbreviated as “SI/F” in FIG. 11) such as a USB interface. The multiplexer 1105 is connected to the bus 1107 and controlled by the microcomputer 1104.

[0130] The video lighting processing device 103, which is a well-known computer, includes a CPU 1122, a ROM 1123, a RAM 1124, a display unit 1125, an operation unit 1126, and a non-volatile storage 1127 which are connected to a bus 1121.

[0131] The CPU 1122 reads a software program for implementing functions of units included in the video lighting processing device 103 from the non-volatile storage 1127 or the ROM 1123 and executes the program.

[0132] Videos captured by all of the cameras 201 and received from the camera array unit 1101, variables generated during arithmetic processing performed in video lighting processing device 103, and the like are temporarily written to the RAM 1124. The CPU 1122 executes programs recorded in the non-volatile storage 1127 or the ROM 1123, and thus various functions of the video lighting processing device 103 are implemented.

[0133] In addition to the above-described units, the serial interface 1128 such as a USB and a network interface card (NIC) 1129 are connected to the bus 1121 as communication interfaces.

[0134] The external monitor 104 is connected to the NIC 1129 through the network 1103. The external monitor 104 includes a well-known computer, and the computer operates a network OS and functions as the external monitor 104 of the video lighting processing device 103.

[0135] Although the display unit 1125 of the video lighting processing device 103 may be used as a monitor as it is, there is generally a restriction on the length of a connection cable between the display unit 1125 and the video lighting processing device 103, and thus it is preferable to connect the external monitor 104 through the network 1103.

[0136] In addition, when the video lighting processing device 103 is constructed as a network video server, a plurality of external monitors 104 can be easily connected, and thus it is possible to display the same desired video on different monitors at the same time. Thus, even when the external monitor 104 breaks down due to some accident, it is possible to prevent an accident of an operation in advance by operating a preliminary external monitor 104 as it is.

Video Lighting Processing Device 103: Software Function

[0137] FIG. 12 is a block diagram illustrating a software function of the video lighting processing device 103.

[0138] The video lighting processing device 103 includes an input/output control unit 1201, a camera calibration processing unit 1202, a presentation viewpoint gaze point determination processing unit 1204, a video processing unit 1205, a lighting determination processing unit 1206, and a display unit 1125.

[0139] The camera array unit 1101 is constituted by a large number of cameras 1 to N embedded in the imaging lighting instrument 102 and disposed surrounding the periphery of the operation field 106a, and images the operation field 106a from various directions at the time of an operation.

[0140] The lighting array unit 1102 is constituted by a large number of LED lights 1 to N usually equal to the number of cameras 201 in the camera array unit 1101, the LED lights being embedded in the imaging lighting instrument 102 together with the cameras 201 and disposed surrounding the periphery of the operation field 106a. In addition, the light emission of the light 202 adjacent to the camera 201 selected by the presentation viewpoint gaze point determination processing unit 1204 is controlled. Further, the light emission of all of the lights is simultaneously controlled as necessary.

[0141] The input/output control unit 1201 receives video data obtained from the camera array unit 1101 and also gives control information for switching the lights 1 to N to the lighting array unit 1102.

[0142] The input/output control unit 1201 and the camera array unit 1101 are connected by a bidirectional serial interface 1128.

[0143] All of the cameras 201 in the camera array unit 1101 transmit video data to the input/output control unit 1201.

[0144] The input/output control unit 1201 and the lighting array unit 1102 are connected via a USB interface. When the input/output control unit 1201 receives a turn-on command from the lighting determination processing unit 1206 to be described later, the input/output control unit 1201 transmits a light emission command for the light 202 designated by the lighting determination processing unit 1206 in response to the turn-on command.

[0145] The microcomputer 1104 having received the light emission command from the input/output control unit 1201 in the lighting array unit 1102 controls the multiplexer 1105 so as to control the light emission of the designated light 202.

[0146] The input/output control unit 1201 and the camera calibration processing unit 1202 are connected by a serial interface. The camera calibration processing unit 1202 estimates camera parameters such as the positions, postures, and focal lengths of the cameras 1 to N constituting the camera array unit 1101 on the basis of video information.

[0147] The camera 201 embedded in the imaging lighting instrument 102 is configured with an angle of view at which a common region is observed in videos obtained by the adjacent cameras 201. In addition, camera parameters such as the position, posture, and focal length of each of the cameras 201 with respect to the operation field 106a, which is a subject, are estimated from a correspondence relationship between common observation regions.

[0148] In a surgical operation, if an operator is right-handed, for example, the operator often holds a surgical instrument such as an electric scalpel and scissors in the right hand and supports work with forceps or his or her fingers in the left hand to perform operation work. Thus, the location of a subject being a target for work to be noted by the surgeon 107 is within a range of approximately 0 to 5 cm beyond the following two locations. [0149] (1) The tip of a surgical instrument such as a scalpel, forceps, or scissors [0150] (2) The tip of the surgeon's finger that is not holding a surgical instrument

[0151] Partial image data and a procedure of setting an observation position for each article will be described below.

(1) Case of Surgical Instrument Such as a Scalpel, Forceps, or Scissors

[0152] A surgical instrument such as a scalpel, forceps, or scissors is imaged by the camera array unit 1101 in advance immediately before an operation, or the like. Video data obtained from the camera array unit 1101 is recorded in the RAM 1124 of the input/output control unit 1201 as multi-viewpoint video data.

[0153] Next, a staff member such as a nurse other than the surgeon 107 selects the tip of a surgical instrument such as a scalpel or the like shown in the multi-viewpoint video data using a mouse or the like which is the operation unit 1126. In addition, the staff member such as a nurse also designates two-dimensional coordinates that serve as an observation position on a screen at the time of performing an operation of selecting a surgical instrument such as a scalpel. For example, a range of approximately 0 to 5 cm beyond the tip of a surgical instrument such as a scalpel or forceps is designated as observation position coordinate information.

[0154] The partial image data and the observation position coordinate information of surgical instruments such as scalpels, forceps, and scissors, which are extracted from the multi-viewpoint video data, are stored in the RAM 1124 or the non-volatile storage 1127 as a gaze point subject video and gaze point coordinates 1207.

(2) Case of Surgeon's Finger

[0155] Prior to an operation, an operator performs several shots by the camera array unit 1101 with his or her finger without holding a surgical instrument such as a scalpel. At the time of the imaging, the operator performs imaging while performing various techniques. Video data obtained from the camera array unit 1101 is recorded in the RAM 1124 of the input/output control unit 1201 as multi-viewpoint video data. Next, a staff member such as a nurse other than the surgeon 107 selects the tip of the operator's finger shown in the multi-viewpoint video data using a mouse or the like which is the operation unit 1126. In addition, the staff member such as a nurse also designates two-dimensional coordinates that serve as an observation position on a screen at the time of performing a selection operation by the operator' hand. For example, in a case where the surgeon is right-handed, a range of approximately 0 to 5 cm from the tip of an index finger of the left hand is designated as observation position coordinate information.

[0156] The extracted partial image data and the observation position coordinate information of the operator's hand are stored in the RAM 1124 or the non-volatile storage 1127 as the gaze point subject video and the gaze point coordinates 1207.

[0157] The reason why a plurality of pieces of multi-viewpoint video data of an operator's finger are prepared is because the finger freely changes the shape thereof, and thus it is necessary to prepare a plurality of pieces of multi-viewpoint video data as the gaze point subject video and the gaze point coordinates 1207 in order to increase the probability of image matching.

[0158] Note that the gaze point refers to a three-dimensional point which is estimated by applying triangulation based on camera parameters to a representative point (two-dimensional point) of a partial region of a subject to be noted by the operator in multi-viewpoint image data obtained from the camera array unit 1101.

[0159] The input/output control unit 1201 and the presentation viewpoint gaze point determination processing unit 1204 are connected by a serial interface. The presentation viewpoint gaze point determination processing unit 1204 detects the location of a subject that the surgeon 107 desires to view from videos received from the input/output control unit 1201 by all of the cameras 201 with reference to the gaze point subject video and the gaze point coordinates 1207, and estimates two-dimensional position coordinates thereof.

[0160] Then, the presentation viewpoint gaze point determination processing unit 1204 further determines a presentation viewpoint which is a viewpoint from which the gaze point can be most appropriately imaged. In the presentation viewpoint determination processing performed by the presentation viewpoint gaze point determination processing unit 1204, the camera 201 that performs imaging from a direction suitable for the observation by the surgeon 107 is selected from among the cameras 1 to N constituting the camera array unit 1101 in consideration of the hand of the surgeon 107 and surgical instruments which are shown in a captured image.

[0161] The camera calibration processing unit 1202 and the presentation viewpoint gaze point determination processing unit 1204 are connected via a serial interface. The presentation viewpoint gaze point determination processing unit 1204 calculate the location of a subject that the assistant 107b of the surgeon 107a desires to view, that is, three-dimensional coordinates of a gaze point, on the basis of stereo vision from camera parameters such as the position, posture, and a focal length of the camera 201 which are received from the camera calibration processing unit 1202, and the estimated two-dimensional positional coordinates.

[0162] When the three-dimensional coordinate information of the gaze point is determined, the presentation viewpoint gaze point determination processing unit 1204 then specifies the camera 201 that clearly images the gaze point.

[0163] The above-described processing is presentation viewpoint determination processing performed by the presentation viewpoint gaze point determination processing unit 1204.

[0164] Basically, the lighting determination processing unit 1206 generates information for controlling the turn-on of the lights 202 disposed on both sides of the camera 201 which is specified as being a camera that clearly images a gaze point by the presentation viewpoint gaze point determination processing unit 1204. Note that, in a case where a specific camera 201 is not designated, the turn-on of all of the lights lighting is controlled.

[0165] The video processing unit 1205 is an interface for outputting the camera 201 that clearly images a gaze point specified by the presentation viewpoint gaze point determination processing unit 1204 to the display unit 1125, the external monitor 104, or the like. The video processing unit 1205 supplies a multi-viewpoint video to the display unit 1125 provided in the operating room and also provides the multi-viewpoint video to the external monitor 104a via the network 1103.

[0166] The external monitor 104a can display videos of the plurality of cameras 201 at the same time. Among them, for example, one camera video that is closest to the line of sight of the surgeon 107a and captures a clear video that is not interfered with by hands of a surgeon and an assistant, and surgical instruments is enlarged and displayed as a main image and can be shared by all participating members. In addition, an external monitor 104b and an external monitor 104c serve as display devices that can present image information other than a direct-view video to an operator. For example, the external monitor 104b presents an image, which is obtained from an angle different from the line of sight of the surgeon 107a, to the surgeon 107a, and the external monitor 104c presents image information other than white light which is captured by near infrared rays.

[0167] Note that, although not particularly illustrated in the drawing, in order for a viewer or the surgeon 107 to view a multi-viewpoint video intuitively or with a simple operation, a function of manually selecting the camera 201 using an operation unit may be added to the presentation viewpoint gaze point determination processing unit 1204 and/or the video processing unit 1205.

Flow of Processing of Presentation Viewpoint Gaze Point Determination Processing Unit 1204

[0168] FIG. 13 is a flowchart illustrating a flow of processing of the presentation viewpoint gaze point determination processing unit 1204.

[0169] When the processing is started (S1301), the presentation viewpoint gaze point determination processing unit 1204 acquires three-dimensional coordinate information of the positions of the cameras 201, the volume of a person region (mainly the position of the hand of the surgeon 107), the position of a gaze point, and the position of an instrument region, on the basis of a multi-viewpoint video and camera parameters received from the input/output control unit 1201 and the camera calibration processing unit 1202 (S1302).

[0170] Next, the presentation viewpoint gaze point determination processing unit 1204 determines whether or not the hand of the surgeon 107 is present between the position of the camera and the position of the gaze point, on the basis of the three-dimensional coordinate information of the gaze point (S1303).

[0171] In a case where it is determined in step S1303 that the hand of the surgeon 107 is present between the position of the camera and the position of the gaze point (YES in S1303), a camera in which the hand of the surgeon 107 disappears between the position of the camera and the position of the gaze point is detected.

[0172] In a case where it is determined in step S1303 that the hand of the surgeon 107 is not present between the position of the camera and the position of the gaze point (NO in S1303), the presentation viewpoint gaze point determination processing unit 1204 subsequently determines whether or not a surgical instrument is present between the position of the camera and the position of the gaze point, on the basis of the three-dimensional coordinate information of the gaze point (S1304).

[0173] In a case where the presentation viewpoint gaze point determination processing unit 1204 determines in step S1304 that a surgical instrument is present between the position of the camera and the position of the gaze point (YES in S1304), a camera in which a surgical instrument is not present between the position of the camera and the position of the gaze point is detected.

[0174] In a case where the presentation viewpoint gaze point determination processing unit 1204 determines in step S1304 that a surgical instrument is not present between the position of the camera and the position of the gaze point (NO in S1304), address information of the camera used and a zoom value among the cameras 1 to N are output (S1306), and the series of processes is terminated (S1307).

[0175] The above-described multi-viewpoint video capturing device 101 according to the embodiment of the present invention presents live magnification information from a viewpoint different from the naked eye to the surgeon 107 in addition to a direct-view manner with the naked eye, for operations that have been performed so far only in a direct-view manner with the naked eye.

[0176] The imaging lighting instrument 102 having the cameras 201 and the lights 202 mounted thereon is positioned between the surgeon 107 and the patient 106, and light of a lighting instrument is not blocked by the heads of the surgeons 107a and 107b, unlike a surgical operation lighting instrument of the related art. Thus, the operation field 106a and the vicinity of the gaze point are always brightly illuminated by the light 202, and the visibility is significantly improved.

[0177] Further, the operation field 106a and the vicinity of the gaze point, which are enlarged and displayed by the multi-viewpoint video capturing device 101, are multi-viewpoint videos, and thus it is possible to three-dimensionally confirm the operation field 106a and the vicinity of the gaze point by moving the viewpoint.

[0178] In particular, the presentation viewpoint gaze point determination processing unit 1204 specifies three-dimensional coordinate information of a gaze point in a multi-viewpoint video being captured from the three-dimensional coordinate information of the gaze point stored in advance in the gaze point subject video and the gaze point coordinates 1207, and thus switching of an appropriate viewpoint is realized centering on an object to be noted (for example, the tip of a scalpel).

[0179] As a result, the object to be noted (for example, the tip of a scalpel) is observed at all times at the same location on the screen, which facilitates observation of the object to be noted.

[0180] The multi-viewpoint video capturing device 101 according to the embodiment of the present invention can significantly improve the certainty and safety of an operation in accordance with the above-described features.

[0181] Note that, even when the presentation viewpoint gaze point determination processing unit 1204, the lighting determination processing unit 1206, and the gaze point subject video and the gaze point coordinates 1207 are not provided, it is possible to move a presentation viewpoint by a manual operation through the operation unit 1126. However, in this case, since three-dimensional coordinate information of the gaze point is unclear, the viewpoint is switched to center on an object other than the object to be noted (for example, the tip of the scalpel). As a result, the observation position of the object to be noted (for example, the tip of the scalpel) moves on the screen as the viewpoint moves.

First Modification Example: Imaging Lighting Instrument 102 with Ring Directly Placed on Operation Field 106a of Patient 106

[0182] FIG. 14 is a schematic view illustrating the state of use of a multi-viewpoint video capturing device 1401 according to a first modification example of the present invention.

[0183] The multi-viewpoint video capturing device 1401 illustrated in FIG. 14 is different from the multi-viewpoint video capturing device 101 illustrated in FIG. 1 in that an imaging lighting instrument 1402 has been miniaturized and placed directly on the operation field 106a of the patient 106.

[0184] As described above, in the imaging lighting instrument 102 illustrated in FIG. 1, the hollow ring having the camera array unit 1101 and the lighting array unit 1102 arranged therein is attached to a frame disposed between the patient 106 and the surgeon 107, for example, at a position approximately 30 cm away from the patient 106. Thereby, the surgeon 107 can perform an operation while visually observing the operation field 106a.

[0185] Also in the example of FIG. 1, the head of the surgeon 107 an generate a shadow by the lights to eliminate the imperfections of imaging, but the problem is that a frame having the hollow ring attached thereto may interfere with the field of vision of the surgeon 107.

[0186] Consequently, in the multi-viewpoint video capturing device 1401 according to the first modification example of the present invention illustrated in FIG. 14, a hollow ring in which the camera array unit 1101 and the lighting array unit 1102 are arranged is installed covering the peripheral edge of the operation field 106a of the patient 106. Also in this case, the surgeon 107 can perform an operation while visually observing the operation field. With this operative method, an operation can be performed in the same state as a normal surgical operation, and thus the surgeon 107 has less hindrance to work as compared with the imaging lighting instrument 102 in FIG. 1. However, since the imaging lighting instrument 1402 comes into contact with the patient 106, there is a possibility that the hands of the surgeons 107a and 107b and surgical instruments will generate shadows by the lights.

Second Modification Example: Imaging Lighting Instrument 102 with Ring Directly Inserted into Operation Field 106a of Patient 106

[0187] FIG. 15 is a schematic view illustrating the state of use of a multi-viewpoint video capturing device 1501 according to a second modification example of the present invention.

[0188] The multi-viewpoint video capturing device 1501 illustrated in FIG. 15 is different from the multi-viewpoint video capturing device 101 illustrated in FIG. 1 and the multi-viewpoint video capturing device 1401 illustrated in FIG. 14 in that an imaging lighting instrument 1502 is smaller than the imaging lighting instrument 1402 according to the first modification example of FIG. 14 and is directly inserted into the operation field 106a of the patient 106.

[0189] The imaging lighting instrument 1502 illustrated in FIG. 15 is an example in which a hollow ring in which the camera array unit 1101 and the lighting array unit 1102 are arranged is inserted into the abdominal cavity of the patient 106 and installed to look down on an operation part from above the abdominal cavity. According to this operative method, the surgeons 107a and 107b can perform an operation while viewing a captured video, and the cameras 201 can image the inside of the abdominal cavity without omission.

[0190] However, the hollow ring in which the cameras 201 and the lighting array unit 1102 are disposed has to be inserted into and removed from an insertion port provided in the abdomen of the patient 106.

[0191] For example, the camera array unit 1101 and the lighting array unit 1102, which are arranged in a line and form a string shape, need to be inserted into the abdominal cavity through the port and deformed into a circle in the abdominal cavity. In addition, after the operation after the imaging is terminated, a mechanism for deforming the camera array unit 1101 and the lighting array unit 1102 into a string shape again and extracting them through the port is required.

Third Modification Example: Imaging Lighting Instrument 1601 for Simultaneously Imaging Blood Vessels in Operation Field 106a of Patient 106 Using Infrared LED 1605 and Cold Mirror 1606 or Hot Mirror

[0192] When a video of blood vessels is displayed on the external monitor 104 together with a visible light video during a surgical operation, it is possible to prevent the occurrence of a medical accident that damages blood vessels and expect a further improvement in safety.

[0193] FIG. 16 is a schematic view illustrating an imaging lighting instrument 1601 according to a third modification example of the present invention, and an LED light 1602 and a camera unit 1603 which are incorporated in the imaging lighting instrument 1601.

[0194] The LED light 1602 includes a visible LED 1604 and an infrared LED 1605. The visible LED 1604 and the infrared LED 1605 continuously emit light at the same time.

[0195] The camera unit 1603 includes a cold mirror 1606 obliquely installed at an angle of 45 degrees on the optical axis, a first camera 1007 installed at a position passing through the cold mirror 1606 on the optical axis, and a second camera 1008 installed at a position orthogonal to the cold mirror 1606 on the optical axis.

[0196] The cold mirror 1606 transmits near-infrared light and reflects visible light. Thus, the first camera 1007 captures an infrared video, and the second camera 1008 captures a visible light video.

[0197] In a case where a hot mirror that transmits visible light and reflects near-infrared light is installed instead of the cold mirror 1606, the first camera 1007 can capture a visible light video, and the second camera 1008 can capture an infrared video.

[0198] In either case, a visible light video and an infrared video can be captured simultaneously from incident light, and thus it is possible to display the visible light video and the infrared video on the display unit 1125 and the external monitor 104 at the same time.

Fourth Modification Example: Multi-Viewpoint Video Capturing Device that Images Blood Vessels in Operation Field 106a of Patient 106 in Time Division Manner Using Infrared LED 1605

[0199] In the third modification example described above, the camera unit 1603 needs to include the cold mirror 1606 or the hot mirror and two cameras 201 therein. There are not many such special products on the market, and it is expensive to make them from ready-made products. However, at the expense of time resolution, it is possible to capture a visible light video and an infrared video at substantially the same time without using a camera having such a special structure.

[0200] FIG. 17A is a schematic view illustrating the LED light 1602 and the camera 201 of the imaging lighting instrument according to a fourth modification example of the present invention, and FIG. 17B is a time chart illustrating light emission timings of the visible LED 1604 and the infrared LED 1605. In addition, FIG. 17C is a block diagram illustrating software functions of an infrared video acquisition unit 1702 provided in the input/output control unit 1201 of the video lighting processing device 103.

[0201] As illustrated in FIG. 17A, the camera 201 is a general imaging element and does not have a special structure as in the third modification example. Instead, the camera 201 outputs a frame timing pulse. The frame timing pulse is input to a light emission timing control unit 1701.

[0202] The light emission timing control unit 1701 synchronizes the light emission timings of the visible LED 1604 and the infrared LED 1605 with a frame timing pulse included in video data output by the camera 201 to control light emission of the visible LED 1604 and the infrared LED 1605 at a timing illustrated in FIG. 17B.

[0203] That is, the light emission timing control unit 1701 turn on only the visible LED 1604 at a first frame timing t1711, turns on only the infrared LED 1605 at a second frame timing t1712, and turns off both the visible LED 1604 and the infrared LED 1605 at a third frame timing t1713. This operation is repeated.

[0204] In addition, infrared LED turn-on captured video data obtained at the second frame timing and infrared LED turn-off captured video data obtained at the third frame timing are input to the infrared video acquisition unit 1702 (see FIG. 17C) formed in the input/output control unit 1201 of the video lighting processing device 103.

[0205] In addition, as illustrated in FIG. 17C, the infrared video acquisition unit 1702 includes an infrared LED turn-on captured video buffer 1703 that holds infrared LED turn-on captured video data, an infrared LED turn-off captured video buffer 1704 that holds infrared LED turn-off captured video data, and an adder 1705.

[0206] The infrared LED turn-on captured video data output from the infrared LED turn-on captured video buffer 1703 and the infrared LED turn-off captured video data output from the infrared LED turn-off captured video buffer 1704 are input to the adder 1705.

[0207] The adder 1705 subtracts the infrared LED turn-off captured video data from the infrared LED turn-on captured video data for each pixel. Then, the infrared LED turn-off captured video data, that is, a noise component due to visible light, which is included in the infrared LED turn-on captured video data is subtracted.

[0208] With the above configuration, although a frame rate is reduced to ⅓ as compared with the third modification example, an infrared image can be acquired by using the general-purpose camera 201 as it is. Thus, as in the third modification example, it is possible to prevent the occurrence of a medical accident that damages blood vessels in a surgical operation and expect an improvement in safety.

[0209] Further, by increasing the type of light that emits not only infrared light but also light having only a specific wavelength spectrum and performing the processing illustrated in FIGS. 17A, 17B, and 17C, it is possible to acquire video data having the specific wavelength spectrum.

[0210] As the video lighting processing device 103, a general personal computer can be used as it is. Thus, not only a multi-viewpoint video based on visible light of the operation field 106a and a multi-viewpoint video based on infrared light which displays blood vessels, but also information output by an external apparatus can be displayed on the display unit 1125 and the external monitor 104.

Fifth Modification Example: Variation of Fixing Instrument 102b

[0211] FIG. 18 is a schematic view illustrating a variation of the fixing instrument 102b according to a fifth modification example of the present invention.

[0212] The fixing instrument 102b illustrated in FIG. 1 is provided to maintain a state where the imaging lighting instrument 102 is suspended from the operation field 106a.

[0213] The fixing instrument 102b is configured to be suspended from the ceiling of an operating room (not illustrated). As a result, the fixing instrument 102b does not interfere with medical treatment by the surgeon 107.

[0214] When a technical idea that does not interfere with medical treatment by the surgeon 107 is disregarded, as illustrated in FIG. 18, legs 1801a and 1801b extending directly below the imaging lighting instrument 102 and an extension arm 1802 extending directly beside the imaging lighting instrument 102 may also be provided. Further, an extension arm 1803 extending obliquely above the imaging lighting instrument 102 can also be provided. In this manner, various variations are conceivable as a method of a fixing instrument for maintaining a state where the imaging lighting instrument 102 is suspended from the operation field 106a.

[0215] Instruments required by the surgeon 107 in a direct-view operation in the related art include three instruments, that is, a shadowless lamp, an operation part imaging camera, and an information display device. These have traditionally existed separately.

[0216] An object of the multi-viewpoint video capturing device 101 according to the present invention is to provide various information essential for an operation to the surgeon 107 in an integrated manner, like a cockpit of an airplane.

[0217] For example, it is possible not only to share effective video information with a plurality of medical professionals in real time, but also to display preoperative simulation information such as CT images, MRI images, and 3D reconstructed images before an operation, navigation information such as appropriate blood vessel running of a main part based on infrared images, and the like on the external monitor 104 in an appropriate layout in accordance with the progress of the operation.

[0218] In addition, vital information of the patient 106 which is acquired in real time such as the blood pressure, pulse, electrocardiogram, and electroencephalogram of the patient 106, medical safety information such as an operative method and operation members, and medical cost information such as equipment to be used may be displayed as shared information.

[0219] In recent years, social demands for medical safety have increased significantly, and it is expected that the multi-viewpoint video capturing device 101 according to the embodiment of the present invention plays a major role in enhancing the safety of an operation.

[0220] For operations that have been performed so far only in a direct-view manner with the naked eye, various information such as live magnification information, infrared information, preoperative radiation image information, 3D simulation information, and medical safety information can be displayed to the surgeon 107 to significantly improve the safety of an operation.

[0221] In addition, by appropriately archive-recording enlarged live information, it is possible to contribute to the education of students and young surgeons and store it as a record that can withstand the disclosure of medical information.

[0222] In addition, although a shadowless lamp is an indispensable equipment in all operating rooms, the camera device with a light used in the embodiment of the present invention can serve as a more compact shadowless lamp. As a result, using the camera device with a light used in the present invention eliminates the need to install a large-scale shadowless lamp in an operating room, allowing the camera device to become a core facility in an operating room, of which there are approximately 20,000 throughout Japan.

[0223] As described above, the multi-viewpoint video capturing device 101 according to the embodiment of the present invention can be regarded as a completely new operation support system, and the multi-viewpoint video capturing device 101 may also have the potential to completely change the state of operating rooms around the world.

[0224] As described above, the present multi-viewpoint video capturing device is intended to be used mainly for surgical operations on living bodies, but the features of the present device that realizes a clear lighting environment in which blocking by the head or body of an operator does not occur, and operation filed video capturing that does not cause blocking cab be effectively applied in the fields of pathological anatomy, judicial anatomy, surgical anatomy, educational anatomy for corpses and also in veterinary surgery for animals.

[0225] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment and also includes other modification examples and application examples without departing from the gist of the present invention described in the claims.

[0226] For example, in the above-described embodiment, the configuration of the device and the system are described in detail and specifically in order to facilitate the understanding of the present invention, and it is not necessarily limited to including all of the configurations described.

[0227] Further, it is possible to replace a portion of a configuration of a certain embodiment with a configuration of another embodiment and to add a configuration of another embodiment to a configuration of a certain embodiment. In addition, it is also possible to perform addition, deletion, or replacement of another configuration for a portion of a configuration of each embodiment.

REFERENCE SIGNS LIST

[0228] 101 Multi-viewpoint video capturing device [0229] 102 Imaging lighting instrument [0230] 102a Housing [0231] 102b Fixing instrument [0232] 103 Video lighting processing device [0233] 104 External monitor [0234] 105 Operating table [0235] 106 Patient [0236] 106a Operation field [0237] 107 Surgeon [0238] 201 Camera [0239] 202 Light [0240] 203 USB hub [0241] 401 Handle [0242] 601 Imaging lighting instrument [0243] 601a Housing [0244] 601b Fixing instrument [0245] 602 Imaging lighting instrument [0246] 602a First housing [0247] 602b First fixing instrument [0248] 602c Second housing [0249] 602d Second fixing instrument [0250] 701 Imaging lighting instrument [0251] 701a First housing [0252] 701b First fixing instrument [0253] 701c Second housing [0254] 701d Second fixing instrument [0255] 702 Imaging lighting instrument [0256] 702a Housing [0257] 702b Fixing instrument [0258] 801 Imaging lighting instrument [0259] 801a Housing [0260] 801b Fixing instrument [0261] 802 Imaging lighting instrument [0262] 802a Housing [0263] 802b Fixing instrument [0264] 901 Imaging instrument [0265] 901a Housing [0266] 1001 Lighting instrument [0267] 1001a Housing [0268] 1007 First camera [0269] 1008 Second camera [0270] 1101 Camera array unit [0271] 1102 Lighting array unit [0272] 1103 Network [0273] 1104 Microcomputer [0274] 1105 Multiplexer [0275] R1106 Current limiting resistor [0276] 1107 Bus [0277] 1108 CPU [0278] 1109 ROM [0279] 1110 RAM [0280] 1111 Serial interface [0281] 1121 Bus [0282] 1122 CPU [0283] 1123 ROM [0284] 1124 RAM [0285] 1125 Display unit [0286] 1126 Operation unit [0287] 1127 Non-volatile storage [0288] 1128 Serial interface [0289] 1129 NIC [0290] 1201 Input/output control unit [0291] 1202 Camera calibration processing unit [0292] 1204 Presentation viewpoint gaze point determination processing unit [0293] 1205 Video processing unit [0294] 1206 Lighting determination processing unit [0295] 1207 Gaze point coordinates [0296] 1401 Multi-viewpoint video capturing device [0297] 1402 Imaging lighting instrument [0298] 1501 Multi-viewpoint video capturing device [0299] 1502 Imaging lighting instrument [0300] 1601 Imaging lighting instrument [0301] 1602 LED light [0302] 1603 Camera unit [0303] 1604 Visible LED [0304] 1605 Infrared LED [0305] 1606 Cold mirror [0306] 1701 Light emission timing control unit [0307] 1702 Infrared video acquisition unit [0308] 1703 Infrared LED turn-on captured video buffer [0309] 1704 Infrared LED turn-off captured video buffer [0310] 1705 Adder [0311] 1801a Leg [0312] 1802, 1803 Extension arm