DIRECTIONALITY CONTROL SYSTEM, CALIBRATION METHOD, HORIZONTAL DEVIATION ANGLE COMPUTATION METHOD, AND DIRECTIONALITY CONTROL METHOD

Abstract

A directionality control system includes: a camera; a microphone provided as a separate body from the camera; a display that displays video data captured by the camera; and a processor that computes a sound collection direction, which is directed from the microphone toward a sound position corresponding to a designated position in the video data. The processor computes the sound collection direction by using parameters including: a first height of the camera from a reference surface, a second height of the microphone from the reference surface, a third height of a computation reference point from the reference surface, the computation reference point being positioned in the sound collection direction at a position different from the sound position, a direction which is directed from the camera toward the sound position, and a fourth height of the sound position from the reference surface.

Claims

1. A directionality control system comprising; an imaging device that captures a video; a sound collector that collects sound, and being provided as a separate body from the imaging device; a display that displays video data captured by the imaging device; and a processor that computes a sound collection direction, which is directed from the sound collector toward a sound position corresponding to a designated position in the video data in response to a designation of a position in the video data displayed on the display, wherein the processor computes the sound collection direction by using parameters including; a first height of the imaging device from a reference surface, a second height of the sound collector from the reference surface, a third height of a computation reference point from the reference surface, the computation reference point being positioned in the sound collection direction at a position different from the sound position corresponding to the designated position in the video data, a direction which is directed from the imaging device toward the sound position corresponding to the designated position in the video data, and a fourth height of the sound position corresponding to the designated position in the video data from the reference surface.

2. The directionality control system according to claim 1, wherein; the processor forms a sound collection directionality of the sound collected by the sound collector in the computed sound collection direction.

3. The directionality control system according to claim 1, wherein the computation reference point is provided on an optical axis of the imaging device, and the parameters further includes; a horizontal component distance between the imaging device and the sound collector; a distance and a depression angle from the imaging device to the computation reference point; and a distance and a depression angle from the sound collector to the computation reference point.

4. The directionality control system according to claim 1, wherein the computation reference point is provided on an optical axis of the imaging device, and the parameters further includes: a horizontal component distance between the imaging device and the sound collector; a distance from the imaging device to the computation reference point; and a horizontal angle and a vertical angle from the sound collector to the computation reference point.

5. The directionality control system according to claim 1, wherein the processor uses a value selected from a plurality of predetermined default values as the fourth height.

6. The directionality control system according to claim 1, wherein the processor computes the fourth height by designating, in the video data, a first designation position corresponding to the sound position, and a second designation position corresponding to a position on the reference surface located in a vertically lower direction of the sound position.

7. A directionality control method in a directionality control system which comprises an imaging device that captures a video, and a sound collector that collects sound and is provided as a separate body from the imaging device, the method comprising: displaying video data captured by the imaging device on a screen; receiving a designation of a position in the video data displayed on the screen; receiving an input of parameters at least including: a first height of the imaging device from a reference surface; a second height of the sound collector from the reference surface; and a third height of a computation reference point from the reference surface, the computation reference point being positioned in a sound collection direction at a position different from a sound position corresponding to the designated position in the video data; setting a fourth height of the sound position corresponding to the designated position in the video data from the reference surface; and computing a direction which is directed from the sound collector toward the sound position corresponding to the designated position in the video data, by using the input parameters, a direction which is directed from the imaging device toward the sound position corresponding to the designated position in the video data, and the fourth height.

8. The directionality control method according to claim 7, further comprising: forming a sound collection directionality of the sound collected by the sound collector in the sound collection direction as computed.

9. The directionality control system according to claim 1, wherein the imaging device includes a camera.

10. The directionality control system according to claim 1, wherein the sound collector includes a microphone.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0025] FIGS. 1(A) and 1(B) are block diagrams illustrating a configuration of a directionality control system of the present embodiment.

[0026] FIGS. 2(A) to 2(E) are exterior views of microphone array apparatuses.

[0027] FIG. 3 is a principle diagram illustrating the content in which the microphone array apparatus forms the sound collection directionality in a predetermined direction as a sound collection direction.

[0028] FIGS. 4(A) and 4(B) illustrate an operation summary of a directionality control system of the present embodiment, in which FIG. 4(A) is a diagram illustrating a state in which a single camera apparatus images subjects reflected in a range of the angle of view, and a state in which the microphone array apparatus collects conversations of subject people present in a sound collection direction and music output from a speaker device which is not present in the sound collection direction, and FIG. 4(B) is a diagram illustrating a state in which collected audio data is output from the speaker device when a direction which is directed from the microphone array apparatus toward a sound collection region central position A corresponding to a position A designated with the finger of the user in a video displayed on a display device is a sound collection direction.

[0029] FIG. 5 is a flowchart illustrating an operation procedure of the initial setting (calibration) in the directionality control system of the present embodiment.

[0030] FIG. 6 is a flowchart illustrating an operation procedure in which a directionality control apparatus of the directionality control system of the present embodiment computes a sound collection direction of the microphone array apparatus.

[0031] FIG. 7 is a diagram illustrating a positional relationship between a reference point O and the designated position A for computing a sound collection direction of the microphone array apparatus on a screen of the display device in a first computation method.

[0032] FIGS. 8(A), 8(B) and 8(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the reference point O, and the sound collection region central position A in the first computation method, in which FIG. 8(A) is a perspective view, FIG. 8(B) is a horizontal direction plan view, and FIG. 8(C) is a vertical direction sectional view taken along the line K-K of FIG. 8(B).

[0033] FIGS. 9(A), 9(B) and 9(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the reference point O, and the sound collection region central position A in the first computation method, in which FIG. 9(A) is a perspective view, FIG. 9(B) is a horizontal direction plan view, and FIG. 9(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 9(B).

[0034] FIGS. 10(A), 10(B) and 10(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the reference point O, and the sound collection region central position A in the first computation method, in which FIG. 10(A) is a perspective view, FIG. 10(B) is a horizontal direction plan view, and FIG. 10(C) is a vertical direction sectional view taken along the line R-R of FIG. 10(B).

[0035] FIG. 11 is a diagram illustrating a positional relationship between the reference point O and the designated position A for computing a sound collection direction of the microphone array apparatus on a screen of the display device in the second computation method.

[0036] FIGS. 12(A), 12(B) and 12(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, a position of the marker (reference point O), and the sound collection region central position A in a second computation method, in which FIG. 12(A) is a perspective view, FIG. 12(B) is a horizontal direction plan view, and FIG. 12(C) is a vertical direction sectional view taken along the line K-K of FIG. 12(B).

[0037] FIGS. 13(A), 13(B) and 13(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the position of the marker (reference point O), and the sound collection region central position A in the second computation method, in which FIG. 13(A) is a perspective view, FIG. 13(B) is a horizontal direction plan view, and FIG. 13(C) is a vertical direction sectional view taken along the line R-R of FIG. 13(B).

[0038] FIGS. 14(A), 14(B) and 14(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, a sound source position (reference point O), and the sound collection region central position A in a third computation method, in which FIG. 14(A) is a perspective view, FIG. 14(B) is a horizontal direction plan view, and FIG. 14(C) is a vertical direction sectional view taken along the line K-K of FIG. 14(B).

[0039] FIGS. 15(A), 15(B) and 15(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the sound source position (reference point O), and the sound collection region central position A in the third computation method, in which FIG. 15(A) is a perspective view, FIG. 15(B) is a horizontal direction plan view, and FIG. 15(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 15(B).

[0040] FIGS. 16(A), 16(B) and 16(C) illustrate each positional relationship between the camera apparatus and the microphone array apparatus of the directionality control system, the sound source position (reference point O), and the sound collection region central position A in the third computation method, in which FIG. 16(A) is a perspective view, FIG. 16(B) is a horizontal direction plan view, and FIG. 16(C) is a vertical direction sectional view taken along the line R-R of FIG. 16(B).

[0041] FIGS. 17(A), 17(B) and 17(C) illustrate a positional relationship between the camera apparatus, the microphone array apparatus, and the sound collection region central position A in a fourth computation method in a case where the microphone array apparatus and the camera apparatus are installed so as to be connected to each other by using a dedicated tool, in which FIG. 17(A) is a perspective view, FIG. 17(B) is a horizontal direction plan view, and FIG. 17(C) is a vertical direction sectional view taken along the line K-K of FIG. 17(B).

[0042] FIGS. 18(A), 18(B) and 18(C) illustrate a positional relationship between the camera apparatus, the microphone array apparatus, and the sound collection region central position A in the fourth computation method in a case where the microphone array apparatus and the camera apparatus are installed so as to be connected to each other by using the dedicated tool, in which FIG. 18(A) is a perspective view, FIG. 18(B) is a horizontal direction plan view, and FIG. 18(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 18(B).

[0043] FIGS. 19(A) and 19(B) are diagrams illustrating a vertical angle of a sound collection direction in a case where a ceiling on which the microphone array apparatus and the camera apparatus are installed is tilted in a direction of .sub.Mv with respect to a horizontal surface, and FIG. 19(C) is a diagram illustrating the sound collection direction .sub.MAv of the microphone array apparatus.

[0044] FIG. 20(A) is a schematic diagram illustrating a calibration method in a sound collection system of a second embodiment, FIG. 20(B) is a plan view in which an omnidirectional camera apparatus is viewed from a vertically lower side, and FIG. 20(C) is a plan view in which an omnidirectional microphone array apparatus is viewed from the vertically lower side.

[0045] FIG. 21(A) is a schematic diagram illustrating a calibration method in a sound collection system of a third embodiment, FIG. 21(B) is a plan view in which an omnidirectional camera apparatus is viewed from a vertically lower side, and FIG. 21(C) is a plan view in which an omnidirectional microphone array apparatus is viewed from the vertically lower side.

[0046] FIG. 22 is a schematic diagram illustrating a calibration method in a sound collection system of a fourth embodiment.

[0047] FIG. 23(A) is a plan view illustrating an omnidirectional camera apparatus and an omnidirectional microphone array apparatus are attached to an attachment member, and FIG. 23(B) is a sectional view taken along the line E-E in FIG. 23(A).

[0048] FIG. 24(A) is a side view illustrating a state in which fixation pins are being engaged with engagement holes, FIG. 24(B) is a plan view and a side view illustrating a state in which the fixation pins inserted into the engagement holes are moved, and FIG. 24(C) is a plan view and a side view illustrating a state in which the fixation pins are engaged with the engagement holes.

[0049] FIG. 25(A) is a side view illustrating a state in which a tool is being attached to an omnidirectional microphone array apparatus in a calibration method of a fifth embodiment, FIG. 25(B) is a side view illustrating a state in which attachment of the tool to the omnidirectional microphone array apparatus is completed, and FIG. 25(C) is an exterior perspective view of a sound collection system in which attachment of the tool to the omnidirectional microphone array apparatus is completed.

[0050] FIG. 26 is a diagram illustrating a state in which the tool is reflected in an image captured by an omnidirectional camera apparatus.

[0051] FIG. 27(A) is a schematic diagram of a sound collection system of a sixth embodiment in a case where a calibration omnidirectional camera apparatus and an omnidirectional microphone array apparatus are integrally installed, and FIG. 27(B) is a schematic diagram of a sound collection system of the sixth embodiment in a case where the omnidirectional microphone array apparatus is installed so that a reference direction of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus matches a reference direction of a horizontal angle of an imaging direction of the calibration omnidirectional camera apparatus.

[0052] FIG. 28(A) is a block diagram illustrating an example of a configuration of the sound collection system illustrated in FIG. 27(A), and FIG. 28(B) is a block diagram illustrating an example of a configuration of the sound collection system illustrated in FIG. 27(B).

[0053] FIG. 29 is a diagram illustrating a state in which collected audio data is output from the speaker device when a direction which is directed from the omnidirectional microphone array apparatus toward the sound collection position A corresponding to the designated position A designated with the finger of the user in an image displayed on the display device is a sound collection direction.

[0054] FIG. 30(A) is a flowchart illustrating an operation procedure related to computation of a first horizontal deviation angle .sub.Ch and a second horizontal deviation angle .sub.Kh and formation of the sound collection directionality in the sound collection system of the sixth embodiment, and FIG. 30(B) is a flowchart specifically illustrating an operation procedure of calibration in step ST20 illustrated in FIG. 30(A).

[0055] FIGS. 31(A), 31(B) and 31(C) are diagrams illustrating each positional relationship between the omnidirectional camera apparatus, the calibration omnidirectional camera apparatus, and the sound collection region central position A in the sixth embodiment, in which FIG. 31(A) is a perspective view, FIG. 31(B) is a plan view in which FIG. 31(A) is viewed in a vertically lower direction from an upper side, and FIG. 31(C) is a vertical direction sectional view taken along the line P-P of FIG. 31(B).

[0056] FIGS. 32(A), 32(B) and 32(C) are diagrams illustrating each positional relationship between the omnidirectional camera apparatus, the calibration omnidirectional camera apparatus, and the sound collection region central position A in the sixth embodiment, in which FIG. 32(A) is a perspective view, FIG. 32(B) is a plan view in which FIG. 32(A) is viewed in a vertically lower direction from an upper side, and FIG. 32(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 31(B).

[0057] FIG. 33 is a block diagram illustrating configurations of a sound collection system of a seventh embodiment.

[0058] FIG. 34(A) is a diagram illustrating a state in which the camera apparatus images target objects and a state in which the omnidirectional microphone array apparatus collects conversations of the target objects who are present in a sound collection directional direction and music output from a speaker device which is not present in the sound collection directional direction, in a sound collection space K in which the sound collection system is installed, and FIG. 34(B) is a diagram illustrating a state in which voice (for example, Hello) collected in a sound collection directional direction which is directed from the omnidirectional microphone array apparatus toward the target sound source position A corresponding to the designated position A which is designated with the finger FG of the user in video data displayed on the display device is output so that a volume level thereof is higher than a volume level of music (for example, ) output from the speaker device.

[0059] FIG. 35(A) is a flowchart illustrating an operation procedure of initial setting in the sound collection system of the seventh embodiment, and FIG. 35(B) is a flowchart illustrating an operation procedure following the initial setting in the sound collection system of the seventh embodiment.

[0060] FIG. 36(A) is a perspective view illustrating each position of the camera apparatus, the omnidirectional microphone array apparatus, a reference point O, and the target sound source position A, FIG. 36(B) is a horizontal direction plan view in which FIG. 36(A) is viewed in a vertically lower direction from a vertically upper direction, and FIG. 36(C) is a vertical direction sectional view taken along the line K-K of FIG. 36(B).

[0061] FIG. 37(A) is a perspective view illustrating each position of the camera apparatus, the omnidirectional microphone array apparatus, the reference point O, and the target sound source position A, FIG. 37(B) is a horizontal direction plan view in which FIG. 37(A) is viewed in a vertically lower direction from a vertically upper direction, and FIG. 37(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 37(B).

[0062] FIG. 38(A) is a block diagram illustrating a configuration of a sound collection system of an eighth embodiment, and FIG. 38(B) is a diagram illustrating a display screen WD1 of video data, and a selection screen WD2 which allows a height of the target sound source position from a floor surface to be selected, displayed on the display device.

[0063] FIG. 39(A) is a flowchart illustrating an operation procedure of initial setting in the sound collection system of the eighth embodiment, and FIG. 39(B) is a flowchart illustrating an operation procedure following the initial setting in the sound collection system of the eighth embodiment.

[0064] FIG. 40(A) is a block diagram illustrating a configuration of a sound collection system of a ninth embodiment, and FIG. 40(B) is a diagram illustrating a display screen WD1 of video data, and a entry form screen WD3 which allows a height of the target sound source position from the floor surface to be input, displayed on the display device.

[0065] FIG. 41(A) is a flowchart illustrating an operation procedure of initial setting in the sound collection system of the ninth embodiment, and FIG. 41(B) is a flowchart illustrating an operation procedure following the initial setting in the sound collection system of the ninth embodiment.

[0066] FIG. 42(A) is a block diagram illustrating a configuration of a sound collection system of a tenth embodiment, and FIG. 42(B) is a diagram illustrating a state in which a first designated position A1 and a second designated position A2 are designated on a display screen WD4 of video data displayed on the display device.

[0067] FIG. 43(A) is a flowchart illustrating an operation procedure of initial setting in the sound collection system of the tenth embodiment, and FIG. 43(B) is a flowchart illustrating an operation procedure following the initial setting in the sound collection system of the tenth embodiment.

[0068] FIG. 44(A) is a diagram illustrating distances and directions from the camera apparatus to the target sound source position A1 of a target object (person) present on the floor surface BL and the position A2 on the floor surface BL located in the vertically lower direction from the target sound source position A1, FIG. 44(B) is a plan view in which the camera apparatus, the target sound source position A1, and the position A2 on the floor surface BL are viewed in a vertically lower direction from a vertically upper direction, and FIG. 44(C) is a sectional view taken along the line A-A of FIG. 44(B).

[0069] FIG. 45(A) is a diagram illustrating a distance and a direction from the camera apparatus 11 to the target sound source position A1 of a target object (person) present on a stand RC placed on the floor surface BL and the position A2 on a stand RC located in the vertically lower direction from the target sound source position A1, FIG. 45(B) is a plan view in which the camera apparatus 11, the target sound source position A1, and the position A2 on the stand RC are viewed in a vertically lower direction from a vertically upper direction, and FIG. 45(C) is a sectional view taken along the line A-A of FIG. 45(B).

[0070] FIG. 46 is a diagram for explaining a problem in a monitoring system of the related art.

[0071] FIG. 47 is a block diagram illustrating a configuration of a sound collection system of an eleventh embodiment.

[0072] FIG. 48(A) is a diagram illustrating an operation summary of a sound collection system of the eleventh embodiment, and FIG. 48(B) is a diagram illustrating a state in which a volume level of voice of a person who is present in a sound collection direction which is directed from the omnidirectional microphone array apparatus toward the target sound source position A corresponding to the designated position A which is designated in captured image data displayed on the display device is output so as to be higher than a volume level of sound output from a speaker device which is not present in the sound collection direction.

[0073] FIG. 49(A) is a flowchart illustrating the entire operation procedure in the sound collection system of the eleventh embodiment, and FIG. 49(B) is a flowchart specifically illustrating a calibration operation procedure in the sound collection system of the eleventh embodiment.

[0074] FIG. 50 is a diagram illustrating a first calibration method in the eleventh embodiment.

[0075] FIG. 51(A) is a diagram illustrating a positional relationship between a PTZ camera apparatus 1, an omnidirectional microphone array apparatus 2, and a calibration marker MAK in the first calibration method, FIG. 51(B) is a horizontal direction plan view of FIG. 51(A), and FIG. 51(C) is a vertical direction sectional view taken along the line K-K of FIG. 51(B).

[0076] FIG. 52(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the first calibration method, FIG. 52(B) is a horizontal direction plan view of FIG. 52(A), and FIG. 52(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 52(B).

[0077] FIG. 53(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the first calibration method, FIG. 53(B) is a horizontal direction plan view of FIG. 53(A), and FIG. 53(C) is a vertical direction sectional view taken along the line R-R of FIG. 53(B).

[0078] FIG. 54 is a diagram illustrating a second calibration method in the eleventh embodiment.

[0079] FIG. 55(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration marker MAK in the second calibration method, FIG. 55(B) is a horizontal direction plan view of FIG. 55(A), and FIG. 55(C) is a vertical direction sectional view taken along the line K-K of FIG. 55(B).

[0080] FIG. 56(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the second calibration method, FIG. 56(B) is a horizontal direction plan view of FIG. 56(A), and FIG. 56(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 56(B).

[0081] FIG. 57(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the second calibration method, FIG. 57(B) is a horizontal direction plan view of FIG. 57(A), and FIG. 57(C) is a vertical direction sectional view taken along the line R-R of FIG. 57(B).

[0082] FIG. 58 is a diagram illustrating a third calibration method in the eleventh embodiment.

[0083] FIG. 59(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and a calibration floor marker MAK2 in the third calibration method, FIG. 59(B) is a horizontal direction plan view of FIG. 59(A), and FIG. 59(C) is a vertical direction sectional view taken along the line K-K of FIG. 59(B).

[0084] FIG. 60 is a diagram illustrating a fourth calibration method in the eleventh embodiment.

[0085] FIG. 61(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK2 in the fourth calibration method, FIG. 61(B) is a horizontal direction plan view of FIG. 61(A), and FIG. 61(C) is a vertical direction sectional view taken along the line K-K of FIG. 61(B).

[0086] FIG. 62 is a diagram illustrating a fifth calibration method in the eleventh embodiment.

[0087] FIG. 63(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and a calibration marker MAK3 in the fifth calibration method, FIG. 63(B) is a horizontal direction plan view of FIG. 63(A), and FIG. 63(C) is a vertical direction sectional view taken along the line K-K of FIG. 63(B).

[0088] FIG. 64 is a diagram illustrating a sixth calibration method in the eleventh embodiment.

[0089] FIG. 65(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration marker MAK in the sixth calibration method, FIG. 65(B) is a horizontal direction plan view of FIG. 65(A), and FIG. 65(C) is a vertical direction sectional view taken along the line K-K of FIG. 65(B).

[0090] FIG. 66(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the sixth calibration method, FIG. 66(B) is a horizontal direction plan view of FIG. 66(A), and FIG. 66(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 66(B).

[0091] FIG. 67(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the sixth calibration method, FIG. 67(B) is a horizontal direction plan view of FIG. 67(A), and FIG. 67(C) is a vertical direction sectional view taken along the line R-R of FIG. 67(B).

[0092] FIG. 68 is a diagram illustrating a seventh calibration method in the eleventh embodiment.

[0093] FIG. 69(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK2 in the seventh calibration method, FIG. 69(B) is a horizontal direction plan view of FIG. 69(A), and FIG. 69(C) is a vertical direction sectional view taken along the line K-K of FIG. 69(B).

[0094] FIG. 70(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the seventh calibration method, FIG. 70(B) is a horizontal direction plan view of FIG. 70(A), and FIG. 70(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 70(B).

[0095] FIG. 71(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the seventh calibration method, FIG. 71(B) is a horizontal direction plan view of FIG. 71(A), and FIG. 71(C) is a vertical direction sectional view taken along the line R-R of FIG. 71(B).

[0096] FIG. 72 is a diagram illustrating an eighth calibration method in the eleventh embodiment.

[0097] FIG. 73(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, the calibration marker MAK, and the calibration floor marker MAK2 in the eighth calibration method, FIG. 73(B) is a horizontal direction plan view of FIG. 73(A), and FIG. 73(C) is a vertical direction sectional view taken along the line K-K of FIG. 73(B).

[0098] FIG. 74 is a diagram illustrating a ninth calibration method in the eleventh embodiment.

[0099] FIG. 75(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK2 in the ninth calibration method, FIG. 75(B) is a horizontal direction plan view of FIG. 75(A), and FIG. 75(C) is a vertical direction sectional view taken along the line K-K of FIG. 75(B).

[0100] FIG. 76 is a diagram illustrating a tenth calibration method in the eleventh embodiment.

[0101] FIG. 77(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration marker MAK in the tenth calibration method, FIG. 77(B) is a horizontal direction plan view of FIG. 77(A), and FIG. 77(C) is a vertical direction sectional view taken along the line K-K of FIG. 77(B).

[0102] FIG. 78(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the tenth calibration method, FIG. 78(B) is a horizontal direction plan view of FIG. 78(A), and FIG. 78(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 78(B).

[0103] FIG. 79(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the target sound source position A in the tenth calibration method, FIG. 79(B) is a horizontal direction plan view of FIG. 79(A), and FIG. 79(C) is a vertical direction sectional view taken along the line R-R of FIG. 79(B).

[0104] FIG. 80 is a diagram illustrating a positional relationship between a camera installed on a wall surface of a room, an omnidirectional microphone array apparatus installed on a ceiling surface of the room, and a sound source position.

[0105] FIG. 81 is a diagram illustrating a horizontal angle and a vertical angle which are computed by using a microphone coordinate system of an omnidirectional microphone array apparatus converted from a camera coordinate system of a PTZ camera and which are directed from the omnidirectional microphone array apparatus toward the sound source position.

[0106] FIG. 82(A) is a plan view illustrating a calibration floor marker MAK4 used in an eleventh calibration method, and FIG. 82(B) illustrates screens of a point O and a point X enlarged by using a focus function of the PTZ camera apparatus 1.

[0107] FIG. 83(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the eleventh calibration method, FIG. 83(B) is a horizontal direction plan view of FIG. 83(A), and FIG. 83(C) is a sectional view taken along the line K-K of FIG. 83(B).

[0108] FIG. 84(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the eleventh calibration method, FIG. 84(B) is a horizontal direction plan view of FIG. 84(A), and FIG. 84(C) is a sectional view taken along the line L-L of FIG. 84(B).

[0109] FIG. 85(A) is a plan view illustrating the calibration floor marker MAK4 used in a twelfth calibration method, and FIG. 85(B) illustrates screens of a point O and a point X enlarged by using the focus function of the PTZ camera apparatus 1.

[0110] FIG. 86(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the twelfth calibration method, FIG. 86(B) is a horizontal direction plan view of FIG. 86(A), and FIG. 86(C) is a sectional view taken along the line K-K of FIG. 86(B).

[0111] FIG. 87(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the twelfth calibration method, FIG. 87(B) is a horizontal direction plan view of FIG. 87(A), and FIG. 87(C) is a sectional view taken along the line L-L of FIG. 87(B).

[0112] FIG. 88(A) is a plan view illustrating a calibration floor marker MAK4 used in a thirteenth calibration method, and FIG. 88(B) illustrates screens of points O and O and a point X enlarged by using the focus function of the PTZ camera apparatus 1.

[0113] FIG. 89(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the thirteenth calibration method, FIG. 89(B) is a horizontal direction plan view of FIG. 89(A), and FIG. 89(C) is a sectional view taken along the line K-K of FIG. 89(B).

[0114] FIG. 90(A) is a diagram illustrating a positional relationship between the PTZ camera apparatus 1, the omnidirectional microphone array apparatus 2, and the calibration floor marker MAK4 in the thirteenth calibration method, FIG. 90(B) is a horizontal direction plan view of FIG. 90(A), and FIG. 90(C) is a sectional view taken along the line L-L of FIG. 90(B).

[0115] FIG. 91(A) is a plan view illustrating a calibration floor marker MAK5 with an angle memory, and FIG. 91(B) illustrates a screen of points O and O enlarged by using the focus function of the PTZ camera apparatus 1.

DESCRIPTION OF EMBODIMENTS

[0116] Hereinafter, embodiments of a directionality control system and a directionality control method related to the present invention will be described with reference to the drawings. The directionality control system of the present embodiment is used as a monitoring system (including a manned monitoring system and an unmanned monitoring system) provided in, for example, a factory, a public facility (for example, a library or an event hall), or a store (for example, a retail store or a bank).

[0117] In addition, the present invention can be expressed as respective apparatuses (for example, a directionality control apparatus to be described later) constituting the directionality control system, or a directionality control method including respective operations (steps) performed by each apparatus constituting the directionality control system.

First Embodiment

Configuration of Directionality Control System

[0118] FIGS. 1(A) and 1(B) are block diagrams illustrating configurations of directionality control systems 10 and 10A of the present embodiment. The directionality control system 10 illustrated in FIG. 1(A) includes at least one camera apparatuses 11 to 1n, a microphone array apparatus (or an omnidirectional microphone array apparatus) 2, and a directionality control apparatus 3. Here, n indicates the number of camera apparatuses, and is an integer of 1 or higher. The camera apparatuses 11 to 1n, the microphone array apparatus 2, and the directionality control apparatus 3 are connected to each other via a network NW1.

[0119] The directionality control system 10A illustrated in FIG. 1(B) includes at least one camera apparatuses 11 to 1n, a microphone array apparatus 2, a directionality control apparatus 3, and a recorder apparatus 4. The camera apparatuses 11 to 1n, the microphone array apparatus 2, the directionality control apparatus 3, and the recorder apparatus 4 are connected to each other via a network NW2.

[0120] Hereinafter, a description will be made focusing on an operation of each unit of the directionality control system 10, and a description of an operation of each unit of the directionality control system 10A will be made regarding the content which is different from the operation of each unit of the directionality control system 10.

[0121] Each of the camera apparatuses 11 to 1n as at least one imaging part is a monitoring camera which is connected to the network NW1 and is installed on, for example, a ceiling surface or a wall surface (refer to FIG. 81) of a room of an event hall or a predetermined stand (refer to FIG. 4(A)) in a fixed manner, and perform a panning operation, a tilting operation, a zooming operation (for example, zoom-in and zoom-out), and a distance-measuring operation and an angle-measuring operation related to a designated specific position in a captured video through a remote operation from a monitoring system control room (not illustrated) which is connected thereto via the network NW1.

[0122] The camera apparatuses 11 to 1n capture a video (including a still image and a moving image; this is also the same for the following description) of a position of a monitoring target which is present in a predefined angle of view CAR centering on an optical axis CX. The camera apparatuses 11 to 1n transmit the captured video data, and input parameters for computing a sound collection direction (.sub.MAh,.sub.MAv) which will be described later to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW1.

[0123] The microphone array apparatus 2 as a sound collection part which is connected to the network NW1 is installed on, for example, a ceiling surface or a wall surface of a room of an event hall or a predetermined stand (refer to FIG. 4(A)) in a fixed manner, and is provided with at least microphone 22 and 23 (refer to FIG. 2) which are provided in a uniform manner and a control unit (not illustrated) which controls an operation of each of the microphones 22 and 23.

[0124] The microphone array apparatus 2 collects sound of a monitoring target in a sound collection direction by using each of the microphone (or microphones) 22 and 23, and transmits audio data collected by each of the microphones 22 and 23 to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW1 or the network NW2.

[0125] The microphone array apparatus 2 forms sound collection directionality of each of the microphones 22 and 23 in a sound collection direction (.sub.MAh,.sub.MAv) which is derived (hereinafter, referred to as computed) by a signal processing unit 33 which will be described later in response to a directionality formation instruction from the signal processing unit 33.

[0126] Consequently, the microphone array apparatus 2 can increase a volume level of audio data which is collected from the sound collection direction (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can reduce a volume level of audio data which is collected from a direction in which the sound collection directionality is not formed. In addition, a method of computing the sound collection direction (.sub.MAh,.sub.MAv) will be described later.

[0127] Exteriors of the microphone array apparatus 2 will be described later with reference to FIG. 2. Further, each of the microphones 22 and 23 may employ a nondirectional microphone, a bidirectional microphone, a unidirectional microphone, a sharply directional microphone, a super-directional microphone (for example, a shotgun microphone), or a combination thereof.

[0128] The networks NW1 and NW2 are wired communication networks (for example, an intranet or the Internet) or wireless communication networks (for example, a local area network (LAN)).

[0129] The directionality control apparatus 3 is connected to the network NW1 or the network NW2, and may be, for example, a stationery personal computer (PC) installed in a monitoring system control room (not illustrated), and may be a mobile phone, a tablet terminal, or a smart phone, which can be carried by a user.

[0130] The directionality control apparatus 3 includes at least a communication unit 31, an operation unit 32, a signal processing unit 33, a display device 36, a speaker device 37, and a memory 38. The signal processing unit 33 includes at least a coordinate transform processing section 34z and an output control section 35.

[0131] The communication unit 31 receives video data or audio data which is transmitted from the camera apparatuses 11 to 1n or the microphone array apparatus 2, and outputs the data to the signal processing unit 33 via the network NW1 or the network NW2.

[0132] The operation unit 32 is a user interface (UI) for notifying the signal processing unit 33 of the content of a user's input operation, and is, for example, a pointing device such as a mouse or a keyboard. In addition, the operation unit 32 may be configured by using a touch panel or a touch pad which is disposed so as to correspond to, for example, a screen of the display device 36 and allows an input operation to be performed with the finger FG of the user or a stylus pen.

[0133] The operation unit 32 outputs coordinate data indicating a region where the user desires to increase or decrease a volume level, that is, a designated position A or a region B illustrated in FIG. 4(B) to the signal processing unit 33 in response to the user's input operation.

[0134] The signal processing unit 33 is configured by using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and performs a control process for collectively controlling operations of the respective units of the directionality control apparatus 3, data input and output processes with other respective units, a data computation (calculation) process, and a data storage process.

[0135] The coordinate transform processing section 34z computes, as a sound collection direction, coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed from an installation position of the microphone array apparatus 2 toward a sound collection region central position (a sound position) A by using coordinate data indicating the position A or the region B which is output from the operation unit 32. In the sound collection direction (.sub.MAh,.sub.MAv), .sub.MAh indicates a horizontal angle of a depression angle .sub.MA which is directed toward the sound collection region central position A from the microphone array apparatus 2, and .sub.MAv indicates a vertical angle of the depression angle .sub.MA which is directed toward the sound collection region central position A from the microphone array apparatus 2. In addition, a process of computing the sound collection direction (.sub.MAh,.sub.MAv) will be described later in detail with reference to the drawings.

[0136] In addition, the sound collection region central position A is a field position which corresponds to the position A designated with the finger FG of the user or a stylus pen on a screen of the display device 36 via the operation unit 32, and is an actual monitoring target.

[0137] The output control section 35 controls operations of the display device 36 and the speaker device 37, so as to cause the display device 36 to reproduce and output video data transmitted from the camera apparatuses 11 to 1n, and to cause the speaker device 37 to output audio data transmitted from the microphone array apparatus 2 as sound.

[0138] The display device 36 as a display part displays video data captured by the camera apparatuses 11 to 1n on a screen.

[0139] The speaker device 37 as a sound output part outputs, as sound, audio data collected by the microphone array apparatus 2 or audio data which is collected by the microphone array apparatus 2 after the sound collection directionality is formed in the sound collection direction (.sub.MAh,.sub.MAv) computed by the coordinate transform processing section 34z. In addition, the display device 36 and the speaker device 27 may be configured separately from the directionality control apparatus 3.

[0140] The memory 38 is configured by using, for example, a random access memory (RAM), and functions as a work memory when the respective units of the directionality control apparatus 3 operate.

[0141] The recorder apparatus 4 records video data captured by the camera apparatuses 11 to 1n and audio data collected by the microphone array apparatus 2. The recorder apparatus 4 records the video data captured by the camera apparatuses 11 to 1n and the audio data collected by the microphone array apparatus 2 in correlation with each other. In addition, the networks NW1 and NW2 may be connected to each other, and various data items may be transmitted between the directionality control systems 10 and 10A.

[0142] FIGS. 2(A) to 2(E) are exterior views of the microphone array apparatus 2. The microphone array apparatuses 2 illustrated in FIGS. 2(A) to 2(E) have different exteriors and arrangement positions of a plurality of microphones, but functions of the microphone array apparatuses 2 are equivalent to each other.

[0143] The microphone array apparatus 2 illustrated in FIG. 2(A) includes a plurality of microphones 22 and 23 disposed in a disc-shaped casing 21. The plurality of microphones 22 and 23 are disposed along a surface of the casing 21. Specifically, a plurality of microphones 22 which are disposed in a large circular shape having the same center as the casing 21 and a plurality of microphones 23 which which are disposed in a large circular shape having the same center as the casing 21 are disposed in a concentric shape.

[0144] The plurality of microphone units 22 which are disposed in the large circular shape have a wide interval therebetween, have a large diameter, and have a characteristic suitable for a low sound range. On the other hand, the respective microphone units 23 have a narrow interval therebetween, have a small diameter, and have a characteristic suitable for a high sound range.

[0145] The microphone array apparatus 2A illustrated in FIG. 2(B) has a configuration in which a plurality of microphones 22 are disposed in a vertical direction and a horizontal direction in a uniform manner along a surface of a disc-shaped casing 21. In the microphone array apparatus 2A, the plurality of microphones 22 are disposed in a straight line in the vertical direction and the horizontal direction, and thus it is possible to reduce a computation amount in a process of forming the sound collection directionality in audio data. In addition, the plurality of microphones 22 may be disposed either in the vertical direction or in the horizontal direction.

[0146] The microphone array apparatus 2B illustrated in FIG. 2(C) has a disc-shaped casing 21B with a smaller diameter than that of the microphone array apparatus 2 illustrated in FIG. 2(A), and has a configuration in which a plurality of microphones 22 are disposed in a uniform manner along a surface of the disc-shaped casing 21B. Since the intervals between the respective microphones 22 are short, the microphone array apparatus 2B has a characteristic suitable for a high sound range.

[0147] The microphone array apparatus 2C illustrated in FIG. 2(D) includes a casing 21C with a doughnut shape in which an opening 21a is formed inside thereof, and a plurality of microphones 22 which are disposed in the casing 21C in a uniform manner. The plurality of microphones 22 are disposed in a concentric shape in the casing 21C. In addition, for example, a camera apparatus (for example, an omnidirectional camera apparatus) may be installed inside the opening 21a in a state of being inserted thereinto.

[0148] The microphone array apparatus 2D illustrated in FIG. 2(E) has a configuration in which a plurality of microphones 22 are disposed in a uniform manner along a surface of a rectangular casing 21D. Since the casing 21D has a rectangular shape, the microphone array apparatus 2D can be easily installed even at a location such as a corner.

[0149] FIG. 3 is a principle diagram illustrating the content in which the microphone array apparatus 2 forms the sound collection directionality in a predetermined direction as a sound collection direction. In FIG. 3, a brief description will be made of a principle of a directionality control process using, for example, a delay sum method. Sound waveforms generated from a sound source 80 are incident to respective microphones 221, 222, 223, . . . , 22(n1) and 22n of the microphone array apparatus 2 with a predetermined angle (incidence angle=(90) [degrees]). An incidence angle illustrated in FIG. 3 may be a horizontal angle .sub.MAh or a vertical angle .sub.MAv of the depression angle .sub.MA which is directed toward the sound collection region central position A from the microphone array apparatus 2.

[0150] The sound source 80 is, for example, a conversation of subjects of the camera apparatus 11, present in a sound collection direction in which the microphone array apparatus 2 collects sound, and is present in a direction of a predetermined angle with respect to the upper surface of the casing 21 of the microphone array apparatus 2. In addition, gaps d between the respective microphones 221, 222, 223, . . . , 22(n1) and 22n are assumed to be constant.

[0151] The sound waveforms generated from the sound source 80 initially arrive at and are collected by the microphone 221, then arrive at and are collected by the microphone 222, similarly, sequentially arrive at and are collected by the microphones, and, finally, arrive at and are collected by the microphone 22n.

[0152] In addition, a direction which is directed toward the sound source 80 from each of the microphones 221, 222, 223, . . . , 22(n1) and 22n of the microphone array apparatus 2 is the same as, for example, a direction which is directed toward a sound collection region central position corresponding to a position designated by the user on a screen of the display device 36 from each microphone of the microphone array apparatus 2 in a case where the sound source 80 is the sound of conversations which people have.

[0153] Here, there are occurrences of arrival time differences 1, 2, 3, . . . and (n1) between time points at which the sound waves arrive at the microphones 221, 222, 223, . . . and 22(n1) and finally arrive at the microphone 22n. For this reason, if audio data of sound collected by the respective microphones 221, 222, 223, . . . , 22(n1) and 22n is added without change, the audio data is added in a state where a phase thereof is shifted, and thus a volume level of the sound waves is completely lowered.

[0154] In addition, 1 indicates a time difference between the time point at which the sound wave arrives at the microphone 221 and the time point at which the sound wave arrives at the microphone 22n, 2 indicates a time difference between the time point at which the sound wave arrives at the microphone 222 and the time point at which the sound wave arrives at the microphone 22n, and, similarly, (n1) indicates a time difference between the time point at which the sound wave arrives at the microphone 22(n1) and the time point at which the sound wave arrives at the microphone 22n.

[0155] In the present embodiment, the microphone array apparatus 2 includes A/D converters 241, 242, 243, . . . , 24(n1) and 24n, delay devices 251, 252, 253, . . . , 25(n1) and 25n which are respectively provided so as to correspond to the microphones 221, 222, 223, . . . , 22(n1) and 22n, and an adder 26 (refer to FIG. 3).

[0156] In other words, in the microphone array apparatus 2, the A/D converters 241, 242, 243, . . . , 24(n1) and 24n A/D-convert analog audio data collected by the respective microphones 221, 222, 223, . . . , 22(n1) and 22n into digital audio data.

[0157] In addition, in the microphone array apparatus 2, the delay devices 251, 252, 253, . . . , 25(n1) and 25n provide delay times corresponding to the arrival time differences in the respective microphones 221, 222, 222 , . . . , 22(n1) and 22n to all phases of the sound waves so that the phases thereof are made to match each other, and then the adder 26 adds the audio data having undergone the delay process together. Accordingly, the microphone array apparatus 2 can form a directionality of the audio data in each of the microphones 221, 222, 223, . . . , 22(n1) and 22n in a direction of the predetermined angel .

[0158] For example, in FIG. 3, delay times D1, D2, D3, . . . , D(n1) and Dn which are respectively set in the delay devices 251, 252, 253, . . . , 25(n1) and 25n respectively correspond to the arrival time differences 1, 2, 3, . . . and (n1), and are expressed by Equation (1)

[00001]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.1\right]& \\ D.Math..Math.1=\frac{L.Math..Math.1}{\mathrm{Vs}}=\frac{\left\{d\left(n-1\right)\cos .Math..Math.\right\}}{\mathrm{Vs}}.Math..Math.D.Math..Math.2=\frac{L.Math..Math.2}{\mathrm{Vs}}=\frac{\left\{d\left(n-2\right)\cos .Math..Math.\right\}}{\mathrm{Vs}}.Math..Math.D.Math..Math.3=\frac{L.Math..Math.3}{\mathrm{Vs}}=\frac{\left\{d\left(n-3\right)\cos .Math..Math.\right\}}{\mathrm{Vs}}.Math..Math.\mathrm{.Math.}.Math..Math.\mathrm{Dn}-1=\frac{L.Math..Math.n-1}{\mathrm{Vs}}=\frac{\left\{d1\cos .Math..Math.\right\}}{\mathrm{Vs}}.Math..Math.\mathrm{Dn}=0& \left(1\right)\end{array}$

[0159] Here, L1 indicates a difference between sound wave arrival distances in the microphone 221 and the microphone 22n. L2 indicates a difference between sound wave arrival distances in the microphone 222 and the microphone 22n. L3 indicates a difference between sound wave arrival distances in the microphone 223 and the microphone 22n, and, similarly, L(n1) indicates a difference between sound wave arrival distances in the microphone 22(n1) and the microphone 22n. Vs indicates a velocity of the sound wave (sound velocity). L1, L2, L3, . . . , and L(n1), and Vs are known values. In FIG. 3, the delay time Dn set in the delay device 25n is 0 (zero).

[0160] In the above-described way, the microphone array apparatus 2 can easily form the sound collection directionality of audio data collected by each of the microphones 22 and 23 by changing the delay times D1, D2, D3, . . . , D(n1) and Dn which are respectively set in the delay devices 251, 252, 253, . . . , 25(n1) and 25n.

[0161] In addition, for convenience of description, the directionality forming process illustrated in FIG. 3 has been described on the premise that the microphone array apparatus 2 performs the process. However, in a case where the signal processing unit 33 includes the same number of A/D converters and the same number of delay devices as the number of microphones, and a single adder, the signal processing unit 33 may perform the above-described process.

[0162] Next, a summary of an operation of the directionality control system 10 of the present embodiment will be described with reference to FIGS. 4(A) and 4(B). FIGS. 4(A) and 4(B) illustrate an operation summary of the directionality control system 10 of the present embodiment. FIG. 4(A) is a diagram illustrating a state in which a single camera apparatus 11 images subjects reflected in a range of the angle of view CAR, and a state in which the microphone array apparatus 2 collects conversations of subject people present in a sound collection direction and music output from a speaker device SP which is not present in the sound collection direction. FIG. 4(B) is a diagram illustrating a state in which collected audio data is output from the speaker device 37 when a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A corresponding to the position A designated with the finger FG of the user in a video displayed on the display device 36 is a sound collection direction.

[0163] In the directionality control system 10, the camera apparatus 11 images the subjects (for example, two people) and the speaker device SP reflected in the predefined angle of view CAR. The microphone array apparatus 2 collects ambient sound. In FIG. 4(A), the two people who are subjects are having conversations, and the speaker device SP is outputting music () as sound. Video data captured by the camera apparatus 11 is displayed on the display device 36 of the directionality control apparatus 3 (refer to FIG. 4(B)).

[0164] Here, if the position A on the display device 36, that is, a substantially central position or the region B of the two people having conversations is designated with the finger FG of the user, the directionality control apparatus 3 computes coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed from the installation position of the microphone array apparatus 2 toward the sound collection region central position A as a sound collection direction of the microphone array apparatus 2 by using coordinate data indicating the position A or the region B. The microphone array apparatus 2 forms sound collection directionality MIX in the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A by using the coordinate data (.sub.MAh,.sub.MAv) computed by the directionality control apparatus 3.

[0165] Therefore, the microphone array apparatus 2 can increase a volume level of the conversation (Hello) of the two people present in the direction in which the sound collection directionality MIX is formed more than a volume level of the music () output from the speaker device SP which is not present in the direction in which the sound collection directionality MIX is formed.

[0166] Consequently, the directionality control apparatus 3 causes the speaker device 37 to output sound with a volume level of the conversation (Hello) of the two people present in the direction in which the sound collection directionality MIX is formed higher than a volume level of the music () output from the speaker device SP which is not present in the direction in which the sound collection directionality MIX is formed (refer to FIG. 4(B)).

[0167] Next, initial setting (calibration) performed in the directionality control systems 10 and 10A of the present embodiment will be described with reference to FIG. 5. FIG. 5 is a flowchart illustrating an operation procedure of the initial setting (calibration) in the directionality control system 10 or 10A of the present embodiment. The initial setting (calibration) includes, for example, an operation in which the signal processing unit 33 of the directionality control apparatus 3 acquires input parameters which are required to compute a direction of the sound collection directionality formed by the microphone array apparatus 2.

[0168] Hereinafter, for simplification of description, the directionality control system 10 will be described of the directionality control systems 10 and 10A, and the directionality control system 10 includes the single camera apparatus 11.

[0169] In FIG. 5, the camera apparatus 11 and the microphone array apparatus 2 constituting the directionality control system 10 are initially installed so as to be fixed at predetermined positions (for example, a ceiling surface or a wall surface of a room of an event hall or a stand) (step ST1). In the present embodiment, the camera apparatus 11 and the microphone array apparatus 2 are respectively installed at different positions.

[0170] After the camera apparatus 11 and the microphone array apparatus 2 are initially installed, the signal processing unit 33 measures each input parameter which is required to compute coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 (step ST2). The process in step ST2 includes a case where the user measures the input parameter by using a measuring device (for example, a laser range finder), a case where the camera apparatus 11 measures and acquires the input parameter by using functions of well-known techniques of the camera apparatus 11, or a case where the input parameter is acquired by using a design drawing (layout) of an installation space (for example, a building). Each input parameter in step ST2 differs in each embodiment which will be described later, and thus detailed content thereof will be described later.

[0171] After step ST2, each input parameter measured in step ST2 is input to the signal processing unit 33 of the directionality control apparatus 3 (step ST3). For example, the camera apparatus 11 transmits an input parameter acquired by using functions of well-known techniques of the camera apparatus 11, to the communication unit 31 of the directionality control apparatus 3. The communication unit 31 outputs the input parameter transmitted by the camera apparatus 11, to the signal processing unit 33.

[0172] In addition, the microphone array apparatus 2 transmits an input parameter acquired by using functions of well-known techniques of the microphone array apparatus 2, to the communication unit 31 of the directionality control apparatus 3. The communication unit 31 outputs the input parameter transmitted by the medical examination support system 2, to the signal processing unit 33.

[0173] Further, the operation unit 32 outputs an input parameter to the signal processing unit 33 in response to a user's input operation for operating the directionality control apparatus 3.

[0174] The signal processing unit 33 temporarily preserves each input parameter acquired in step ST3, in the memory 38 (step ST4). Through the above-described steps, the operation of the initial setting (calibration) in the directionality control system 10 is finished.

[0175] Next, with reference to FIGS. 6, 80 and 81, a description will be made of an operation procedure in which the directionality control apparatus 3 of the directionality control system 10 of the present embodiment computes a sound collection direction of the microphone array apparatus 2. FIG. 6 is a flowchart illustrating an operation procedure in which the directionality control apparatus 3 of the directionality control system 10 of the present embodiment computes a sound collection direction of the microphone array apparatus 2. FIG. 6 illustrates the entire operation procedure in which the directionality control apparatus 3 of the directionality control system 10 of the present embodiment computes a sound collection direction of the microphone array apparatus 2, and FIGS. 80 and 81 illustrate a fundamental approach to the computation. A detailed computation process will be described later with reference to FIG. 7 and the subsequent drawings.

[0176] In FIG. 6, the directionality control apparatus 3 receives designation of any position A or region B in video data which is being displayed on a screen of the display device 36, via the operation unit 32 (step ST11). The directionality control apparatus 3 transmits a notification indicating that the designation of any position A or region B in the video data which is being displayed on the screen of the display device 36 has been received, to the camera apparatus 11.

[0177] After step ST11, in a case where the camera apparatus 11 receives the notification indicating that the designation of the position A or the region B has been received from the directionality control apparatus 3, the camera apparatus 11 acquires all or some coordinates of a distance, a horizontal angle, and a vertical angle (L.sub.CA,.sub.CAh,.sub.CAv) to the sound collection region central position A corresponding to the position A or the region B on the screen designated in step ST11, with the installation position of the camera apparatus 11 as a start point (step ST12).

[0178] The camera apparatus 11 transmits all or some of the coordinates of the distance, the horizontal angle, and the vertical angle (L.sub.CA,.sub.CAh,.sub.CAv) to the sound collection region central position A corresponding to the position A or the region B on the screen designated in step ST11 with the installation position of the camera apparatus 11 as a start point, to the directionality control apparatus 3.

[0179] The signal processing unit 33 of the directionality control apparatus 3 computes coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 by using all or some of the coordinates of the distance, the horizontal angle, and the vertical angle (L.sub.CA,.sub.CAh,.sub.CAv) to the sound collection region central position A corresponding to the position A or the region B on the screen designated in step ST11 with the installation position of the camera apparatus 11 acquired in step ST12 as a start point, and each input parameter which is temporarily preserved in the memory 38 in step ST4 illustrated in FIG. 5 (step ST13).

[0180] The directionality control apparatus 3 transmits a directionality formation instruction including the coordinates (.sub.MAh,.sub.MAv) computed in step ST13 to the microphone array apparatus 2. The microphone array apparatus 2 forms the sound collection directionality of each of the microphones 22 and 23 in a sound collection direction indicated by the coordinates (.sub.MAh,.sub.MAv) computed by the directionality control apparatus 3 in response to the directionality formation instruction from the directionality control apparatus 3 (step ST14).

[0181] Consequently, the microphone array apparatus 2 can increase a volume level of audio data which is collected from the sound collection direction (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can reduce a volume level of audio data which is collected from a direction in which the sound collection directionality is not formed. In the above-described way, the operation is finished in which the directionality control apparatus 3 of the directionality control system 10 computes a sound collection direction of the microphone array apparatus 2.

In Case of Camera Apparatus Sends Only Panning/Tilting/Zooming Information

[0182] In the description hitherto, the camera apparatus 11 performs the operation in step ST12 illustrated in FIG. 6, but the camera apparatus 11 may send information regarding an angle in a panning direction, information regarding an angle in a tilting direction, and zooming information to the directionality control apparatus 3, and the directionality control apparatus 3 may compute a distance from the camera apparatus 11 to the sound collection region central position A, a horizontal angle, and a vertical angle (L.sub.CA,.sub.CAh,.sub.CAv), which is also the same for each of the following embodiments.

[0183] In addition, in the directionality control system 10 of the present embodiment, a timing at which the microphone array apparatus 2 collects sound is not limited to the time right after step ST14, and may be, for example, the time after power is supplied to the microphone array apparatus 2.

Description of Generalization of Coordinate Transform Process from Camera Coordinate System to Microphone Coordinate System

[0184] FIG. 80 is a diagram illustrating a positional relationship between a camera CX installed on a wall surface of a room, an omnidirectional microphone array apparatus MX installed on a ceiling surface of the room, and a position of a sound source P. FIG. 81 is a diagram illustrating a horizontal angle and a vertical angle which are computed by using a microphone coordinate system of an omnidirectional microphone array apparatus 2 converted from a camera coordinate system of a PTZ camera apparatus 1 and which are directed from the omnidirectional microphone array apparatus 2 toward the sound source position P.

[0185] As illustrated in FIG. 80, for example, a case is assumed in which the camera CX (for example, the PTZ camera apparatus 1) is installed on the wall surface of the room, and the omnidirectional microphone array apparatus MX (for example, the omnidirectional microphone array apparatus 2) is installed on the ceiling surface of the room. In this case, when the entire space is observed as a single coordinate system, if a position (that is, a vector V.sub.CXMX) from the camera CX to the omnidirectional microphone array apparatus MX is known, a position (that is, a vector V.sub.MXP) of the sound source P viewed from the omnidirectional microphone array apparatus MX can be specified by using a position (that is, a vector V.sub.CXP) of the sound source P viewed from the camera CX. In other words, Equation (2) is established (refer to FIG. 80).

[Equation 2]

{right arrow over (V.sub.MXP)}={right arrow over (V.sub.CXP)}{right arrow over (V.sub.CXMX)}(2)

[0186] However, a position of the sound source P on an image captured by the camera CX is represented by coordinates corresponding to an independent coordinate system (hereinafter, referred to as a camera coordinate system) of the camera CX, and a position of the sound source P which is a sound collection target of the omnidirectional microphone array apparatus MX is represented by coordinates corresponding to an independent coordinate system (hereinafter, referred to as a microphone coordinate system) of the omnidirectional microphone array apparatus MX. For this reason, a coordinate transform process is required to compute a position of the sound source P viewed from the omnidirectional microphone array apparatus MX by using a position of the sound source P based on the camera coordinate system, viewed from the camera CX.

[0187] Coordinates (X,Y,Z) corresponding to the microphone coordinate system are expressed by Equation (3) by using coordinates (x,y,z) corresponding to the camera coordinate system, and a predetermined coordinate transform matrix U.

[00002]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.3\right]& \\ \left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=\left[U\right]\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]=\left[\begin{array}{cccc}{U}_{11}& U_{12}& U_{13}& U_{14}\\ U_{21}& U_{22}& U_{23}& U_{24}\\ U_{31}& U_{32}& U_{33}& U_{34}\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]& \left(3\right)\end{array}$

[0188] As a specific example, in a case where the x axis, the y axis, and the z axis of the camera coordinate system are rotated about the z axis by degrees, rotated about the x axis by a degrees, and rotated about the y axis by degrees so that the coordinates of the camera CX are moved in parallel by (x.sub.0,y.sub.0,z.sub.0), the coordinate transform matrix U is expressed by Equation (4). In addition, in Equation (4), a parallel movement, an order of integration in a rotation matrix of the coordinate axes, or selection of a rotation axis is not limited thereto.

[00003]$\begin{array}{cc}.Math.\left[\mathrm{Equation}.Math..Math.4\right]& \\ \left[\begin{array}{cccc}{U}_{11}& U_{12}& U_{13}& U_{14}\\ U_{21}& U_{22}& U_{23}& U_{24}\\ U_{31}& U_{32}& U_{33}& U_{34}\\ 0& 0& 0& 1\end{array}\right]=\left[\begin{array}{cccc}1& 0& 0& -x_0\\ 0& 1& 0& -y_0\\ 0& 0& 1& -z_0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}\cos .Math..Math.& 0& \sin .Math..Math.& 0\\ 0& 1& 0& 0\\ -\sin .Math..Math.& 0& \cos .Math..Math.& 0\\ 0& 0& 0& 1\end{array}\right].Math.\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& \cos .Math..Math.& -\sin .Math..Math.& 0\\ 0& \sin .Math..Math.& \cos .Math..Math.& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}\cos .Math..Math.& -\sin .Math..Math.& 0& 0\\ \sin .Math..Math.& \cos .Math..Math.& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]& \left(4\right)\end{array}$

[0189] In order to compute the coordinate transform matrix U, a method using a design drawing (layout) of a space (for example, a room) in which the camera CX and the omnidirectional microphone array apparatus MX are installed may be employed (refer to FIG. 81, for example,), and a method using an actually measured result may be employed in a case where there is no design drawing. Further, the coordinate transform matrix U is computed in a case where a single omnidirectional microphone array apparatus 2 is installed for a single camera CX. However, also in a case where a plurality of cameras CX and a single omnidirectional microphone array apparatus MX are installed or a case where a plurality of cameras CX and a plurality of omnidirectional microphone array apparatuses MX are installed, the coordinate transform matrix U between each camera CX and each omnidirectional microphone array apparatus MX is required to be computed.

[0190] Here, for example, it is assumed that data regarding coordinates (x.sub.0,y.sub.0,z.sub.0) in the camera coordinate system, indicating a distance between the PTZ camera apparatus 1 as the camera CX and the omnidirectional microphone array apparatus 2 as the omnidirectional microphone array apparatus MX, and the camera coordinate system and the microphone coordinate system are predefined, and an angle formed by a horizontal reference axis of the camera is known. The user obtains data regarding coordinates of the PTZ camera apparatus 1 in an orthogonal coordinate system and the angle by using a layout diagram and inputs the data and the angle via the operation unit 32.

[0191] Hereinafter, with reference to FIG. 81, a description will be made of an example in which the coordinate transform processing section 34z of the signal processing unit 33 of the directionality control apparatus 3 computes the coordinate transform matrix U and sound collection direction coordinates (,) in the microphone coordinate system by using the coordinate transform matrix U.

[0192] As illustrated in FIG. 81, the PTZ camera apparatus 1 is installed on the wall surface of the sound collection space (for example, a room), and the omnidirectional microphone array apparatus 2 is installed on the ceiling surface of the sound collection space (for example, a room). It is assumed that the wall surface is vertical (perpendicular) to a horizontal floor surface, and the ceiling surface is parallel (horizontal) to the horizontal floor surface.

[0193] In the camera coordinate system of the PTZ camera apparatus 1, an origin O.sub.C is a rotation center of the PTZ camera apparatus 1 in the panning direction and the tilting direction, an x-y plane is defined in parallel to a pedestal of the PTZ camera apparatus 1, and this x-y plane is parallel to the wall surface. In addition, the x axis is directed vertically upward.

[0194] First, the PTZ camera apparatus 1 transforms coordinates (r,,) in a spherical coordinate system of the camera coordinate system into coordinates (x,y,z) in the orthogonal coordinate system according to Equation (5), and transmits data regarding the coordinates (x,y,z) having undergone the transform process to the directionality control apparatus 3.

[00004]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.5\right]& \\ \left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{c}r.Math..Math.\sin .Math..Math..Math..Math.\cos .Math..Math.\\ r.Math..Math.\sin .Math..Math..Math..Math.\sin .Math..Math.\\ r.Math..Math.\cos .Math..Math.\end{array}\right]& \left(5\right)\end{array}$

[0195] An origin .sub.M of the omnidirectional microphone array apparatus 2 viewed from the PTZ camera apparatus 1 is provided at the position of (x.sub.0,y.sub.0,z.sub.0) when viewed from the orthogonal coordinate system of the PTZ camera apparatus 1. The microphone coordinate system of the omnidirectional microphone array apparatus 2 has the origin O.sub.M as a center of the omnidirectional microphone array apparatus 2, an array formation plane of the omnidirectional microphone array apparatus 2 is an X-Y plane, and the X-Y plane is parallel to the ceiling surface. In addition, the Z axis is perpendicular to the array formation plane (X-Y plane) and is directed vertically downward.

[0196] The coordinate transform processing section 34z of the signal processing unit 33 of the directionality control apparatus 3 moves in the origin O.sub.C of the PTZ camera apparatus 1 parallel by (x.sub.0,y.sub.0,z.sub.0) by using the data regarding the coordinates (x,y,z) in the orthogonal coordinate system of the PTZ camera apparatus 1 received from the PTZ camera apparatus 1, rotates the origin O.sub.C about the y axis by 90 degrees, and then rotates the origin O.sub.C about the z axis by degrees, so as to compute data regarding the coordinates (X,Y,Z) in the orthogonal coordinate system of the omnidirectional microphone array apparatus 2 (refer to Equation (6)). In addition, in Equation (6), a parallel movement or an order of integration in a rotation matrix of the coordinate axes is not particularly limited, and the content of each vector changes depending on rotation about a rotation axis and an order of parallel movement according to an approach to coordinate transform in mathematics.

[00005]$\begin{array}{cc}.Math.\left[\mathrm{Equation}.Math..Math.6\right]& \\ \left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=\left[\begin{array}{cccc}\cos .Math..Math.& \sin .Math..Math.& 0& 0\\ -\sin .Math..Math.& \cos .Math..Math.& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}0& 0& 1& 0\\ 0& 1& 0& 0\\ -1& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& -x_0\\ 0& 1& 0& -y_0\\ 0& 0& 1& -z_0\\ 0& 0& 0& 1\end{array}\right].Math.\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]& \left(6\right)\end{array}$

[0197] The coordinate transform processing section 34z of the signal processing unit 33 of the directionality control apparatus 3 transforms the coordinates (X,Y,Z) in the orthogonal coordinate system of the omnidirectional microphone array apparatus 2 into coordinates (R,,) in the spherical coordinate system of the omnidirectional microphone array apparatus 2 according to Equation (7) based on the computation result using Equation (6). Thus, the directionality control apparatus 3 can compute the coordinates (,) in the spherical coordinate system of the omnidirectional microphone array apparatus 2 as the sound collection direction coordinates (,). Further, the output control section 35 of the signal processing unit 33 of the directionality control apparatus 3 forms the directionality in the sound collection direction coordinates (,).

[00006]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.7\right]& \\ \left[\begin{array}{c}R\\ \\ \end{array}\right]=\left[\begin{array}{c}\sqrt{X^2+Y^2+Z^2}\\ {\cos }^{-1}.Math.\{Z/\sqrt{X^2+Y^2+Z^2}\\ {\tan }^{-1}.Math.\left\{Y/Z\right\}\end{array}\right]& \left(7\right)\end{array}$

[0198] In addition, the above-described example of computing the sound collection direction coordinates (,) has been described with the position of the PTZ camera apparatus 1 as a reference, but is not limited to a case of using the position of the PTZ camera apparatus 1 as a reference as illustrated in FIG. 81, and, for example, a position of the sound source P may be used as a reference.

[0199] Further, a description has been made of a case where the PTZ camera apparatus 1 sends data which is transformed into orthogonal coordinates, but data of the camera coordinate system may be transmitted to the directionality control apparatus 3, and the directionality control apparatus 3 may perform coordinate transform on the data.

[0200] Hereinafter, a description will be made of an example of a method in which the camera apparatus 11 and the omnidirectional microphone array apparatus 2 specifically compute a sound collection direction of the omnidirectional microphone array apparatus 2 for a sound position viewed from the camera apparatus 11 based on an actually measured value from the camera apparatus 11 to the sound position without a design drawing (layout diagram) of an installation space (for example, a room).

Method of Computing Coordinates (.SUB.MAh.,.SUB.MAv.) Indicating Sound Collection Direction of Microphone Array Apparatus 2

[0201] Next, with reference to FIGS. 7 to 18, a detailed description will be made of a method of computing the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 in the signal processing unit 33 of the directionality control apparatus 3. Herein, a description will be made of a total of four types of computation methods.

First Computation Method

[0202] In a first computation method, a reference point O is provided in a direction of the optical axis CX of the camera apparatus 11.

[0203] The signal processing unit 33 computes the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 based on:

[0204] (1) a horizontal component (direction) distance L.sub.CMh of a distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2;

[0205] (2) a distance L.sub.CO and a depression angle .sub.CO from the camera apparatus 11 to the reference point O;

[0206] (3) a distance L.sub.MO and a depression angle .sub.MO from the microphone array apparatus 2 to the reference point O;

[0207] (4) respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from a horizontal surface;

[0208] (5) a horizontal angle .sub.CAh and a vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A; and

[0209] (6) a height H.sub.A of the sound collection region central position A from the horizontal surface.

[0210] In the first computation method, the input parameters in step ST2 illustrated in FIG. 5 include:

[0211] (1) the horizontal component (direction) distance L.sub.CMh of the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2;

[0212] (2) the distance L.sub.CO and the depression angle .sub.CO from the camera apparatus 11 to the reference point O;

[0213] (3) the distance L.sub.MO and the depression angle .sub.MO from the microphone array apparatus 2 to the reference point O; and

[0214] (4) the respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from the horizontal surface.

[0215] (1) The horizontal component (direction) distance L.sub.CMh of the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2 is a fixed value defined when the camera apparatus 11 and the microphone array apparatus 2 are initially installed.

[0216] (2) The distance L.sub.CO and the depression angle .sub.CO from the camera apparatus 11 to the reference point O can be easily measured, for example, by the user causing a laser range finder to be directed toward the camera apparatus 11 at the position of the reference point O.

[0217] (3) The distance L.sub.MO and the depression angle .sub.MO from the microphone array apparatus 2 to the reference point O can be easily measured, for example, by the user causing the laser range finder to be directed toward the medical examination support system 2 at the position of the reference point O.

[0218] (4) The respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from the horizontal surface are fixed values defined when the camera apparatus 11 and the microphone array apparatus 2 are initially installed, and are fixed values defined when the position of the reference point O is determined.

[0219] In addition, in the first computation method, in the coordinates (L.sub.CA,.sub.CAh,.sub.CAv) in step ST12 illustrated in FIG. 6,

[0220] (5) the horizontal angle .sub.CAh and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A are used, and are acquired by using a function of a well-known technique of the camera apparatus 11.

[0221] In addition, in the first computation method,

[0222] (6) the height H.sub.A of the sound collection region central position A from the horizontal surface is a fixed value which is set in advance, and is a selected value or an input value with a size of a person as H.sub.A, for example, in a case where there is the person around the sound collection region central position A when the position A is designated with the finger FG of the user. Alternatively, when the position A is designated with the finger FG of the user, a default value (for example, 1.5 m or 0.8 m) may be used in a case where the directionality control apparatus 3 determines that there is a person (for example, an adult or a child) at the designated position.

[0223] FIG. 7 is a diagram illustrating a positional relationship between the reference point O and the designated position A for computing a sound collection direction of the microphone array apparatus 2 on a screen of the display device 36 in the first computation method. The reference point O in the first computation method is present in the direction of the optical axis CX of the camera apparatus 11 and is thus located at the central point of the screen of the display device 36.

[0224] In addition, in the following description of the first computation method, the position A designated with the finger FG of the user is different from the position of the reference point O and is a position in the lower right direction of the reference point O (refer to FIG. 7).

[0225] FIGS. 8(A), 8(B) and 8(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the reference point O, and the sound collection region central position A in the first computation method. FIG. 8(A) is a perspective view. FIG. 8(B) is a horizontal direction plan view. FIG. 8(C) is a vertical direction sectional view taken along the line K-K of FIG. 8(B).

[0226] FIGS. 9(A), 9(B) and 9(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the reference point O, and the sound collection region central position A in the first computation method. FIG. 9(A) is a perspective view. FIG. 9(B) is a horizontal direction plan view. FIG. 9(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 9(B).

[0227] FIGS. 10(A), 10(B) and 10(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the reference point O, and the sound collection region central position A in the first computation method. FIG. 10(A) is a perspective view. FIG. 10(B) is a horizontal direction plan view. FIG. 10(C) is a vertical direction sectional view taken along the line R-R of FIG. 10(B).

[0228] Hereinafter, a detailed description will be made of the first computation method of the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 in the signal processing unit 33. In the first computation method, a reference line in a direction of 0 degrees of a horizontal angle of the microphone array apparatus 2 is directed toward the camera apparatus 11. In addition, a computation operation in the following description of each computation method will be described as being performed by the signal processing unit 33, but the signal processing unit 33 may be replaced with the coordinate transform processing section 34z.

[0229] The signal processing unit 33 computes a horizontal component distance L.sub.COh of the distance L.sub.CO from the camera apparatus 11 to the reference point O according to Equation (8) by using the distance L.sub.CO and the depression angle .sub.CO from the camera apparatus 11 to the reference point O.

[Equation 8]

L.sub.COh=L.sub.COcos .sub.CO (8)

[0230] The signal processing unit 33 computes a horizontal component distance L.sub.MOh of the distance L.sub.MO from the medical examination support system 2 to the reference point O according to Equation (9) by using the distance L.sub.MO and the depression angle .sub.MO from the microphone array apparatus 2 to the reference point O.

[Equation 9]

L.sub.MOh=L.sub.MOcos .sub.MO (9)

[0231] The signal processing unit 33 computes a cosine value cos .sub.COh of a horizontal angle .sub.COh of the depression angle .sub.CO from the camera apparatus 11 to the reference point O according to Equation (10) based on the cosine theorem for the triangle COM illustrated in FIG. 8(B) by using the respective computation results of Equations (8) and (9).

[00007]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.10\right]& \\ \cos .Math..Math.{}_{\mathrm{COh}}=\frac{L_{\mathrm{COh}}^2+L_{\mathrm{CMh}}^2-L_{\mathrm{MOh}}^2}{2.Math.L_{\mathrm{COh}}{L}_{\mathrm{CMh}}}& \left(10\right)\end{array}$

[0232] The signal processing unit 33 computes a cosine value cos .sub.MOh of a horizontal angle .sub.MOh of the depression angle .sub.MO from the camera apparatus 11 to the reference point O according to Equation (11) based on the cosine theorem for the triangle COM illustrated in FIG. 8(B) by using the respective computation results of Equations (8) and (9).

[00008]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.11\right]& \\ \cos .Math..Math.{}_{\mathrm{MOh}}=\frac{L_{\mathrm{MOh}}^2+L_{\mathrm{CMh}}^2-L_{\mathrm{COh}}^2}{2.Math.L_{\mathrm{MOh}}{L}_{\mathrm{CMh}}}& \left(11\right)\end{array}$

[0233] The signal processing unit 33 computes a vertical component distance L.sub.COv of the distance L.sub.CO from the camera apparatus 11 to the reference point O according to Equation (12) by using the respective heights H.sub.C and H.sub.O of the camera apparatus 11 and the reference point O from the horizontal surface and the respective computation results of Equations (8) and (10).

[Equation 12]

L.sub.COv={square root over ((L.sub.COhcos .sub.COh).sup.2+(H.sub.CH.sub.O).sup.2)}(12)

[0234] The signal processing unit 33 computes a vertical component distance L.sub.MOv of the distance L.sub.MO from the microphone array apparatus 2 to the reference point O according to Equation (13) by using the respective heights H.sub.M and H.sub.O of the microphone array apparatus 2 and the reference point O from the horizontal surface and the respective computation results of Equations (9) and (11).

[Equation 13]

L.sub.MOv={square root over ((L.sub.MOhcos .sub.MOh).sup.2+(H.sub.MH.sub.O).sup.2)}(13)

[0235] The signal processing unit 33 computes a vertical component distance L.sub.CMv (=L.sub.CM) of the distance L.sub.CM from the camera apparatus 11 to microphone array apparatus 2 according to Equation (14) by using the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the microphone array apparatus 2 from the horizontal surface and the horizontal component distance L.sub.CMh of the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

Equation [14]

L.sub.CMv=L.sub.CM={square root over (L.sub.CMh.sup.2+(H.sub.MH.sub.C).sup.2)}(14)

[0236] The signal processing unit 33 computes a cosine value cos .sub.COv of the vertical angle .sub.COv of the depression angle .sub.CO from the camera apparatus 11 to the reference point O according to Equation (15) based on the cosine theorem for the triangle COM illustrated in FIG. 8(C) by using the respective computation results of Equations (12) to (14).

[00009]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.15\right]& \\ \cos .Math..Math.{}_{\mathrm{COv}}=\frac{L_{\mathrm{COv}}^2+L_{\mathrm{CMv}}^2-L_{\mathrm{MOv}}^2}{2.Math.L_{\mathrm{COv}}{L}_{\mathrm{CMv}}}& \left(15\right)\end{array}$

[0237] The signal processing unit 33 computes a sine value sin .sub. of an angle .sub. between a direction which is directed toward the microphone array apparatus 2 from the camera apparatus 11 illustrated in FIG. 8(C) and the horizontal surface according to Equation (16) by using the computation result of Equation (14) and the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the microphone array apparatus 2 from the horizontal surface.

[00010]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.16\right]& \\ \sin .Math..Math.{}_{}=\frac{H_M-H_C}{L_{\mathrm{CMv}}}& \left(16\right)\end{array}$

[0238] Next, the signal processing unit 33 computes a distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (17) by using the respective computation results of Equations (15) and (16), the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A, the height H.sub.A of the sound collection region central position A from the horizontal surface, and the height H.sub.C of the camera apparatus 11 from the horizontal surface.

[00011]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.17\right]& \\ L_{\mathrm{CA}}=\frac{H_C-H_A}{\sin \left({}_{\mathrm{COv}}+{}_{\mathrm{CAv}}-{}_{}\right)}& \left(17\right)\end{array}$

[0239] The signal processing unit 33 computes a horizontal component distance L.sub.CAh of the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (18) by using the respective computation results of Equations (15) to (17) and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A.

[00012]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.18\right]& \\ L_{\mathrm{CAh}}=L_{\mathrm{CA}}\cos \left({}_{\mathrm{COv}}+{}_{\mathrm{CAv}}-{}_{}\right)=\frac{H_C-H_A}{\tan \left({}_{\mathrm{COv}}+{}_{\mathrm{CAv}}-{}_{}\right)}& \left(18\right)\end{array}$

[0240] The signal processing unit 33 computes a horizontal component distance L.sub.MAh of the distance from the microphone array apparatus 2 to the sound collection region central position A according to Equation (19) based on the cosine theorem for the triangle CAM illustrated in FIG. 8(B) by using the respective computation results of Equations (10) and (18), the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position A, and the horizontal component distance L.sub.CMh of the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

[00013]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.19\right]& \\ L_{\mathrm{MAh}}=\sqrt{(L_{\mathrm{CAh}}^2+L_{\mathrm{CMh}}^2-2.Math.L_{\mathrm{CAh}}{L}_{\mathrm{CMh}}\cos \left({}_{\mathrm{COh}}+{}_{\mathrm{CAh}}\right)}& \left(19\right)\end{array}$

[0241] The signal processing unit 33 computes a cosine value cos .sub.MAh of the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (20) based on the cosine theorem for the triangle CAM illustrated in FIG. 8(B) by using the respective computation results of Equations (18) and (19), and the horizontal direction distance L.sub.CMh from the camera apparatus 11 to the microphone array apparatus 2.

[0242] Consequently, the signal processing unit 33 can compute the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (21).

[00014]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.20\right]& \\ \cos .Math..Math.{}_{\mathrm{MAh}}=\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CMh}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CMh}}}& \left(20\right)\\ \left[\mathrm{Equation}.Math..Math.21\right]& \\ {}_{\mathrm{MAh}}=\arccos \left(\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CMh}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CMh}}}\right)& \left(21\right)\end{array}$

[0243] In addition, the signal processing unit 33 computes a tangent value tan .sub.MAv of the vertical angle .sub.MAv of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (22) based on a tangent for the triangle MAP illustrated in FIG. 10(C) by using the computation result of Equation (21), the height H.sub.M of the microphone array apparatus 2 from the horizontal surface and the height H.sub.A of the sound collection region central position A from the horizontal surface.

[0244] Consequently, the signal processing unit 33 can compute the vertical angle .sub.MAv of the the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (23).

[00015]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.22\right]& \\ \tan .Math..Math.{}_{\mathrm{MAv}}=\frac{H_M-H_A}{L_{\mathrm{MAh}}}& \left(22\right)\\ \left[\mathrm{Equation}.Math..Math.23\right]& \\ {}_{\mathrm{MAv}}=\arctan \left(\frac{H_M-H_A}{L_{\mathrm{MAh}}}\right)& \left(23\right)\end{array}$

[0245] In the above-described way, in the first computation method, the reference point O is provided in the direction of the optical axis CX of the camera apparatus 11, and the directionality control apparatus 3 uses, as the respective input parameters, the distance L.sub.CO and the depression angle .sub.CO between the camera apparatus 11 and the reference point O, the distance L.sub.MO and the depression angle .sub.MO between the microphone array apparatus 2 and the reference point O, the horizontal component distance L.sub.CMh of the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2, and the respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from the horizontal surface.

[0246] Further, the directionality control apparatus 3 computes, as a sound collection direction, the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed toward the sound collection region central position A corresponding to the position A designated with the finger FG of the user in a video of a predetermined region captured as a monitoring target of the camera apparatus 11, that is, a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A, with the position of the microphone array apparatus 2 as a reference, by using the respective input parameters.

[0247] Consequently, according to the first computation method, the directionality control system 10 of the present embodiment can form the sound collection directionality in the direction of the sound collection region central position A designated with the position of the microphone array apparatus 2 as a reference with high accuracy and can thus collect audio data in the corresponding direction with high accuracy.

Second Computation Method

[0248] In a second computation method, instead of the optical axis CX of the camera apparatus 11, a position of a marker MAK which is suspended vertically downward from the microphone array apparatus 2 by a string STR as the reference point O.

[0249] In addition, a length of the string STR is less than the height H.sub.M of the microphone array apparatus 2 from the horizontal surface. Further, the marker MAK is, for example, a ball in a color which can be easily recognized by the user when imaged by the camera apparatus 11.

[0250] The signal processing unit 33 computes the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 based on:

[0251] (1) a distance L.sub.CO from the camera apparatus 11 to the marker MAK which is suspended vertically downward from the microphone array apparatus 2;

[0252] (2) a distance L.sub.MO from the microphone array apparatus 2 to the marker MAK;

[0253] (3) a horizontal angle .sub.CMh (=.sub.COh) and a vertical angle .sub.COv between a direction of the optical axis CX of the camera apparatus 11 and a direction which is directed from the camera apparatus 11 toward the marker MAK; and

[0254] (4) a distance L.sub.CA, a horizontal angle .sub.CAh, and a vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A.

[0255] In the second computation method, the input parameters in step ST2 illustrated in FIG. 5 include:

[0256] (1) the distance L.sub.CO from the camera apparatus 11 to the marker MAK which is suspended vertically downward from the microphone array apparatus 2;

[0257] (2) the distance L.sub.MO from the microphone array apparatus 2 to the marker MAK; and

[0258] (3) the horizontal angle .sub.CMh and the vertical angle .sub.COv between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the marker MAK.

[0259] (1) The distance L.sub.CO from the camera apparatus 11 to the marker MAK which is suspended vertically downward from the microphone array apparatus 2 is acquired by using a function of a well-known technique of the camera apparatus 11. For example, a focal length obtained when the camera apparatus 11 focuses and images the marker MAK may be used as L.sub.CO.

[0260] (2) The distance L.sub.MO from the microphone array apparatus 2 to the marker MAK is the same as the length of the string STR.

[0261] (3) The horizontal angle .sub.CMh and the vertical angle .sub.COv between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the marker MAK is acquired by using a function of a well-known technique of the camera apparatus 11.

[0262] In addition, in the second computation method,

[0263] (4) the distance L.sub.CA, the horizontal angle .sub.CAh, and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A are acquired by using a function of a well-known technique of the camera apparatus 11 in step ST12 illustrated in FIG. 6.

[0264] FIG. 11 is a diagram illustrating a positional relationship between the reference point O and the designated position A for computing a sound collection direction of the microphone array apparatus 2 on a screen of the display device 36 in the second computation method. The reference point O in the second computation method is not present in the direction of the optical axis CX of the camera apparatus 11 and thus is not present at the central point of the screen of the display device 36 and is located in the upper left direction from the central point of the screen of the display device 36, for example.

[0265] In addition, in the following description of the second computation method, the position A designated with the finger FG of the user is different from the positions of the reference point O and the central point of the screen indicating the optical axis direction and is a position in the lower right direction of the reference point O (refer to FIG. 11).

[0266] FIGS. 12(A), 12(B) and 12(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, a position of the marker MAK (reference point O), and the sound collection region central position A in the second computation method. FIG. 12(A) is a perspective view. FIG. 12(B) is a horizontal direction plan view. FIG. 12(C) is a vertical direction sectional view taken along the line K-K of FIG. 12(B).

[0267] FIGS. 13(A), 13(B) and 13(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the position of the marker MAK (reference point O), and the sound collection region central position A in the second computation method. FIG. 13(A) is a perspective view. FIG. 13(B) is a horizontal direction plan view. FIG. 13(C) is a vertical direction sectional view taken along the line R-R of FIG. 13(B).

[0268] Hereinafter, a detailed description will be made of the second computation method of the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 in the signal processing unit 33. In the second computation method, a reference line in a direction of 0 degrees of a horizontal angle of the microphone array apparatus 2 is directed toward the camera apparatus 11, and the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the microphone array apparatus 2 from the horizontal surface are assumed to be the same as each other.

[0269] The signal processing unit 33 computes a sine value sin .sub.COv of an angle .sub.COv between a direction which is directed toward the marker MAK from the camera apparatus 11 and a direction which is directed toward the microphone array apparatus 2 from the camera apparatus 11 according to Equation (24) by using the distance L.sub.CO from the camera apparatus 11 to the marker MAK which is suspended vertically downward from the microphone array apparatus 2, and the distance L.sub.MO from the microphone array apparatus 2 to the marker MAK.

[00016]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.24\right]& \\ \sin .Math..Math.{}_{\mathrm{COv}}^{}=\frac{L_{\mathrm{MO}}}{L_{\mathrm{CO}}}& \left(24\right)\end{array}$

[0270] The signal processing unit 33 computes a distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2 according to Equation (25) by using the computation result of Equation (24) and the distance L.sub.CO from the camera apparatus 11 to the marker MAK which is suspended vertically downward from the microphone array apparatus 2.

[Equation 25]

L.sub.CM=L.sub.COcos .sub.COv (25)

[0271] The signal processing unit 33 computes a horizontal component distance L.sub.CAh of the the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (26) by using the respective computation results of Equations (24) and (25), the distance L.sub.CA and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A, and the vertical angle .sub.COv between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the marker MAK.

[Equation 26]

L.sub.CAh=L.sub.CAcos(.sub.COv+.sub.COv+.sub.CAv) (26)

[0272] The signal processing unit 33 computes a horizontal component distance L.sub.MAh of the distance from the microphone array apparatus 2 to the sound collection region central position A according to Equation (27) based on the cosine theorem for the triangle CAM illustrated in FIG. 12(B) by using the respective computation results of Equations (25) and (26), the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position A, and the horizontal angle .sub.CMh between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the marker MAK.

[00017]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.27\right]& \\ L_{\mathrm{MAh}}=\sqrt{(L_{\mathrm{CAh}}^2+L_{\mathrm{CMh}}^2-2.Math.L_{\mathrm{CAh}}{L}_{\mathrm{CMh}}\cos \left({}_{\mathrm{CMh}}+{}_{\mathrm{CAh}}\right)}& \left(27\right)\end{array}$

[0273] The signal processing unit 33 computes a cosine value cos .sub.MAh of the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (28) based on the cosine theorem for the triangle CAM illustrated in FIG. 12(B) by using the respective computation results of Equations (25), (26) and (27), and the horizontal direction distance L.sub.CMh from the camera apparatus 11 to the microphone array apparatus 2.

[0274] Consequently, the signal processing unit 33 can compute the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (29).

[00018]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.28\right]& \\ \cos .Math..Math.{}_{\mathrm{MAh}}=\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}& \left(28\right)\\ \left[\mathrm{Equation}.Math..Math.29\right]& \\ {}_{\mathrm{MAh}}=\arccos \left(\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}\right)& \left(29\right)\end{array}$

[0275] The signal processing unit 33 computes a vertical component distance L.sub.CAv of the the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (30) by using the distance L.sub.CA and the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position A, and the horizontal angle .sub.CMh between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the marker MAK.

[Equation 30]

L.sub.CAv=L.sub.CAcos(.sub.CMh+.sub.CAh) (30)

[0276] The signal processing unit 33 computes a vertical component distance L.sub.MAv from the microphone array apparatus 2 to the sound collection region central position A according to Equation (31) based on the cosine theorem for the triangle CAM illustrated in FIG. 12(B) by using the respective computation results of Equations (25), (26) and (30).

[00019]$\begin{array}{cc}.Math.\left[\mathrm{Equation}.Math..Math.31\right]& \\ L_{\mathrm{MAv}}=\sqrt{(L_{\mathrm{CAv}}^2+L_{\mathrm{CM}}^2-2.Math.L_{\mathrm{CAv}}{L}_{\mathrm{CM}}\cos \left({}_{\mathrm{COv}}+{}_{\mathrm{COv}}^{}+{}_{\mathrm{CAv}}\right)}& \left(31\right)\end{array}$

[0277] The signal processing unit 33 computes a cosine value cos .sub.MAv of the angle .sub.MAv between the direction which is directed from the microphone array apparatus 2 toward the camera apparatus 11 and the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A according to Equation (32) based on the cosine theorem for the triangle MAP illustrated in FIG. 12(C) by using the respective computation results of Equations (25), (30) and (31).

[00020]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.32\right]& \\ \cos .Math..Math.{}_{\mathrm{MAv}}^{}=\frac{L_{\mathrm{MAv}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAv}}^2}{2.Math.L_{\mathrm{MAv}}{L}_{\mathrm{CM}}}& \left(32\right)\end{array}$

[0278] The signal processing unit 33 computes a sine value sin .sub.MAv for the angle .sub.MAv between the direction which is directed from the microphone array apparatus 2 toward the camera apparatus 11 and the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A according to Equation (33) based on a sine for the triangle MAS illustrated in FIG. 13(C).

[0279] The signal processing unit 33 computes a tangent value tan .sub.MAv of the vertical angle .sub.MAv of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (34) by using the respective computation results of Equations (27) and (33).

[0280] Consequently, the signal processing unit 33 can compute the vertical angle .sub.MAv of the the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (35).

[00021]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.33\right]& \\ \sin .Math..Math.{}_{\mathrm{MAv}}^{}=\frac{H_M-H_A}{L_{\mathrm{MAv}}}& \left(33\right)\\ \left[\mathrm{Equation}.Math..Math.34\right]& \\ \tan .Math..Math.{}_{\mathrm{MAv}}=\frac{H_M-H_A}{L_{\mathrm{MAh}}}& \left(34\right)\\ \left[\mathrm{Equation}.Math..Math.35\right]& \\ {}_{\mathrm{MAv}}=\arctan \left(\frac{H_M-H_A}{L_{\mathrm{MAh}}}\right)& \left(35\right)\end{array}$

[0281] In the above-described way, in the second computation method, the marker MAK which is suspended vertically downward from the microphone array apparatus 2 by using the string is provided as the reference point O, and the directionality control apparatus 3 uses, as the respective input parameters, the distance L.sub.CO and the vertical angle .sub.COv of the depression angle .sub.CO from the camera apparatus 11 to the marker MAK, the angle .sub.CMh between the direction of the optical axis CX of the camera apparatus 11 and the direction which is directed from the camera apparatus 11 toward the medical examination support system 2, and the distance L.sub.MO from the microphone array apparatus 2 to the marker MAK.

[0282] Further, the directionality control apparatus 3 computes, as a sound collection direction, the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed toward the sound collection region central position A corresponding to the position A designated with the finger FG of the user in a video of a predetermined region captured as a monitoring target of the camera apparatus 11, that is, a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A, with the position of the microphone array apparatus 2 as a reference, by using the respective input parameters.

[0283] Consequently, according to the second computation method, the directionality control system 10 of the present embodiment can more easily compute the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed toward the sound collection region central position A from the microphone array apparatus 2 since the number of input parameters is smaller than that in the first computation method. In addition, the directionality control system 10 can form the sound collection directionality in the direction of the sound collection region central position A designated with the position of the microphone array apparatus 2 as a reference with high accuracy and can thus collect audio data in the corresponding direction with high accuracy.

[0284] In addition, in the second computation method, a position of the marker MAK may be a position of an intersection between the direction of the optical axis CX of the camera apparatus 11 and the vertically downward direction of the microphone array apparatus 2. In this case, since the angle .sub.COv between the direction of the optical axis CX and the direction which is directed from the camera apparatus 11 toward the marker MAK is 0 (zero), an amount of computation of the coordinates (.sub.MAh,.sub.MAv) indicating the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A is reduced in the signal processing unit 33.

[0285] Further, in the second computation method, in a case where the microphone array apparatus 2 has a function of performing irradiation with laser light, the microphone array apparatus 2 performs irradiation with laser light vertically downward without using the marker MAK, and the camera apparatus 11 analyzes a captured image of an irradiation point (for example, a point with a certain height in the vertically downward direction of the microphone array apparatus 2) of the laser light. Thus, the signal processing unit 33 can compute the coordinates (.sub.MAh,.sub.MAv) indicating the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A in the same manner as in the above-described second computation method.

Third Computation Method

[0286] In a third computation method, a sound source is provided as the reference point O in the direction of the optical axis CX of the camera apparatus 11.

[0287] The signal processing unit 33 computes the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 based on:

[0288] (1) a distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2;

[0289] (2) a distance L.sub.CO from the camera apparatus 11 to a predetermined sound source position (reference point O) in the optical axis direction;

[0290] (3) a horizontal angle .sub.MOh and a vertical angle .sub.MOv from the microphone array apparatus 2 to the sound source position (reference point O);

[0291] (4) respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the sound source position (reference point O) from a horizontal surface; and

[0292] (5) a distance L.sub.CA, a horizontal angle .sub.CAh, and a vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A.

[0293] In the third computation method, the input parameters in step ST2 illustrated in FIG. 5 include:

[0294] (1) the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2;

[0295] (2) the distance L.sub.CO from the camera apparatus 11 to the predetermined sound source position (reference point O) in the optical axis direction;

[0296] (3) the horizontal angle .sub.MOh and the vertical angle .sub.MOv from the microphone array apparatus 2 to the sound source position (reference point O); and

[0297] (4) the respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the sound source position (reference point O) from the horizontal surface.

[0298] (1) The distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2 is a fixed value defined when the camera apparatus 11 and the microphone array apparatus 2 are initially installed.

[0299] (2) The distance L.sub.CO from the camera apparatus 11 to the predetermined sound source position (reference point O) in the optical axis direction can be easily measured, for example, by the user causing a laser range finder to be directed toward the camera apparatus 11 at the position of the reference point O.

[0300] (3) The horizontal angle .sub.MOh and the vertical angle .sub.MOv from the microphone array apparatus 2 to the sound source position (reference point O) may be measured by using a function (for example, a sound source detection function) of a well-known technique of the microphone array apparatus 2.

[0301] (4) The respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from the horizontal surface are fixed values defined when the camera apparatus 11 and the microphone array apparatus 2 are initially installed, and are fixed values defined when the position of the reference point O is determined.

[0302] In addition, in the third computation method,

[0303] (5) the distance L.sub.CA, the horizontal angle .sub.CAh, and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A are are acquired by using a function of a well-known technique of the camera apparatus 11 in step ST12 illustrated in FIG. 6.

[0304] Further, in the third computation method, since the sound source serving as the reference point O is present in the direction of the optical axis CX of the camera apparatus 11, a positional relationship between the reference point O and the designated position A for computing a sound collection direction of the microphone array apparatus 2 on a screen of the display device 36 is the positional relationship illustrated in FIG. 7, and thus description thereof will be omitted.

[0305] FIGS. 14(A), 14(B) and 14(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the sound source position (reference point O), and the sound collection region central position A in the third computation method. FIG. 14(A) is a perspective view. FIG. 14(B) is a horizontal direction plan view. FIG. 14(C) is a vertical direction sectional view taken along the line K-K of FIG. 14(B).

[0306] FIGS. 15(A), 15(B) and 15(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the sound source position (reference point O), and the sound collection region central position A in the third computation method. FIG. 15(A) is a perspective view. FIG. 15(B) is a horizontal direction plan view. FIG. 15(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 15(B).

[0307] FIGS. 16(A), 16(B) and 16(C) illustrate each positional relationship between the camera apparatus 11 and the microphone array apparatus 2 of the directionality control system 10, the sound source position (reference point O), and the sound collection region central position A in the third computation method. FIG. 16(A) is a perspective view. FIG. 16(B) is a horizontal direction plan view. FIG. 16(C) is a vertical direction sectional view taken along the line R-R of FIG. 16(B).

[0308] Hereinafter, a detailed description will be made of the third computation method of the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 in the signal processing unit 33. In the third computation method, a reference line in a direction of 0 degrees of a horizontal angle of the microphone array apparatus 2 is not directed toward the camera apparatus 11, and the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the microphone array apparatus 2 from the horizontal surface are assumed to be the same as each other.

[0309] First, the signal processing unit 33 computes a horizontal component distance L.sub.COh of the distance L.sub.CO from the camera apparatus 11 to the predetermined sound source position (reference point O) in the optical axis direction according to Equation (36) by using the distance L.sub.CO from the camera apparatus 11 to the predetermined sound source position (reference point O) in the optical axis direction, and the respective heights H.sub.C and H.sub.O of the camera apparatus 11 and the sound source position (reference point O) in the optical axis direction from the horizontal surface.

[Equation 36]

L.sub.COh={square root over ((L.sub.CO).sup.2(H.sub.CH.sub.O).sup.2)}(36)

[0310] The signal processing unit 33 computes a horizontal component distance L.sub.MOh of the distance L.sub.MO from the microphone array apparatus 2 to the predetermined sound source position (reference point O) in the optical axis direction according to Equation (37) by using the vertical angle .sub.MOv from the microphone array apparatus 2 to the sound source position (reference point O), and the respective heights H.sub.M and H.sub.O of the microphone array apparatus 2 and the sound source position (reference point O) from the horizontal surface.

[00022]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.37\right]& \\ L_{\mathrm{MOh}}=\frac{\left(H_M-H_O\right)}{\tan .Math..Math.{}_{\mathrm{MOv}}}& \left(37\right)\end{array}$

[0311] The signal processing unit 33 computes a cosine value cos .sub.COh of a horizontal angle .sub.COh of the depression angle .sub.CO from the camera apparatus 11 to the sound source (reference point O) according to Equation (38) based on the cosine theorem for the triangle COM illustrated in FIG. 14(B) by using the respective computation results of Equations (36) and (37), and the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2, and, similarly, computes a cosine value cos(.sub.MOh.sub.MCh) of a horizontal angle (.sub.MOh.sub.MCh) of the depression angle (.sub.MO.sub.MC) in a direction which is directed toward the sound source (reference point O) from the reference line for the 0-degree direction of the microphone array apparatus 2 according to Equation (39).

[0312] Consequently, the signal processing unit 33 can compute an angle .sub.MCh between the reference line for the 0-degree direction of the microphone array apparatus 2 and the direction which is directed from the microphone array apparatus 2 toward the camera apparatus 11 according to Equation (40).

[00023]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.38\right]& \\ \cos .Math..Math.{}_{\mathrm{COh}}=\frac{L_{\mathrm{COh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{MOh}}^2}{2.Math.L_{\mathrm{COh}}{L}_{\mathrm{CM}}}& \left(38\right)\\ \left[\mathrm{Equation}.Math..Math.39\right]& \\ \cos .Math..Math.\left({}_{\mathrm{MOh}}-{}_{\mathrm{MCh}}\right)=\frac{L_{\mathrm{MOh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{COh}}^2}{2.Math.L_{\mathrm{CM}}{L}_{\mathrm{MOh}}}& \left(39\right)\\ \left[\mathrm{Equation}.Math..Math.40\right]& \\ {}_{\mathrm{MCh}}={}_{\mathrm{MOh}}-\arccos \left(\frac{L_{\mathrm{MOh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{COh}}^2}{2.Math.L_{\mathrm{CM}}{L}_{\mathrm{MOh}}}\right)& \left(40\right)\end{array}$

[0313] Next, the signal processing unit 33 computes a vertical component distance L.sub.COv of the distance L.sub.CO from the camera apparatus 11 to the sound source position (reference point O) according to Equation (41) by using the respective computation results of Equations (36) and (38) and the respective heights H.sub.C and H.sub.O of the camera apparatus 11 and the sound source (reference point O) from the horizontal surface.

[Equation 41]

L.sub.COv={square root over ((L.sub.COhcos .sub.COh).sup.2+(H.sub.CH.sub.O).sup.2)}(41)

[0314] The signal processing unit 33 computes a vertical component distance L.sub.MOv of the distance L.sub.MO from the microphone array apparatus 2 to the sound source (reference point O) according to Equation (42) by using the respective computation results of Equations (37) and (39) and the respective heights H.sub.M and H.sub.O of the microphone array apparatus 2 and the sound source position (reference point O) from the horizontal surface.

[Equation 42]

L.sub.MOv={square root over ((L.sub.MOhcos(.sub.MOh.sub.MCh)).sup.2+(H.sub.MH.sub.O).sup.2)}(42)

[0315] The signal processing unit 33 computes a cosine value cos .sub.COv of the angle .sub.COv between the direction which is directed from the camera apparatus 11 toward sound source (reference point O) and the direction which is directed from the camera apparatus 11 toward the microphone array apparatus 2 according to Equation (43) based on the cosine theorem for the triangle COM illustrated in FIG. 14(C) by using the respective computation results of Equations (41) and (42) and the distance L.sub.CM (=L.sub.CMv) between the camera apparatus 11 and the microphone array apparatus 2.

[00024]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.43\right]& \\ \cos .Math..Math.{}_{\mathrm{COv}}=\frac{L_{\mathrm{COv}}^2+L_{\mathrm{CMv}}^2-L_{\mathrm{MOv}}^2}{2.Math.L_{\mathrm{COv}}{L}_{\mathrm{CMv}}}& \left(43\right)\end{array}$

[0316] Next, the signal processing unit 33 computes a horizontal component distance L.sub.CAh of the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (44) by using the computation result of Equation (43), and the distance L.sub.CA and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A.

[Equation 44]

L.sub.CAh=L.sub.CAcos(.sub.COv+.sub.CAv) tm (44)

[0317] The signal processing unit 33 computes a horizontal component distance L.sub.MAh of the distance L.sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (45) based on the cosine theorem for the triangle CAM illustrated in FIG. 14(B) by using the respective computation results of Equations (39) and (44), and the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

[00025]$\begin{array}{cc}.Math.\left[\mathrm{Equation}.Math..Math.45\right]& \\ L_{\mathrm{MAh}}=\sqrt{L_{\mathrm{CAh}}^2+L_{\mathrm{CM}}^2-2.Math.L_{\mathrm{CAh}}{L}_{\mathrm{CM}}\cos \left({}_{\mathrm{COh}}+{}_{\mathrm{CAh}}\right)}& \left(45\right)\end{array}$

[0318] The signal processing unit 33 computes a cosine value cos(.sub.MAh.sub.MCh) of a horizontal angle (.sub.MAh.sub.MCh) of the depression angle (.sub.MA.sub.MC) in a direction which is directed toward the sound collection region central position A from the reference line for the 0-degree direction of the microphone array apparatus 2 according to Equation (46) based on the cosine theorem for the triangle CAM illustrated in FIG. 14(B) by using the respective computation results of Equations (44) and (45), and the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

[0319] Consequently, the signal processing unit 33 can compute a horizontal angle .sub.MAh of the depression angle .sub.MA in the direction which is directed from the reference line for the 0-degree direction of the microphone array apparatus 2 toward the sound collection region central position A according to Equation (47).

[00026]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.46\right]& \\ \cos \left({}_{\mathrm{MAh}}-{}_{\mathrm{MCh}}\right)=\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{CM}}{L}_{\mathrm{MAh}}}& \left(46\right)\\ \left[\mathrm{Equation}.Math..Math.47\right]& \\ {}_{\mathrm{MAh}}={}_{\mathrm{MCh}}+\arccos \left(\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{CM}}{L}_{\mathrm{MAh}}}\right)& \left(47\right)\end{array}$

[0320] Further, the signal processing unit 33 computes a vertical component distance L.sub.CAh of the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (48) by using the computation result of Equations (40), and the distance L.sub.CA and the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position A.

[Equation 48]

L.sub.CAv=L.sub.CAcos(.sub.MCh+.sub.CAh) (48)

[0321] The signal processing unit 33 computes a vertical component distance L.sub.MAv of the distance L.sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (49) based on the cosine theorem for the triangle CAM illustrated in FIG. 14(C) by using the respective computation results of Equations (41) and (48), the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position A, and the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

[00027]$\begin{array}{cc}.Math.\left[\mathrm{Equation}.Math..Math.49\right]& \\ L_{\mathrm{MAv}}=\sqrt{(L_{\mathrm{CAv}}^2+L_{\mathrm{CM}}^2-2.Math.L_{\mathrm{CAv}}{L}_{\mathrm{CM}}\cos \left({}_{\mathrm{COv}}+{}_{\mathrm{CAv}}\right)}& \left(49\right)\end{array}$

[0322] The signal processing unit 33 computes a cosine value cos .sub.MAv of the angle .sub.MAv between the direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A and the direction which is directed from the microphone array apparatus 2 toward the camera apparatus 11 in the K-K section illustrated in FIG. 14(B) according to Equation (50) based on the cosine theorem for the triangle CAM illustrated in FIG. 14(C) by using the respective computation results of Equations (48) and (49) and the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2.

[00028]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.50\right]& \\ \cos .Math..Math.{}_{\mathrm{MAv}}^{}=\frac{L_{\mathrm{MAv}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAv}}^2}{2.Math.L_{\mathrm{MAv}}{L}_{\mathrm{CM}}}& \left(50\right)\end{array}$

[0323] The signal processing unit 33 computes a difference (H.sub.MH.sub.A) between the heights H.sub.M and H.sub.A of the microphone array apparatus 2 and the sound collection region central position A from the horizontal surface according to Equation (51) by using the respective computation results of Equations (49) and (50).

[Equation 51]

H.sub.MH.sub.A=L.sub.MAvsin .sub.MAv (51)

[0324] The signal processing unit 33 computes a tangent value tan .sub.MAv of the vertical angle .sub.MAv of the depression angle .sub.MA which is directed toward the sound collection region central position A from the reference line for the 0-degree direction of the microphone array apparatus 2 according to Equation (52) based on a tangent for the triangle MSA illustrated in FIG. 16(C) by using the respective computation results of Equations (45) and (51).

[0325] Consequently, the signal processing unit 33 can compute the vertical angle .sub.MAv of the depression angle .sub.MA from the reference line for the 0-degree direction of the microphone array apparatus 2 to the sound collection region central position A according to Equation (53).

[00029]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.52\right]& \\ \tan .Math..Math.{}_{\mathrm{MAv}}=\frac{H_M-H_A}{L_{\mathrm{MAh}}}& \left(52\right)\\ \left[\mathrm{Equation}.Math..Math.53\right]& \\ {}_{\mathrm{MAv}}=\arctan \left(\frac{H_M-H_A}{L_{\mathrm{MAh}}}\right)& \left(53\right)\end{array}$

[0326] In the above-described way, in the third computation method, the sound source is provided as the reference point O in the direction of the optical axis CX of the camera apparatus 11, and the directionality control apparatus 3 uses, as the respective input parameters, the distance L.sub.CO from the camera apparatus 11 to the sound source (reference point O), the horizontal angle .sub.MOh and the vertical angle .sub.MOv of the depression angle .sub.MO from the microphone array apparatus 2 to the sound source (reference point O), the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2, and the respective heights H.sub.C, H.sub.M and H.sub.O of the camera apparatus 11, the microphone array apparatus 2, and the reference point O from the horizontal surface.

[0327] Further, the directionality control apparatus 3 computes, as a sound collection direction, the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed toward the sound collection region central position A corresponding to the position A designated with the finger FG of the user in a video of a predetermined region captured as a monitoring target of the camera apparatus 11, that is, a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A, with the position of the microphone array apparatus 2 as a reference, by using the respective input parameters.

[0328] Consequently, according to the third computation method, even if the reference line for the 0-degree direction of the microphone array apparatus 2 is not set in advance in the direction toward the camera apparatus 11, the directionality control system 10 of the present embodiment can form the sound collection directionality in the direction of the sound collection region central position A designated with the position of the microphone array apparatus 2 as a reference with high accuracy and can thus collect audio data in the corresponding direction with high accuracy.

[0329] In addition, in each of the first to third computation methods, in order to simplify calculation of coordinates indicating a sound collection direction of the microphone array apparatus 2, a vertical angle and a horizontal angle representing the coordinates are computed in an approximate manner in some portions, but may be more accurately computed by using, for example, a geometric positional relationship.

Fourth Computation Method

[0330] In a fourth computation method, the camera apparatus 11 and the microphone array apparatus 2 are connected and fixed to each other by using a dedicated tool 50 and are installed on, for example, an indoor ceiling surface (refer to FIGS. 17(A), 17(B) and 17(C) or FIGS. 18(A), 18(B) and 18(C)).

[0331] The dedicated tool 50 connects and fixes the microphone array apparatus 2 and the camera apparatus 11 to each other so that, for example, a reference line for the 0-degree direction of the microphone array apparatus 2 opposes, for example, a reference line for the 0-degree direction of the camera apparatus 11 and heights thereof from a horizontal surface are the same as each other. In addition, a shape of the dedicated tool 50 is not particularly limited as long as the camera apparatus 11 and the microphone array apparatus 2 are connected and fixed to each other with a predetermined horizontal angle and vertical angle.

[0332] Since the camera apparatus 11 and the microphone array apparatus 2 are connected and fixed to each other by using the dedicated tool 50, in the fourth computation method, the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2 is a fixed value corresponding to a parameter of the dedicated tool.

[0333] The signal processing unit 33 computes the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 based on:

[0334] (1) a length of the dedicated tool 50; and

[0335] (2) a distance L.sub.CA, a horizontal angle .sub.CAh, and a vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position (point A).

[0336] In the fourth computation method, the input parameters in step ST2 illustrated in FIG. 5 include:

[0337] (1) the length of the dedicated tool 50, that is, information regarding the type of dedicated tool 50 to be used. For example, if the length of the dedicated tool 50 is 5 m, the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2 is also 5 m, and if the length of the dedicated tool 50 is 10 m, the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2 is also 10 m.

[0338] In addition, in the fourth computation method,

[0339] (2) the distance L.sub.CA, the horizontal angle .sub.CAh, and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A are acquired by using a function of a well-known technique of the camera apparatus 11 in step ST12 illustrated in FIG. 6.

[0340] FIGS. 17(A), 17(B) and 17(C) illustrate a positional relationship between the camera apparatus 11, the microphone array apparatus 2, and the sound collection region central position A in the fourth computation method in a case where the microphone array apparatus 2 and the camera apparatus 11 are installed so as to be connected to each other by using the dedicated tool 50. FIG. 17(A) is a perspective view. FIG. 17(B) is a horizontal direction plan view. FIG. 17(C) is a vertical direction sectional view taken along the line K-K of FIG. 17(B).

[0341] FIGS. 18(A), 18(B) and 18(C) illustrate a positional relationship between the camera apparatus 11, the microphone array apparatus 2, and the sound collection region central position A in the fourth computation method in a case where the microphone array apparatus 2 and the camera apparatus 11 are installed so as to be connected to each other by using the dedicated tool 50. FIG. 18(A) is a perspective view. FIG. 18(B) is a horizontal direction plan view. FIG. 18(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 18(B).

[0342] Hereinafter, a detailed description will be made of the fourth computation method of the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 in the signal processing unit 33.

[0343] The signal processing unit 33 computes a horizontal component distance L.sub.CAh of the the distance L.sub.CA from the camera apparatus 11 to the sound collection region central position A according to Equation (54) by using the distance L.sub.CA and the vertical angle .sub.CAv from the camera apparatus 11 to the sound collection region central position A.

[Equation 54]

L.sub.CAh=L.sub.CAcos .sub.CAv (54)

[0344] The signal processing unit 33 computes a horizontal component distance L.sub.MAh of the distance from the microphone array apparatus 2 to the sound collection region central position A according to Equation (55) based on the cosine theorem for the triangle CAM illustrated in FIG. 17(B) by using the computation result of Equation (54), the horizontal angle .sub.CAh from the camera apparatus 11 to the sound collection region central position (point A), and the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2.

[Equation 55]

L.sub.MAh={square root over ((L.sub.CAh.sup.2+L.sub.CM.sup.22L.sub.CAhL.sub.CMcos .sub.CAh)}(55)

[0345] The signal processing unit 33 computes a cosine value cos .sub.MAh of the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (56) based on the cosine theorem for the triangle CAM illustrated in FIG. 17(B) by using the respective computation results of Equations (54) and (55), and the distance L.sub.CM between the camera apparatus 11 and the microphone array apparatus 2.

[0346] Consequently, the signal processing unit 33 can compute the horizontal angle .sub.MAh of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (57).

[00030]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.56\right]& \\ \cos .Math..Math.{}_{\mathrm{MAh}}=\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}& \left(56\right)\\ \left[\mathrm{Equation}.Math..Math.57\right]& \\ {}_{\mathrm{MAh}}=\arccos \left(\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}\right)& \left(57\right)\end{array}$

[0347] The signal processing unit 33 computes a difference (H.sub.MH.sub.A) between the heights H.sub.M and H.sub.A of the microphone array apparatus 2 and the sound collection region central position A from the horizontal surface according to Equation (58) based on a sine for the triangle CAP illustrated in FIG. 17(C).

[Equation 58]

H.sub.MH.sub.A=H.sub.CH.sub.A=L.sub.CAsin .sub.CAv (58)

[0348] The signal processing unit 33 computes a tangent value tan .sub.MAv of the vertical angle .sub.MAv of the depression angle .sub.MA which is directed toward the sound collection region central position A from the microphone array apparatus 2 according to Equation (59) based on a tangent for the triangle MAS illustrated in FIG. 18(C).

[0349] Consequently, the signal processing unit 33 can compute the vertical angle .sub.MAv of the depression angle .sub.MA from the microphone array apparatus 2 to the sound collection region central position A according to Equation (60).

[00031]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.59\right]& \\ \tan .Math..Math.{}_{\mathrm{MAv}}=\frac{H_M-H_A}{L_{\mathrm{MAh}}}& \left(59\right)\\ \left[\mathrm{Equation}.Math..Math.60\right]& \\ {}_{\mathrm{MAv}}=\arctan \left(\frac{H_M-H_A}{L_{\mathrm{MAh}}}\right)& \left(60\right)\end{array}$

[0350] In the above-described way, in the fourth computation method, the microphone array apparatus 2 and the camera apparatus 11 are connected and fixed to each other by using the dedicated tool 50, and are installed on, for example, an indoor ceiling surface, and, in this state, the directionality control apparatus 3 uses the length of the dedicated tool 50, that is, the distance L.sub.CM between the microphone array apparatus 2 and the camera apparatus 11 as the input parameter.

[0351] Further, the directionality control apparatus 3 computes, as a sound collection direction, the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed toward the sound collection region central position A corresponding to the position A designated with the finger FG of the user in a video of a predetermined region captured as a monitoring target of the camera apparatus 11, that is, a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A, with the position of the microphone array apparatus 2 as a reference, by using the respective input parameter.

[0352] Consequently, according to the fourth computation method, the directionality control system 10 of the present embodiment can more easily compute the coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed from the microphone array apparatus 2 toward the sound collection region central position A since the number of input parameters is smaller than that in the first to third computation methods and the reference lines for the 0-degree direction of the camera apparatus 11 and the microphone array apparatus 2 oppose each other. In addition, the directionality control system 10 can form the sound collection directionality in the direction of the sound collection region central position A designated with the position of the microphone array apparatus 2 as a reference with high accuracy and can thus collect audio data in the corresponding direction with high accuracy.

[0353] In addition, in the above-described respective embodiments, a description has been made a case where a timing at which the directionality control apparatus 3 starts to compute the coordinates (.sub.MAh,.sub.MAv) indicating a sound collection direction of the microphone array apparatus 2 is the time when any position A or region B on a screen of the display device 36 is designated with the finger FG of the user, but the timing is not limited thereto.

[0354] For example, the timing may be the time when ranges of the angles of view CAR of the camera apparatuses 11 to 1n change due to the camera apparatuses 11 to 1n being rotated in panning directions, tilting directions, or the panning and tilting directions at predetermined intervals which are predefined. Consequently, if a sound collection direction is predefined, the directionality control system 10 can form the sound collection directionality in the predefined sound collection direction without the user designating a sound collection direction of the microphone array apparatus 2.

[0355] In the above-described respective computation methods, the description has been made based on the microphone array apparatus 2 being installed along the surface parallel to the horizontal surface. However, there is a case where a ceiling surface on which the microphone array apparatus 2 is installed may be obliquely tilted.

[0356] In this case, the horizontal angle .sub.MAh and the vertical angle .sub.MAv of a sound collection direction of the microphone array apparatus 2 are required to be corrected by an angle with which the ceiling surface is tilted since values thereof computed by the signal processing unit 33 according to the first to fourth computation methods cannot be used without being changed.

[0357] Hereinafter, with reference to FIG. 19, a description will be made of correction of, for example, vertical angle .sub.MAv of the sound collection direction (.sub.MAh,.sub.MAv) of the microphone array apparatus 2 in the signal processing unit 33 in a case where the indoor ceiling surface on which the microphone array apparatus 2 is installed is not parallel to the horizontal surface but is obliquely tilted. FIGS. 19(A) and 19(B) are diagrams illustrating a vertical angle of a sound collection direction in a case where a ceiling on which the microphone array apparatus 2 and the camera apparatus 11 are installed is tilted in a direction of .sub.Mv with respect to the horizontal surface. FIG. 19(C) is a diagram illustrating the sound collection direction .sub.MAv of the microphone array apparatus 2.

[0358] For example, a vertical angle of a sound collection direction of the microphone array apparatus 2 computed by the signal processing unit 33 according to each of the above-described first to fourth computation methods is a vertical angle .sub.MAv of a sound collection direction with respect to a plane HR parallel to the horizontal surface. In FIG. 19(B), a direction of the vertical angle .sub.MAv is not an angle with a longitudinal direction NT of the casing of the microphone array apparatus 2 as a reference.

[0359] For this reason, the signal processing unit 33 computes the vertical angle .sub.MAv and further computes the tilt angle .sub.Mv of the indoor ceiling surface. Specifically, the signal processing unit 33 computes a cosine value cos .sub.Mv of the tilt angle .sub.Mv of the indoor cesf according to Equation (61) by using the horizontal component distance L.sub.CMh of the distance L.sub.CM from the camera apparatus 11 to the microphone array apparatus 2 and the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the microphone array apparatus 2 from the horizontal surface.

[0360] Consequently, the signal processing unit 33 computes the tilt angle .sub.Mv of the indoor ceiling surface by using the computation result of Equation (61) (refer to Equation (62)).

[00032]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.61\right]& \\ \cos .Math..Math.{}_{\mathrm{Mv}}=\frac{L_{\mathrm{CMh}}}{\sqrt{L_{\mathrm{CMh}}^2+{\left(H_C-H_M\right)}^2}}& \left(61\right)\\ \left[\mathrm{Equation}.Math..Math.62\right]& \\ {}_{\mathrm{Mv}}=\arccos \left(\frac{L_{\mathrm{CMh}}}{\sqrt{L_{\mathrm{CMh}}^2+{\left(H_C-H_M\right)}^2}}\right)& \left(62\right)\end{array}$

[0361] The signal processing unit 33 computes the vertical angle .sub.MAv of the sound collection direction of the microphone array apparatus 2 according to Equation (63) by using the computation result of Equation (62) and the vertical angle .sub.MAv which is computed according to each of the above-described first to fourth computation methods.

[00033]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.63\right]& \\ {}_{\mathrm{MAv}}={}_{\mathrm{MAv}}^{}+{}_{\mathrm{Mv}}={}_{\mathrm{MAv}}^{}+\arccos \left(\frac{L_{\mathrm{CMh}}}{\sqrt{L_{\mathrm{CMh}}^2+{\left(H_C-H_M\right)}^2}}\right)& \left(63\right)\end{array}$

[0362] Consequently, even in a case where the indoor ceiling surface on which the microphone array apparatus 2 is installed is not parallel to the horizontal surface but is tilted with a predetermined angle, the directionality control apparatus 3 can appropriately compute a sound collection direction of the microphone array apparatus 2, can form the sound collection directionality in the direction of the sound collection region central position A designated with the position of the microphone array apparatus 2 as a reference with high accuracy, and can thus collect audio data in the corresponding direction with high accuracy.

[0363] Each of second to fifth embodiments described below relates to a calibration method of matching a reference direction of a horizontal angle of an imaging direction from the camera apparatus with a reference direction of a horizontal angle of a sound collection direction from the microphone array apparatus.

[0364] Here, there is a control apparatus which controls an operation of each of a camera apparatus and a microphone array apparatus and obtains audio data in a direction in which the camera apparatus performs imaging (for example, refer to a reference Patent Literature 1). The control apparatus disclosed in Patent Literature 1 controls an operation of each, for example, the camera apparatus which is installed in a conference room which is available to a TV conference system and can acquire omnidirectional images and the microphone array apparatus which can change a sound collection region.

[0365] The control apparatus disclosed in the reference Patent Literature 1 detects a direction of a speaker based on voice, compresses a data amount by cutting out only a region centering on the speaker from an omnidirectional image, and directs a beam direction of the microphone array apparatus toward the speaker so as to reduce noise other than conversations of the speaker.

[0366] (Reference Patent Literature 1) Japanese Patent No. 4252377

[0367] For example, in a case where the camera apparatus and the microphone array apparatus are integrally assembled and are disposed on the same axis, an optical axis of the camera apparatus and a physical central axis of the microphone array apparatus are common to each other. Therefore, in a case where the microphone array apparatus collects conversation voice of a subject in a video captured by the camera apparatus, a vertical angle of coordinates (horizontal angle, vertical angle) indicating a sound collection direction of the microphone array apparatus is the same as a vertical angle of coordinates (horizontal angle, vertical angle) indicating an imaging direction of the camera apparatus.

[0368] However, in a case where the camera apparatus and the microphone array apparatus are integrally assembled and are disposed on the same axis, and the camera apparatus and the microphone array apparatus are used through a combination of separate operations thereof, the horizontal angle of of the coordinates (horizontal angle, vertical angle) indicating the sound collection direction of the microphone array apparatus is the same as the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the imaging direction of the camera apparatus only when reference directions (for example, a 0 direction) of the horizontal angles of the microphone array apparatus and the camera apparatus match each other.

[0369] In the above-described reference Patent Literature 1, the camera apparatus and the microphone array apparatus are integrally connected to each other via a cylinder which is acoustically transparent, but it is not disclosed that reference directions of the horizontal angles of the camera apparatus and the microphone array apparatus match each other. In addition, the camera apparatus and the microphone array apparatus do not have a structure in which the apparatuses can be used separately from each other.

[0370] Therefore, when the camera apparatus and the microphone array apparatus which operate separately from each other are used integrally with each other, if the reference directions of the horizontal angles of the camera apparatus and the microphone array apparatus do not match each other, an imaging region of the camera apparatus in the panning direction and a sound collection region of the microphone array apparatus do not match each other. For this reason, there is a problem in that conversation voice of a subject in a video captured by the camera apparatus is not appropriately collected by the microphone array apparatus.

[0371] Therefore, in each of the second to fifth embodiments related to the present invention, in order to solve the above-described problem, a description will be made of examples of a calibration method of matching a reference directions of a horizontal angle of coordinates indicating an imaging region of the camera apparatus with a reference direction of a horizontal angle of coordinates indicating a sound collection direction of the microphone array apparatus in a case where the camera apparatus and the microphone array apparatus are integrally used.

[0372] Hereinafter, each of the second to fifth embodiments of a calibration method related to the present invention will be described with reference to the drawings. The following calibration method of each embodiment is, for example, a method of matching a horizontal angle of coordinates indicating an imaging region of an omnidirectional camera apparatus and a horizontal angle of coordinates indicating a sound collection direction of an omnidirectional microphone array apparatus in a sound collection system in which the omnidirectional camera apparatus and the omnidirectional microphone array apparatus are integrally disposed on the same axis. The sound collection system of each embodiment is installed on an installation surface (for example, a ceiling surface of an event hall) in a predetermined sound collection space.

Second Embodiment

[0373] A calibration method of the second embodiment will be described with reference to FIGS. 20(A), 20(B) and 20(C). FIG. 20(A) is a schematic diagram illustrating a calibration method in a sound collection system 1Z of the second embodiment. FIG. 20(B) is a plan view in which an omnidirectional camera apparatus 3z is viewed from a vertically lower side. FIG. 20(C) is a plan view in which an omnidirectional microphone array apparatus 5 is viewed from the vertically lower side.

[0374] The omnidirectional camera apparatus 3z includes a casing into which an optical system (for example, a fish-eye lens) and an imaging unit (for example, an image sensor) (not illustrated) are built and which is covered with a dome-shaped transparent cover 3a, and is fitted into an inner circumferential space inside an opening 13 which is formed at the center of a casing of the omnidirectional microphone array apparatus 5. The omnidirectional camera apparatus 3z, which has a function of, for example, a monitoring camera, is connected to a host computer (not illustrated) of a central control room via a network (not illustrated), and displays omnidirectional videos on a monitor (not illustrated). In addition, the omnidirectional camera apparatus 3z may cut out a video in a designated direction and may display the video on the monitor (not illustrated) in response to a remote operation from the host computer.

[0375] The omnidirectional microphone array apparatus 5 includes a ring-shaped casing 17 in which the casing of the omnidirectional camera apparatus 3z is fitted into the inner circumferential space inside the opening 13, and a plurality of microphone units 18 are disposed in a concentric shape around the opening 13 in a circumferential direction of the casing 17. The microphone unit 18 employs, for example, a high-quality small-sized electret condenser microphone (ECM), and this is also the same for the following respective embodiments. The omnidirectional microphone array apparatus 5 forms the directionality in a predetermined sound collection direction, and emphasizes and collects sound in the sound collection direction in which the directionality is formed.

[0376] In addition, in the respective embodiments including the present embodiment, configurations and operations of the omnidirectional camera apparatus and the omnidirectional microphone array apparatus are the same as the above-described content, but different content in each embodiment will be described as necessary.

[0377] Here, in a case of installing the sound collection system 1Z in which the omnidirectional microphone array apparatus 5 and the omnidirectional camera apparatus 3z are integrally attached to each other in an i direction on the same axis, it is necessary to match reference directions (for example, a 0 direction) of horizontal angles of the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 with each other in order to align an imaging region of the omnidirectional camera apparatus 3z with a sound collection direction of the omnidirectional microphone array apparatus 5.

[0378] In the present embodiment, in order to match the reference directions of the horizontal angles of the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 with each other on a plane perpendicular to the i direction on the same axis, a key 11y as an example of an engagement member which protrudes upwardly in FIG. 20(B) is formed on an outer circumference of the cover 3a of the omnidirectional camera apparatus 3z. The key 11y is formed in a reference direction g of the horizontal angle of coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3z, that is, in a direction of the horizontal angle of 0.

[0379] A key groove 15 as an engagement groove to the key 11y is fitted when the cover 3a of the omnidirectional camera apparatus 3z is inserted into the opening 13 is formed at a circumferential edge of the opening 13 of the omnidirectional microphone array apparatus 5. The key groove 15 is formed in a reference direction h of the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the sound collection direction of the omnidirectional microphone array apparatus 5, that is, in the direction of the horizontal angle of 0. In addition, in the present embodiment, a sectional shape of each of the key 11y and the key groove 15 is a rectangular shape but may be a polygonal shape or a semicircular shape other than the rectangular shape.

[0380] In a case where the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 are installed in the i direction on the same axis, the omnidirectional camera apparatus 3z is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5 so that the key 11y is fitted to the key groove 15, and thus the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 are integrally combined with each other.

[0381] In the above-described way, in the calibration method of the present embodiment, the omnidirectional camera apparatus 3z is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5 so that, for example, the key 11y is fitted to the key groove 15, and thus it is possible to easily match the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3z with the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5 and therefore to alleviate a restriction when the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 are installed. As a result, in the sound collection system 1Z of the present embodiment, the omnidirectional microphone array apparatus 5 can collect sound in the sound collection direction with high accuracy by using the same horizontal angle as the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3z.

[0382] In addition, the present embodiment relates to a case where the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 are integrally provided by fitting the omnidirectional camera apparatus 3z into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5.

[0383] Further, the omnidirectional camera apparatus 3z illustrated in the present embodiment is a camera apparatus which includes a fish-eye lens and can capture an image in all directions, but may be a camera apparatus with a hemispheric transparent dome shape which has a rotation function in a panning direction, a rotation function in a tilting direction, and a zooming function.

Third Embodiment

[0384] Next, a calibration method of the third embodiment will be described with reference to FIGS. 21(A), 21(B) and 21(C). FIG. 21(A) is a schematic diagram illustrating a calibration method in a sound collection system 1A of the third embodiment. FIG. 21(B) is a plan view in which an omnidirectional camera apparatus 3AZ is viewed from a vertically lower side. FIG. 21(C) is a plan view in which an omnidirectional microphone array apparatus 5A is viewed from the vertically lower side.

[0385] In addition, in the respective embodiments including the present embodiment, the same constituent elements as those in the above-described second embodiment are given the same reference numerals so that description thereof will be omitted or made briefly, and different content will be described.

[0386] In the present embodiment, for example, a triangular marker 21z illustrated in FIG. 21(A) is added to the outer circumference of the cover 3a of the omnidirectional camera apparatus 3AZ. In addition, a shape of the marker 21z is not limited to a triangular shape and may be, for example, a rectangular shape. The marker 21z is added in the reference direction g of the horizontal angle of coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3AZ, that is, in a direction of the horizontal angle of 0.

[0387] Further, a similar triangular marker 23z is added at a position facing the marker 21z when the cover 3a of the omnidirectional camera apparatus 3AZ is inserted into the opening 13 on the circumferential edge of the opening 13 of the omnidirectional microphone array apparatus 5A. In addition, a shape of the marker 23z is not limited to a triangular shape and may be, for example, a rectangular shape. The marker 23z is added in the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5A, that is, in the direction of the horizontal angle of 0.

[0388] In a case where the omnidirectional camera apparatus 3AZ and the omnidirectional microphone array apparatus 5A are installed in the i direction on the same axis, the omnidirectional camera apparatus 3AZ is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5A so that the tip of the marker 21z faces the tip of the marker 23z, and thus the omnidirectional camera apparatus 3AZ and the omnidirectional microphone array apparatus 5A are integrally combined with each other.

[0389] In the above-described way, in the calibration method of the present embodiment, the omnidirectional camera apparatus 3AZ is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5A so that, for example, the respective tips of the markers 21z and 23 face each other, and thus it is possible to easily match the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3AZ with the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5A. Further, in the calibration method of the present embodiment, it is possible to alleviate a restriction when the omnidirectional camera apparatus 3AZ and the omnidirectional microphone array apparatus 5A are installed. As a result, in the sound collection system 1A of the present embodiment, the omnidirectional microphone array apparatus 5A can collect sound in the sound collection direction with high accuracy by using the same horizontal angle as the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3AZ.

[0390] In addition, in the present embodiment, the omnidirectional camera apparatus 3AZ may not be fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5A unlike the above-described second embodiment, and, for example, a female screw portion formed in advance on the inside of the opening 13 of the omnidirectional microphone array apparatus 5A is screwed with a male screw portion formed on the outer circumference of the cover 3a of the omnidirectional camera apparatus 3AZ so that the omnidirectional camera apparatus 3AZ and the omnidirectional microphone array apparatus 5A are integrally combined with each other.

Fourth Embodiment

[0391] Next, a calibration method of the fourth embodiment will be described with reference to FIGS. 22, 23(A) and 23(B). FIG. 22 is a schematic diagram illustrating a calibration method in a sound collection system 1B of the fourth embodiment. FIG. 23(A) is a plan view illustrating an omnidirectional camera apparatus and an omnidirectional microphone array apparatus are attached to an attachment member 7. FIG. 23(B) is a sectional view taken along the line E-E in FIG. 23(A).

[0392] In the sound collection system of each of the above-described second and third embodiments, the omnidirectional microphone array apparatus and the omnidirectional camera apparatus are directly attached to a predetermined installation surface.

[0393] In the sound collection system 1B of the present embodiment, the attachment member 7 (attachment tool) is first attached and fixed to a predetermined installation surface (for example, a ceiling surface 8), and both an omnidirectional microphone array apparatus 5B and an omnidirectional camera apparatus 3BZ are attached to the attachment member 7 (refer to FIG. 22). Consequently, in the sound collection system 1B of the present embodiment, the omnidirectional microphone array apparatus 5B and the omnidirectional camera apparatus 3BZ are integrally combined with each other.

[0394] FIG. 23(A) illustrates an attachment state of the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B when viewed from a surface of the attachment member 7, that is, when viewed from the ceiling surface 8 as an example of the predetermined installation surface in a downward direction illustrated in FIG. 23(B). The attachment member 7 is a metallic member which has an uneven surface and is formed in a substantially disc shape. In addition, the attachment member 7 may be a member made of ceramics or a synthetic resin (for example, plastic or elastomer).

[0395] Engagement pieces 7a which protrude in the i direction on the same axis and are used for attaching and fixing the omnidirectional camera apparatus 3BZ are formed at three concentric locations on the surface of the attachment member 7, that is, the surface of the attachment member 7 facing the ceiling surface 8. Further, engagement pieces 7b which protrude in the i direction on the same axis and are used for attaching and fixing the omnidirectional microphone array apparatus 5B are formed at three concentric locations which have diameters larger than those of the concentric locations where the engagement pieces 7a are formed, on the surface of the attachment member 7.

[0396] FIG. 24(A) is a side view illustrating a state in which fixation pins 33z and 35z are being engaged with engagement holes 71 and 73. FIG. 24(B) is a plan view and a side view illustrating a state in which the fixation pins 33z and 35z inserted into the engagement holes 71 and 73 are moved. FIG. 24(C) is a plan view and a side view illustrating a state in which the fixation pins 33z and 35 are engaged with the engagement holes 71 and 73.

[0397] Each of the engagement pieces 7a is provided with the engagement hole 71 which is engaged with the fixation pin 33z provided on a bottom of the omnidirectional camera apparatus 3BZ and is formed in a gourd shape of which a diameter of one end is larger than a diameter of the other end. Similarly, each of the engagement pieces 7b is provided with the engagement hole 73 which is engaged with the fixation pin 35z provided on a bottom of the omnidirectional microphone array apparatus 5B and is formed in a gourd shape of which a diameter of one end is larger than a diameter of the other end.

[0398] Each of the fixation pins 33z and 35z includes a head having a thickness (diameter) between one end and the other end of each of the engagement holes 71 and 73 and a body thinner than the head.

[0399] Each of fan-shaped hole portions 7c and 7d is formed at three locations so as to expand outside the engagement pieces 7a and the engagement pieces 7b on the surface of the attachment member 7. Shapes and positions of the fan-shapes hole portions 7c and 7d are designed so that the reference directions g and h of the horizontal angles of the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B match each other when the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B are attached to the attachment member 7.

[0400] Screw holes 7e into which screws 31z are inserted are formed at three locations at the center of the surface of the attachment member 7. The screws 31z are screwed with the ceiling surface 8 via the screw holes 7e and thus the attachment member 7 is fixed to the ceiling surface 8.

[0401] When the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B are attached to the attachment member 7, first, the omnidirectional camera apparatus 3BZ is attached to the attachment member 7. In this case, the fixation pin 33z is engaged with the engagement hole 71 formed at the engagement piece 7a (refer to FIG. 24(A)).

[0402] In other words, as illustrated in FIG. 24(A), the fixation pin 33z which protrudes from the bottom of the omnidirectional camera apparatus 3BZ is inserted into one end side of the engagement hole 71 with a larger diameter. In addition, as illustrated in FIG. 24(B), the fixation pin 33z is moved in the engagement hole 71 by rotating the omnidirectional camera apparatus 3BZ in a state in which the head of the fixation pin 33z protrudes out of the engagement hole 71. Further, as illustrated in FIG. 24(C), the fixation pin 33z is engaged with the engagement hole 71 in a state in which the head of the fixation pin 33z is moved to the other end side of the engagement hole 71, and thus the omnidirectional camera apparatus 3BZ is fixed in the i direction on the same axis.

[0403] After the omnidirectional camera apparatus 3BZ is attached to the attachment member 7, the omnidirectional microphone array apparatus 5B attached to the attachment member 7 from the inside of the opening 13 of the omnidirectional microphone array apparatus 5B so as to expose the omnidirectional camera apparatus 3BZ. In this case, the fixation pin 35z is engaged with the engagement hole 73 formed at the engagement piece 7b. In addition, a procedure of fixing the fixation pin 35z to the engagement hole 73 is the same as the procedure of fixing the fixation pin 33z to the engagement hole 71.

[0404] In the above-described way, in the calibration method of the present embodiment, the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B are integrally attached to the attachment member 7 which is attached and fixed to the ceiling surface 8. Consequently, in the calibration method of the present embodiment, the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B can be easily installed in a state in which the reference directions of the horizontal angles of the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B match each other. Therefore, in the sound collection system 1B of the present embodiment, the omnidirectional microphone array apparatus 5B can collect sound in the sound collection direction with high accuracy by using the same horizontal angle as the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3BZ.

Fifth Embodiment

[0405] Next, a calibration method of the fifth embodiment will be described with reference to FIGS. 25(A), 25(B) and 25(C) and FIG. 26. FIG. 25(A) is a side view illustrating a state in which a tool 61 is being attached to an omnidirectional microphone array apparatus 5C in the calibration method of the fifth embodiment. FIG. 25(B) is a diagram illustrating a state in which attachment of the tool 61 to the omnidirectional microphone array apparatus 5C is completed. FIG. 25(C) is an exterior perspective view of a sound collection system 1C in which attachment of the tool 61 to the omnidirectional microphone array apparatus 5C is completed. FIG. 26 is a diagram illustrating a state in which the tool 61 is reflected in an image 80z captured by an omnidirectional camera apparatus 3CZ.

[0406] In the calibration method of the present embodiment, configurations of the omnidirectional camera apparatus 3CZ and the omnidirectional microphone array apparatus 5C in the sound collection system 1C and a procedure of attaching the apparatuses to an installation surface are the same as the configurations and the attachment procedure of the omnidirectional camera apparatus 3AZ and the omnidirectional microphone array apparatus 5A in the third embodiment, and thus description thereof will be omitted.

[0407] The tool 61 indicating the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5C is attached to both ends which oppose to each other in a casing 17 of the omnidirectional microphone array apparatus 5C. In the present embodiment, a wire is hung so as to cross over a casing of the omnidirectional camera apparatus 3CZ as the tool 61. Both tip ends of the wire are attached to holes which are formed in advance in the casing 17 of the omnidirectional microphone array apparatus 5C, and thus the tool 61 is attached to the omnidirectional microphone array apparatus 5C.

[0408] In addition, a mark 63 indicating a reference direction (for example, a 0 direction) of a horizontal angle is added to a part of the tool 61. The mark 63 may be colored differently from other parts of the tool 61. A material of the tool 61 is not particularly limited, such as ceramics, synthetic resin (for example, plastic or elastomer), and metal, and a shape thereof is not limited to a spherical shape and may be a columnar shape or other shapes.

[0409] In the present embodiment, as illustrated in FIGS. 25(A) and 25(B), after the omnidirectional camera apparatus 3CZ is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5C, the tool 61 is attached to opposing both ends of the casing 17 of the omnidirectional microphone array apparatus 5C.

[0410] In addition, in the present embodiment, as illustrated in FIG. 25(C), the omnidirectional camera apparatus 3CZ captures a subject in a predetermined imaging region, for example, in all directions (360) in a state in which the tool 61 is attached so as to cross over the casing of the omnidirectional camera apparatus 3CZ. In this case, the omnidirectional camera apparatus 3CZ may cover a background portion of a captured image with a specific color so that objects other than the tool 61 are not reflected much in the captured image.

[0411] Further, the tool 61 is not limited to a single wire. As long as the mark 63 indicating a reference of a horizontal angle is fixed to a predetermined position and is recognized in a video captured by the camera apparatus, the tool may be made of a sheet metal regardless of a shape thereof, and may have a three-legged shape.

[0412] Further, the tool 61 may be formed in an opaque dome shape covering the entire casing of the omnidirectional camera apparatus 3 and including an opening which is used as a mark in a reference direction of a horizontal angle of the omnidirectional microphone array apparatus 5C.

[0413] A tool image 61A indicating the reference direction his reflected through the center of the captured image 80z in the circular captured image 80z obtained by the omnidirectional camera apparatus 3CZ illustrated in FIG. 26. A mark image 63A of the mark 63 indicating the reference direction h (for example, a direction of 0) of the horizontal angle of the omnidirectional microphone array apparatus 5C is reflected so as to overlap the tool image 61A. The mark image 63A may be recognized through image processing in the omnidirectional camera apparatus 3CZ, and may be recognized with the naked eyes by an operator of the sound collection system 1C.

[0414] In the captured image 80z, the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3CZ shows a vertical direction in FIG. 26. Therefore, based on the captured image 80z, the omnidirectional camera apparatus 3CZ can compute an angle difference between the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5C shown as the tool image 61A and the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3CZ, as a deviation amount E of the horizontal angle.

[0415] Therefore, in a case where the omnidirectional camera apparatus 3CZ and the omnidirectional microphone array apparatus 5C are connected to each other via a network (not illustrated), the omnidirectional microphone array apparatus 5C may set the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5 to a value which is deviated relative to the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3CZ by the deviation amount . In other words, the omnidirectional microphone array apparatus 5C can appropriately compute coordinates of a sound collection direction by using an angle which is reversely returned by the deviation amount of a horizontal angle as a horizontal angle of coordinates (horizontal angle, vertical angle) indicating an actual sound collection direction.

[0416] In the above-described way, in the calibration method of the present embodiment, the omnidirectional microphone array apparatus 5C can match a horizontal angle indicating an imaging direction of the omnidirectional camera apparatus 3CZ with a horizontal angle indicating a sound collection direction of the omnidirectional microphone array apparatus 5C by using the deviation amount E of the horizontal angle computed by the omnidirectional camera apparatus 3CZ, and can thus adjust coordinates (horizontal angle, vertical angle) of the sound collection direction of the omnidirectional microphone array apparatus 5C.

[0417] Hereinafter, configurations, operations, and effects of the above-described calibration methods will be described.

[0418] According to an embodiment of the present invention, there is provided a calibration method including a step of positioning a camera apparatus which captures a video of a predetermined imaging region and a microphone array apparatus which collects sound of the imaging region on the same axis; a step of attaching the camera apparatus to a circumferential edge of an opening which is formed at a center of the casing of the microphone array apparatus; and a step of matching reference directions of horizontal angles of the camera apparatus and the microphone array apparatus with each other on a plane perpendicular to the same axis by attaching the camera apparatus to inside of the opening.

[0419] In the above-described method, it is possible to easily match the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3z with the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5 and therefore to alleviate a restriction when the omnidirectional camera apparatus 3z and the omnidirectional microphone array apparatus 5 are installed. As a result, in the sound collection system 1Z of the present embodiment, the omnidirectional microphone array apparatus 5 can collect sound in the sound collection direction with high accuracy by using the same horizontal angle as the horizontal angle of the coordinates (horizontal angle, vertical angle) indicating the imaging region of the omnidirectional camera apparatus 3z.

[0420] In addition, in the calibration method according to the embodiment of the present invention, in the matching step, the reference direction of the horizontal angle of the microphone array apparatus matches the reference direction of the horizontal angle of the camera apparatus by engaging an engagement member formed on an outer circumference of a casing of the camera apparatus with an engagement groove formed at the circumferential edge of the opening.

[0421] In the above-described method, the omnidirectional camera apparatus 3z is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5 so that, for example, the key 11y is fitted to the key groove 15, and thus it is possible to easily match the reference direction of the horizontal angle of the omnidirectional camera apparatus 3z with the reference direction of the horizontal angle of the omnidirectional microphone array apparatus 5.

[0422] Further, in the calibration method according to the embodiment of the present invention, in the matching step, the reference direction of the horizontal angle of the microphone array apparatus matches the reference direction of the horizontal angle of the camera apparatus by opposing a first marker portion added to the circumferential edge of the opening to a second marker portion added on the casing of the camera apparatus.

[0423] In the above-described method, the omnidirectional camera apparatus 3AZ is fitted into the inner circumferential space inside the opening 13 of the omnidirectional microphone array apparatus 5A so that, for example, the respective tips of the markers 21 and 23 face each other, and thus it is possible to easily match the reference direction g of the horizontal angle of the omnidirectional camera apparatus 3AZ with the reference direction h of the horizontal angle of the omnidirectional microphone array apparatus 5A.

[0424] Further, in the calibration method according to the embodiment of the present invention, in the attachment step, the casing of the camera apparatus is attached to a predetermined attachment tool, and the casing of the microphone array apparatus is attached to the predetermined attachment tool by inserting the casing of the camera apparatus attached to the predetermined attachment tool into the inside of the opening so that the casing of the camera apparatus is fitted into the inside thereof.

[0425] In the above-described calibration method, the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B are integrally attached to the attachment member 7 which is attached and fixed to the ceiling surface 8, and thus the reference directions of the horizontal angles of the omnidirectional camera apparatus 3BZ and the omnidirectional microphone array apparatus 5B can match each other.

[0426] In addition, according to another embodiment of the present invention, there is provided a calibration method including a step of positioning a camera apparatus which captures a video of a predetermined imaging region and a microphone array apparatus which collects sound of the imaging region on the same axis; a step of attaching the camera apparatus to a circumferential edge of an opening which is formed at a center of the casing of the microphone array apparatus; a step of attaching a tool indicating a reference direction of a horizontal angle of the microphone array apparatus to both opposing ends of a casing of the microphone array apparatus; a step of causing the camera apparatus to capture an image of the tool; a step of computing a deviation amount between a reference direction of a horizontal angle of the camera apparatus and a reference direction of a horizontal angle of the microphone array apparatus; and a step of matching the reference directions of the horizontal angles of the camera apparatus and the microphone array apparatus with each other on a plane perpendicular to the same axis by adjusting a horizontal angle of a sound collection direction of the microphone array apparatus by using the computed deviation amount.

[0427] In the calibration method of the present embodiment, the omnidirectional microphone array apparatus 5C can match a horizontal angle indicating an imaging direction of the omnidirectional camera apparatus 3CZ with a horizontal angle indicating a sound collection direction of the omnidirectional microphone array apparatus 5C by using the deviation amount E of the horizontal angle computed by the omnidirectional camera apparatus 3CZ, and can thus adjust coordinates (horizontal angle, vertical angle) of the sound collection direction of the omnidirectional microphone array apparatus 5C.

[0428] In the above-described respective embodiments, a reference direction of a horizontal angle has been described as a direction in which a horizontal angle is 0, but any angle may be set as a reference direction.

[0429] Further, each microphone disposed inside the casing of the omnidirectional microphone array apparatus in the above-described respective embodiments may employ a nondirectional microphone, a bidirectional microphone, a unidirectional microphone, a sharply directional microphone, a super-directional microphone (for example, a shotgun microphone), or a combination thereof.

[0430] A sixth embodiment described below relates to a directionality control system and a horizontal deviation angle computation method, in which a direction connecting a camera apparatus to a microphone array apparatus is set as each reference direction, a deviation angle between a front direction (0 direction) of the camera apparatus and the reference direction of the camera apparatus and a deviation angle between a front direction (0 direction) of the microphone array apparatus and the reference direction of the microphone array apparatus, and thus calibration of a horizontal angle is performed.

[0431] In the above-described monitoring system, in a case where the camera apparatus and the microphone array apparatus are disposed at different positions as separate apparatuses, an optical axis of the camera apparatus is different from a physical central axis of the microphone array apparatus. For this reason, in a case where the microphone array apparatus collects conversation voice of a subject who is present in an imaging direction of the camera apparatus, it is necessary to adjust, to appropriate values, coordinates (horizontal angle, vertical angle) indicating the direction (hereinafter, simply referred to as a sound collection direction) in which the microphone array apparatus collects the conversation voice and coordinates (horizontal angle, vertical angle) indicating the imaging direction of the camera apparatus.

[0432] In order to perform the adjustment, for example, a front direction (for example, a 0 direction) of a horizontal angle of coordinates (hereinafter, referred to as imaging direction coordinates) indicating the imaging direction of the camera apparatus is required to be a known value in the camera apparatus, and a front direction (for example, a 0 direction) of a horizontal angle of coordinates (hereinafter, referred to as sound collection direction coordinates) indicating a sound collection direction of the microphone array apparatus is required to be a known value in the microphone array apparatus.

[0433] Also in order that a positional relationship between the camera apparatus and the microphone array apparatus are required to be appropriately specified, and a direction of the microphone array apparatus with the camera apparatus as a reference and a direction of the camera apparatus with the microphone array apparatus as a reference are known values, for example, when the microphone array apparatus is installed, a front direction (for example, a 0 direction) of a horizontal angle of the microphone array apparatus is preferably directed toward the camera apparatus.

[0434] However, Patent Literature 1 does not disclose that a direction of the microphone array apparatus with the camera apparatus as a reference and a direction of the camera apparatus with the microphone array apparatus as a reference are computed in a case where a front direction (for example, a 0 direction) of a horizontal angle of the camera apparatus and a front direction (for example, a 0 direction) of a horizontal angle of the microphone array apparatus are unknown. Therefore, it is not clear an angle from which direction each horizontal angle is, and thus a positional relationship between the camera apparatus and the microphone array apparatus cannot be appropriately specified.

[0435] Therefore, for example, since a sound collection direction of the microphone array apparatus is not directed in a direction in which a subject is present while the microphone array apparatus collects voice of the subject, there is a problem in that it is hard for the microphone array apparatus to appropriately collect conversation voice of the subject who is present in an imaging direction of the camera apparatus.

[0436] In addition, in most cases where the camera apparatus is installed later at a location where the microphone array apparatus has been installed, it is hard to install the camera apparatus so that a front direction (for example, a 0 direction) of a horizontal angle of the microphone array apparatus is directed toward the camera apparatus. Further, in a case where the microphone array apparatus is additionally installed later at a location where a plurality of camera apparatuses have been installed, there is a problem in which a front direction (for example, a 0 direction) of a horizontal angle of the microphone array apparatus cannot match directions of the plurality of camera apparatuses.

[0437] Therefore, in the sixth embodiment related to the present invention, in order to solve the above-described problems, a description will be made of examples of a directionality control system and a horizontal deviation angle computation method, in which a horizontal deviation angle indicating an angle between a 0 direction of each horizontal angle of imaging direction coordinates of the camera apparatus and sound collection direction coordinates of the microphone array apparatus and mutual reference directions connecting both the apparatuses to each other is computed, and thus the microphone array apparatus can appropriately collect conversation voice of a subject who is present in an imaging direction of the camera apparatus.

[0438] Hereinafter, with reference to the drawings, a description will be made of the sixth embodiment related to the directionality control system and the horizontal deviation angle computation method according to the present invention. The directionality control system of the present embodiment is used as a monitoring system (including a manned monitoring system and an unmanned monitoring system) provided in, for example, a factory, a public facility (for example, a library or an event hall), or a store (for example, a retail store or a bank).

[0439] In addition, the present invention can be expressed as respective apparatuses (for example, a directionality control apparatus to be described later) constituting the directionality control system, or a horizontal deviation angle computation method including respective operations (steps) performed by each apparatus (for example, a directionality control apparatus to be described later) constituting the directionality control system.

Sixth Embodiment

Configuration of Directionality Control System

[0440] FIG. 27(A) is a schematic diagram of a directionality control system 10 of the present embodiment in a case where a calibration omnidirectional camera apparatus C1 and an omnidirectional microphone array apparatus 2 are integrally installed. FIG. 27(B) is a schematic diagram of a directionality control system 10A of the present embodiment in a case where the omnidirectional microphone array apparatus 2 is installed so that a reference direction of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a reference direction of a horizontal angle of an imaging direction of the calibration omnidirectional camera apparatus C1.

[0441] The directionality control system 10 illustrated in FIG. 27(A) includes an omnidirectional camera apparatus 11z as a first imaging part which images subjects (for example, two people in FIG. 27(A); this is also the same for the following description), the calibration omnidirectional camera apparatus C1 as a second imaging part which images the same subjects as those imaged by the omnidirectional camera apparatus 11z, and the omnidirectional microphone array apparatus 2 as a sound collection part which collects voice of the same subjects (for example, conversation voice of the two people). In the directionality control system 10 illustrated in FIG. 27(A), for example, a columnar opening 21a (refer to FIG. 2(D)) formed at the center of a casing of the omnidirectional microphone array apparatus 2, and the calibration omnidirectional camera apparatus C1 is fitted into an inner circumferential space inside the opening 21a so that the omnidirectional microphone array apparatus 2 and the calibration omnidirectional camera apparatus C1 are integrally formed with each other.

[0442] The omnidirectional camera apparatus 11z functions as, for example, a monitoring camera, includes a casing into which an optical system (for example, a fish-eye lens or a wide angle lens) and an imaging system (for example, an image sensor) (not illustrated) are built, and is installed on a predetermined installation surface (for example, a ceiling surface of a room of an event hall or a stand). The omnidirectional camera apparatus 11z is connected to a host computer (not illustrated) of a central control room via a network (not illustrated), and performs a panning direction operation, a tilting direction operation, a zooming operation, an imaging operation, and a distance-measuring operation and an angle-measuring operation related to an actual position corresponding to a designated position (for example, a designated position A to be described later) in a captured image in response to a remote operation from the host computer. The omnidirectional camera apparatus 11z images, for example, a subject which is present in a first imaging direction CAX1 which is directed from the omnidirectional camera apparatus 11z toward a sound collection position (or a sound position) A which will be described later (refer to FIG. 27(A)).

[0443] The calibration omnidirectional camera apparatus C1 functions as, for example, a calibration camera which computes front direction (for example, a 0 direction) of each horizontal angle of imaging direction coordinates of the omnidirectional camera apparatus 11z and sound collection direction coordinates of the omnidirectional microphone array apparatus 2, and a horizontal deviation angle relative to mutual reference directions connecting the omnidirectional camera apparatus 11z and the omnidirectional microphone array apparatus 2 to each other. The calibration omnidirectional camera apparatus C1 includes a casing into which an optical system (for example, a fish-eye lens or a wide angle lens) and an imaging system (for example, an image sensor) (not illustrated) are built, and is installed on a predetermined installation surface (for example, a ceiling surface of a room of an event hall or a stand). The calibration omnidirectional camera apparatus C1 is connected to the host computer (not illustrated) of the central control room via a network (not illustrated), and performs a panning direction operation, a tilting direction operation, a zooming operation, an imaging operation, and a distance-measuring operation and an angle-measuring operation related to an actual position corresponding to a designated position (for example, a designated position A to be described later) in a captured image in response to a remote operation from the host computer. The calibration omnidirectional camera apparatus C1 images, for example, a subject which is present in a second imaging direction CAX2 which is directed from the calibration omnidirectional camera apparatus C1 toward the sound collection position A which will be described later (refer to FIG. 27(A)).

[0444] The omnidirectional microphone array apparatus 2 of the directionality control system 10 illustrated in FIG. 27(A) includes, for example, a doughnut-shaped or ring-shaped casing 21 (refer to FIG. 2(D)) in which a casing of the calibration omnidirectional camera apparatus C1 is fitted into an inner circumferential space inside the opening 21a. In the omnidirectional microphone array apparatus 2, a plurality of microphone units 22 are disposed in a concentric shape around the opening 21a in a circumferential direction of the casing 21. The microphone unit 18 employs, for example, a high-quality small-sized electret condenser microphone (ECM), and this is also the same for the following description. The omnidirectional microphone array apparatus 2 forms, for example, the sound collection directionality in a sound collection direction MIX which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A, and collects voice of a subject present in the sound collection direction MIX.

[0445] On the other hand, in the directionality control system 10A illustrated in FIG. 27(B), the calibration omnidirectional camera apparatus C1 and the omnidirectional microphone array apparatus 2 are not integrally formed with each other, and the calibration omnidirectional camera apparatus C1 and the microphone array apparatus 2 operate separately from each other. For this reason, in the directionality control system 10A illustrated in FIG. 27(B), after a directionality control apparatus 3 which will be described later computes a horizontal deviation angle, the calibration omnidirectional camera apparatus C1 is detached, and the omnidirectional microphone array apparatus 2 is installed so that a horizontal angle 0 direction of an imaging direction CAX2 (a second front direction; refer to FIG. 31(B) or 32(B)) of the calibration omnidirectional camera apparatus C1 matches a horizontal angle 0 direction of the omnidirectional microphone array apparatus 2.

[0446] Consequently, in the same manner as in the directionality control system 10 illustrated in FIG. 27(A), the omnidirectional microphone array apparatus 2 can use the horizontal angle 0 direction (second front direction) of the calibration omnidirectional camera apparatus C1 in common, and can thus appropriately form the sound collection directionality in the sound collection direction MIX which is directed toward the sound collection position A and can appropriately collect voice of a subject present in the sound collection direction MIX. Hereinafter, a description will be made of a configuration of the directionality control system of the present invention by using the directionality control system 10 illustrated in FIG. 27(A), but the same effect can also be achieved through replacement with the directionality control system 10A illustrated in FIG. 27(B).

[0447] In addition, in the present embodiment, the omnidirectional camera apparatus 11z including a fish-eye lens or a wide angle lens is used, but a pan-tilt-zoom (PTZ) camera apparatus which includes a standard lens or a telephoto lens and mechanically performs operation in a panning direction and a tilting direction and a zooming operation may be used.

[0448] FIG. 28(A) is a block diagram illustrating an example of a configuration of the directionality control system 10 illustrated in FIG. 27(A). FIG. 28(B) is a block diagram illustrating an example of a configuration of the directionality control system 10A illustrated in FIG. 27(B). The directionality control system 10 illustrated in FIG. 28(A) includes the omnidirectional camera apparatus 11z, the omnidirectional microphone array apparatus 2, the calibration omnidirectional camera apparatus C1, the directionality control apparatus 3, and a recorder apparatus 4. The omnidirectional camera apparatus 11z, the omnidirectional microphone array apparatus 2, the calibration omnidirectional camera apparatus C1, the directionality control apparatus 3, and the recorder apparatus 4 are connected to each other via a network NW. The network NW may be a wired network (for example, an intranet or the Internet), and may be a wireless network (for example, a wireless local area network (LAN)), which is also the same for the following description.

[0449] In addition, the directionality control system 10A illustrated in FIG. 28(B) has the same configuration as the configuration of the directionality control system 10 illustrated in FIG. 28(A) except for the calibration omnidirectional camera apparatus C1 and the omnidirectional microphone array apparatus 2 are formed as separate apparatuses.

[0450] The omnidirectional camera apparatus 11z is connected to the network NW, measures and acquires input parameters (for example, a distance L.sub.CK between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1) for computing a sound collection direction (.sub.MAh,.sub.MAv) of the omnidirectional microphone array apparatus 2, which will be described later, and transmits the measured input parameters and captured image data to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW.

[0451] The calibration omnidirectional camera apparatus C1 is connected to the network NW, similarly, measures and acquires input parameters (for example, the distance L.sub.CK between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1) for computing the sound collection direction (.sub.MAh,.sub.MAv) of the omnidirectional microphone array apparatus 2, which will be described later, and transmits the measured input parameters and captured image data to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW.

[0452] The microphone array apparatus 2 are connected to the network NW and includes at least microphone units 22 and 23 in which microphones are provided at equal intervals (refer to FIGS. 2(A) to 2(E)) and a control unit (not illustrated) which controls an operation of each of the microphone units 22 and 23.

[0453] The omnidirectional microphone array apparatus 2 collects sound in a sound collection direction in which a subject serving as a sound collection target is present by using each of the microphone units 22 and 23, and transmits audio data collected by each of the microphone units 22 and 23 to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW.

[0454] The omnidirectional microphone array apparatus 2 forms sound collection directionality of each of the microphone units 22 and 23 in the sound collection direction (.sub.MAh,.sub.MAv) which is computed by a coordinate computation section 34x of a signal processing unit 33 of the directionality control apparatus 3 in response to a directionality formation instruction from the directionality control apparatus 3 which will be described later.

[0455] Consequently, the omnidirectional microphone array apparatus 2 can relatively increase a volume level of audio data collected from the sound collection direction (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can relatively reduce a volume level of audio data collected from a direction in which the sound collection directionality is not formed. In addition, a method of computing the sound collection direction (.sub.MAh,.sub.MAv) will be described later.

[0456] An exterior of the omnidirectional microphone array apparatus 2 has been described with reference to FIG. 2, and thus description thereof will be omitted. In addition, each of the microphone units 22 and 23 of the omnidirectional microphone array apparatus 2 may employ a nondirectional microphone, a bidirectional microphone, a unidirectional microphone, a sharply directional microphone, a super-directional microphone (for example, a shotgun microphone), or a combination thereof.

[0457] The directionality control apparatus 3 is connected to the network NW, and may be, for example, a stationery personal computer (PC) installed in a monitoring system control room (not illustrated), and may be a mobile phone, a tablet terminal, or a smart phone, which can be carried by a user.

[0458] The directionality control apparatus 3 includes at least a communication unit 31, an operation unit 32, a signal processing unit 33, a display device 36, a speaker device 37, and a memory 38. The signal processing unit 33 includes a horizontal deviation angle computation section 34w, a coordinate computation section 34x, and an output control section 34c.

[0459] The communication unit 31 outputs image data or audio data which is transmitted from the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, or the microphone array apparatus 2, to the signal processing unit 33 via the network NW.

[0460] The operation unit 32 is a user interface (UI) for notifying the signal processing unit 33 of the content of a user's input operation, and is, for example, a pointing device such as a mouse or a keyboard. In addition, the operation unit 32 may be configured by using a touch panel or a touch pad which is disposed so as to correspond to, for example, a screen of the display device 36 and allows an input operation to be performed with the finger FG of the user or a stylus pen.

[0461] The operation unit 32 acquires coordinate data indicating a region where the user desires to increase or decrease a volume level, that is, a designated position A illustrated in FIG. 29 and outputs the coordinate data to the signal processing unit 33 in response to the user's input operation.

[0462] The signal processing unit 33 is configured by using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and performs a control process for collectively controlling operations of the respective units of the directionality control apparatus 3, data input and output processes with other respective units, a data computation (calculation) process, and a data storage process.

[0463] The horizontal deviation angle computation section 34w as a deviation angle computation part computes a first horizontal deviation angle .sub.Ch of the omnidirectional camera apparatus 11z in a first front direction (horizontal angle 0 direction) and a second horizontal deviation angle .sub.Kh of the calibration omnidirectional camera apparatus C1 in a second front direction (horizontal angle 0 direction) relative to a reference position (a line K-K illustrated in FIG. 31(B)) connecting the omnidirectional camera apparatus 11z to the calibration omnidirectional camera apparatus C1, based on direction angle information of the first imaging direction CAX1 which is directed from the omnidirectional camera apparatus 11z to the sound collection position A corresponding to the designated position A, direction angle information of the second imaging direction CAX2 which is directed from the calibration omnidirectional camera apparatus C1 to the sound collection position A, and a distance between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1, in response to a user's designation of any position (=designated position A) in image data displayed on the display device 36. A specific computation method in the horizontal deviation angle computation section 34w will be described later with reference to FIGS. 31 and 32.

[0464] The coordinate computation section 34x computes a horizontal angle .sub.MAh and a vertical angle .sub.MAv of the sound collection direction MIX which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A based on the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh computed by the horizontal deviation angle computation section 34w, as coordinates (sound collection direction coordinates) indicating a sound collection direction in which the omnidirectional microphone array apparatus 2 collects voice of a subject.

[0465] In the sound collection direction (.sub.MAh,.sub.MAv), .sub.MAh indicates a horizontal angle of the sound collection direction MIX which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A, and .sub.MAv indicates a vertical angle of the sound collection direction MIX which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A.

[0466] Since a relationship between coordinate axes of the omnidirectional camera apparatus 11z and coordinate axes of the microphone array apparatus 2 is known based on the first horizontal deviation angle .sub.Ch, the second horizontal deviation angle .sub.Kh, and the distance L.sub.ck therebetween, the directionality control apparatus 3 computes the sound collection direction (.sub.MAh,.sub.MAv) in response to designation of a sound collection position in image data displayed on the display device 36.

[0467] In addition, the sound collection position A is a field position which corresponds to the designated position A which is designated with the finger FG of the user or a stylus pen on a screen of the display device 36 via the operation unit 32, and is an actual monitoring target (refer to FIGS. 27(A) and 29).

[0468] The output control section 34c controls operations of the display device 36 and the speaker device 37, so as to cause the display device 36 to reproduce and output video data transmitted from the omnidirectional camera apparatus 11z, and to cause the speaker device 37 to output audio data transmitted from the omnidirectional microphone array apparatus 2 as sound. In addition, the output control section 34c controls an operation of the omnidirectional microphone array apparatus 2 so as to cause the omnidirectional microphone array apparatus 2 to form the sound collection directionality of audio data in the sound collection direction MIX corresponding to sound collection direction coordinates (.sub.MAh,.sub.MAv) computed by, for example, the coordinate computation section 34x.

[0469] The display device 36 as a display part displays image data captured by the omnidirectional camera apparatus 11z or the calibration omnidirectional camera apparatus C1 on a screen.

[0470] The speaker device 37 as a sound output part outputs, as sound, audio data collected by the omnidirectional microphone array apparatus 2 or audio data which is collected by the omnidirectional microphone array apparatus 2 after the sound collection directionality is formed in the sound collection direction (.sub.MAh,.sub.MAv) computed by the coordinate computation section 34x. In addition, the display device 36 and the speaker device 37 may be configured separately from the directionality control apparatus 3.

[0471] The memory 38 is configured by using, for example, a random access memory (RAM), and functions as a work memory when the respective units of the directionality control apparatus 3 operate.

[0472] The recorder apparatus 4 records image data captured by the omnidirectional camera apparatus 11z or the calibration omnidirectional camera apparatus C1 and audio data collected by the omnidirectional microphone array apparatus 2. The recorder apparatus 4 records the image data captured by the omnidirectional camera apparatus 11z and the audio data collected by the omnidirectional microphone array apparatus 2 in correlation with each other.

[0473] Next, a summary of an operation of the directionality control system 10 of the present embodiment will be described with reference to FIGS. 27(A) and 29. FIG. 29 is a diagram illustrating a state in which collected audio data is output from the speaker device 37 when a direction which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A corresponding to the designated position A designated with the finger of the user in an image displayed on the display device 36 is a sound collection direction.

[0474] In the directionality control system 10, the omnidirectional camera apparatus 11z images the subjects (for example, two people) illustrated in FIG. 27(A). The microphone array apparatus 2 collects ambient sound including conversation voice of the subjects. In FIG. 27(A), the two people are having conversations. For example, image data captured by omnidirectional camera apparatus 11z is displayed on the display device 36 of the directionality control apparatus 3 (refer to FIG. 29).

[0475] Here, if the designated position A on the display device 36, that is, a central position or a substantially central position the two people having conversations is designated with the finger FG of the user, the directionality control apparatus 3 acquires coordinate data (.sub.CAh,.sub.CAv) of the first imaging direction CAX1 indicating the designated position A.

[0476] The directionality control apparatus 3 computes sound collection direction coordinates (.sub.MAh,.sub.MAv) indicating a direction which is directed from the installation position of the omnidirectional microphone array apparatus 2 toward the sound collection position A, that is, a sound collection direction, as a sound collection direction of the omnidirectional microphone array apparatus 2 based on a relationship between a coordinate system of the omnidirectional camera apparatus 11z and a coordinate system of the omnidirectional microphone array apparatus 2, which is computed through calibration. The omnidirectional microphone array apparatus 2 forms the sound collection directionality in the direction which is directed from the omnidirectional microphone array apparatus 2 toward the sound collection position A by using the coordinate data (.sub.MAh,.sub.MAv) computed by the directionality control apparatus 3.

[0477] Therefore, the omnidirectional microphone array apparatus 2 can increase a volume level of the conversation (Hello) of the two people present in the direction in which the sound collection directionality is formed more than a volume level of sound output from other objects which are not present in the direction in which the sound collection directionality is formed.

[0478] Consequently, the directionality control apparatus 3 causes the speaker device 37 to output sound with a volume level of the conversation (Hello) of the two people present in the direction in which the sound collection directionality is formed higher than a volume level of sound output from other objects which are not present in the direction in which the sound collection directionality is formed (refer to FIG. 29).

[0479] Next, a specific operation procedure of the directionality control system 10 or the directionality control system 10A of the present embodiment will be described with reference to FIGS. 30(A) and 30(B). FIG. 30(A) is a flowchart illustrating an operation procedure related to computation of the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh and formation of the sound collection directionality in the directionality control system 10 or 10A of the present embodiment. FIG. 30(B) is a flowchart specifically illustrating an operation procedure of calibration in step S20 illustrated in FIG. 30(A).

[0480] In description of the calibration illustrated in FIG. 30(B), initial setting includes an operation of installing or attaching the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, the omnidirectional microphone array apparatus 2 constituting the directionality control system 10 or the directionality control system 10A on or to a predetermined installation surface. Hereinafter, for simplification of description of the calibration illustrated in FIG. 30(B), for example, an operation procedure of the directionality control system 10 illustrated in FIG. 27(A) will be described, and an operation procedure of the directionality control system 10A will be described as necessary in a case where there is content which is different from that of the operation procedure of the directionality control system 10 illustrated in FIG. 27(A).

[0481] In FIG. 30(A), the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, the omnidirectional microphone array apparatus 2 are installed, and calibration for defining a relationship between coordinate axes of the omnidirectional camera apparatus 11z and the omnidirectional microphone array apparatus 2 is performed (step ST20). In addition, details of the operation in step S20 will be described later with reference to FIG. 30(B).

[0482] After the calibration is completed in step ST20, a position where sound is desired to be collected from a video (or an image) captured by the omnidirectional camera apparatus 11z is designated on a screen of the display device 36 (step ST21).

[0483] The coordinate computation section 34x of the directionality control apparatus 3 computes a horizontal angle and a vertical angle (.sub.CAh and .sub.CAv) of the first imaging direction CAX1 designated in step ST21 as coordinates (sound collection direction coordinates, that is, (.sub.MAh,.sub.MAv)) indicating a sound collection direction in which a voice of a subject is collected by the omnidirectional microphone array apparatus 2, by using an input parameter L.sub.CK measured in step ST12 (which will be described later) and the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh computed in step ST15 (step ST22).

[0484] The output control section 34c of the directionality control apparatus 3 forms the sound collection directionality of each of the microphone units 22 and 23 in the sound collection direction indicated by the sound collection direction coordinates (.sub.MAh,.sub.MAv) computed in step ST22 (step ST23).

[0485] Consequently, the omnidirectional microphone array apparatus 2 can relatively increase a volume level of audio data collected from the sound collection direction defined by the sound collection coordinates (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can relatively reduce a volume level of audio data collected from a direction in which the sound collection directionality is not formed.

[0486] In addition, in the directionality control system 10 of the present embodiment, a timing at which the omnidirectional microphone array apparatus 2 collects sound is not limited to the time right after step ST20, and may be, for example, the time after power is supplied to the omnidirectional microphone array apparatus 2.

[0487] The calibration in step S20 will be described in detail. In FIG. 30(B), the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the omnidirectional microphone array apparatus 2 constituting the directionality control system 10 are initially installed so as to be fixed to a predetermined installation surface (for example, a ceiling surface of a room of an event hall or a stand) (step ST11; refer to FIG. 31(A)).

[0488] In step ST11, the calibration omnidirectional camera apparatus C1 and the omnidirectional microphone array apparatus 2 are integrally installed as follows, for example.

[0489] Specifically, first, an attachment tool (not illustrated; for example, an attachment tool made of metal, an attachment tool made of ceramics, or an attachment tool made of a synthetic resin (for example, plastic or elastomer)) is attached and fixed to a predetermined installation surface (for example, a stand).

[0490] After the attachment tool is attached to the predetermined installation surface, both of the calibration omnidirectional camera apparatus C1 and the omnidirectional microphone array apparatus 2 are attached to the attachment tool. As described above, the calibration omnidirectional camera apparatus C1 is fitted into the inner circumferential space inside the opening 21a formed at the center of the casing of the omnidirectional microphone array apparatus 2 so that the omnidirectional microphone array apparatus 2 and the calibration omnidirectional camera apparatus C1 are integrally formed with each other. In addition, the calibration omnidirectional camera apparatus C1 and the microphone array apparatus 2 are integrally formed with each other so that a front direction (0 direction) of each horizontal angle is used in common.

[0491] Therefore, since a horizontal angle and a vertical angle of the second imaging direction CAX2 of the calibration omnidirectional camera apparatus C1 are the same as a horizontal angle and a vertical angle of a sound collection direction of the omnidirectional microphone array apparatus 2, if the second front direction is determined by computing at least the second horizontal deviation angle .sub.Kh, sound collection direction coordinates in the omnidirectional microphone array apparatus 2 can be appropriately computed.

[0492] After the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the omnidirectional microphone array apparatus 2 are initially installed, there is the measurement of an input parameter (for example, the distance L.sub.CK) which is required for the horizontal deviation angle computation section 34w to compute the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh (step ST12). The distance L.sub.CK indicates a distance between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1.

[0493] The process in step ST12 includes a case where the user measures the distance L.sub.CK by using a measuring device (for example, a laser range finder), or a case where the omnidirectional camera apparatus 11z measures and acquires the distance L.sub.CK by using functions of well-known techniques of the omnidirectional camera apparatus 11z. Hereinafter, for simplification of description, it is assumed that the user measures the distance L.sub.CK by using a measuring device (for example, a laser range finder) in step S12. The signal processing unit 33 acquires data regarding the distance L.sub.CK which is an input parameter output from the operation unit 32 in response to a user's input operation.

[0494] After step ST12, the directionality control apparatus 3 receives designation of any designated position A in image data which is being displayed on a screen of the display device 36, via the operation unit 32 (step ST13). The directionality control apparatus 3 transmits a notification indicating that the designation of the designated position A in the image data which is being displayed on the screen of the display device 36 has been received, to the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1.

[0495] After step ST13, in a case where the omnidirectional camera apparatus 11z receives the notification indicating that the designation of the designated position A has been received from the directionality control apparatus 3, the omnidirectional camera apparatus 11z measures and acquires coordinate data including a horizontal angle and a vertical angle (.sub.CAh,.sub.CAv) of the first imaging direction CAX1 which is directed toward the sound collection position A corresponding to the designated position A on the screen designated in step ST13, with the installation position of the omnidirectional camera apparatus 11z as a start point (step ST14). However, in the coordinate data including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) obtained in step ST14, the horizontal angle .sub.CAh is data in a state in which it is not determined which direction is the first front direction.

[0496] Further, in a case where the calibration omnidirectional camera apparatus C1 receives the notification indicating that the designation of the designated position A has been received from the directionality control apparatus 3, the calibration omnidirectional camera apparatus C1 measures and acquires coordinate data including a horizontal angle and a vertical angle (.sub.KAh,.sub.KAv) of the second imaging direction CAX2 which is directed toward the sound collection position A corresponding to the designated position A on the image data designated in step ST13, with the installation position of the calibration omnidirectional camera apparatus C1 as a start point (step ST14). Similarly, in the coordinate data including the horizontal angle and the vertical angle (.sub.KAh,.sub.KAv) obtained in step ST14, the horizontal angle .sub.KAh is data in a state in which it is not determined which direction is the second front direction.

[0497] The omnidirectional camera apparatus 11z transmits the coordinate data including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) of the first imaging direction CAX1 to the directionality control apparatus 3. The calibration omnidirectional camera apparatus C1 transmits the coordinate data including the horizontal angle and the vertical angle (.sub.KAh,.sub.KAv) of the second imaging direction CAX2 to the directionality control apparatus 3.

[0498] The horizontal deviation angle computation section 34w of the directionality control apparatus 3 computes the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh based on the input parameter L.sub.CK measured in step S12, and the coordinate data including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) of the first imaging direction CAX1 and the coordinate data including the horizontal angle and the vertical angle (.sub.KAh,.sub.KAv) of the second imaging direction CAX2 measured in step S14 (step ST15). Details of the operation in step S15 will be described later with reference to FIGS. 31 and 32.

[0499] Consequently, the directionality control apparatus 3 can determine which direction is the first front direction (horizontal angle of 0) of a horizontal angle of the omnidirectional camera apparatus 11z and can further determine which direction is the second front direction (horizontal angle of 0) of a horizontal angle of the calibration omnidirectional camera apparatus C1 before computing sound collection direction coordinates of the omnidirectional microphone array apparatus 2.

[0500] Here, in the initial installation in step ST11, in a case where the omnidirectional microphone array apparatus 2 is installed so that a front direction (for example, a 0 direction) of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a front direction (for example, a 0 direction) of a horizontal angle of the calibration omnidirectional camera apparatus C1 (YES in step ST16), the calibration is completed, and the operation procedure proceeds to step ST21.

[0501] On the other hand, in a case where the omnidirectional microphone array apparatus 2 is not installed so that a reference direction (for example, a 0 direction) of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a front direction (for example, a 0 direction) of a horizontal angle of the calibration omnidirectional camera apparatus C1 in the initial installation in step ST11 (NO in step ST16), the operation procedure of the directionality control system 10 proceeds to step ST21.

[0502] In other words, in step ST17, the omnidirectional microphone array apparatus 2 is installed so that a front direction (for example, a 0 direction) of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a front direction (for example, a 0 direction) of a horizontal angle of the calibration omnidirectional camera apparatus C1 (step ST17).

[0503] In step ST17, the omnidirectional microphone array apparatus 2 is installed as follows, for example, so that a reference direction (for example, a 0 direction) of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a reference direction of a horizontal angle of the second imaging direction CAX2 of the calibration omnidirectional camera apparatus C1.

[0504] Specifically, for example, a triangular or triangular marker (not illustrated) is added to an outer circumference of the casing of the calibration omnidirectional camera apparatus C1. The marker is added in a direction indicating the front direction (0 direction) of a horizontal angle of the second imaging direction of the calibration omnidirectional camera apparatus C1. In addition, for example, a marker with the same shape (not illustrated) is added at a position facing the marker of calibration omnidirectional camera apparatus C1 on a circumferential edge of the opening 21a formed at the center of the casing of the omnidirectional microphone array apparatus 2.

[0505] Therefore, if the omnidirectional microphone array apparatus 2 is installed so that the marker of the calibration omnidirectional camera apparatus C1 faces the marker of the omnidirectional microphone array apparatus 2, the omnidirectional microphone array apparatus 2 is installed so that a front direction (for example, a 0 direction) of a horizontal angle of a sound collection direction of the omnidirectional microphone array apparatus 2 matches a front direction of a horizontal angle of the second imaging direction CAX2 of the calibration omnidirectional camera apparatus C1.

Method of Computing Coordinates (.SUB.MAh.,.SUB.MAv.) Indicating Sound Collection Direction of Omnidirectional Microphone Array Apparatus 2

[0506] Next, with reference to FIGS. 31 and 32, a detailed description will be made of a method of computing the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh in the horizontal deviation angle computation section 34w of the directionality control apparatus 3.

[0507] FIGS. 31(A), 31(B) and 31(C) are diagrams illustrating each positional relationship between the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the sound collection position A in the present embodiment. FIG. 31(A) is a perspective view. FIG. 31(B) is a plan view in which FIG. 7(A) is viewed in a vertically downward direction from an upper side. FIG. 31(C) is a vertical direction sectional view taken along the line K-K of FIG. 7(B).

[0508] FIGS. 32(A), 32(B) and 32(C) are diagrams illustrating each positional relationship between the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the sound collection position A in the present embodiment. FIG. 32(A) is a perspective view. FIG. 32(B) is a plan view in which FIG. 32(A) is viewed in a vertically lower direction from an upper side. FIG. 32(C) is a vertical direction sectional view taken along the line K-K of FIG. 32(B).

[0509] The horizontal deviation angle computation section 34w computes the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh in response to designation of any designated position A from the user in image data displayed on the display device 36 based on:

[0510] (1) the distance L.sub.CK between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 measured in step ST12;

[0511] (2) the coordinates including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) of the first imaging direction CAX1 measured in step ST14;

[0512] (3) the coordinates including the horizontal angle and the vertical angle (.sub.KAh,.sub.KAv) of the second imaging direction CAX2 measured in the initial installation of step ST14;

[0513] (4) the respective heights H.sub.C and H.sub.K of the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 from the horizontal surface measured in the initial installation of step ST11; and

[0514] (5) the height H.sub.A of the sound collection position A from the horizontal surface.

[0515] (4) The respective heights H.sub.C, H.sub.K and H.sub.M of the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the omnidirectional microphone array apparatus 2 from the horizontal surface are fixed values defined when the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the omnidirectional microphone array apparatus 2 are initially installed. For example, the respective heights H.sub.C, H.sub.K and H.sub.M of the omnidirectional camera apparatus 11z, the calibration omnidirectional camera apparatus C1, and the omnidirectional microphone array apparatus 2 from the horizontal surface are the same as each other.

[0516] (5) The height H.sub.A of the sound collection position A from the horizontal surface is a predefined fixed value, and, for example, in a case where there is a person around the sound collection position A when the designated position A is designated with the finger FG of the user, the height H.sub.A thereof is a selected value or an input value corresponding to a size of the person. Alternatively, when the position A is designated with the finger FG of the user, a default value (for example, 1.5 m or 0.8 m) may be used in a case where the directionality control apparatus 3 determines that there is a person (for example, an adult or a child) at the designated position.

[0517] Hereinafter, a detailed description will be made of a method of computing the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh in the horizontal deviation angle computation section 34w. First, as the premise for describing the present computation method, the first front direction indicating a front direction (0 direction) of a horizontal angle which is required to define a horizontal angle of the first imaging direction CAX1 of the omnidirectional camera apparatus 11z is not known, that is, unknown (refer to FIG. 31(B) or 32(B)). Similarly, the second front direction indicating a front direction (0 direction) of a horizontal angle which is required to define a horizontal angle of the second imaging direction CAX2 of the calibration omnidirectional camera apparatus C1 is not known, that is, unknown (refer to FIG. 31(B) or 32(B)).

[0518] The horizontal deviation angle computation section 34w computes a horizontal component distance L.sub.CAh of the distance from the omnidirectional camera apparatus 11z to the sound collection position A according to Equation (64) by using the height H.sub.C of the omnidirectional camera apparatus 11z from the horizontal surface, the height H.sub.A of the sound collection position A from the horizontal surface, and the vertical angle .sub.CAv of the first imaging direction CAX1 in the triangle CAS illustrated in FIG. 31(C).

[00034]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.64\right]& \\ L_{\mathrm{CAh}}=\frac{H_C-H_A}{\tan .Math..Math.\left({}_{\mathrm{CAv}}\right)}& \left(64\right)\end{array}$

[0519] The horizontal deviation angle computation section 34w computes a horizontal component distance L.sub.KAh of the distance from the calibration omnidirectional camera apparatus C1 to the sound collection position A according to Equation (65) by using the height H.sub.K of the calibration omnidirectional camera apparatus C1 from the horizontal surface, the height H.sub.A of the sound collection position A from the horizontal surface, and the vertical angle .sub.KAv of the second imaging direction CAX2 in the triangle KAS illustrated in FIG. 32(C).

[00035]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.65\right]& \\ L_{\mathrm{KAh}}=\frac{H_K-H_A}{\tan .Math..Math.\left({}_{\mathrm{KAv}}\right)}& \left(65\right)\end{array}$

[0520] The horizontal deviation angle computation section 34w computes a cosine value cos .sub.KCAh of a horizontal angle .sub.KCAh formed between a straight line connecting the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 to each other and a straight line K-K from the omnidirectional camera apparatus 11z to the sound collection position A according to Equation (66) based on the cosine theorem for the triangle KCA illustrated in FIG. 31(B) by using the distance L.sub.CK, and the distances L.sub.CAh and L.sub.KAh which are computed according to Equations (64) and (65).

[00036]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.66\right]& \\ \cos .Math..Math.{}_{\mathrm{KCAh}}=\frac{L_{\mathrm{CK}}^2+L_{\mathrm{CAh}}^2-L_{\mathrm{KAh}}^2}{2.Math..Math.L_{\mathrm{CK}}{L}_{\mathrm{CAh}}}& \left(66\right)\end{array}$

[0521] The horizontal deviation angle computation section 34w computes the horizontal angle .sub.KCAh according to Equation (67) by using the computation result of Equation (66).

[Equation 67]

.sub.KCAh=arc cos(L.sub.CK.sup.2+L.sub.CAh.sup.2L.sub.KAh.sup.2/2L.sub.CKL.sub.CAh) (67)

[0522] The horizontal deviation angle computation section 34w computes a cosine value cos .sub.CKAh of a horizontal angle .sub.CKAh formed between the straight line connecting the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 to each other and a straight line K-K from the calibration omnidirectional camera apparatus C1 to the sound collection position A according to Equation (68) based on the cosine theorem for the triangle KCA illustrated in FIG. 31(B) by using the distance L.sub.CK, and the distances L.sub.CAh and L.sub.KAh which are computed according to Equations (64) and (65).

[00037]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.68\right]& \\ \cos .Math..Math.{}_{\mathrm{CKAh}}=\frac{L_{\mathrm{CK}}^2+L_{\mathrm{KAh}}^2-L_{\mathrm{CAh}}^2}{2.Math..Math.L_{\mathrm{CK}}{L}_{\mathrm{KAh}}}& \left(68\right)\end{array}$

[0523] The horizontal deviation angle computation section 34w computes the horizontal angle .sub.KCAh according to Equation (69) by using the computation result of Equation (68).

[00038]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.69\right]& \\ {}_{\mathrm{CKAh}}=\mathrm{arc}.Math..Math.\cos \left(\frac{L_{\mathrm{CK}}^2+L_{\mathrm{KAh}}^2-L_{\mathrm{CAh}}^2}{2.Math..Math.L_{\mathrm{CK}}{L}_{\mathrm{KAh}}}\right)& \left(69\right)\end{array}$

[0524] The horizontal deviation angle computation section 34w computes the first horizontal deviation angle .sub.Ch according to Equation (70) by using the horizontal angle .sub.CAh of the first imaging direction CAX1, and the horizontal angle .sub.KCAh computed according to Equation (67).

[Equation 70]

.sub.Ch=.sub.CAh.sub.KCAh (70)

[0525] The horizontal deviation angle computation section 34w computes the second horizontal deviation angle cm, according to Equation (71) by using the horizontal angle .sub.KAh of the second imaging direction CAX2, and the horizontal angle .sub.CKAh computed according to Equation (68).

[Equation 71]

.sub.Kh=.sub.KAh.sub.CKAh (71)

[0526] In the above-described way, in the directionality control system 10 or the directionality control system 10A of the present embodiment, the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 capture images of the same subject, and the omnidirectional microphone array apparatus 2 collects voice of the subject whose image is captured. If any designated position A is designated in image data which is obtained by the omnidirectional camera apparatus 11z and is displayed on the display device 36, the horizontal deviation angle computation section 34w of the directionality control apparatus 3 computes the first horizontal deviation angle .sub.Ch which is a deviation angle of a front direction of the omnidirectional camera apparatus 11z relative to mutual reference directions connecting the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 to each other, and the second horizontal deviation angle .sub.Kh which is a deviation angle formed between the second front direction of a horizontal angle of the calibration omnidirectional camera apparatus C1 and the reference direction of the calibration omnidirectional camera apparatus C1.

[0527] Consequently, in the directionality control system 10, the directionality control apparatus 3 can compute sound collection direction coordinates of the omnidirectional microphone array apparatus 2 which is attached so as to surround the casing of the calibration omnidirectional camera apparatus C1 or the omnidirectional microphone array apparatus 2 which is attached at a position of the calibration omnidirectional camera apparatus C1 so as to match a reference direction of a horizontal angle of the calibration omnidirectional camera apparatus C1, based on imaging direction coordinates which are directed from the omnidirectional camera apparatus 11z to the sound collection position A.

[0528] In other words, the directionality control apparatus 3 can determine to what extent a deviation of the first front direction of a horizontal angle of the omnidirectional camera apparatus 11z occurs relative to a reference direction of the omnidirectional camera apparatus 11z, and can further determine to what extent a deviation of the second front direction of a horizontal angle of the calibration omnidirectional camera apparatus C1 occurs relative to a reference direction of the calibration omnidirectional camera apparatus C1, before computing sound collection direction coordinates of the omnidirectional microphone array apparatus 2. Therefore, in the directionality control system 10, a horizontal angle and a vertical angle of the second imaging direction CAX2 of the calibration omnidirectional camera apparatus C1 are respectively the same as a horizontal angle and a vertical angle of a sound collection direction of the omnidirectional microphone array apparatus 2. Thus, if the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh are computed, the sound collection direction coordinates of the omnidirectional microphone array apparatus 2 can be appropriately computed based on the computed distance L.sub.ck.

[0529] In other words, the directionality control apparatus 3 can appropriately compute sound collection direction coordinates which are directed from omnidirectional microphone array apparatus 2 to a certain sound collection position in image data captured by the omnidirectional camera apparatus 11z.

[0530] Hereinafter, configurations, operations, and effects of the above-described directionality control system and horizontal deviation angle calibration method related of the present invention will be described.

[0531] According to an embodiment of the present invention, there is provided a directionality control system including a first imaging part that captures an image of a subject; a second imaging part that captures an image of the subject; a sound collection part that collects voice of the subject; a display part that displays image data captured by the first imaging part; and a deviation amount computation part that computes a first horizontal deviation angle of a horizontal angle of a first imaging direction which is directed from the first imaging part toward a sound collection position corresponding to a designated position in the image data relative to a first reference horizontal angle, and a second horizontal deviation angle of a horizontal angle of a second imaging direction which is directed from the second imaging part toward the sound collection position relative to a second reference direction, in response to designation of any position in the displayed image data.

[0532] In the above-described configuration, the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 capture images of the same subject, and the omnidirectional microphone array apparatus 2 collects voice of the subject whose image is captured. If any designated position A is designated in image data which is obtained by the omnidirectional camera apparatus 11z and is displayed on the display device 36, the horizontal deviation angle computation section 34w of the directionality control apparatus 3 computes the first horizontal deviation angle .sub.Ch which is a deviation angle of the first front direction (for example, a 0 direction) of a horizontal angle of the omnidirectional camera apparatus 11z relative to mutual reference directions connecting the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 to each other, and the second horizontal deviation angle .sub.Kh which is a deviation angle formed between the second front direction of a horizontal angle of the calibration omnidirectional camera apparatus C1 and the reference direction of the calibration omnidirectional camera apparatus C1.

[0533] Consequently, in the directionality control system 10, the directionality control apparatus 3 can compute sound collection direction coordinates of the omnidirectional microphone array apparatus 2 which is attached so as to surround the casing of the calibration omnidirectional camera apparatus C1 or the omnidirectional microphone array apparatus 2 which is attached at a position of the calibration omnidirectional camera apparatus C1 so as to match a reference direction of a horizontal angle of the calibration omnidirectional camera apparatus C1, based on imaging direction coordinates which are directed from the omnidirectional camera apparatus 11z to the sound collection position A, and can thus compute each deviation amount between a front direction (for example, a 0 direction) of a horizontal angle of the sound collection direction coordinates and the mutual reference directions. Therefore, in the directionality control system 10, the directionality control apparatus 3 can appropriately compute sound collection direction coordinates which are directed from omnidirectional microphone array apparatus 2 to a certain sound collection position in image data captured by the omnidirectional camera apparatus 11z.

[0534] In addition, in the directionality control system according to the embodiment of the present invention, the deviation amount computation part computes the first horizontal deviation angle and the second horizontal deviation angle based on a distance from the first imaging part to the second imaging part, a horizontal angle and a vertical angle which are directed from the first imaging part toward the sound collection position, a horizontal angle and a vertical angle which are directed from the second imaging part toward the sound collection position, a height of the first imaging part from a horizontal surface, a height of the second imaging part from the horizontal surface, and a height of the sound collection position from the horizontal surface.

[0535] In the above-described configuration, the horizontal deviation angle computation section 34w of the directionality control apparatus 3 computes the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh based on the distance L.sub.CK between the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1, the coordinates including the horizontal angle .sub.CAh and the vertical angle .sub.CAv which are directed from the omnidirectional camera apparatus 11z toward the sound collection position A, the coordinates including the horizontal angle .sub.KAh and the vertical angle .sub.KAv which are directed from the calibration omnidirectional camera apparatus C1 toward the sound collection position A, the height H.sub.C of the omnidirectional camera apparatus 11z from the horizontal surface, the height H.sub.K of the calibration omnidirectional camera apparatus C1 from the horizontal surface, and the height H.sub.A of the sound collection position A from the horizontal surface. Consequently, the directionality control apparatus 3 can easily compute the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh.

[0536] Further, in the directionality control system according to the embodiment of the present invention, the directionality control system further includes a coordinate computation part that computes a horizontal angle and a vertical angle of a direction which is directed from the sound collection part to the sound collection position as coordinates indicating a sound collection direction in which the sound collection part collects the voice of the subject based on the first horizontal deviation angle and the second horizontal deviation angle.

[0537] In this above-described configuration, the coordinate computation section 34x of the directionality control apparatus 3 can compute a horizontal angle and a vertical angle of sound collection direction coordinates of the omnidirectional microphone array apparatus 2 which is attached so as to surround the casing of the calibration omnidirectional camera apparatus C1 or the omnidirectional microphone array apparatus 2 which is attached at a position of the calibration omnidirectional camera apparatus C1 so as to match a reference direction of a horizontal angle of the calibration omnidirectional camera apparatus C1, based on the first horizontal deviation angle .sub.Ch and the second horizontal deviation angle .sub.Kh computed by the horizontal deviation angle computation section 34w.

[0538] In addition, in the directionality control system according to the embodiment of the present invention, the directionality control system further includes an output control part that causes the sound collection part to form sound collection directionality of audio data in the sound collection direction corresponding to the computed coordinates indicating the sound collection direction.

[0539] In the above-described configuration, the output control section 34c of the directionality control apparatus 3 causes the omnidirectional microphone array apparatus 2 to form the sound collection directionality of audio data in the sound collection direction MIX corresponding to the sound collection direction coordinates of the omnidirectional microphone array apparatus 2 computed by the coordinate computation section 34x. Consequently, the directionality control apparatus 3 can cause the omnidirectional microphone array apparatus 2 to appropriately collect conversation voice of a subject who is present in an imaging direction of the omnidirectional camera apparatus 11z.

[0540] Further, in the directionality control system according to the embodiment of the present invention, a columnar opening is formed at a center of a casing of the sound collection part, and the sound collection part and the second imaging part are integrally formed with each other as a result of the second imaging part being fitted into an inner circumferential space of the opening.

[0541] In the above-described configuration, the columnar opening is formed at the center of the casing of the omnidirectional microphone array apparatus 2, and the omnidirectional microphone array apparatus 2 and the calibration omnidirectional camera apparatus C1 are integrally formed with each other as a result of the calibration omnidirectional camera apparatus C1 being fitted into the inner circumferential space of the opening 21a. Consequently, the omnidirectional microphone array apparatus 2 can use a horizontal angle and a vertical angle of the second imaging direction CAX2 which is directed from the calibration omnidirectional camera apparatus C1 toward the sound collection position A in common as a horizontal angle and a vertical angle of the sound collection direction MIX of the omnidirectional microphone array apparatus 2.

[0542] Further, according to another embodiment of the present invention, there is provided a horizontal deviation angle computation method for a directionality control system including a first imaging part, a second imaging part, and a sound collection part, the method including a step of causing the first imaging part to capture an image of a subject; a step of causing the second imaging part to capture an image of the subject; a step of causing the sound collection part to collect voice of the subject; a step of displaying image data captured by the first imaging part on a display part; and a step of computing a first horizontal deviation angle of a horizontal angle of a first imaging direction which is directed from the first imaging part toward a sound collection position corresponding to a designated position in the image data relative to a first reference angle, and a second horizontal deviation angle of a horizontal angle of a second imaging direction which is directed from the second imaging part toward the sound collection position relative to a second reference direction, in response to designation of any position in the image data displayed on the display part.

[0543] In the above-described method, the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 capture images of the same subject, and the omnidirectional microphone array apparatus 2 collects voice of the subject whose image is captured. If any designated position A is designated in image data which is obtained by the omnidirectional camera apparatus 11z and is displayed on the display device 36, the horizontal deviation angle computation section 34w of the directionality control apparatus 3 computes the first horizontal deviation angle .sub.Ch which is a deviation angle of the first front direction (for example, a 0 direction) of a horizontal angle of the omnidirectional camera apparatus 11z relative to mutual reference directions connecting the omnidirectional camera apparatus 11z and the calibration omnidirectional camera apparatus C1 to each other, and the second horizontal deviation angle .sub.Kh which is a deviation angle formed between the second front direction of a horizontal angle of the calibration omnidirectional camera apparatus C1 and the reference direction of the calibration omnidirectional camera apparatus C1.

[0544] Consequently, in the directionality control system 10, the directionality control apparatus 3 can compute sound collection direction coordinates of the omnidirectional microphone array apparatus 2 which is attached so as to surround the casing of the calibration omnidirectional camera apparatus C1 or the omnidirectional microphone array apparatus 2 which is attached at a position of the calibration omnidirectional camera apparatus C1 so as to match a reference direction of a horizontal angle of the calibration omnidirectional camera apparatus C1, based on imaging direction coordinates which are directed from the omnidirectional camera apparatus 11z to the sound collection position A, and can thus compute each deviation amount between a front direction (for example, a 0 direction) of a horizontal angle of the sound collection direction coordinates and the mutual reference directions. Therefore, in the directionality control system 10, the directionality control apparatus 3 can appropriately compute sound collection direction coordinates which are directed from omnidirectional microphone array apparatus 2 to a certain sound collection position in image data captured by the omnidirectional camera apparatus 11z.

[0545] Each of seventh to tenth embodiments described below relates to a directionality control system and a directionality control method, in which a height of a target object present in a sound collection space as a sound source from a reference surface is determined, and sound collection directionality of sound collected by a microphone array apparatus is formed by using the height of the target object from the reference surface.

[0546] Patent Literature 1 is based on, for example, a camera apparatus, a microphone array apparatus, and a target object (for example, a person) being present on the same plane assuming usage forms in a television conference system. However, in the above-described monitoring system, the camera apparatus, the microphone array apparatus, and the target object (for example, a person) are all seldom present on the same plane in practice.

[0547] For example, as illustrated in FIG. 46, since a camera apparatus CA and a microphone array apparatus Mic-A are frequently installed on an upper side (for example, a ceiling surface of a store) relative to target objects (two people) as sound collection targets, the camera apparatus CA, the microphone array apparatus Mic-A, the target objects (two people) are present on stereoscopic three-dimensional coordinates. FIG. 46 is a diagram for explaining a problem in the monitoring system of the related art.

[0548] Therefore, in the monitoring system illustrated in FIG. 46, in a case where the microphone array apparatus Mic-A collects conversations of the target objects (two people) imaged by the camera apparatus CA, if a method of Patent Literature 1 is used, there is a problem in that coordinates (horizontal angle, vertical angle) indicating a direction in which the microphone array apparatus Mic-A collect sounds cannot be appropriately computed.

[0549] In addition, in the monitoring system illustrated in FIG. 46, in a case where the microphone array apparatus Mic-A collects conversations of the target objects (two people) which are currently being imaged by the camera apparatus CA, even if horizontal angles and vertical angles indicating directions which are directed from the camera apparatus CA toward the target objects are respectively the same as each other, if heights of target object sound positions (hereinafter, simply referred to as target sound source positions or sound positions) from a reference surface (for example, a floor surface) are different from each other, there is a problem in that a sound collection directional direction which is directed from the microphone array apparatus Mic-A toward a target sound source position is not uniquely specified.

[0550] For example, in FIG. 46, a point A, a point A, and a point A are positions where horizontal angles and vertical angles from the camera apparatus CA are the same as each other, but if one of heights H.sub.A, H.sub.A and H.sub.A of the point A, the point A, and the point A from a reference surface (for example, a floor surface) is not defined, sound collection directional directions which are directed from the microphone array apparatus Mic-A toward the point A, the point A, and the point A are all different from each other. In other words, it is hard for the microphone array apparatus Mic-A to collect the conversations of the target objects (two people) who are subjects of the camera apparatus CA with high accuracy.

[0551] Therefore, in each of the seventh to tenth embodiments of the present invention, in order to solve the above-described problems, a description will be made of examples of a directionality control system and a directionality control method, in which a height of a target sound source position present in a sound collection space from a reference surface is determined, and sound collection directionality is formed in a sound collection directional direction which is directed from a microphone array apparatus toward the target sound source position based on the height of the target sound source position from the reference surface.

[0552] Hereinafter, with reference to the drawings, description will be made of each of the seventh to tenth embodiments of a directionality control system and a directionality control method related to the present invention. The directionality control system of each embodiment is used as a monitoring system (including a manned monitoring system and an unmanned monitoring system) provided in, for example, a factory, a company, a public facility (for example, an event hall), or a store (for example, a retail store), but an installation location is not particularly limited. In the following respective embodiments, description will be made assuming that the directionality control system of each embodiment is installed in, for example, a store.

[0553] In addition, the present invention can be expressed as respective apparatuses (for example, a directionality control apparatus to be described later) constituting the directionality control system, or a directionality control method including respective operations (steps) performed by each apparatus constituting the directionality control system.

Seventh Embodiment

Configuration of Directionality Control System

[0554] FIG. 33 is a block diagram illustrating configurations of a directionality control system 10 of the seventh embodiment. The directionality control system 10 illustrated in FIG. 33 includes at least one camera apparatuses 11 to 1n, an omnidirectional microphone array apparatus 2, a directionality control apparatus 3, and a recorder apparatus 4. Here, n indicates the number of camera apparatuses, and is an integer of 1 or higher. The camera apparatuses 11 to 1n, the omnidirectional microphone array apparatus 2, the directionality control apparatus 3, and the recorder apparatus 4 are connected to each other via a network NW. The network NW may be a wired network (for example, an intranet or the Internet), and may be a wireless network (for example, a wireless local area network (LAN)), which is also the same for the following embodiments.

[0555] The camera apparatuses 11 to 1n as at least one imaging part includes a casing into which an optical system (for example, a wide angle lens) and an imaging system (for example, an image sensor) (not illustrated) are built, and is fixed to and installed on, for example, a ceiling surface of the store or a stand (refer to FIG. 34(A)) so as to function as a monitoring camera. The camera apparatuses 11 to 1n are connected to the directionality control apparatus 3 of a central control room (not illustrated) via the network NW, and performs a panning direction operation, a tilting direction operation, a zooming operation, an imaging operation, and a distance-measuring operation and an angle-measuring operation related to an actual target sound source position A corresponding to a designated position (for example, a designated position A illustrated in FIG. 34(B) to be described later) in a captured video in response to a remote operation from the directionality control apparatus 3.

[0556] In addition, the camera apparatuses 11 to 1n capture a video (including a still image and a moving image; this is also the same for the following description) of a target object which is present in a predefined angle of view centering on an optical axis. The camera apparatuses 11 to 1n transmit the captured video data, and input parameters for computing a horizontal angle .sub.MAh and a vertical angle .sub.MAv of a sound collection directional direction which will be described later to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW.

[0557] The omnidirectional microphone array apparatus 2 as a sound collection part includes, for example, a doughnut-shaped or ring-shaped (annular) casing 21C (refer to FIG. 2(D)) in which an opening 21a is formed at its center, and is fixed to and installed on, for example, a ceiling surface of the store or a predetermined stand (refer to FIG. 34(A)). The omnidirectional microphone array apparatus 2 forms sound collection directionality for collecting sound with high accuracy in a sound collection directional direction which is directed from an installation position M of the omnidirectional microphone array apparatus 2 toward a target sound source position A, and collects conversation voice (for example, Hello) of target objects (two people) present in the sound collection directional direction with high accuracy. In addition, a shape of the casing of the omnidirectional microphone array apparatus 2 is not limited to a doughnut shape or a ring shape (annular shape), and description thereof will be omitted since the description thereof has been made with reference to FIG. 2.

[0558] In the omnidirectional microphone array apparatus 2, a plurality of microphone units 22 are disposed in a concentric shape around the opening 21a in a circumferential direction of the casing 21C. The microphone unit 22 employs, for example, a high-quality small-sized electret condenser microphone (ECM), and this is also the same for the following respective embodiments.

[0559] The omnidirectional microphone array apparatus 2 is connected to the network NW and includes at least microphone units 22 and 23 in which microphones are provided at equal intervals (refer to FIGS. 2(A) to 2(E)) and a control unit (not illustrated) which controls an operation of each of the microphone units 22 and 23.

[0560] The omnidirectional microphone array apparatus 2 collects sound in a sound collection directional direction in which a target object (sound source) as a sound collection target is present by using each of the microphone units 22 and 23, performs predetermined sound processing on audio data collected by each of the microphone units 22 and 23, and transmits a processed result to the directionality control apparatus 3 or the recorder apparatus 4 via the network NW.

[0561] The omnidirectional microphone array apparatus 2 forms sound collection directionality of each of the microphone units 22 and 23 in sound collection directional coordinates (.sub.MAh,.sub.MAv) which are computed by a sound collection directional direction computation section 34b of a signal processing unit 33 of the directionality control apparatus 3 in response to a directionality formation instruction from the directionality control apparatus 3 which will be described later.

[0562] Consequently, the omnidirectional microphone array apparatus 2 can relatively increase a volume level of sound collected from the sound collection directional coordinates (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can relatively reduce a volume level of sound collected from a direction in which the sound collection directionality is not formed. In addition, a method of computing the sound collection directional coordinates (.sub.MAh,.sub.MAv) will be described later.

[0563] The directionality control apparatus 3 is connected to the network NW, and may be, for example, a stationery personal computer (PC) installed in a central control room (not illustrated) of a company, and may be a mobile phone, a tablet terminal, or a smart phone, which can be carried by a user.

[0564] The directionality control apparatus 3 includes at least a communication unit 31, an operation unit 32, a signal processing unit 33, a display device 36, a speaker device 37, and a memory 38. The signal processing unit 33 includes at least a sound source height determination section 34a, a sound collection directional direction computation section 34b, and an output control section 34c.

[0565] The communication unit 31 outputs video data or audio data which is transmitted from the camera apparatuses 11 to 1n or the omnidirectional microphone array apparatus 2, to the signal processing unit 33 via the network NW.

[0566] The operation unit 32 is a user interface (UI) for notifying the signal processing unit 33 of the content of a user's input operation, and is, for example, a pointing device such as a mouse or a keyboard. In addition, the operation unit 32 may be configured by using a touch panel or a touch pad which is disposed so as to correspond to, for example, a screen of the display device 36 and allows an input operation to be performed with the finger FG of the user or a stylus pen.

[0567] The operation unit 32 acquires coordinate data indicating a location where the user desires to increase or decrease a volume level, that is, a designated position A illustrated in FIG. 34(B) and outputs the coordinate data to the signal processing unit 33 in response to the user's input operation.

[0568] The signal processing unit 33 is configured by using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and performs a control process for collectively controlling operations of the respective units of the directionality control apparatus 3, data input and output processes with other respective units, a data computation (calculation) process, and a data storage process.

[0569] The sound source height determination section 34a as a height determination part determines a height H.sub.A of the target sound source position A from a floor surface BL, corresponding to the designated position A if the designated position A is designated with the finger FG of the user or a stylus pen in video data captured by, for example, the camera apparatus 11 as video data displayed on the display device 36. In the following respective embodiments, as long as there is no particular description, for example, a reference surface is the floor surface BL in the store.

[0570] Specifically, if the designated position A in the video data displayed on the display device 36 is designated with the finger FG of the user, the sound source height determination section 34a reads data regarding the height of the target sound source position A from the floor surface BL, corresponding to coordinate data regarding the designated position A, from a configuration file CF1. The sound source height determination section 34a determines the read data regarding the height of the target sound source position A from the floor surface BL, corresponding to the coordinate data regarding the designated position A as a height H.sub.A of the target sound source position A corresponding to the designated position A from the floor surface BL.

[0571] The sound collection directional direction computation section 34b computes coordinates (.sub.MAh,.sub.MAv) (hereinafter, simply referred to as sound collection directional coordinates) indicating a sound collection directional direction which is directed from the installation position M of the omnidirectional microphone array apparatus 2 toward the target sound source position A corresponding to the designated position A in response to the designation of the designated position A in the video data displayed on the display device 36.

[0572] In the sound collection directional coordinates (.sub.MAh,.sub.MAv), .sub.MAh indicates a horizontal angle of the sound collection directional direction which is directed from the omnidirectional microphone array apparatus 2 toward the target sound source position A, and .sub.MAv indicates a vertical angle of the sound collection directional direction which is directed from the omnidirectional microphone array apparatus 2 toward the target sound source position A. In addition, the target sound source position A is a field position which corresponds to the designated position A which is designated with the finger FG of the user or a stylus pen in the video data displayed on the display device 36 via the operation unit 32, and is an actual monitoring target.

[0573] The output control section 34c as a control part controls operations of the camera apparatuses 11 to 1n, the omnidirectional microphone array apparatus 2, the display device 36, and the speaker device 37, so as to reproduce and output video data transmitted from the camera apparatuses 11 to 1n on the display device 36 and to output audio data transmitted from the omnidirectional microphone array apparatus 2 from the speaker device 37 as sound. The output control section 34c causes the omnidirectional microphone array apparatus 2 to form sound collection directionality of audio data in a sound collection directional direction corresponding to sound collection direction coordinates (.sub.MAh,.sub.MAv) computed by the sound collection directional direction computation section 34b.

[0574] The display device 36 as a display part displays video data captured by the camera apparatuses 11 to 1n on a screen thereof.

[0575] The speaker device 37 as a sound output part outputs, as sound, audio data collected by the omnidirectional microphone array apparatus 2 or audio data which is collected by the omnidirectional microphone array apparatus 2 after the sound collection directionality is formed in the sound collection directional coordinates (.sub.MAh,.sub.MAv) computed by the sound collection directional direction computation section 34b. In addition, the display device 36 and the speaker device 27 may be configured separately from the directionality control apparatus 3.

[0576] The memory 38 as a storage part is configured by using, for example, a random access memory (RAM), and functions as a program memory, a data memory, and a work memory when the respective units of the directionality control apparatus 3 operate. In addition, the memory 38 stores the configuration file CF1 illustrated in FIG. 33. The configuration file CF1 includes at least data regarding the height H.sub.C of, for example, the camera apparatus 11 from the floor surface BL, data regarding the height H.sub.M of the omnidirectional microphone array apparatus 2 from the floor surface BL, and data (first configuration data) regarding the height H.sub.A of the target sound source position A corresponding to a predetermined designated position A in video data which is being captured by, for example, the camera apparatus 11 from the floor surface BL.

[0577] The recorder apparatus 4 records the video data captured by the camera apparatuses 11 to 1n and the audio data collected by the omnidirectional microphone array apparatus 2 in correlation with each other.

[0578] Next, a summary of an operation of the directionality control system 10 of the present embodiment will be described with reference to FIGS. 34(A) and 34(B). FIG. 34(A) is a diagram illustrating a state in which the camera apparatus 11 images target objects (two people) and a state in which the omnidirectional microphone array apparatus 2 collects conversations of the target objects (two people) who are present in a sound collection directional direction and music output from a speaker device SP which is not present in the sound collection directional direction, in a sound collection space K in which the directionality control system 10 is installed.

[0579] FIG. 34(B) is a diagram illustrating a state in which voice (for example, Hello) collected in a sound collection directional direction which is directed from the omnidirectional microphone array apparatus 2 toward the target sound source position A corresponding to the designated position A which is designated with the finger FG of the user in video data displayed on the display device 36 is output so that a volume level thereof is higher than a volume level of music (for example, ) output from the speaker device SP.

[0580] In the directionality control system 10 illustrated in FIG. 34(A), the camera apparatus 11 images subjects (for example, two people illustrated in FIG. 34(A)) reflected in a range of an angle of view unique to the camera apparatus 11. The omnidirectional microphone array apparatus 2 collects sound around the installation position M of the omnidirectional microphone array apparatus 2 in the sound collection space K. In FIG. 34(A), the two people as target objects are having conversations, and Hello is an example of conversation content. Video data captured by the camera apparatus 11 is displayed on the display device 36 of the directionality control apparatus 3 (refer to FIG. 34(B)), and, for example, the two people as target objects and the speaker device SP are displayed thereon.

[0581] In FIG. 34(B), if the designated position A on the display device 36 is designated with the finger FG of the user, the directionality control apparatus 3 reads the data regarding the height H.sub.A of the target sound source position A corresponding to coordinate data indicating the designated position A from the floor surface BL from the configuration file CF1, and computes the sound collection directional coordinates (.sub.MAh,.sub.MAv) which are directed from the installation position M of the omnidirectional microphone array apparatus 2 toward the target sound source position A by using the read data regarding the height H.sub.A. The omnidirectional microphone array apparatus 2 forms the sound collection directionality in the direction which is directed from the installation position M of the omnidirectional microphone array apparatus 2 toward the target sound source position A by using the coordinate data regarding the sound collection directional coordinates (.sub.MAh,.sub.MAv) computed by the directionality control apparatus 3.

[0582] Next, an operation procedure of initial setting performed in the directionality control system 10 of the present embodiment will be described with reference to FIG. 35(A). FIG. 35(A) is a flowchart illustrating an operation procedure of the initial setting in the directionality control system 10 of the seventh embodiment. The initial setting includes, for example, an operation in which the camera apparatuses 11 to 1n or the omnidirectional microphone array apparatus 2 is initially installed, and an operation in which the sound collection directional direction computation section 34b acquires input parameters which are required to compute a sound collection directional direction, which is also the same for the following respective embodiments.

[0583] In FIG. 35(A), the camera apparatus 11 and the omnidirectional microphone array apparatus 2 constituting the directionality control system 10 are initially installed so as to be fixed at predetermined positions (for example, a ceiling surface of the store or a stand) (step ST1). The camera apparatus 11 and the omnidirectional microphone array apparatus 2 are respectively installed at different positions (refer to FIG. 34(A)).

[0584] After the camera apparatus 11 and the omnidirectional microphone array apparatus 2 are initially installed, the sound collection directional direction computation section 34b measures each input parameter which is required to compute sound collection directional coordinates (.sub.MAh,.sub.MAv) (step ST2). The process in step ST2 includes a case where the user measures the input parameter by using a measuring device (for example, a laser range finder), and a case where the camera apparatus 11 measures and acquires the input parameter by using functions of well-known techniques of the camera apparatus 11. Each input parameter in step ST2 differs in each method of computing sound collection directional coordinates, and thus detailed content thereof will be described with reference to FIGS. 36 to 38.

[0585] After step ST2, each input parameter measured in step ST2 is input to the signal processing unit 33 of the directionality control apparatus 3 from the camera apparatus 11 or is input to the signal processing unit 33 from the operation unit 32 (step ST3). For example, the camera apparatus 11 transmits an input parameter acquired by using functions of well-known techniques of the camera apparatus 11, to the communication unit 31 of the directionality control apparatus 3. The communication unit 31 outputs the input parameter transmitted by the camera apparatus 11, to the signal processing unit 33. In addition, the operation unit 32 outputs data regarding the height H.sub.A of the target sound source position A from the floor surface BL as an example of the input parameter to the signal processing unit 33 in response to a user's input operation.

[0586] The signal processing unit 33 generates the configuration file CF1 including the respective input parameters acquired in step ST3 and preserves the configuration file CF1 in the memory 38 (step ST4). In the above-described way, the operation of the initial setting in the directionality control system 10 is finished.

[0587] Next, an operation procedure following the initial setting in the directionality control system 10 of the present embodiment will be described with reference to FIG. 35(B). FIG. 35(B) is a flowchart illustrating an operation procedure following the initial setting in the directionality control system 10 of the seventh embodiment.

[0588] In FIG. 35(B), the directionality control apparatus 3 receives designation of the designated position A in video data which is being displayed on the display device 36 illustrated in FIG. 34(B), via the operation unit 32 (step ST11). The directionality control apparatus 3 transmits a notification indicating that the designation of the designated position A in the video data which is being displayed on the display device 36 has been received, to the camera apparatus 11.

[0589] After step ST11, in a case where the camera apparatus 11 receives the notification indicating that the designation of the designated position A has been received from the directionality control apparatus 3, the camera apparatus 11 acquires coordinate data including a horizontal angle and a vertical angle (.sub.CAh,.sub.CAv) to the target sound source position A corresponding to the position A designated in step ST11, with an installation position C of the camera apparatus 11 as a start point (step ST12).

[0590] The camera apparatus 11 transmits the coordinate data including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) to the target sound source position A corresponding to the position A designated in step ST11 with the installation position C of the camera apparatus 11 as a start point, to the directionality control apparatus 3.

[0591] The sound source height determination section 34a of the signal processing unit 33 of the directionality control apparatus 3 reads data regarding the height of the target sound source position A from the floor surface BL, corresponding to the coordinate data regarding the designated position A, from the configuration file CF1 stored in the memory 38 in step ST4. The sound source height determination section 34a determines the read data regarding the height of the target sound source position A from the floor surface BL, corresponding to the coordinate data regarding the designated position A as a height H.sub.A of the target sound source position A corresponding to the designated position A from the floor surface BL.

[0592] Further, the sound collection directional direction computation section 34b of the signal processing unit 33 of the directionality control apparatus 3 computes the sound collection directional coordinates (.sub.MAh,.sub.MAv) which are directed from the installation position M of the omnidirectional microphone array apparatus 2 toward the target sound source position A by using the coordinate data including the horizontal angle and the vertical angle (.sub.CAh,.sub.CAv) from the camera apparatus 11 to the target sound source position A, and the respective input parameters (including the height H.sub.A of the target sound source position A corresponding to the designated position A from the floor surface BL) read from the configuration file CF1 (step ST13). A process of computing the sound collection directional coordinates (.sub.MAh,.sub.MAv) will be described in detail with reference to FIGS. 36 and 37.

[0593] The directionality control apparatus 3 transmits a directionality formation instruction including the sound collection directional coordinates (.sub.MAh,.sub.MAv) computed in step ST13 to the omnidirectional microphone array apparatus 2. The omnidirectional microphone array apparatus 2 forms the sound collection directionality of each of the microphones 22 and 23 in a sound collection directional direction indicated by the sound collection directional coordinates (.sub.MAh,.sub.MAv) computed by the directionality control apparatus 3 in response to the directionality formation instruction from the directionality control apparatus 3 (step ST14).

[0594] Consequently, the microphone array apparatus 2 can increase a volume level of audio data which is collected from the sound collection directional direction indicated by the sound collection directional coordinates (.sub.MAh,.sub.MAv) in which the sound collection directionality is formed, and can reduce a volume level of audio data which is collected from a direction in which the sound collection directionality is not formed. In the above-described way, the operation following the initial setting in the directionality control system 10 is finished.

[0595] In addition, in the directionality control system 10 of the present embodiment, a timing at which the omnidirectional microphone array apparatus 2 collects sound is not limited to the time right after step ST14, and may be, for example, the time after power is supplied to the omnidirectional microphone array apparatus 2.

Method of Computing Coordinates (.SUB.MAh.,.SUB.MAv.) Indicating Sound Collection Directional Direction of Omnidirectional Microphone Array Apparatus 2

[0596] Here, with reference to FIGS. 36 and 37, a detailed description will be made of a method of computing the sound collection directional coordinates (.sub.MAh,.sub.MAv) of the omnidirectional microphone array apparatus 2 in the sound collection directional direction computation section 34b of the signal processing unit 33 of the directionality control apparatus 3.

[0597] FIG. 36(A) is a perspective view illustrating each position of the camera apparatus 11, the omnidirectional microphone array apparatus 2, a reference point O, and the target sound source position A. FIG. 36(B) is a horizontal direction plan view in which FIG. 36(A) is viewed in a vertically lower direction from a vertically upper direction. FIG. 36(C) is a vertical direction sectional view taken along the line K-K of FIG. 36(B).

[0598] FIG. 37(A) is a perspective view illustrating each position of the camera apparatus 11, the omnidirectional microphone array apparatus 2, the reference point O, and the target sound source position A. FIG. 37(B) is a horizontal direction plan view in which FIG. 37(A) is viewed in a vertically lower direction from a vertically upper direction. FIG. 37(C) is a vertical direction sectional view taken along the line Q-Q of FIG. 37(B).

[0599] The sound collection directional direction computation section 34b computes the sound collection directional coordinates (.sub.MAh,.sub.MAv) of the omnidirectional microphone array apparatus 2 based on:

[0600] (1) a distance L.sub.CM between the camera apparatus 11 and the omnidirectional microphone array apparatus 2;

[0601] (2) a horizontal angle .sub.CAh and a vertical angle .sub.CAv from the camera apparatus 11 to the target sound source position A;

[0602] (3) respective heights H.sub.C and H.sub.M (H.sub.C=H.sub.M) of the camera apparatus 11 and the omnidirectional microphone array apparatus 2 from the floor surface BL; and

[0603] (4) a height H.sub.A of the target sound source position A from the floor surface BL.

[0604] In the present computation method, the input parameters in step ST2 illustrated in FIG. 35(A) include:

[0605] (1) the distance L.sub.CM between the camera apparatus 11 and the omnidirectional microphone array apparatus 2;

[0606] (3) the respective heights H.sub.C and H.sub.M (H.sub.C=H.sub.M) of the camera apparatus 11 and the omnidirectional microphone array apparatus 2 from the floor surface BL.

[0607] In addition, among the respective input parameters in the present computation method,

[0608] (1) the distance L.sub.CM between the camera apparatus 11 and the omnidirectional microphone array apparatus 2 is a fixed value measured, for example, in the initial setting in step ST1 illustrated in FIG. 35(A).

[0609] (3) The respective heights H.sub.C and H.sub.M (H.sub.C=H.sub.M) of the camera apparatus 11 and the omnidirectional microphone array apparatus 2 from the floor surface BL are fixed values measured, for example, in the initial setting in step ST1 illustrated in FIG. 35(A). In addition, for simplification of description, the description will be made assuming that the heights H.sub.C and H.sub.M of the camera apparatus 11 and the omnidirectional microphone array apparatus 2 from the floor surface BL are the same as each other, but the heights may be different from each other.

[0610] In addition, among the respective input parameters in the present computation method,

[0611] (2) the horizontal angle .sub.CAh and the vertical angle .sub.CAv from the camera apparatus 11 to the target sound source position A are acquired by using a function of a well-known technique of the camera apparatus 11 in step ST12 illustrated in FIG. 35(B).

[0612] Further, in the present computation method,

[0613] (4) the height H.sub.A of the target sound source position A from the floor surface BL is a fixed value determined by the sound source height determination section 34a of the signal processing unit 33 of the directionality control apparatus 3, that is, a fixed value (predetermined value) written in the configuration file CF1 in step ST4 illustrated in FIG. 35(A).

[0614] Hereinafter, a detailed description will be made of a method of computing the sound collection directional coordinates (.sub.MAh,.sub.MAv) of the omnidirectional microphone array apparatus 2 in the sound collection directional direction computation section 34b.

[0615] The sound collection directional direction computation section 34b computes a horizontal component distance L.sub.CAh of the distance L.sub.CA from the camera apparatus 11 to the target sound source position A according to Equation (58) by using the respective heights H.sub.C and H.sub.M of the camera apparatus 11 and the omnidirectional microphone array apparatus 2 from the floor surface BL, and the vertical angle .sub.CAv from the camera apparatus 11 to the target sound source position A, in the triangle CAP illustrated in FIG. 36(C).

[0616] The sound collection directional direction computation section 34b computes a horizontal component distance L.sub.MAh of the distance from the omnidirectional microphone array apparatus 2 to the target sound source position A according to Equation (72) based on the cosine theorem for the triangle CAM illustrated in FIG. 36(B) by using the computation result of Equation (64), the horizontal angle .sub.CAh from the camera apparatus 11 to the target sound source position A, and the distance L.sub.CM from the camera apparatus 11 to the omnidirectional microphone array apparatus 2.

[Equation 72]

L.sub.MAh={square root over ((L.sub.CAh.sup.2+L.sub.CM.sup.22L.sub.CAhL.sub.CMcos .sub.CAh)}(72)

[0617] The sound collection directional direction computation section 34b computes a cosine value cos .sub.MAh of the horizontal angle .sub.MAh of the depression angle .sub.MA from the omnidirectional microphone array apparatus 2 to the target sound source position A according to Equation (73) based on the cosine theorem for the triangle CAM illustrated in FIG. 36(B) by using the respective computation results of Equations (64) and (72), and the distance L.sub.CM from the camera apparatus 11 to the omnidirectional microphone array apparatus 2.

[0618] Consequently, the sound collection directional direction computation section 34b can compute the horizontal angle .sub.MAh of the depression angle .sub.MA from the omnidirectional microphone array apparatus 2 to the target sound source position A according to Equation (74).

[00039]$\begin{array}{cc}\left[\mathrm{Equation}.Math..Math.73\right]& \\ \cos .Math..Math.{}_{\mathrm{MAh}}=\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math..Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}& \left(73\right)\\ \left[\mathrm{Equation}.Math..Math..Math.74\right]& \\ {}_{\mathrm{MAh}}=\mathrm{arc}.Math..Math.\cos \left(\frac{L_{\mathrm{MAh}}^2+L_{\mathrm{CM}}^2-L_{\mathrm{CAh}}^2}{2.Math..Math.L_{\mathrm{MAh}}{L}_{\mathrm{CM}}}\right)& \left(74\right)\end{array}$