Speech processing apparatus, method, and program

Abstract

The speech processing apparatus 100 includes an air microphone speech recognition unit 101 which recognizes speech from an air microphone 200 acquiring speech through air, a wearable microphone speech recognition unit 102 which recognizes speech from a wearable microphone 300, a sensing unit 103 which measures environmental conditions, a weight decision unit 104 which calculates the weights for recognition results of the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102 on the basis of the environmental conditions, and a combination unit 105 which combines the recognition results outputted from the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102, using the weights.

Claims

1. A speech processing apparatus comprising: an air microphone speech recognition unit which recognizes speech from an air microphone acquiring speech through air, a wearable microphone speech recognition unit which recognizes speech from a wearable microphone, a sensing unit which measures environmental conditions including at least a distance between the air microphone and a speaker, a weight decision unit which calculates weights for recognition results of the air microphone speech recognition unit and the wearable microphone speech recognition unit on the basis of the environmental conditions, and a combination unit which combines the recognition results outputted from the air microphone speech recognition unit and the wearable microphone speech recognition unit, using the weights, wherein the weight decision unit determines the weight for the recognition results of the wearable microphone speech recognition unit, in accordance with the distance between the air microphone and the speaker, and wherein the weight for the recognition results of the air microphone speech recognition unit is a monotonically non-decreasing function, with respect to the distance between the air microphone and the speaker.

2. The speech processing apparatus according to claim 1, wherein the air microphone speech recognition unit and the wearable microphone speech recognition unit perform either of speech recognition, speaker recognition, language recognition, dialect recognition, emotion recognition, gender recognition, age recognition.

3. A speech processing method comprising: recognizing speech from an air microphone acquiring speech through air, recognizing speech from a wearable microphone, measuring environmental conditions including at least a distance between the air microphone and a speaker, calculating weights for results of recognizing the speech from the air microphone and from the wearable microphone on the basis of the environmental conditions, and combining the results of recognizing the speech from the air microphone and from the wearable microphone, using the weights, wherein the weight for the results of recognizing the speech from the wearable microphone is determined in accordance with the distance between the air microphone and the speaker, and wherein the weight for the results of recognizing the speech from the wearable microphone is a monotonically non-decreasing function, with respect to the distance between the air microphone and the speaker.

4. A non-transitory computer readable information recording medium storing a speech processing program, when executed by a processor, executes: recognizing speech from an air microphone acquiring speech through air, recognizing speech from a wearable microphone, measuring environmental conditions including at least a distance between the air microphone and a speaker, calculating weights for results of recognizing the speech from the air microphone and from the wearable microphone on the basis of the environmental conditions, and combining the results of recognizing the speech from the air microphone and from the wearable microphone, using the weights, w herein the weight for the results of recognizing the speech from the wearable microphone is determined in accordance with the distance between the air microphone and the speaker, and wherein the weight for the results of recognizing the speech from the wearable microphone is a monotonically non-decreasing function, with respect to the distance between the air microphone and the speaker.

5. The speech processing apparatus according to claim 1, wherein the air microphone speech recognition unit and the wearable microphone speech recognition unit perform either of speech recognition, speaker recognition, language recognition, dialect recognition, emotion recognition, gender recognition, age recognition.

6. The speech processing apparatus according to claim 1, wherein each of the air microphone speech recognition unit and the wearable microphone speech recognition unit calculates a score for predetermined potential speech, potential emotion, potential words, potential dialect words, potential words for each age, or potential words for each gender, and the combination unit combines scores calculated by the air microphone speech recognition unit and the wearable microphone speech recognition unit, using the weights.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a block diagram of a speech processing apparatus of the first example embodiment.

(2) FIG. 2 is a flowchart illustrating an operation of the speech processing apparatus of the first example embodiment.

(3) FIG. 3 is a block diagram illustrating a configuration example of an information processing system capable of implementing the functions of the speech processing apparatus.

(4) FIG. 4 is a block diagram illustrating main parts of the speech processing apparatus.

DESCRIPTION OF EMBODIMENTS

(5) One example embodiment of the present invention will be described below with reference to the figures. The following detailed descriptions are merely exemplary in nature and are not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background of the invention or the following detailed description. In addition, the drawings together with the detailed description, serve to explain the principles for the inventive method. The drawings are for illustration and do not limit the application of the technique.

First Example Embodiment

(6) A speech processing apparatus of the first example embodiment can provide high performance of speech processing in severe conditions of noise or far distance between the speaker and the air microphone.

(7) <Configuration of the Speech Processing Apparatus>

(8) FIG. 1 illustrates a block diagram of a speech processing apparatus 100 of the first embodiment. The speech processing apparatus 100 includes an air microphone speech recognition unit (recognition unit using regular air microphone) 101, a wearable microphone speech recognition unit 102, a sensing unit 103, a weight decision unit 104, and a combination unit 105.

(9) The air microphone speech recognition unit 101 applies speech processing for recognition using the speech data from an air microphone 200. In the case of speaker recognition which is to predict who is speaking, the air microphone speech recognition unit 101 uses the speech data from the air microphone 200 and calculates scores for the potential speakers. Log likelihood is often used as the score here. In case of speech recognition which is to predict the speech contents, the air microphone speech recognition unit 101 uses the speech from the air microphone 200 and calculates the posteriors of linguistic units, e.g. phonemes, syllables, words, as the scores. Any kind of speech processing is possible: speech recognition, emotion recognition, language recognition, dialect recognition, age recognition, gender recognition, etc. Details of all recognition are well-known for a person skilled in the art.

(10) The wearable microphone speech recognition unit 102 applies speech processing for recognition using the speech data from a wearable microphone 300. An example of the wearable microphone 300 is an in-ear microphone. In the case of speaker recognition, the wearable microphone speech recognition unit 102 uses the speech data from the wearable microphone 300 and calculates scores for the potential speakers. In case of speech recognition, the wearable microphone speech recognition unit 102 uses the speech data from the wearable microphone 300 and calculates the posteriors of linguistic units as the scores.

(11) The sensing unit 103 measures the environmental conditions, including signal-noise-ratio (SNR) and distance between the air microphone 200 and the speaker. For SNR measurement, the sensing unit 103 first detects speech and non-speech segments in the air microphone speech, and then calculates the ratio between average amplitudes of speech and non-speech segments. As for distance measurement, plural ways can be used here. For example, in-door positioning sensor, which is embedded together with the wearable microphone 300 and not shown in the figure, acquires the position of the speaker and calculates the distance between it and the known position of the air microphone 200. Another way is speaker distance detection using a single microphone as is disclosed in NPL 3.

(12) The weight decision unit 104 decides weights according to the sensing result by the sensing unit 103. The weights here refer to the weights of using the results outputted from the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102. For example, the weights w.sub.a and w.sub.s is calculated using following equations (1) and (2).
w.sub.a=a+c*SNR  (1)
w.sub.s=b+d*distance  (2)

(13) Here, “distance” is a distance between the air microphone 200 and the speaker measured by the sensing unit 103. w.sub.a is a weight for each score s.sub.a from the air microphone speech recognition unit 101. w.sub.s is a weight for each score s.sub.s from the wearable microphone speech recognition unit 102. a, b, c and d are constants which can be determined experimentally. c and d are positive.

(14) The example shown here is based on 1.sup.st-order polynomial functions in calculating the weights w.sub.a and w.sub.s. However, it is of course possible to use any other functions that are monotonically non-decreasing with respect to SNR and distance.

(15) Another example of such functions is a step function that can take only two values of 0 or 1. In that case, either of the air microphone speech recognition unit 101 or the wearable microphone speech recognition unit 104 is exclusively chosen in accordance with SNR and distance.

(16) The combination unit 105 combines the scores from the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102, using the weights outputted from the weight decision unit 104, and output the final recognition result. In the case of speaker recognition, for each potential speakers, the combination unit 105 calculates a combined score s using following equation (3), for example.
s=(w.sub.as.sub.a+w.sub.ss.sub.s)/(w.sub.a+w.sub.s)  (3)

(17) By comparing the scores of all potential speakers, the combination unit 105 outputs information (data) indicating the speaker who has the highest score as the recognition result. In the case of speech recognition, Recognizer Output Voting Error Reduction (ROVER) in NPL4 can be used in the combination unit 105.

(18) <Operation of the Speech Processing Apparatus>

(19) Next, the operation of the speech processing apparatus 100 will be described with reference to FIG. 2.

(20) In step S11, the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102 receive speech data.

(21) In step S12, the air microphone speech recognition unit 101 estimates recognition results by using the speech data from the air microphone 200. Namely, the air microphone speech recognition unit 101 calculates score s.sub.a, for example. In the case of speaker recognition, the air microphone speech recognition unit 101 calculates scores for the potential speakers. In the case of speech recognition, emotion recognition, language recognition, dialect recognition, age recognition, or gender recognition, the air microphone speech recognition unit 101 calculates scores for predetermined potential speech, potential emotion, potential words, potential dialect words, potential words for each age, or potential words for each gender.

(22) In step S13, the wearable microphone speech recognition unit 102 estimates recognition results for example, by using the speech data from the wearable microphone 300 in the same way as the air microphone speech recognition unit 101. Namely, the wearable microphone speech recognition unit 102 calculates score s.sub.s, for example.

(23) In step S14, the sensing unit 103 measures environmental conditions including SNR and distance between the air microphone and the speaker.

(24) In step S15, the weight decision unit 104 decides the weights for recognition results from the air microphone speech recognition unit 101 and the wearable microphone speech recognition unit 102.

(25) Note that the order of steps A02-A03 and steps A04-A05 can be switched, not limited to the form presented in FIG. 2. Step S12 and step S13 can be also switched.

(26) In step S16, the combination unit 105 combines the recognition results outputted from the air microphone recognition speech unit 101 and the wearable microphone speech recognition unit 102, while reflecting the weights outputted from the weight decision unit 104.

(27) As explained above, the first example embodiment can improve performance of speech processing for recognition in severe conditions. It takes the advantages of both the air microphone 200 that picks up full-band speech and the wearable microphone 300 that is robust to surrounding noise. In other words, the speech processing apparatus, speech processing method, and program of the above example embodiment can provide better performance for speech processing in severe conditions by taking advantage of each type of microphones that the air microphone 200 picks up full-band speech and the wearable microphone 300 is robust to surrounding noise.

(28) As a final point, it should be clear that the process, techniques and methodology described and illustrated here are not limited or related to a particular apparatus. It can be implemented using a combination of components. Also various types of general purpose devise may be used in accordance with the instructions herein. The present invention has also been described using a particular set of examples. However, these are merely illustrative and not restrictive. For example the described software may be implemented in a wide variety of languages such as C/C++, Java, MATLAB and Python etc. Moreover other implementations of the inventive technology will be apparent to those skilled in the art.

(29) The aforementioned example embodiment can be configured in hardware, but it is also possible to implement an example embodiment by a computer program.

(30) An information processing system shown in FIG. 3 includes one or more processors such as CPUs (Central Processing Unit) 1000, an I/O (Input/Output) device 1001 and a memory 1002. A magnetic storage medium such as a hard disk, or a semiconductor memory such as a ROM (read only memory) or a flash memory can be used as the memory 1002. The I/O device 1001 can realize functions of an output unit for inputting speech data and an input unit for outputting recognition results.

(31) The memory 1002 stores a program for realizing the function of each unit in the speech processing apparatus 100 shown in FIG. 1. In the memory 1002, an area in which at least a program is stored is a non-transitory tangible storage area (non-transitory tangible media). In addition, the memory 1002 may realize a temporary information storage unit. The CPU 1000 executes processes according to the program stored in the memory 1002 to realize functions of the air microphone speech recognition unit 101, the wearable microphone speech recognition unit 102, the sensing unit 103, the weight decision unit 104, and the combination unit 105 shown in FIG. 1.

(32) FIG. 4 is a block diagram showing main parts of a speech processing apparatus according to the present invention. As shown in FIG. 4, the speech processing apparatus 10 includes air microphone speech recognition means 11 (realized by the air microphone speech recognition unit 101 in the example embodiment) for recognizing speech from an air microphone acquiring speech through air, wearable microphone speech recognition means 12 (realized by the wearable microphone speech recognition unit 102 in the example embodiment) for recognizing speech from a wearable microphone, sensing means13 (realized by the sensing unit 103 in the example embodiment) for measuring environmental conditions including at least a distance between the air microphone and the speaker, weight decision means 14 (realized by the weight decision unit 104 in the example embodiment) for calculating the weights for recognition results of the air microphone speech recognition means 11 and the wearable microphone speech recognition means 12 on the basis of the environmental conditions, and combination means 15 (realized by the combination unit 105 in the example embodiment) for combining the recognition results outputted from the air microphone speech recognition means 11 and the wearable microphone speech recognition means 12, using the weights.

(33) The foregoing exemplary embodiments may be partly or wholly described in the following supplementary notes, though the structure of the present invention is not limited to such.

(34) (Supplementary note 1) A speech processing apparatus comprising:

(35) air microphone speech recognition means for recognizing speech from an air microphone acquiring speech through air,

(36) wearable microphone speech recognition means for recognizing speech from a wearable microphone,

(37) sensing means for measuring environmental conditions including SNR and a distance between the air microphone and the speaker,

(38) weight decision means for calculating the weights for recognition results of the air microphone speech recognition means and the wearable microphone speech recognition means on the basis of the environmental conditions, and

(39) combination means for combining the recognition results outputted from the air microphone speech recognition means and the wearable microphone speech recognition means, using the weights.

(40) (Supplementary note 2) The speech processing apparatus according to Supplementary note 1,

(41) wherein the weight decision means determines the weight for the recognition results of the wearable microphone speech recognition means, in accordance with the distance between the air microphone and the speaker.

(42) (Supplementary note 3) The speech processing apparatus according to Supplementary note 2,

(43) wherein the weight for the recognition results of the air microphone speech recognition means is monotonically non-decreasing functions, with respect to the distance between the air microphone and the speaker.

(44) (Supplementary note 4) The speech processing apparatus according to any one of Supplementary notes 1 to 3,

(45) wherein the air microphone speech recognition means and the wearable microphone speech recognition means perform either of speech recognition, speaker recognition, language recognition, dialect recognition, emotion recognition, gender recognition, age recognition.

(46) (Supplementary note 5) A speech processing method comprising:

(47) recognizing speech from an air microphone acquiring speech through air,

(48) recognizing speech from a wearable microphone,

(49) measuring environmental conditions including SNR and a distance between the air microphone and the speaker,

(50) calculating the weights for results of recognizing the speech from the air microphone and from the wearable microphone on the basis of the environmental conditions, and

(51) combining the results of recognizing the speech from the air microphone and from the wearable microphone, using the weights.

(52) (Supplementary note 6) The speech processing method according to Supplementary note 5,

(53) wherein the weight for the results of recognizing the speech from the wearable microphone is determined in accordance with the distance between the air microphone and the speaker.

(54) (Supplementary note 7) The speech processing method according to Supplementary note 6,

(55) wherein the weight for the results of recognizing the speech from the air microphone is monotonically non-decreasing functions, with respect to the distance between the air microphone and the speaker.

(56) (Supplementary note 8) A speech processing program for causing a computer to execute:

(57) recognizing speech from an air microphone acquiring speech through air,

(58) recognizing speech from a wearable microphone,

(59) measuring environmental conditions including SNR and a distance between the air microphone and the speaker,

(60) calculating the weights for results of recognizing the speech from the air microphone and from the wearable microphone on the basis of the environmental conditions, and

(61) combining the results of recognizing the speech from the air microphone and from the wearable microphone, using the weights.

(62) (Supplementary note 9) The speech processing program according to Supplementary note 8,

(63) wherein the weight for the results of recognizing the speech from the wearable microphone is determined in accordance with the distance between the air microphone and the speaker.

(64) (Supplementary note 10) The speech processing program according to Supplementary note 9,

(65) wherein the weight for the results of recognizing the speech from the air microphone is monotonically non-decreasing functions, with respect to the distance between the air microphone and the speaker.

(66) (Supplementary note 11) A non-transitory computer readable information recording medium storing a speech processing program, when executed by a processor, performs,

(67) recognizing speech from an air microphone acquiring speech through air,

(68) recognizing speech from a wearable microphone,

(69) measuring environmental conditions including at least a distance between the air microphone and the speaker,

(70) calculating the weights for results of recognizing the speech from the air microphone and from the wearable microphone on the basis of the environmental conditions, and

(71) combining the results of recognizing the speech from the air microphone and from the wearable microphone, using the weights.

(72) (Supplementary note 12) The computer readable information recording medium according to Supplementary note 11,

(73) wherein the weight for the results of recognizing the speech from the wearable microphone is determined in accordance with the distance between the air microphone and the speaker.

(74) (Supplementary note 13) The computer readable information recording medium according to Supplementary note 12,

(75) wherein the weight for the results of recognizing the speech from the air microphone is monotonically non-decreasing functions, with respect to the distance between the air microphone and the speaker.

(76) While the present invention has been described with reference to the exemplary embodiments and examples, the present invention is not limited to these exemplary embodiments and examples. Various changes understandable by those skilled in the art can be made to the structures and details of the present invention within the scope of the present invention.

REFERENCE SIGNS LIST

(77) 10 speech processing apparatus 11 air microphone speech recognition means 12 wearable microphone speech recognition means 13 sensing mean 14 weight decision means 15 combination means 100 speech processing apparatus 101 air microphone speech recognition unit 102 wearable microphone speech recognition unit 103 sensing unit 104 weight decision unit 105 combination unit 200 air microphone 300 wearable microphone 1000 CPU 1001 I/O device 1002 memory