TAG ESTIMATION DEVICE, TAG ESTIMATION METHOD, AND PROGRAM

Abstract

A tag estimation device capable of estimating, for an utterance made among several persons, a tag representing a result of analyzing the utterance is provided. The tag estimation device includes an utterance sequence information vector generation unit that adds a t-th utterance word feature vector and a t-th speaker vector to a (t−1)-th utterance sequence information vector u.sub.t-1 that includes an utterance word feature vector that precedes the t-th utterance word feature vector and a speaker vector that precedes the t-th speaker vector to generate a t-th utterance sequence information vector u.sub.t, where t is a natural number, and a tagging unit that determines a tag l.sub.t that represents a result of analyzing a t-th utterance from a model parameter set in advance and the t-th utterance sequence information vector u.sub.t.

Claims

1. A tag estimation device comprising: a hardware processor that: adds a t-th utterance word feature vector and a t-th speaker vector to a (t−1)-th utterance sequence information vector u.sub.t-1 that includes an utterance word feature vector that precedes the t-th utterance word feature vector and a speaker vector that precedes the t-th speaker vector to generate a t-th utterance sequence information vector u.sub.t, where t is a natural number; and determines a tag l.sub.t that represents a result of analyzing a t-th utterance from a model parameter set in advance and the t-th utterance sequence information vector u.sub.t.

2. The tag estimation device according to claim 1, comprising: the hardware processor that: transforms a word sequence of the t-th utterance into the utterance word feature vector of the t-th utterance; and a speaker vector transform unit that transforms a speaker label of the t-th utterance into the speaker vector of the t-th utterance.

3. The tag estimation device according to claim 1, comprising: the hardware processor that: transforms a word sequence of the t-th utterance into the utterance word feature vector of the t-th utterance; and transforms voice of the t-th utterance into the speaker vector of the t-th utterance.

4. The tag estimation device according to claim 1, comprising: the hardware processor that: transforms voice of the t-th utterance into the utterance word feature vector of the t-th utterance; and transforms a speaker label of the t-th utterance into the speaker vector of the t-th utterance.

5. The tag estimation device according to claim 1, comprising: the hardware processor that: transforms voice of the t-th utterance into the utterance word feature vector of the t-th utterance; and transforms the voice of the t-th utterance into the speaker vector of the t-th utterance.

6. The tag estimation device according to claim 1, wherein the tag includes at least any one of an utterance scene, an utterance type, an emotion of an utterance utterer, and utterance paralinguistic information.

7. The tag estimation device according to claim 1, wherein the hardware processor uses an estimation model that is learned on a basis of teacher data including a set of an utterance sequence and a sequence of labels that are right tags corresponding to the utterance sequence.

8. A tag estimation method to be executed by a tag estimation device, comprising: an utterance sequence information vector generation step of adding a t-th utterance word feature vector and a t-th speaker vector to a (t−1)-th utterance sequence information vector u.sub.t-1 that includes an utterance word feature vector that precedes the t-th utterance word feature vector and a speaker vector that precedes the t-th speaker vector to generate a t-th utterance sequence information vector u.sub.t, where t is a natural number; and a tagging step of determining a tag l.sub.t that represents a result of analyzing a t-th utterance from a model parameter set in advance and the t-th utterance sequence information vector u.sub.t.

9. A non-transitory computer-readable storage medium storing a program for causing a computer to function as a tag estimation device according to claim 1.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0012] FIG. 1 is a block diagram showing a configuration of a tag estimation device according to a first embodiment.

[0013] FIG. 2 is a flowchart showing operation of the tag estimation device according to the first embodiment.

[0014] FIG. 3 is a block diagram showing a configuration of a model parameter learning device according to the first embodiment.

[0015] FIG. 4 is a flowchart showing operation of the model parameter learning device according to the first embodiment.

[0016] FIG. 5 is a block diagram showing a configuration of a tag estimation device according to a second embodiment.

[0017] FIG. 6 is a flowchart showing operation of the tag estimation device according to the second embodiment.

[0018] FIG. 7 is a block diagram showing a configuration of a tag estimation device according to a third embodiment.

[0019] FIG. 8 is a flowchart showing operation of the tag estimation device according to the third embodiment.

[0020] FIG. 9 is a block diagram showing a configuration of a tag estimation device according to a fourth embodiment.

DESCRIPTION OF EMBODIMENTS

[0021] Embodiments of the present invention will be described below in detail. Note that constitutional units having the same capabilities are denoted by the same reference numerals and that a redundant description thereof will be omitted.

First Embodiment

[0022] A configuration of a tag estimation device according to a first embodiment will be described below with reference to FIG. 1. As shown in FIG. 1, a tag estimation device 11 includes an utterance word feature vector transform unit 111, a speaker vector transform unit 112, an utterance sequence information vector generation unit 113, a vector storage unit 113A, and a tagging unit 114.

[0023] Note that a tag is any one among a group of tags including two or more tags determined in advance. For example, at the time of tagging a scene at a call center, a group of tags including five tags of “opening,” “requirement grasping,” “requirement addressing,” “personal information grasping,” and “closing” can be set. At the time of tagging a meeting scene, a group of tags including four tags of “opening,” “subject raising,” “debate,” and “closing” can be set. In this case, a problem with tagging can be set as the problem of determining any one tag for each utterance.

[0024] Operation of the tag estimation device 11 according to the present embodiment will be described below with reference to FIG. 2. The flowchart in FIG. 2 shows a process on a t-th utterance in an utterance sequence. Assume that t is an arbitrary natural number. Assume that the utterance sequence includes two or more utterances and that the utterance sequence is in chronological order.

[0025] [Utterance Word Feature Vector Transform Unit 111]

Input: a word sequence of the t-th utterance
Output: an utterance word feature vector w.sub.t of the t-th utterance
Operation: The utterance word feature vector transform unit 111 transforms the word sequence of the t-th utterance into the utterance word feature vector w.sub.t of the t-th utterance (S111).

[0026] A word sequence of an utterance may be obtained by transforming the utterance into text using voice recognition or may be obtained by manually transcribing the utterance. Note that, if the utterance is not divided into words, the utterance may be transformed into a word sequence using, e.g., morphological analysis.

[0027] The utterance word feature vector transform unit 111 can express an utterance word feature vector in an arbitrary manner. For example, a word feature vector called Bag-of-Words or the like can be used. Note that since Bag-of-Words is a publicly known technique, a description thereof will be omitted. The utterance word feature vector transform unit 111 can also use a recurrent neural network in order to use an utterance word feature vector holding the order of words. A recurrent neural network has the capability of transforming a variable-length symbol sequence or vector sequence into a single vector. Recurrent neural networks include variants, such as a GRU and an LSTM, and one having the same capabilities can be used. Note that, if a recurrent neural network is used, the utterance word feature vector transform unit 111 has a model parameter obtained through learning. A recurrent neural network is a publicly known technique, and a description thereof will be omitted.

[0028] [Speaker Vector Transform Unit 112]

Input: a speaker label of the t-th utterance
Output: a speaker vector r.sub.t of the t-th utterance
Operation: The speaker vector transform unit 112 transforms the speaker label of the t-th utterance into the speaker vector r.sub.t of the t-th utterance (S112).

[0029] The speaker vector transform unit 112 can use an arbitrary method as long as the method vectorizes speaker information. For example, the speaker vector transform unit 112 can use a one-hot vector in which only a dimension corresponding to a given speaker label is set at 1, and the other dimensions are set at 0. The speaker vector transform unit 112 may adopt a vector obtained by performing a linear transform or a non-linear transform on a one-hot vector as a speaker vector. Note that, if a linear transform or a non-linear transform is performed, the speaker vector transform unit 112 has a model parameter obtained through learning.

[0030] [Utterance Sequence Information Vector Generation Unit 113]

Input: the utterance word feature vector w.sub.t of the t-th utterance, the speaker vector r.sub.t of the t-th utterance, and an utterance sequence information vector u.sub.t-1 in which utterance word feature vectors preceding the t-th utterance word feature vector and speaker vectors preceding the t-th speaker vector are embedded Output: an utterance sequence information vector u.sub.t in which first to t-th utterance word feature vectors and first to t-th speaker vectors are embedded
Operation: The utterance sequence information vector generation unit 113 adds the utterance word feature vector w.sub.t of the t-th utterance and the speaker vector r.sub.t of the t-th utterance to the (t−1)-th utterance sequence information vector u.sub.t-1 including utterance word feature vectors w.sub.1, . . . , w.sub.t-1 preceding the t-th utterance word feature vector and speaker vectors r.sub.1, . . . , r.sub.t-1 preceding the t-th speaker vector to generate the t-th utterance sequence information vector u.sub.t (S113).

[0031] That is, the utterance sequence information vector generation unit 113 recursively embeds pieces of utterance information up to the present. The utterance sequence information vector generation unit 113 first constructs a combined vector of a current utterance word feature vector and a speaker vector of a current utterance. A combined vector c.sub.t of the utterance word feature vector w.sub.t of the t-th utterance and the speaker vector r.sub.t of the t-th utterance is represented by the formula below.

c.sub.t=[w.sub.t.sup.T,r.sub.t.sup.T].sup.T [Formula 1]

[0032] In Formula 1, T is a symbol representing transposition of a vector. The combined vector c.sub.t and the utterance sequence information vector u.sub.t-1, in which the preceding utterance word feature vectors and the preceding speaker vectors are embedded, are handled in a recurrent neural net. The use of the recurrent neural network makes it possible to construct the vector u.sub.t, in which the utterance word feature vectors up to the present and the speaker vectors up to the present are embedded. That is, u.sub.t is constructed in accordance with the formula below.

u.sub.t=RNN(c.sub.t,u.sub.t-1) [Formula 2]

[0033] In Formula 2, RNN( ) is a function having capabilities of a recurrent neural network and includes a model parameter obtained through learning. Since a recurrent neural network is a publicly known technique, as described earlier, a description thereof will be omitted.

[0034] [Tagging Unit 114]

Input: the utterance sequence information vector u.sub.t, in which the first to t-th utterance word feature vectors and the first to t-th speaker vectors are embedded
Output: a tag l.sub.t of the t-th utterance
Operation: The tagging unit 114 determines the tag l.sub.t representing a result of analyzing the t-th utterance from a model parameter set in advance and the t-th utterance sequence information vector u.sub.t (S114). A tag may be a tag for identification of a scene for a conversation or a tag for identification of an utterance type, emotions of an utterer, paralinguistic information, and the like. The tagging unit 114 may select and determine multiple types of ones from among the different types of tags.

[0035] The tagging unit 114 first generates a posterior probability vector o.sub.t corresponding to each label through a transform using a softmax function. That is, o.sub.t follows the formula below.

o.sub.t=SOFTMAX(u.sub.t) [Formula 3]

[0036] In Formula 3, SOFTMAX( ) is a transform function using a softmax function and includes a model parameter obtained through learning. A softmax function is a publicly known technique. Each element of o.sub.t represents a probability of each tag, and a value o.sub.tk for a k-th dimension represents a probability of a k-th tag. Note that o.sub.t is a vector having two or more elements. Finally, the tag l.sub.t of the current utterance can be determined in accordance with the formula below.

[00001] $\begin{matrix} l_{t} = \underset{k}{\arg \max} o_{tk} & [Formula 4] \end{matrix}$

[0037] That is, the tagging unit 114 determines a tag corresponding to an element with a maximum probability.

[0038] <Model Parameter Learning Device 12>

[0039] A configuration of a model parameter learning device according to the first embodiment will be described below with reference to FIG. 3. As shown in FIG. 3, a model parameter learning device 12 includes a model parameter estimation unit 121.

[0040] [Model Parameter Estimation Unit 121]

Input: N (≥1) sets of utterance sequences (sequences as word sequences and speaker label sequences) and sequences of labels which are right tags corresponding to the utterance sequences
Output: a model parameter θ Operation: The model parameter estimation unit 121 determines a model parameter of the tagging unit 114 from the N (≥1) sets of utterance sequences (sequences as word sequences and speaker label sequences) and label sequences (S121). Note that, if the utterance word feature vector transform unit 111, the speaker vector transform unit 112, and the utterance sequence information vector generation unit 113 each have a model parameter, the model parameters are also determined through learning in step S121.

[0041] Although detailed model parameters differ depending on the configurations of the utterance word feature vector transform unit 111, the speaker vector transform unit 112, the utterance sequence information vector generation unit 113, and the tagging unit 114, all the model parameters are denoted here by θ. Specifically, the model parameters are model parameters which recurrent neural networks of the utterance word feature vector transform unit 111 and the utterance sequence information vector generation unit 113 have, a model parameter used for a linear transform or a non-linear transform in the speaker vector transform unit 112, a model parameter which the softmax function of the tagging unit 114 has, and the like. Letting here a t-th utterance (a word sequence and a speaker label) of n-th voice data be U.sub.t.sub.n; and a t-th label, l.sub.t.sub.n, all the model parameters can be estimated in accordance with the learning criterion below.

[00002] $\begin{matrix} \hat{θ} = \underset{θ}{argmax} {.Math.}_{n = 1}^{N} {.Math.}_{t = 1}^{T_{n}} P (l_{t}^{n} | U_{t}^{n}, θ) & [Formula 5] \end{matrix}$

[0042] In Formula 5, θ{circumflex over ( )} is an estimated model parameter, T.sub.n represents the number of utterances of the n-th voice data, and P(l.sub.t.sub.n|U.sub.t.sub.n, θ) represents a probability value of l.sub.t.sub.n obtained in accordance with θ in a case where U.sub.tn is given, and corresponds to a value of a corresponding element of o.sub.t described earlier if n is omitted. That is, the model parameter estimation unit 121 estimates the model parameter so as to achieve the best behavior for given data (such that a prediction probability is the highest).

Second Embodiment

[0043] A configuration of a tag estimation device according to a second embodiment using a speaker vector construction unit which directly constructs a vector representing information on a speaker from voice instead of the speaker vector transform unit described earlier will be described below with reference to FIG. 5. As shown in FIG. 5, a tag estimation device 21 includes an utterance word feature vector transform unit 111, a speaker vector construction unit 212, an utterance sequence information vector generation unit 113, a vector storage unit 113A, and a tagging unit 114 and is the same in configuration as in the first embodiment except for the speaker vector construction unit 212. The tag estimation device 21 according to the present embodiment can estimate a tag, as in the first embodiment, even if a speaker label is not obtained, and only voice thereof is obtained.

[0044] Operation of the speaker vector construction unit 212 different from the first embodiment will be described with reference to FIG. 6.

[0045] [Speaker Vector Construction Unit 212]

Input: voice of a t-th utterance
Output: a speaker vector r.sub.t of the t-th utterance
Operation: The speaker vector construction unit 212 transforms the voice of the t-th utterance into the speaker vector r.sub.t of the t-th utterance (S212). The speaker vector construction unit 212 can use an arbitrary method (e.g., an end-to-end model) as long as the method extracts a vector representing speaker information from voice. The speaker vector construction unit 212 can use, for example, a method for extracting a vector for use in speaker identification, such as an i-vector. Since an i-vector is publicly known, a description thereof will be omitted.

[0046] <Model Parameter Learning Device 12>

[Model Parameter Estimation Unit 121]

[0047] In the present embodiment, input to and operation of a model parameter estimation unit 121 of a model parameter learning device 12 are changed in the manner below.

Input: N sets of utterance sequences (sequences as word sequences and voice sequences) and label sequences
Operation: The model parameter estimation unit 121 determines a model parameter of the tagging unit 114 (and optionally model parameters of the utterance word feature vector transform unit 111, the speaker vector construction unit 212, and the utterance sequence information vector generation unit 113) from the N sets of utterance sequences (sequences as word sequences and voice sequences) and label sequences.

Third Embodiment

[0048] A configuration of a tag estimation device according to a third embodiment using an utterance content vector construction unit which directly constructs a vector representing utterance content from voice instead of the utterance word feature vector transform unit described earlier will be described below with reference to FIG. 7. As shown in FIG. 7, a tag estimation device 31 includes an utterance content vector construction unit 311, a speaker vector transform unit 112, an utterance sequence information vector generation unit 113, a vector storage unit 113A, and a tagging unit 114 and is the same in configuration as in the first embodiment except for the utterance content vector construction unit 311. The tag estimation device 31 according to the present embodiment can estimate a tag, as in the first embodiment, even if a word sequence of an utterance is not obtained, and only voice thereof is obtained.

[0049] Operation of the utterance content vector construction unit 311 different from the first embodiment will be described with reference to FIG. 8.

[0050] [Utterance Content Vector Construction Unit 311]

Input: voice of a t-th utterance
Output: an utterance word feature vector w.sub.t of the t-th utterance
Operation: The utterance content vector construction unit 311 transforms the voice of the t-th utterance into the utterance word feature vector w.sub.t of the t-th utterance (S311). The utterance content vector construction unit 311 can use an arbitrary method as long as the method extracts a vector representing utterance content from voice. The utterance content vector construction unit 311 can use a method for extracting a vector for use in voice recognition, such as a mean vector of mel-cepstral coefficients. Since a mel-cepstral coefficient is publicly known, a description thereof will be omitted.

[0051] <Model Parameter Learning Device 12>

[0052] [Model Parameter Estimation Unit 121]

[0053] In the present embodiment, input to and operation of a model parameter estimation unit 121 of a model parameter learning device 12 are changed in the manner below.

Input: N sets of utterance sequences (voice sequences and speaker label sequences) and label sequences
Operation: The model parameter estimation unit 121 determines a model parameter of the tagging unit 114 (and optionally model parameters of the utterance content vector construction unit 311, the speaker vector transform unit 112, and the utterance sequence information vector generation unit 113) from the N sets of utterance sequences (voice sequences and speaker label sequences) and label sequences.

Fourth Embodiment

[0054] A configuration of a tag estimation device according to a fourth embodiment simultaneously using the speaker vector construction unit 212 according to the second embodiment that is a substitute for the speaker vector transform unit 112 described earlier, and the utterance content vector construction unit 311 according to the third embodiment that is a substitute for the utterance word feature vector transform unit 111 described earlier will be described below with reference to FIG. 9. As shown in FIG. 9, a tag estimation device 41 includes an utterance content vector construction unit 311, a speaker vector construction unit 212, an utterance sequence information vector generation unit 113, a vector storage unit 113A, and a tagging unit 114.

[0055] Since the components of the tag estimation device 41 according to the present embodiment operate in the same manner as those denoted by the same reference numerals of the tag estimation devices 11 to 31 according to the first to third embodiments, a description of operation of the components will be omitted.

[0056] <Model Parameter Learning Device 12>

[0057] [Model Parameter Estimation Unit 121]

[0058] In the present embodiment, input to and operation of a model parameter estimation unit 121 of a model parameter learning device 12 are changed in the manner below.

Input: N sets of utterance sequences (voice sequences) and label sequences
Operation: The model parameter estimation unit 121 determines a model parameter of the tagging unit 114 (and optionally model parameters of the utterance content vector construction unit 311, the speaker vector construction unit 212, and the utterance sequence information vector generation unit 113) from the N sets of utterance sequences (voice sequences) and label sequences.

[0059] A tag estimation device and a tag estimation method according to the embodiments are capable of implementing tagging with interaction information of a multiple persons conversation in mind. Specifically, at the time of estimation of a tag for a current utterance, tagging can be implemented with word information and speaker information of the current utterance and word information and speaker information of an utterance prior to the present in mind. This makes it possible to make tagging performance higher than in conventional techniques.

[0060] <Supplement>

[0061] A device according to the present invention as a single hardware entity has, for example, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a communication unit to which a communication device (e.g., a communication cable) capable of communicating with an outside of the hardware entity can be connected, a CPU (which stands for Central Processing Unit and may include a cache memory, a register, and the like), a RAM and a ROM which are memories, an external storage device which is a hard disk, and a bus which makes connections so as to allow data exchange among the input unit, the output unit, the communication unit, the CPU, the RAM, the ROM, and the external storage device. A device (drive) capable of reading from and writing to a recording medium, such as a CD-ROM, or the like may be provided in the hardware entity as needed. Physical substances provided with such hardware resources include a general-purpose computer.

[0062] A program required to implement the above-described capabilities, data required in processing of the program, and the like are stored in the external storage device for the hardware entity (the program may be stored in not only the external storage device but also, for example, a ROM which is a read-only storage device). Data and the like obtained through the processing of the program are appropriately stored in the RAM, the external storage device, and the like.

[0063] In the hardware entity, each program stored in the external storage device (or the ROM or the like) and data required in processing of the program are loaded into a memory as needed and are appropriately interpreted, executed, and processed by the CPU. As a result, the CPU implements a predetermined capability (each component represented above as a “*** unit,” “*** means,” or the like).

[0064] The present invention is not limited to the above-described embodiments and can be appropriately changed without departing from the spirit of the present invention. The processes described in the embodiments are not always executed in chronological order in order of description and may be executed in parallel or individually in accordance with processing performance of a device which is to execute processing or as needed.

[0065] As described above, if processing capabilities in a hardware entity (a device according to the present invention) described in the embodiments are implemented by a computer, processing details of capabilities which the hardware entity needs to have are described in a program. With execution of the program by the computer, the processing capabilities in the hardware entity are implemented on the computer.

[0066] The program describing the processing details can be recorded on a computer-readable recording medium. As the computer-readable recording medium, anything, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory, may be adopted. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape, or the like can be used as a magnetic recording device, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R/RW (Recordable/ReWritable), or the like can be used as an optical disc, an MO (Magneto-Optical disc) or the like can be used as a magneto-optical recording medium, and an EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) or the like can be used as a semiconductor memory.

[0067] Distribution of the program is performed by, for example, selling, giving, or renting a portable recording medium, such as a DVD or a CD-ROM, having the program recorded thereon. Additionally, a configuration in which the program is stored in a storage device of a server computer and is distributed through transfer of the program from the server computer to another computer over a network may be adopted.

[0068] For example, a computer which executes a program as described above first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its storage device. At the time of execution of processing, the computer reads the program stored in its recording medium and executes processing in accordance with the read program. As another form of execution of the program, the computer may directly read the program from a portable recording medium and execute processing in accordance with the program. Alternatively, each time a program is transferred from the server computer to the computer, the computer may successively execute processing in accordance with the received program. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) service which does not perform transfer of the program from the server computer to the computer and implements processing capabilities only by giving an instruction for execution and acquiring a result may be adopted. Note that programs according to the present form are assumed to include information made available for processing by an electronic computer which is equivalent to a program (e.g., data which is not a direct instruction to a computer but has the property of prescribing processing of the computer).

[0069] Additionally, in this form, a hardware entity is constructed through execution of a predetermined program on a computer. However, at least a part of the processing details may be implemented by means of hardware.

TAG ESTIMATION DEVICE, TAG ESTIMATION METHOD, AND PROGRAM

Assignee

Inventors

Cpc classification

Classification Explorer

G10L15/02

PHYSICS

Classification Explorer

G06F16/00

PHYSICS

Classification Explorer

G06F40/274

PHYSICS

Classification Explorer

G10L25/51

PHYSICS

Classification Explorer

G06F16/90

PHYSICS

Classification Explorer

G10L25/30

PHYSICS

Classification Explorer

G10L25/48

PHYSICS

Classification Explorer

G06F16/906

PHYSICS

Classification Explorer

G10L15/10

PHYSICS

Classification Explorer

G10L15/26

PHYSICS

Classification Explorer

G06F16/65

PHYSICS

International classification

Classification Explorer

G10L25/30

PHYSICS

Classification Explorer

G10L15/02

PHYSICS

Classification Explorer

G10L15/10

PHYSICS

Classification Explorer

G10L25/48

PHYSICS

Abstract

Claims

Description