System for generating topic inference information of lyrics

Abstract

A system for generating topic inference information of lyrics that can provide more useful for topic interpretation of lyrics. A device for learning topic numbers performs an operation of updating and learning topic numbers, which performs an operation of updating topic numbers on all of a plurality of lyrics data of each of a plurality of artists, for a predetermined number of times. The operation of updating topic numbers updates the topic number assigned to a given lyrics data of a given artist using a random number generator having a deviation of appearance probability corresponding to a probability distribution over topic numbers. An outputting device outputs the topic numbers of the plurality of lyrics data for each of the plurality artists, and a probability distribution over words for each of the topic numbers.

Claims

1. A system for generating topic inference information of lyrics for obtaining reliable information for inferring a topic that is a subject, a main point or a theme of lyrics as determined by lyric contents, the system comprising: a means for obtaining lyrics data, operable to obtain a plurality of lyrics data each including a song name and the lyrics of each of a plurality of artists; a means for generating a given number of one or more topic numbers k (1≤k≤K) where k is a number in a range of 1 to K (a positive integer); an analysis means for extracting a plurality of words by performing morpheme analysis of a plurality of lyrics in the plurality of the lyrics data, using a morpheme analysis engine; a means for learning the one or more topic numbers, operable to first assign a topic number to each of the plurality lyrics data of each of the plurality of artists in a random or arbitrary manner; then to calculate a probability p that the topic number of a given lyrics data S.sub.ar is k, based on a number R.sub.ak of lyrics data other than the lyrics data S.sub.ar of a given artist a, to which the topic number k is assigned and a number N.sub.kv of times that the topic number k is assigned to a word v in the plurality of lyrics data of the plurality of artists except the given lyrics data S.sub.ar; to generate a probability distribution over the one or more topic numbers of the given lyrics data S.sub.ar, based on the calculated probability p; next to perform an operation of updating the one or more topic numbers to update the topic number assigned to the given lyrics data S.sub.ar of the given artist a using a random number generator having a deviation of appearance probability corresponding to the probability distribution over the one or more topic numbers; and to perform an operation of updating and learning the one or more topic numbers for a predetermined number of times, the operation of updating and learning the one or more topic numbers performing the operation of updating the one or more topic numbers on all of the plurality of lyrics data of each of the plurality of artists; a means for learning values of one or more switch variables, operable to assign values of the one or more switch variables to the plurality of words included in the plurality of lyrics data of each of the plurality of artists in a random or arbitrary manner; then to generate a probability distribution λ.sub.a over values of the one or more switch variables by calculating a probability whether the value of a switch variable x assigned to a given word v.sub.arj is a topic word or a background word, based on the values of the one or more switch variables assigned to the plurality of words in the plurality of lyrics data of the given artist a; next to perform an operation of updating values of the one or more switch variables to update the value of the switch variable assigned to the given word v.sub.arj using a random number generator having a deviation of appearance probability corresponding to the probability distribution over values of the one or more switch variables; and to perform an operation of updating and learning values of the one or more switch variables for a predetermined number of times, the operation of updating and learning values of the one or more switch variables performing the operation of updating values of the one or more switch variables on all of the plurality of words included in the plurality of lyrics data of each of the plurality of artists; and an outputting means operable to identify the one or more topic numbers of each of the plurality of lyrics data and the probability distributions over words for each of the one or more topic numbers, based on learning results obtained from the means for learning the one or more topic numbers and learning results obtained by the means for learning values of the one or more switch variables; wherein the means for generating the given number of the one or more topic numbers, the analysis means, the means for learning the one or more topic numbers, the means for learning values of the one or more switch variables and the outputting means are implemented on a computer by a computer program installed in the computer.

2. The system for generating topic inference information of lyrics according to claim 1, wherein: in the means for learning the one or more topic numbers, it is assumed that the one or more topic numbers assigned to all of the plurality of lyrics but the topic number assigned to the given lyrics data of the given artist are correct when generating the probability distribution over the one or more topic numbers.

3. The system for generating topic inference information of lyrics according to claim 1, wherein: in the means for learning values of the one or more switch variables, it is assumed that values of the one or more switch variables assigned to all of words but the value of the switch variable x assigned to a given word in the plurality of words of the given lyrics data of the given artist are correct when performing the operation of updating values of the one or more switch variables.

4. The system for generating topic inference information of lyrics according to claim 1, wherein the means for learning the one or more topic numbers: calculates a first probability p.sub.1 that the topic number of the given lyrics data S.sub.ar is k, based on the number R.sub.ak of lyrics data other than the given lyrics data S.sub.ar of the given artist a when generating the probability distribution over the one or more topic numbers; calculates a second probability p.sub.2 that the topic number of the given lyrics data S.sub.ar is k, based on the number N.sub.kv of times that the topic number k is assigned to the word v in the plurality of lyrics data of the plurality of artists other than the given lyrics data S.sub.ar; calculates the probability p that the topic number of the given lyrics data S.sub.ar is k, from the first probability p.sub.1 and the second probability p.sub.2; and determines the probability distribution over the one or more topic numbers of the given lyrics data S.sub.ar by performing the above-identified calculations on all of the one or more topic numbers and normalizing probabilities that the topic number of the given lyrics data S.sub.ar is any one of 1 (one) to K such that normalized probabilities sum up to 1 (one).

5. The system for generating topic inference information of lyrics according to claim 1, wherein: the outputting means is configured to output a probability distribution over words for each topic number, based on the number N.sub.kv of times that the topic number k is assigned to a given word v, wherein the outputting means are implemented on the computer by the computer program installed in the computer.

6. The system for generating topic inference information of lyrics according to claim 5, wherein: in the outputting means, an occurrence probability θ.sub.kv of the word v to which the topic number k is assigned is calculated as follows:
θ.sub.kv=(N.sub.kv+β)/(N.sub.k+β|V|) where N.sub.kv denotes a number of times that a topic number k is assigned to a given word v, N.sub.k denotes a number of all of words to which the topic number k is assigned, β denotes a smoothing parameter, and |V| denotes a number of kinds of words.

7. The system for generating topic inference information of lyrics according to claim 4, wherein the means for learning values of the one or more switch variables: calculates a third probability p.sub.3 that the value of the switch variable of the word v.sub.arj is 0 (zero), based on a number N.sub.a0 of words to which a value of 0 (zero) is assigned as the value of switch variable in all of lyrics data of all of songs of the given artist a; calculates a fourth probability p.sub.4 that the value of the switch variable of the word v.sub.arj is 0 (zero), based on a number Nz.sub.arv.sub.arj of times that 0 (zero) is assigned to the value of the switch variable of the word v.sub.arj in all of songs of all of artists to which the same topic number Z.sub.ar as the lyrics including the word v.sub.arj is assigned; calculates a fifth probability p.sub.5 that the value of the switch variable is 0 (zero) from the third probability p.sub.3 and the fourth probability p.sub.4; calculates a sixth probability p.sub.6 that the value of the switch variable of the word v.sub.arj is 1 (one), based on a number N.sub.a1 of times that 1 (one) is assigned as the value of the switch variable in the plurality lyrics data of the given artist; calculates a seventh probability p.sub.7 that the value of the switch variable of the word v.sub.arj is 1 (one), based on a number N.sub.1varj of times that 1 (one) is assigned as the value of the switch variable of the word v.sub.arj in the plurality of lyrics data of the plurality of artists; calculates an eighth probability pa that the value of the switch variable is 1 (one) from the sixth probability p.sub.6 and the seventh probability p.sub.7; and normalize the probabilities from the fifth probability p.sub.5 and the eighth probability pa such that a sum of the probability that the value of the switch variable of the word v.sub.arj is 0 (zero) and the probability that the value of the switch variable of the word v.sub.arj is 1 (one) is 1 (one) to obtain a probability distribution over values of the one or more switch variables.

8. The system for generating topic inference information of lyrics according to claim 1, wherein: the topic number for each of the plurality of lyrics data in the outputting means is a topic number that is last assigned to each of the plurality of lyrics data after the operation of updating and learning the one or more topic numbers is performed for a predetermined number of times in the means for learning the one or more topic numbers.

9. The system for generating topic inference information of lyrics according to claim 1, further comprising: a first means for generating a first word probability distribution, operable to generate a probability distribution over words included in lyrics data of a new song s of an artist that has not been used in learning; a second means for generating one or more second word probability distributions over words included respectively in lyrics data of the plurality of songs of the plurality of artists; a means for computing similarities operable to respectively obtain similarities between the first word probability distribution over words included in the lyrics data of the new song s as calculated by the first means for generating the first word probability distribution and the one or more second word probability distributions over words respectively included in the lyrics data of the plurality of songs as calculated by the second means for generating the one or more second word probability distributions; a means for generating a weight distribution by adding the similarities of the lyrics data of the plurality of songs corresponding to the lyrics data of the plurality of songs to the one or more topic numbers as a weight; and a means for determining a topic number, operable to determine a topic number having a largest weight as a topic number of the lyrics data of the new song s.

10. The system for generating topic inference information of lyrics according to claim 9, further comprising: a third means for generating a third word probability distribution, operable to generate the third word probability distribution over words included in the lyrics data of all of songs of the artist that have not been used in learning and for which an occurrence probability of background words are to be calculated; a fourth means for generating one or more fourth word probability distributions, operable to generate probability distributions over words included in the lyrics data of all of the songs of each of the artists; a fifth means for generating one or more fifth word probability distributions, operable to generate probability distributions over background words included in the lyrics data of all of songs for each of the artists; a means for computing similarities, operable to obtain similarities respectively between the third word probability distribution over words included in the lyrics data of the new song s as calculated by the third means for generating the third word probability distribution and the one or more fourth word probability distributions over words included in the lyrics data of the plurality of songs as calculated by the fourth means for generating the one or more fourth word probability distributions; and a means for generating an occurrence probability distribution over background words, operable to multiply the respective probability distributions over background words included in the lyrics data of all of the songs of each of the artists as calculated by the fifth means for generating the one or more fifth word probability distributions, by the similarities for each of the artists as computed by the means for computing similarities to obtain probability distributions, and normalizing the obtained probability distributions such that the weights sum up to 1 (one) for the respective artists, and then determining a resulting probability distribution as an occurrence probability distribution over background words.

11. A system for generating topic inference information of lyrics for obtaining reliable information for inferring a topic that is a subject, a main point or a theme of lyrics as determined by lyric contents, the system comprising: a means for obtaining lyrics data, operable to obtain a plurality of lyrics data each including a song name and the lyrics of each of a plurality of artists; a means for generating a given number of one or more topic numbers k (1≤k≤K) where k is a number in a range of 1 to K (a positive integer); an analysis means for extracting a plurality of words by performing morpheme analysis of a plurality of lyrics in the plurality of the lyrics data, using a morpheme analysis engine; a means for learning the one or more topic numbers, operable to first assign a topic number to each of the plurality lyrics data of each of the plurality of artists in a random or arbitrary manner; then to calculate a probability p that the topic number of a given lyrics data S.sub.ar is k, based on a number R.sub.ak of lyrics data other than the lyrics data S.sub.ar of a given artist a, to which the topic number k is assigned and a number N.sub.kv of times that the topic number k is assigned to a word v in the plurality of lyrics data of the plurality of artists except the given lyrics data S.sub.ar; to generate a probability distribution over the one or more topic numbers of the given lyrics data S.sub.ar, based on the calculated probability p; next to perform an operation of updating the one or more topic numbers to update the topic number assigned to the given lyrics data S.sub.ar of the given artist a using a random number generator having a deviation of appearance probability corresponding to the probability distribution over the one or more topic numbers; and to perform an operation of updating and learning the one or more topic numbers for a predetermined number of times, the operation of updating and learning the one or more topic numbers performing the operation of updating the one or more topic numbers on all of the plurality of lyrics data of each of the plurality of artists; and an outputting means operable to identify the one or more topic numbers of each of the plurality of lyrics data and the probability distributions over words for each of the one or more topic numbers, based on learning results obtained from the means for learning the one or more topic numbers; wherein the means for generating the given number of the one or more topic numbers, the analysis means, the means for learning the one or more topic numbers, and the outputting means are implemented on a computer by a computer program installed in the computer.

12. The system for generating topic inference information of lyrics according to claim 11, wherein: in the means for learning the one or more topic numbers, it is assumed that the one or more topic numbers assigned to all of the plurality of lyrics but the topic number assigned to the given lyrics data of the given artist are correct when generating the probability distribution over the one or more topic numbers.

13. The system for generating topic inference information of lyrics according to claim 11, wherein the means for learning the one or more topic numbers: calculates a first probability p.sub.1 that the topic number of the given lyrics data S.sub.ar is k, based on the number R.sub.ak of lyrics data other than the given lyrics data S.sub.ar of the given artist a when generating the probability distribution over the one or more topic numbers; calculates a second probability p.sub.2 that the topic number of the given lyrics data S.sub.ar is k, based on the number N.sub.kv of times that the topic number k is assigned to the word v in the plurality of lyrics data of the plurality of artists other than the given lyrics data S.sub.ar; calculates the probability p that the topic number of the given lyrics data S.sub.ar is k, from the first probability p.sub.1 and the second probability p.sub.2; and determines the probability distribution over the one or more topic numbers of the given lyrics data S.sub.ar by performing the above-identified calculations on all of the one or more topic numbers and normalizing probabilities that the topic number of the given lyrics data S.sub.ar is any one of 1 (one) to K such that normalized probabilities sum up to 1 (one).

14. The system for generating topic inference information of lyrics according to claim 13, wherein: the outputting means is configured to output a probability distribution over words for each topic number, based on the number N.sub.kv of times that the topic number k is assigned to a given word v, wherein the outputting means are implemented on the computer by the computer program installed in the computer.

15. The system for generating topic inference information of lyrics according to claim 14, wherein: in the outputting means, an occurrence probability θ.sub.kv of the word v to which the topic number k is assigned is calculated as follows:
θ.sub.kv=(N.sub.kv+β)/(N.sub.k+β|V|) where N.sub.kv denotes a number of times that a topic number k is assigned to a given word v, N.sub.k denotes a number of all of words to which the topic number k is assigned, β denotes a smoothing parameter, and |V| denotes a number of kinds of words.

16. The system for generating topic inference information of lyrics according to claim 11, further comprising: a first means for generating a first word probability distribution, operable to generate a probability distribution over words included in lyrics data of a new song s of an artist that has not been used in learning; a second means for generating one or more second word probability distributions over words included respectively in lyrics data of the plurality of songs of the plurality of artists; a means for computing similarities, operable to obtain similarities respectively between the probability distribution over words included in the lyrics data of the new song s as calculated by the first means for generating the first word probability distribution and the probability distributions over words included in the lyrics data of the plurality of songs as calculated by the second means for generating the one or more second word probability distributions; a means for generating a weight distribution by adding the similarities of the lyrics data of the plurality of songs corresponding to the lyrics data of the plurality of songs to the one or more topic numbers as a weight; and a means for determining a topic number, operable to determine a topic number having a largest weight as a topic number of the lyrics data of the new song s.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a block diagram illustrating a first embodiment of a system for generating topic inference information of lyrics according to the present invention.

(2) FIG. 2 is a flowchart illustrating an example algorithm of a computer program when implementing the system of the first embodiment of the present invention on a computer.

(3) FIG. 3 is used for explaining a model for a lyrics generative process.

(4) FIG. 4 illustrates how songs are collected for each of artists.

(5) FIG. 5 illustrates an example of morpheme analysis.

(6) FIG. 6A is a flowchart of an algorithm for automated assignment of topic numbers; and FIG. 6B illustrates an example of assignment of an initial topic number.

(7) FIG. 7 illustrates an example algorithm when a means of learning topic numbers is implemented as software.

(8) FIG. 8 is a detailed flowchart illustrating Step ST408 in the step illustrated in FIG. 7.

(9) FIG. 9 is a block diagram illustrating a second embodiment of a system for generating topic inference information of lyrics according to the present invention.

(10) FIG. 10 is a flowchart illustrating an example algorithm of a computer program when implementing the system of the second embodiment of the present invention on a computer.

(11) FIG. 11 is used for explaining a model for the lyrics generative process.

(12) FIG. 12A is a flowchart illustrating an algorithm for assigning a value of a switch variable to each word included a plurality of lyrics data of each of a plurality of artists in a random or arbitrary manner; and FIG. 12B illustrates an example initial switch variable.

(13) FIG. 13 is a detailed flowchart illustrating Step ST1412 “Generate Probability Distribution over Values of Switch Variables” in FIG. 10.

(14) FIG. 14A illustrates example occurrence probabilities of words for each of a plurality of topics; and FIG. 14B illustrates example identifications over topic numbers of each of a plurality of lyrics data of each of a plurality of artists.

(15) FIG. 15 is a block diagram illustrating a system configuration for obtaining topic numbers of lyric data of a new song of a given artist that has not been used in learning.

(16) FIG. 16A is a flowchart illustrating an algorithm for implementing the system of FIG. 15 using software; and FIG. 16B schematically illustrates an idea of the algorithm of FIG. 16A.

(17) FIG. 17 is a block diagram illustrating a system configuration for generating an occurrence probability of background words as further information for determining a topic of lyrics data of a song of an artist.

(18) FIG. 18A is a flowchart illustrating an algorithm for implementing the system of FIG. 17 using software; and FIG. 18B schematically illustrates an idea of the algorithm of FIG. 18A.

(19) FIGS. 19A and 19B illustrate example word probability distributions for each of the topics obtained in the embodiments of the present invention; and FIG. 19C illustrates an example word probability distribution over background words.

(20) FIGS. 20A and 20B respectively illustrate an example occurrence probability distribution over background words of a song for each artist.

(21) FIGS. 21A to 21C are used to explain problems when using the LDA to infer topics of lyrics.

(22) FIGS. 22A to 22C are used to explain problems when using a clustering method to infer topics of lyrics.

DESCRIPTION OF EMBODIMENTS

(23) Now, embodiments of the present invention will be described below in detail with reference to accompanying drawings.

First Embodiment

(24) FIG. 1 is a block diagram illustrating a first embodiment of a system for generating topic inference information of lyrics according to the present invention. Each block of the illustrated embodiment is implemented on a computer by a computer program installed in the computer, or are implemented by a plurality of processors and a plurality of memories. FIG. 2 is a flowchart illustrating an example algorithm of a method for generating topic inference information of lyrics and a computer program when implementing a basic configuration of the system of the first embodiment using a computer.

(25) As illustrated in FIG. 1, a system 1 for generating topic inference information of lyrics as a basis of the first embodiment comprises a lyrics database 3, a means 5 for obtaining lyrics data, a means 7 for generating topic numbers, a means 9 for learning topic numbers, an analysis means 11, and an outputting means 13.

(26) In the present invention, structural elements as illustrated as a block in FIG. 1 are employed, based on a lyrics generative process as modeled from the lyrics generative process illustrated in FIG. 3. Here, a model of lyrics generative process illustrated in FIG. 3 is described first. When generating lyrics, an artist and a song are chosen at step S1. For example, an artist is “Hiroshi Sekiya” and a song is “Modoranai Natsu (not returning summer)”. Next, a topic number of the song is generated (step S2). Here, the topic refers to a subject, a main point, or a theme of lyrics as determined by the contents of the lyrics. For example, “Natsu (summer)”, “Josei no Koiuta (love song for women)”, “Tabi (journey)”, etc. “A topic number” is assigned to the topic when classifying topics into groups. In the first embodiment, “the topic number” is simply a number and does not have a meaning. Roughly assuming that the topics of lyrics of artists' songs are grouped into 20 kinds (a plurality of kinds of lyrics are classified into 20 groups, for example), the topic number is a number of 1 to 20. The lyrics of a plurality of songs of a given artist are grouped into some kinds of topics, and a number is assigned to each topic. Then, an occurrence probability is calculated for each topic number. This is a probability distribution over topic numbers. Next, a word is sequentially generated from the word probability distribution of the topic when identifying the jth word in the lyrics (steps S4 to S5). Once the jth word is identified, then the j+1th word will be identified. When all the words are identified, lyrics generation is completed.

(27) Now, the theories used in implementing the first embodiment on hardware such as a computer will be described using mathematical equations and expressions. The model is represented by the following equation. Here, the number of topics given as an input is K, a collection of artists in the collection of lyrics data is A, and a collection of nouns or given parts of speech is V. The topic k (1≤k≤K) has a word probability distribution φk=(φ.sub.k1, φ.sub.k2, . . . , φ.sub.kv), and an occurrence probability of a word v∈V is φ.sub.kv≥0 and satisfies the following equation.
Σ.sub.v=1.sup.|v|ϕ.sub.kv=1

(28) An artist a∈A has a topic probability distribution θa=(θa.sub.1, θa.sub.2, . . . , θa.sub.k), and a topic occurrence probability θa≥0 and satisfies the following equation.
Σ.sub.k=1.sup.Kθ.sub.ak=1

(29) The artist a∈A has a probability distribution λa=(λa.sub.0; λa.sub.1) for choosing a value of the switch variable. λa.sub.0 is a probability having a value of the switch variable of 0 (zero), and indicates that a word is chosen from the topics. λa.sub.1 is a probability having a value of the switch variable of 1 (one), and indicates that a word is chosen from the background words. λa.sub.0≥0 and λa.sub.1≥0 and λa.sub.0+λa.sub.1=1 are satisfied. The background word v∈V has a word probability distribution φ=φ.sub.1, φ.sub.2, . . . φ.sub.|V|), and a word occurrence probability φ.sub.V≥0 and satisfies the following equation.
Σ.sub.v=1.sup.|V|ψ.sub.v=1

(30) The system of the present invention automatically generates information useful for determining or inferring the topics of lyrics, based on the model illustrated in FIG. 3. The means 5 for obtaining lyrics data obtains a plurality of lyrics data (a collection of lyrics data) including song names and lyrics for each of a plurality of artists from a lyric database 3, as illustrated in step ST1 of FIG. 2 and FIG. 4. In an example of FIG. 4, the means for obtaining lyrics data obtains respective lyrics data of songs of two artists (Hiroshi Sekiya and Hiromi Yoshii). As the database 3, for example, MySQL may be used.

(31) To implement this using a computer, the total number of lyrics data of an artist a is defined as R.sub.a, and the r(1≤r≤R.sub.a)th lyric as S.sub.ar, a collection D.sub.a of lyrics of the artist a is represented by the following equation.
D.sub.a={S.sub.ar}.sub.r=1.sup.R.sup.a

(32) Further, a collection of D of lyrics of all of the artists is represented by D={D.sub.a}.sub.a∈.sub.A.

(33) The means 7 for generating topic numbers generates a topic number k of 1 to K (a positive integer) as illustrated in step ST2 of FIG. 2. In the first embodiment, the means 7 for generating topic numbers generates topic numbers of 1 to 20.

(34) As illustrated in step ST3 of FIG. 2 and FIG. 5, the analysis means 11 analyzes a plurality of lyrics in a plurality of lyrics data by means of morpheme analysis to extract a plurality of words. The morpheme analysis is performed using a morpheme analysis engine that extracts a group of nouns or particular parts of speech from sentences as words. There have currently been proposed various morpheme analysis engines. With a morpheme analysis engine, it is possible to readily extract words from even a considerable amount of songs. In the first embodiment, as illustrated in FIG. 5, an open source morpheme analysis engine called MeCab (HYPERLINK “http://taku910.github.io/mecab/) is used. Here, the number of nouns or particular parts of speech included in the lyric S.sub.ar is mathematically defined as V.sub.ar.

(35) As illustrated in step ST4 of FIG. 2 and steps ST41 to ST47 of FIG. 6A, the means 9 for learning topic numbers assigns topic numbers respectively to a plurality of lyrics data of each of the artists i in a random or arbitrary manner. In the first embodiment, as illustrated in FIG. 6B, an occurrence probability of a plurality of lyrics data to which a topic number is assigned is initially set as 0.05=(1/K).

(36) FIG. 7 illustrates an example algorithm of software when implementing the means 9 for learning topic numbers using a computer. In the algorithm of FIG. 7, an operation of updating and learning topic numbers (steps ST403 to ST411) is performed for a predetermined number of times (ST402). The operation of updating and learning topic numbers performs an operation of updating topic numbers (steps ST404 to ST411 of FIG. 7) on all of a plurality of lyrics data of each of the artists i. In the first embodiment, it is assumed that topic numbers assigned to all of a plurality of lyrics but the topic number k (an integer of k=1 to K) assigned to the given lyrics data S.sub.ar of the given artist a are correct. A probability that the topic number assigned to the given lyrics data S.sub.ar of the given artist a is k is calculated to generate a probability distribution over topic numbers (step ST408).

(37) FIG. 8 is a detailed flowchart illustrating Step 408. When generating a probability distribution over the topic numbers, first, a first probability p.sub.1 that the topic number of the given lyrics data S.sub.ar is k is calculated, based on the number R.sub.ak of all of the lyrics data but the given lyrics data S.sub.ar of the given artist a (step ST408D). Next, a second probability p.sub.2 that the topic number of the given lyrics data S.sub.ar is k is calculated, based on the number N.sub.kv of times that the topic number k is assigned to the word v in a plurality of lyrics data of a plurality of artists but the given lyrics data S.sub.ar (steps ST408E to 51408H). In step ST408F, it is determined whether a given word v is present in a collection V of words. In step ST408G, it is determined whether that word is present in the given lyrics data S.sub.ar. In step ST408G, if “N”, the process goes to step ST408I where a next word is processed. In step ST408F, if “N”, any word is not present anymore and the process goes to step ST408J. In step ST408J, a probability p that the topic number of the given lyrics data S.sub.ar is k is calculated, from the first probability p.sub.1 and the second probability p.sub.2. The above-mentioned calculations are performed on all of the topic numbers (1 to K), and probabilities that the topic number of the given lyrics data S.sub.ar is any one of 1 to K are normalized such that normalized probabilities sum up to 1 (one). Thus, a probability distribution over topic numbers of the given lyrics data S.sub.ar is determined (step ST408C). With these calculations, the accuracy is increased for the probability distribution over topic numbers.

(38) Next, in step ST409, the topic number of the song, namely, the given lyrics data S.sub.ar is updated. In updating the topic number, the topic number assigned to the given lyrics data S.sub.ar of the given artist a is updated using a random number generator having a deviation of appearance probability corresponding to the probability distribution over topic numbers (step ST409). The operation of updating topic numbers (steps ST403, ST409) is performed on all of a plurality of lyrics data of each of a plurality of artists (steps ST 404, ST411). Then, the operation of updating and learning topic numbers (steps ST403 to ST411) is performed for a predetermined number of times [in an example of FIG. 7, 500 times]. Imaginarily, the probability of topic numbers is illustrated in FIG. 6B. The “random number generator” used herein is conceptually described as follows. In the first embodiment, assume an imaginary polyhedron dice having 20 faces corresponding to the topic numbers, and each face having an area proportional to its appearance probability. When rolling the imaginary dice, the number (a number of 1 to 20) assigned to the appearing face of the dice is defined as an updated topic number.

(39) As illustrated in step ST5 of FIG. 2, the outputting means 13 of FIG. 1 outputs the respective topic numbers of a plurality of lyrics data of each of a plurality of artists i, and the probability distributions over words for each of the topic numbers, based on learning results obtained by the means 9 for learning topic numbers. In the outputting means 13, the topic number of each of a plurality of lyrics data is a topic number that is last assigned to each of a plurality of lyrics data after the operation of updating and learning topic numbers is performed for a predetermined number of times (in the first embodiment, 500 times) in the means 9 for learning topic numbers. Thus, outputting the results of last assignment enables assignment of topic numbers suitable for a plurality of lyrics data.

(40) Specifically, a last updated value is determined as the topic number assigned to lyrics data in step ST409 of FIG. 7 “Update Topic Number”. The outputting means 13 outputs a word probability distribution for each of the topic numbers, based on the number N.sub.kv of times that the topic number k is assigned to a word v. Specifically, the occurrence probability θ.sub.kv of the word v to which the topic number k is assigned is calculated by the following equation. From the calculated occurrence probability, a word probability distribution for each topic number is identified.
θ.sub.kv=(N.sub.kv+β)/(N.sub.k+β|V|)

(41) Where N.sub.kv denotes the number of times that the topic number k is assigned to the given word v, N.sub.k denotes the number of all the words to which the topic number k is assigned, β denotes a smoothing parameter for the number times of word appearing, and |V| denotes the number of kinds of words.

(42) (Equation-Based Updating of Topic Numbers)

(43) Updating of topic numbers as mentioned above will be theoretically described below. First, it is assumed that each of θa, φk, φ, and λa has a Dirichlet distribution of parameters α, β, γ, and pas a prior distribution. Defining that the topic number of the song S.sub.ar of the artist a as Z.sub.ar, the value of the switch variable of the jth word in the lyrics S.sub.ar of the artist a as X.sub.arj, a collection D of lyrics and a collection Z of topic numbers are represented by the following equation.
Z={{z.sub.ar}.sub.r=1.sup.R.sup.a}a∈A

(44) A collection X of switch variables is represented by the following equation.
X={{{x.sub.arj}.sub.j=1.sup.V.sup.ar}.sub.r=1.sup.R.sup.a}a∈A

(45) The joint distributions is represented by the following equation.
P(D,Z,X|α,β,γ,ρ)=∫∫∫∫P(D,Z,X|Θ,Φ,Ψ,Λ)P(Θ|α)P(Φ|β)P(ψ|γ)×P(Λ|ρ)dΘdΦdψdΛ  (1)

(46) Here, the following equation holds.
Θ={θ.sub.a}a∈A Φ={ϕ.sub.k}.sub.k=1.sup.K Λ={λ.sub.a}a∈A

(47) P(D, Z, X|α,β, γ, ρ) represents a probability that the following combination occurs: words (D) of all of the lyrics, all of the topic numbers (Z), and assignment of all of the switch variables (X) when topic number assignment for all of the songs of all of the artists and values of switch variables assignment for all of the words of all of the songs of all of the artists are determined. Equation (1) is calculated by integrating out these parameters as follows.

(48) $\begin{array}{cc}P\left(D,Z,X.Math.\alpha ,\beta ,\gamma ,\rho \right)={\left(\frac{\Gamma \left(\alpha K\right)}{{\Gamma \left(\alpha \right)}^K}\right)}^{.Math.A.Math.}\underset{a\in A}{.Math.}\frac{\underset{k=1}{\overset{K}{.Math.}}\Gamma \left(R_{\mathrm{ak}}+\alpha \right)}{\Gamma \left(R_a+\alpha K\right)}\times {\left(\frac{\Gamma \left(\beta .Math.V.Math.\right)}{{\Gamma \left(\beta \right)}^{.Math.V.Math.}}\right)}^K\underset{k=1}{\overset{K}{.Math.}}\frac{\underset{v\in V}{.Math.}\Gamma \left(N_{\mathrm{kv}}+\beta \right)}{\Gamma \left(N_k+\beta .Math.V.Math.\right)}\times \frac{\Gamma \left(\gamma .Math.V.Math.\right)}{{\Gamma \left(\gamma \right)}^{.Math.V.Math.}}\frac{\underset{v\in V}{.Math.}\Gamma \left(N_{1v}+\gamma \right)}{\Gamma \left(N_1+\gamma .Math.V.Math.\right)}\times {\left(\frac{\Gamma \left(2\rho \right)}{{\Gamma \left(\rho \right)}^2}\right)}^{.Math.A.Math.}\underset{a\in A}{.Math.}\frac{\Gamma \left(\rho +N_{a0}\right)\Gamma \left(\rho +N_{a1}\right)}{\Gamma \left(2\rho +N_a\right)}.& \left(2\right)\end{array}$

(49) Here, N.sub.a0 and N.sub.a1 respectively denote the number of words for which the value of switch variable is 0 (zero) and the number of words for which the value of switch variable is 1 (one) in the lyrics of the artist a, and N.sub.a=N.sub.a0+N.sub.a1. N.sub.1 denotes the number of words v for which the value of switch variable is one, N.sub.1=Σ.sub.v∈.sub.vN.sub.1v. Here, N.sub.k=Σ.sub.v∈.sub.vN.sub.kv, and N.sub.kv denotes the number of times that the topic number k is assigned to the word v under the condition of a switch variable of 0 (zero). R.sub.ak denotes the number of lyrics to which the topic number k is assigned in the lyrics of the artist a.
R.sub.a=Σ.sub.k=1.sup.KR.sub.ak

(50) The term of expression (3) in equation (2) denotes a probability that when assignment of topic numbers to all of the lyrics is determined, that assignment is observed.

(51) $\begin{array}{cc}{\left(\frac{\Gamma \left(\alpha K\right)}{{\Gamma \left(\alpha \right)}^K}\right)}^{.Math.A.Math.}\underset{a\in A}{.Math.}\frac{\underset{k=1}{\overset{K}{.Math.}}\Gamma \left(R_{\mathrm{ak}}+\alpha \right)}{\Gamma \left(R_a+\alpha K\right)}.& \left(3\right)\end{array}$

(52) The term of the following expression (4) in equation (2) denotes a probability that when assignment of values of switch variables to all of the words in all of the lyrics is determined, that assignment is observed.

(53) $\begin{array}{cc}{\left(\frac{\Gamma \left(2\rho \right)}{{\Gamma \left(\rho \right)}^2}\right)}^{.Math.A.Math.}\underset{a\in A}{.Math.}\frac{\Gamma \left(\rho +N_{a0}\right)\Gamma \left(\rho +N_{a1}\right)}{\Gamma \left(2\rho +N_a\right)}.& \left(4\right)\end{array}$

(54) The term of the following expression (5) in equation (2) denotes a probability that when assignment of all of the topic numbers to all of the lyrics and assignment of all of the values of switch variables to all of the words in all of lyrics are determined, all of the words in all of the lyrics are observed.

(55) $\begin{array}{cc}{\left(\frac{\Gamma \left(\beta .Math.V.Math.\right)}{{\Gamma \left(\beta \right)}^{.Math.V.Math.}}\right)}^K\underset{k=1}{\overset{K}{.Math.}}\frac{\underset{v\in V}{.Math.}\Gamma \left(N_{\mathrm{kv}}+\beta \right)}{\Gamma \left(N_k+\beta .Math.V.Math.\right)}\times \frac{\Gamma \left(\gamma .Math.V.Math.\right)}{{\Gamma \left(\gamma \right)}^{.Math.V.Math.}}\frac{\underset{v\in V}{.Math.}\Gamma \left(N_{1v}+\gamma \right)}{\Gamma \left(N_1+\gamma .Math.V.Math.\right)}.& \left(5\right)\end{array}$

(56) A probability of Z.sub.ar=k is represented by equation (6) when the topic number of a song S.sub.ar of the artist a is defined as Z.sub.ar.

(57) $\begin{array}{cc}p\left(z_{\mathrm{ar}}=k.Math.D,X,Z_{\backslash \mathrm{ar}}\right)\propto \frac{R_{\mathrm{ak}\backslash \mathrm{ar}}+\alpha }{R_a-1+\alpha K}\frac{\Gamma \left(N_{k\backslash \mathrm{ar}}+\beta .Math.V.Math.\right)}{\Gamma \left(N_{k\backslash \mathrm{ar}}+N_{\mathrm{ar}}+\beta .Math.V.Math.\right)}\times \underset{v\in V}{.Math.}\frac{\Gamma \left(N_{\mathrm{kv}\backslash \mathrm{ar}}+N_{\mathrm{arv}}+\beta \right)}{\Gamma \left(N_{\mathrm{kv}\backslash \mathrm{ar}}+\beta \right)}.& \left(6\right)\end{array}$

(58) In the above equation, \.sub.ar denotes a value when rth lyrics of the artist a is excluded. N.sub.ar denotes the number of words in rth lyrics of the artist a, and N.sub.arv denotes the number of words v appearing in rth lyrics of the artist a. In equation (6), the term of expression (7) in equation (6) denotes how many topic numbers k are assigned to the lyrics other than rth lyric of the artist a. In other words, the more the topic number k is assigned to the songs of the artist a, the higher the probability that the topic number assigned to rth lyric of the artist a is k will be.

(59) $\begin{array}{cc}\frac{R_{\mathrm{ak}\backslash \mathrm{ar}}+\alpha }{R_a-1+\alpha K}.& \left(7\right)\end{array}$

(60) The term of expression (8) in equation (6) denotes how many words the topic number k is assigned to in the rth lyrics of the artist a when looking into the songs other than the rth song of the artist a. For example, if a word “Natsu (summer)” is presented in the rth song of the artist a, it is taken into consideration how many times the topic number k is assigned to the word “Natsu (summer)” in all of the songs but the rth song of the artist a. Here, when the topic number of the song is k, it is considered that the topic number k is assigned to all of the words in the lyrics of that song. Namely, the more words the topic number k is assigned in the lyrics of the rth song of the artist a, the higher the probability that the topic number of the rth song of the artist a is k will be.

(61) $\begin{array}{cc}\frac{\Gamma \left(N_{k\backslash \mathrm{ar}}+\beta .Math.V.Math.\right)}{\Gamma \left(N_{k\backslash \mathrm{ar}}+N_{\mathrm{ar}}+\beta .Math.V.Math.\right)}\underset{v\in V}{.Math.}\frac{\Gamma \left(N_{\mathrm{kv}\backslash \mathrm{ar}}+N_{\mathrm{arv}}+\beta \right)}{\Gamma \left(N_{\mathrm{kv}\backslash \mathrm{ar}}+\beta \right)}.& \left(8\right)\end{array}$

(62) Updating of the topic number is performed so as to increase the value of equation (2). In parallel with updating of the topic numbers for each of the lyrics, a probability distribution over words for each topic number is also updated.

(63) The switch variable as described above is theoretically a switch variable output by a means 115 for learning values of switch variables in a second embodiment as described later. In the first embodiment, the switch variable is assumed to be 0 (zero), and the value of a switch variable is not updated. Accordingly, background words are not taken into consideration.

Second Embodiment

(64) FIG. 9 is a block diagram illustrating a second embodiment of a system for generating topic inference information according to the present invention. Each block of the illustrated embodiment is implemented on a computer by a computer program installed in the computer, or are implemented by a plurality of processors and a plurality of memories. FIG. 10 is a flowchart illustrating an example algorithm of a computer program when implementing the system of the second embodiment of the present invention on a computer.

(65) The second embodiment is different from the first embodiment as illustrated in FIGS. 1 to 8 in that the means 115 for learning values of switch variables is provided in the second embodiment. The two embodiments are common in other aspects. In FIG. 9, reference signs for the elements in the second embodiment are obtained by adding 100 to reference signs of the counterpart elements having the same or similar functions in the first embodiment illustrated in FIG. 1, and descriptions thereof are omitted here. In the flowchart of FIG. 10, likewise, reference signs for the steps in the second embodiment are obtained by adding 1000 to reference signs of the counterpart steps having the same or similar functions in the first embodiment illustrated in FIG. 7, and descriptions thereof are omitted here. In the model of lyrics generative process of FIG. 11, further, reference signs for the steps in the second embodiment are obtained by adding 10 to reference signs of the counterpart steps having the same or similar functions in the first embodiment illustrated in FIG. 3, and descriptions thereof are omitted here.

(66) In the second embodiment, as illustrated in FIG. 9, the means 115 for learning values of switch variables is provided. Here, when determining the jth word included in the lyrics, the “value of a switch variable” determines whether the word concerned is a word related to the topic or a background word unrelated to the topic. Namely, the switch variable is a variable for specifying whether a given word is related to the subject of an assumed topic or the background. Identification of the switch variable by means of computation eliminates the need of manually specifying a collection of words related to the topics or a collection of unrelated words. The value of 0 (zero) for the switch variable indicating that the word concerned is fully or 100% related to the topic has an occurrence probability of 1 (one). The value of 0 (zero) for the switch variable indicating that the word concerned is 50% related to the topic has an occurrence probability of 0.5. When determining the jth word, the probability distribution over values of the switch variables of the jth word is used. When manually generating lyrics, the probability distribution over values of switch variables will be manually specified.

(67) In steps S14 to S18 of FIG. 11, based on the probability of the value of the switch variable of the jth word, depending upon whether the probability distribution indicates the topic or not (step S16), the jth word is generated from the word probability distribution for the topic (step S17) or a background word is generated from the word probability distribution (step S18). Once the jth word is determined, the j+1th word is determined (step S19), and thereafter lyrics generation is completed when all of the words are determined.

(68) The means 115 for learning values of switch variables performs an operation of updating and learning of values of switch variables [steps ST1409 to ST1415 of FIG. 10] for a predetermined number of times [in the illustrated example, 500 times]. In the operation of updating and learning of values of switch variables, values are assigned, as prior preparations, to a plurality of words included in a plurality of lyrics data of each of a plurality of artists in a random or arbitrary manner as illustrated in the flowchart of FIG. 12A (steps ST141 to ST150). As illustrated in FIG. 12B, in the illustrated example, it is initially assumed that a probability that a word is related to the topic or a probability that a word is related to the background are respectively 0.5.

(69) As illustrated in FIG. 10, the means 115 for learning values of switch variables assumes that values of switch variables assigned to all of words but the value of the switch variable x (x=a probability variable indicative of a topic word or a background word) assigned to a given word v in a plurality of words of a given lyrics data of a given artist are correct. The means 115 calculates a probability whether a value of the switch variable x assigned to a given word v is 0 (zero) or 1 (one), and generates a probability distribution over values of switch variables λ.sub.a (steps ST1410 to ST1412 in FIG. 10).

(70) FIG. 13 is a detailed flowchart illustrating Step ST1412 “Generate Probability Distribution over Values of Switch Variables” in FIG. 10. First, a third probability p.sub.3 that the value of the switch variable for the word v.sub.arj is 0 (zero) is calculated, based on the number N.sub.a0 of words V.sub.arj to which a value of 0 (zero) is assigned as the value of the switch variable in all of lyric data of all of songs for a given artist a (step ST1412A). Next, a fourth probability p.sub.4 that the value of the switch variable of the word v.sub.arj is 0 (zero) is calculated, based on the number Nz.sub.arv.sub.arj of times that 0 (zero) is assigned to the value of the switch variable of the word v.sub.arj in all of songs of all of artists to which the same topic number Z.sub.ar as the lyrics including the word v.sub.arj is assigned (step ST1412B). Then, a fifth probability p.sub.5 that the value of the switch variable is 0 (zero) is calculated from the third probability p.sub.3 and the fourth probability p.sub.4 (step ST1412C). Further, a sixth probability p.sub.6 that the value of the switch variable of the word V.sub.arj is 1 (one) is calculated, based on the number N.sub.a1 of times that 1 (one) is assigned as the value of the switch variable in the plurality lyrics data of the given artist (step ST1412D). A seventh probability p.sub.7 that the value of the switch variable of the word v.sub.arj is 1 (one) is calculated, based on the number N.sub.1varj of times that 1 (one) is assigned as the value of the switch variable of the word v.sub.an in the plurality of lyrics data of the plurality of artists (step ST1412E). Then, an eighth probability p.sub.8 that the switch variable value is 1 (one) is calculated from the sixth probability p.sub.6 and the seventh probability p.sub.7 (step ST1412F). The probabilities are normalized from the fifth probability p.sub.5 and the eighth probability p.sub.8 such that the probability that the value of the switch variable of the word v.sub.arj is 0 (zero) and the probability that the value of the switch variable of the word v.sub.ar is 1 (one) sum up to 1 (one), and then a normalization result is defined as the probability distribution over values of switch variables (step ST1412G).

(71) (Equation-Based Updating of Switch Variables)

(72) Updating of switch variables as mentioned above will be theoretically described below. First, it is assumed a value of the switch variable for the jth word in the lyrics data S.sub.ar of the artist a is x.sub.arj. A probability that x.sub.arj=0 is represented by the following equation.

(73) $\begin{array}{cc}P\left(x_{\mathrm{arj}}=0.Math.D,X_{\backslash \mathrm{arj}},Z,\alpha ,\beta ,\gamma ,\rho \right)\propto \frac{\rho +N_{a0\backslash \mathrm{arj}}}{2\rho +N_a-1}\frac{N_{z_{\mathrm{ar}}{v}_{\mathrm{arj}}\backslash \mathrm{arj}}+\beta }{N_{z_{\mathrm{ar}}\backslash \mathrm{arj}}+\beta .Math.V.Math.}.& \left(9\right)\end{array}$

(74) In the above equation, \.sub.ar denotes a value when the jth word of the rth lyrics of the artist a is excluded. The term of expression (10) in equation of (9) denotes how readily the artist a generates words from the topics. The larger the value is, the higher the probability that the value of the switch variable of the jth word in the rth lyrics of the artist a is 0 (zero) will be.

(75) $\begin{array}{cc}\frac{\rho +N_{a0\backslash \mathrm{arj}}}{2\rho +N_a-1}.& \left(10\right)\end{array}$

(76) The term of expression (11) in equation of (9) denotes how readily the jth word in the rth lyrics of the artist a occurs at the topic number Z.sub.ar. The larger the value is, the higher the probability that the value of the switch variable of the jth word in the rth lyrics of the artist a is 0 (zero) will be. For example, when the jth word of the rth lyrics of the artist a is “Natsu (summer)”, it is taken into consideration how often 0 (zero) is assigned to the word “Natsu (summer)” as the value of the switch variable in all of the words of all of the songs of all of the artists to which the topic number z.sub.ar has been assigned.

(77) 0$\begin{array}{cc}\frac{N_{z_{\mathrm{ar}}{v}_{\mathrm{arj}}\backslash \mathrm{arj}}+\beta }{N_{z_{\mathrm{ar}}\backslash \mathrm{arj}}+\beta .Math.V.Math.}.& \left(11\right)\end{array}$

(78) Likewise the probability that x.sub.arj=1 is represented as follows.

(79) $\begin{array}{cc}P\left(x_{\mathrm{arj}}=1.Math.D,X_{\backslash \mathrm{arj}},Z,\alpha ,\beta ,\gamma ,\rho \right)\propto \frac{\rho +N_{a1\backslash \mathrm{arj}}}{2\rho +N_a-1}\frac{N_{1v_{\mathrm{arj}}\backslash \mathrm{arj}}+\gamma }{N_{1\backslash \mathrm{ark}}+\gamma .Math.V.Math.}.& \left(12\right)\end{array}$

(80) The term of expression (13) in equation (12) denotes how readily the artist a generates a word from the background words. The larger the value is, the higher the probability that the value of the switch variable of the jth word in the rth lyrics of the artist a is 1 (one) will be.

(81) $\begin{array}{cc}\frac{\rho +N_{a1\backslash \mathrm{arj}}}{2\rho +N_a-1}.& \left(13\right)\end{array}$

(82) The term of expression (14) in equation (12) denotes how readily the jth word in the rth lyrics of the artist a generates a word from the background words. The larger the value is, the higher the probability that the value of the switch variable of the jth word in the rth lyrics of the artist a is 1 (one) will be.

(83) $\begin{array}{cc}\frac{N_{1v_{\mathrm{arj}}\backslash \mathrm{arj}}+\gamma }{N_{1\backslash \mathrm{arj}}+\gamma .Math.V.Math.}.& \left(14\right)\end{array}$

(84) Updating of the values of switch variables of the words as illustrated in step ST1413 of FIG. 10 is preferably performed so as to increase the value of equation (2). In parallel with updating the value of the switch variable of each word, a probability distribution over values of switch variables for each of the artists is also updated.

(85) Specifically, an operation of updating the value of switch variable assigned to a given word is performed on all of a plurality of words included in a plurality of lyrics of each of a plurality of artists, using a random Number generator having a deviation of appearance probability corresponding to the probability distribution over values of the switch variables (step ST1412 to ST1416). The “random number generator” used herein is conceptually described as follows. In the second embodiment, assume an imaginary dihedron dice having two faces corresponding to two switch variables, and each face having an area proportional to its appearing probability. When rolling the imaginary dice, the number assigned to the appearing face of the dice is defined as an updated value of the switch variable.

(86) As illustrated in step ST5 of FIG. 2 and FIGS. 14A and 14B, the outputting means 113 identifies the respective topic numbers of a plurality of lyrics data of each of a plurality of artists, based on the learning results of the means 109 for learning topic numbers and the results of the means 115 for learning values of switch variables (FIG. 14B), and generates an occurrence probability of words for each of a plurality of topic numbers (FIG. 14A). With this, it is possible to grasp the topics of lyrics of a plurality of songs of each of the artists that reflect the artists' personalities. Thus, it is possible to provide artist-based information to a person who chooses a song.

(87) The topic number of each of a plurality of lyrics output by the outputting means 113 is a topic number last assigned to the lyrics data of each of the artists [the topic number last updated in step ST1409 of FIG. 10] after performing the operation of updating and learning topic numbers for a predetermined number of times [in FIG. 10, 500 times; step ST1402 in FIG. 10] in the means 109 for learning topic numbers. According to the last assignment, the most suitable topic number can be assigned to each of a plurality of lyrics data.

(88) Likewise, the word probability distribution for each topic number output by the outputting means 113 is the last stored word probability distribution for each topic number after performing the operation of updating and learning topic numbers for a predetermined number of times [In FIG. 10, 500 times; step ST1402 in FIG. 10] in the means 109 for learning topic numbers. In the outputting means 113, an occurrence probability θ.sub.kv of words for each topic number is preferably calculated as follows.

(89) $\begin{array}{cc}{\theta }_{\mathrm{kv}}=\frac{N_{\mathrm{kv}}+\beta }{N_k+\beta .Math.V.Math.}.& \left(15\right)\end{array}$

(90) Where N.sub.kv denotes the number of times that the topic number k is assigned to the given word v, N.sub.k denotes the number of all of the words to which the topic number k is assigned, β denotes a smoothing parameter, and |V| denotes the number of kinds of words. Here, the smoothing parameter refers to the number of times of pseudo-occurrence of each word at each topic number. The number of kinds of words refers to the number of unique words included in the lyrics in the lyrics database illustrated in FIG. 4. In the computation using the above equation, a higher probability than 0 (zero) is assigned to a word that has never been assigned to any topic, thereby getting close to the human intuition.

(91) (Effect Obtainable from Second Embodiment)

(92) According to the second embodiment, once an arbitrary number of topics is determined, the last updated topic number for each of a plurality of lyrics data can be identified with the topic number as updated by the means for learning topic numbers. Further, an occurrence probability of words for each topic number is generated, based on the values of switch variables as last updated by the means for learning values of switch variables. Once the topic number of each of a plurality of lyrics data and the occurrence probability of words for each topic number have been determined, the word having a high occurrence probability can be known for each of topic number, thereby eliminating the need of manually specifying a collection of words related to the topics and a collection of words unrelated to the topics. Further, once a plurality of words having a high occurrence probability have been grasped, reliable information for determining the topics can be obtained from these words, thereby grasping likely meanings of the topics of the lyrics of each song.

(93) [System for Topic Inference Information of Lyrics that have not been Used in Learning]

(94) When obtaining a topic number of a lyrics data of a new song of a given artist that has not been used in learning, the system may be configured as illustrated in FIG. 15. Steps ST201 to ST211 describe an algorithm for implementing the system of FIG. 15 using software. FIG. 16B schematically illustrates an idea of the algorithm of FIG. 16A. In the present embodiment, the system may further comprise a first means 17 for generating a word probability distribution for implementing step ST202 that generates a probability distribution over words included in lyrics data of a new song s of an artist that has not been used in learning; a second means 19 for implementing step ST207 that generates a word probability distributions over words included respectively in lyrics data of the plurality of songs of the plurality of artists; a means 21 for implementing step ST208 that computes cosine similarities and similarities according to an arbitrary scale respectively between the probability distribution over words included in the lyrics data of the new song s as calculated by the first means 17 for generating a word probability and the probability distributions over words included in the lyrics data of the plurality of songs as calculated by the second means 19 for generating word probability distributions; a means 23 for implementing step ST 211 that generates a weight distribution by adding the similarities of the lyrics data of the plurality of songs corresponding to the lyrics data of the plurality of songs to the topic numbers as a weight; and a means 25 for determining a topic number that determines a topic number having the largest weight as a topic number for the lyrics data of the new song s (remaining steps of step ST211). Thus, the topic can readily be determined for lyrics that have not been used in learning.

(95) (System for Generating Occurrence Probability for Background Word)

(96) FIG. 17 is a block diagram illustrating a system configuration for generating an occurrence probability of background words as further information for determining a topic of lyrics data of a song of an artist that has not been used in learning. Steps ST301 to ST309 of FIG. 18A describe an algorithm for implementing the system of FIG. 17 using software. FIG. 18B schematically illustrates an idea of the algorithm of FIG. 18A. In FIG. 17, reference signs for parts are the same as those of the counterparts having the same or similar functions in FIG. 1,

(97) In the present embodiment, the system comprises a third means 27 for generating a word probability distribution to a fifth means 31 for generating a word probability distribution; a means 33 for computing similarities, and a means 35 for generating an occurrence probability distribution. The third means 27 for generating a word probability distribution generates a probability distribution over words included in the lyric data of all of songs of the artist that have not been used in learning and for which an occurrence probability of background words are to be calculated (step ST302); a fourth means 29 for generating a word probability distribution generates probability distributions over words in the lyrics data of all of the songs of each of the artist that have been used in learning (step ST306); and a fifth means 31 for generating a word probability distribution generates a probability distribution for background words included in the lyrics data of all of songs for each of the artists that have been used in learning (step ST306). In the present embodiment, the word distribution over background words for each artist can be obtained by determining a word distribution over background words for each artist, not a common word probability over background words for all of the artists as illustrated in FIG. 11. The means 33 for computing similarities obtains cosine similarities or similarities according to an arbitrary scale respectively between the probability distribution over words included in the lyrics data of the new song s as calculated by the third means 27 for generating a word probability and the probability distributions over words included in the lyrics data of the plurality of songs as calculated by the fourth means 29 for generating word probability distributions (step ST307). The means 35 for generating an occurrence probability distribution over background words multiplies the respective probability distributions over background words included in the lyrics data of all of the songs of each of the artists as the fifth means 31 for generating an occurrence probability of words by the similarities for each of the artists as computed by the means 33 for computing similarities to obtain probability distributions, and normalizing the obtained probabilities such as the weights of the respective artists sum up to 1 (one). Thus, a resulting probability is determined as an occurrence probability of background words (step ST309). The determined occurrence probability is determined as the occurrence probability distribution over background words for a given artist. Also from the probability distribution over background words, meanings of the topics can be known.

(98) [Example Results]

(99) FIGS. 19A and 19B illustrate example word probability distributions for each of the topics obtained in the embodiments of the present invention; and FIG. 19C illustrates an example word probability distribution over background words for all of the songs. As known from the word probability distributions for each topic in FIG. 19A, the meaning of “positive” is specified for the topic 17 since there are words such as “Kimi (you)”, “Yume (dream)”, “Boku (I)”, and “Ima (now)” having high occurrence probabilities. As known from the word probability distribution over words for each topic in FIG. 19B, the meaning of “Love song for adult women” is specified for the topic 19 since there are words such as “Anata (you)”, “Watashi (I)”, “Hito (person)”, and “Koi (love)” having high occurrence probabilities. FIGS. 20A and 20B respectively illustrate an example occurrence probability distribution over background words of a song for each artist. FIG. 20A is an example occurrence probability distribution over background words in a song of an artist A; and FIG. 20B is an example occurrence probability distribution over background words in a song of an artist B. Looking at them, it is possible to obtain information on topic tendency of songs of artists from background words for each artist.

(100) [Method and Computer Program]

(101) The present invention may be implemented as a method for generating topic inference information of lyrics and a computer program therefor as follows,

(102) The method comprises:

(103) (1) A step of obtaining a plurality of lyrics data each including a song name and lyrics for each of a plurality of artists;

(104) a step of generating a given number of topic numbers k of 1 to K (1≤k≤K);

(105) an analysis step of analyzing the plurality of lyrics in the plurality of lyrics data by means of morpheme analysis to extract a plurality of words;

(106) a step of learning topic numbers by first assigning the topic number k to the plurality of lyrics data for each of the plurality of artists in a random or arbitrary manner, then calculating a probability p that the topic number of a given lyrics data S.sub.ar is k, based on a number R.sub.ak of lyrics data other than a lyrics data S.sub.ar for a given artist a, to which the topic number k is assigned and a number N.sub.kv of times that the topic number k is assigned to the word v in the plurality of lyrics data of the plurality of artists except the given lyrics data S.sub.ar, calculating a probability distribution over topic numbers of the given lyrics data S.sub.ar, based on the calculated probability p, next performing an operation of updating topic numbers to update the topic number assigned to the given lyrics data S.sub.ar of the given artist a using a random number generator having a deviation of appearance probability corresponding to the probability distribution over topic numbers, and performing an operation of updating and learning topic numbers on all of the plurality of lyrics data of each of the plurality of artists for a predetermined number of times; and

(107) an outputting step of identifying the topic numbers of each of the plurality of lyrics data and the probability distributions over words for each of the topic numbers, based on learning results obtained in the step of learning topic numbers.

(108) (2) The method for generating topic inference information of lyrics as described in (1) further comprises:

(109) a step of learning values of switch variables, wherein a value of the switch variable is assigned to each of the plurality of words included in the plurality of lyrics data of each of the plurality of artists in a random or arbitrary manner; then a probability distribution A.sub.a over values of switch variables is generated by calculating a probability whether the value of the switch variable x assigned to the given word v.sub.arj is a topic word or a background word, based on values of switch variables assigned to the plurality of words in the plurality of lyrics data of the given artist a; next an operation of updating switch variables is performed to update the value of the switch variable assigned to the given word using a random number generator having a deviation of appearance probability corresponding to the probability distribution over values of the switch variables; and the operation of updating and learning values of switch variables, which performs the operation of updating values of switch variables on all of the plurality of words included in the plurality of lyrics data of each of the plurality of artists, is performed for a predetermined number of times.

(110) (3) The method for generating topic inference information of lyrics as described in (1), wherein:

(111) in the step of learning topic numbers, it is assumed that topic numbers assigned to all of the plurality of lyrics but the topic number assigned to the given lyrics data of the given artist are correct when generating the probability distribution over topic numbers.

(112) (4) The method for generating topic inference information of lyrics as described in (2), wherein:

(113) in the step of learning values of switch variables, it is assumed that values of switch variables assigned to all of words but the value of the switch variable x assigned to the given word in the plurality of words of the given lyrics data of the given artist are correct when performing the operation of updating switch variables.

(114) (5) The method for generating topic inference information of lyrics as described in (1), wherein the step of learning topic numbers:

(115) calculates a first probability p.sub.1 that the topic number of the given lyrics data S.sub.ar is k, based on the number R.sub.ak of lyrics data other than the given lyrics data S.sub.ar of the given artist a when generating a probability distribution over topic numbers;

(116) calculates a second probability p.sub.2 that the topic number of the given lyrics data S.sub.ar is k, based on the number N.sub.kv of times that the topic number k is assigned to the word v in the plurality of lyrics data of the plurality of artists other than the given lyrics data S.sub.ar;

(117) calculates the probability p that the topic number of the given lyrics data S.sub.ar is k, from the first probability p.sub.1 and the second probability p.sub.2; and

(118) determines a probability distribution over topic numbers of the given lyrics data S.sub.ar by performing the above-identified calculations on all of the topic numbers and normalizing probabilities that the topic number of the given lyrics data S.sub.ar is any one of 1 to K such that normalized probabilities sum up to 1 (one).

(119) (6) The method for generating topic inference information of lyrics as described in (1), wherein the outputting step is configured to output a probability distribution over words for each topic number, based on the number N.sub.kv of times that the topic number k is assigned to a given word v as used in the step of calculating the second probability p.sub.2.
(7) The method for generating topic inference information of lyrics as described in (6), wherein:

(120) in the outputting step, an occurrence probability θ.sub.kv of a word v to which the topic number k is assigned is calculated as follows:
θ.sub.kv=(N.sub.kv+β)/(N.sub.k+β|V|)

(121) where N.sub.kv denotes a number of times that a topic number k is assigned to a given word v, N.sub.k denotes a number of all of words to which the topic number k is assigned, β denotes a smoothing parameter, and |V| denotes a number of kinds of words.

(122) (8) The method for generating topic inference information of lyrics as described in (2), wherein the step of learning values of switch variables:

(123) calculates a third probability p.sub.3 that the value of switch variable of the word v.sub.arj is 0 (zero), based on a number N.sub.a0 of words to which a value of 0 (zero) is assigned as the value of the switch variable in all of lyric data of all of songs of the given artist a;

(124) calculates a fourth probability p.sub.4 that the value of the switch variable of the word v.sub.arj is 0 (zero), based on a number Nz.sub.arv.sub.arj of times that 0 (zero) is assigned to the value of the switch variable of the word v.sub.an in all of sons of all of artists to which the same topic number Z.sub.ar as the lyrics including the word v.sub.arj is assigned;

(125) calculates a fifth probability p.sub.5 that the value of the switch variable is 0 (zero) from the third probability p.sub.3 and the fourth probability p.sub.4;

(126) calculates a sixth probability p.sub.6 that the value of the switch variable of the word v.sub.arj is 1 (one), based on a number N.sub.a1 of times that 1 (one) is assigned as the value of the switch variable in the plurality of lyrics data of the given artist;

(127) calculates a seventh probability p.sub.7 that the value of the switch variable of the word v.sub.arj is 1 (one), based on a number N.sub.1varj of times that 1 (one) is assigned as the value of the switch variable of the word v.sub.arj in the plurality of lyrics data of the plurality of artists;

(128) calculates an eighth probability p.sub.8 that the value of switch variable is 1 (one) from the sixth probability p.sub.6 and the seventh probability p.sub.7; and

(129) normalize the probabilities from the fifth probability p.sub.5 and the eighth probability p.sub.8 such that a sum of the probability that the value of the switch variable of the word v.sub.arj is 0 (zero) and the probability that the value of the switch variable of the word v.sub.arj is 1 (one) is 1 (one) to obtain a probability distribution over values of switch variables.

(130) (9) The method for generating topic inference information of lyrics as described in (1), wherein:

(131) the topic number of each of the plurality of lyrics data in the outputting means is a topic number that is last assigned to each of the plurality of lyrics data after the operation of updating and learning topic numbers is performed for a predetermined number of times in the step of learning topic numbers.

(132) (10) The method for generating topic inference information of lyrics as described in (1) or (2), further comprises:

(133) a first step of generating a word probability distribution over words included in lyrics data of a new song s of an artist that has not been used in learning;

(134) a second step of generating a word probability distributions over words included respectively in lyrics data of the plurality of songs of the plurality of artists;

(135) a step of computing similarities, respectively obtain similarities between the probability distribution of the words included in the lyrics data of the new song s as calculated by the first step of generating a word probability and the probability distributions over words included in the lyrics data of the plurality of songs as calculated by the second step of generating word probability distributions;

(136) a step of generating a weight distribution by adding the similarities of the lyrics data of the plurality of songs corresponding to the lyrics data of the plurality of songs to the topic numbers as a weight; and

(137) a step of determining a topic number, determining a topic number having a largest weight as the topic number of the lyrics data of the new song s.

(138) (11) The method for generating topic inference information of lyrics as described in (10), further comprises:

(139) a third step of generating a word probability distribution over words included in the lyric data of all of songs of the artist that have not been used in learning and for which an occurrence probability of background words are to be calculated;

(140) a fourth step of generating word probability distributions over words in the lyrics data of all of the songs of each of the artist;

(141) a fifth step of generating a probability distribution over background words included in the lyrics data of all of songs of each of the artists;

(142) a step of computing similarities, respectively obtaining similarities between the probability distribution over words included in the lyrics data of the new song s as calculated by the third step of generating a word probability and the probability distributions over words included in the lyrics data of the plurality of songs as calculated by the forth step of generating word probability distributions; and

(143) a step of generating an occurrence probability distribution over background words, multiplying the respective probability distributions over the background words included in the lyrics data of all of the songs of each of the artists as calculated by the fifth step of generating a word probability distribution by the similarities of each of the artists as computed by the step of computing similarities to obtain probability distributions, and normalizing the obtained probability distributions such that the weights for each of the artists sum up to 1 (one), and then determining a resulting probability distribution as the occurrence probability distribution over background words.

(144) (12) A computer program for implementing the steps of the method for generating topic inference information of lyrics as described in any one of (1) to (11) using a computer.

(145) (13) The computer program for generating topic inference information of lyrics as described in (12) is recorded in a computer-readable medium.

INDUSTRIAL APPLICABILITY

(146) According to the present invention, once an arbitrary number of topics are determined, the respective topic numbers are identified for a plurality of lyrics data with the topic numbers for the lyrics data that are finally updated by the means of learning topic numbers. Once the topic number of each of the lyrics data is grasped, a word probability distribution can be known for each topic number. This accordingly eliminates the need of manually specifying a collection of words related to the topics and a collection of unrelated words. Further, once a plurality of words having a high occurrence probability are grasped, reliable information for determining the topics can be obtained from the thus grasped words, thereby obtaining likely meaning of the topic of each lyrics.

DESCRIPTION OF REFERENCE SIGNS

(147) 1, 101 System for generating topic inference information of lyrics 3, 103 Lyrics database 5, 1 05 Means for obtaining topic numbers 7, 107 Means for generating topic numbers 9, 109 Means for learning topic numbers 11, 111 Analysis means 13, 113 Outputting means 115 Means for learning values of switch variables