Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

An audio signal analysis system comprises an electronic controller that is configured to execute a plurality of modules including an acquisition module configured to acquire a first spectrum, which is a time average of a plurality of frequency spectra of an audio signal, a specification module configured to acquire a plurality of reference values corresponding to different pitches that follow a prescribed temperament and configured to specify, by a problem-solving search algorithm, a frequency difference corresponding to a second spectrum which includes a plurality of components each having a frequency difference with respect to each of the plurality of reference values, the second spectrum being similar to the first spectrum with a degree of similarity exceeding a prescribed threshold value, and a correction module configured to correct the frequency difference so as to reduce systematic error included in the frequency difference specified by the specification module.

Disclosed herein are method, system, and computer program product embodiments for performing the continuous tuning of received audio input from an earpiece or microphone especially customized for karaoke singing, wherein the audio input may be mixed with user-selected song input, and the joint mixed input is independently altered in the frequency domain for output to an earpiece worn by a user as well as separately for an additional audio output to an external connected speaker, for an optimal karaoke experience.

Despite many practical limitations imposed by mobile device platforms and application execution environments, vocal musical performances may be captured and continuously pitch-corrected for mixing and rendering with backing tracks in ways that create compelling user experiences. Based on the techniques described herein, even mere amateurs are encouraged to share with friends and family or to collaborate and contribute vocal performances as part of virtual “glee clubs.” In some implementations, these interactions are facilitated through social network- and/or eMail-mediated sharing of performances and invitations to join in a group performance. Using uploaded vocals captured at clients such as a mobile device, a content server (or service) can mediate such virtual glee clubs by manipulating and mixing the uploaded vocal performances of multiple contributing vocalists.

Embodiments described herein relate generally to systems comprising a display device, a display device-coupled computing platform, a mobile device in communication with the computing platform, and a content server in which methods and techniques of capture and/or processing of audiovisual performances are described and, in particular, description of techniques suitable for use in connection with display device connected computing platforms for rendering vocal performance captured by a handheld computing device.

Various methods for representing audio suitable for use in variational audio encoding are disclosed. A method comprises maintaining, by a computing system, state information for multiple resonator models with different resonant frequencies. The method further comprises iteratively performing a number of different operations, by the computing system for multiple respective samples in a set of audio samples in the time domain. These operations include updating the state information for the multiple resonator models based on the sample amplitude. The operations also include determining respective resonator amplitudes and phases for the updated multiple resonator models and storing, respective resonator amplitude and change-in-phase information for the sample.

A pitch adjustment method and apparatus, and a computer storage medium are provided. the fundamental frequency sequence of the singing sound of the user is obtained, the pitch difference between each candidate melody file and the fundamental frequency sequence at each corresponding time point is calculated, and the sum of all pitch differences of each candidate melody file is calculated. The candidate melody file with the minimum sum is determined as the target melody file, and the pitch of the accompaniment file of the target song is adjusted according to the pitch difference between the target melody file and the original melody file of the target song.

Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

To generate a melody, one or more machine-readable constraints are accepted from a user through a user interface. The constraints include rhythm constraints and pitch constraints. A sequence of musical elements is generated based on the constraints, each of the musical elements specifying, in machine-readable data, a musical pitch or silence and a duration of the musical pitch or silence. The pitch constraints prescribe pitches in the sequence of musical elements and the rhythm constraints prescribe rhythm of the sequence of musical elements. The sequence of musical elements is rendered in human-perceivable form as a melody.

A musical performance correction method executable by a computer, the musical performance correction method includes: estimating a playing position in a musical composition by analyzing musical performance data indicating a played pitch played by a user; and correcting the played pitch indicated by the musical performance data to a reference pitch, from among a plurality of reference pitches corresponding to the playing position in the musical composition, in a case where a difference between the played pitch and the reference pitch is less than a predetermined value.