In accordance with an embodiment, described herein is a system and method for providing a live lyrics overlay in a social messaging environment. The system can utilize advances in three-dimensional mapping technology that allow social messaging services, to offer real time video lenses or overlays to their users, and extends this three-dimensional mapping technology to support for lyrics. During creation of a video with lyrics lens overlay, the lyrics corresponding to a selected song are retrieved from a lyrics source, and are displayed within the video. For example, with the lyrics lens, a user can record an image of themselves on live video, singing along to a song clip, with the lyrics of the song displayed as if they appear to be coming from their mouths. The created live lyrics content can also be shared with other users of a social messaging environment.

Visual effects schedules are applied to audiovisual performances with differing visual effects applied in correspondence with differing elements of musical structure. Segmentation techniques applied to one or more audio tracks (e.g., vocal or backing tracks) are used to compute some of the components of the musical structure. In some cases, applied visual effects schedules are mood-denominated and may be selected by a performer as a component of his or her visual expression or determined from an audiovisual performance using machine learning techniques.

Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

In general terms the present disclosure proposes computer-implemented method for context-based scrolling of tablature. The computer-implemented method comprises receiving, at computer, tablature from device at which tablature is stored or is generated, wherein computer comprises at least one processor, determining structure of tablature using at least one feature recognition algorithm, using at least one processor, determining one or more parameters of display environment, using at least one processor, building abstract syntax tree of tablature comprising an array of structural elements of tablature, using at least one processor, determining scrolling time period in which portion of the tablature that is displayed on given region of a viewport of display environment is to be replaced by a next portion of tablature, using at least one processor, and scrolling tablature according to at least abstract syntax tree of tablature and scrolling time period, in display environment, using the at least one processor.

Techniques have been developed to facilitate the livestreaming of group audiovisual performances. Audiovisual performances including vocal music are captured and coordinated with performances of other users in ways that can create compelling user and listener experiences. For example, in some cases or embodiments, duets with a host performer may be supported in a sing-with-the-artist style audiovisual livestream in which aspiring vocalists request or queue particular songs for a live radio show entertainment format. The developed techniques provide a communications latency-tolerant mechanism for synchronizing vocal performances captured at geographically-separated devices (e.g., at globally-distributed, but network-connected mobile phones or tablets or at audiovisual capture devices geographically separated from a live studio).

Techniques have been developed to facilitate the livestreaming of group audiovisual performances. Audiovisual performances including vocal music are captured and coordinated with performances of other users in ways that can create compelling user and listener experiences. For example, in some cases or embodiments, duets with a host performer may be supported in a sing-with-the-artist style audiovisual livestream in which aspiring vocalists request or queue particular songs for a live radio show entertainment format. The developed techniques provide a communications latency-tolerant mechanism for synchronizing vocal performances captured at geographically-separated devices (e.g., at globally-distributed, but network-connected mobile phones or tablets or at audiovisual capture devices geographically separated from a live studio).

Vocal audio of a user together with performance synchronized video is captured and coordinated with audiovisual contributions of other users to form composite duet-style or glee club-style or window-paned music video-style audiovisual performances. In some cases, the vocal performances of individual users are captured (together with performance synchronized video) on mobile devices, television-type display and/or set-top box equipment in the context of karaoke-style presentations of lyrics in correspondence with audible renderings of a backing track. Contributions of multiple vocalists are coordinated and mixed in a manner that selects for presentation, at any given time along a given performance timeline, performance synchronized video of one or more of the contributors. Selections are in accord with a visual progression that codes a sequence of visual layouts in correspondence with other coded aspects of a performance score such as pitch tracks, backing audio, lyrics, sections and/or vocal parts.

A method according to one aspect of the present disclosure includes acquiring verbal data representing a verbal expression corresponding to a sound reproduced by an acoustic device, and displaying, on a display device, motion graphics including the verbal expression corresponding to the sound reproduced by the acoustic device in a form of a text in accordance with the verbal data. The displaying the motion graphics on the display device includes selecting a type of motion graphics that relates to the verbal expression corresponding to the reproduced sound from among various types of motion graphics and displaying the selected type of motion graphics on the display device.

The disclosure provides a method and an apparatus for song generation. A text input may be received. A topic and an emotion may be extracted from the text input. A melody may be determined according to the topic and the emotion. Lyrics may be generated according to the melody and the text input. A song may be generated at least according to the melody and the lyrics.

An electronic device includes a microphone, a speaker, a processor operatively connected to the microphone and the speaker, and a memory electrically connected to the processor and storing instructions that, when executed by the processor, cause the processor to receive a sound through the microphone, analyze the received sound, determine a song associated with the sound, and a kind of output sound based on at least in part on a result of the analyzing, generate an output sound based on the kind of output sound, and output the generated output sound through the speaker.