A method relating to synchronize MIDI file with video includes acquiring a video and a MIDI file, and identifying timing of a video frame. The method also includes converting timing into tick information and editing a tick of the MIDI file. The method further includes detecting the MIDI file corresponding to the video frame, and playing a musical instrument based on the MIDI file corresponding to the video.

The present disclosure provides a method for generating a video of a body moving in synchronization with music by applying a first artificial neural network (ANN) to a sequence of samples of an audio waveform of the music to generate a first latent vector describing the waveform and a sequence of coordinates of points of body parts of the body, by applying a first stage of a second ANN to the sequence of coordinates to generate a second latent vector describing movement of the body, by applying a second stage of the second ANN to static images of a person in a plurality of different poses to generate a third latent vector describing an appearance of the person, and by applying a third stage of the second ANN to the first latent vector, the second latent vector, and the third latent vector to generate the video.

The present disclosure provides a method for generating a video of a body moving in synchronization with music by applying a first artificial neural network (ANN) to a sequence of samples of an audio waveform of the music to generate a first latent vector describing the waveform and a sequence of coordinates of points of body parts of the body, by applying a first stage of a second ANN to the sequence of coordinates to generate a second latent vector describing movement of the body, by applying a second stage of the second ANN to static images of a person in a plurality of different poses to generate a third latent vector describing an appearance of the person, and by applying a third stage of the second ANN to the first latent vector, the second latent vector, and the third latent vector to generate the video.

The present disclosure relates to a method and system for providing tutorial functions. The method may include obtaining a piece of music selected by a user; determining a display mode that configures one or more display categories and a layout of the one or more display categories on a user interface; obtaining one or more content items corresponding to the one or more display categories, respectively, the one or more content items being associated with the piece of music; and generating an instruction to display the one or more content items on the user interface according to the layout.

Exemplary embodiments relate to the application of media effects such as facial mask overlays, to visual data (such as a video or photo). Publicly-available images may be found and mapped to a mask. In the mapping process, a user may type in the name of a celebrity or public figure, and a system may perform a public image search. In some embodiments, candidate images may be filtered in order to remove images unsuitable for use in masks. Typically, only a single forward-facing image is required for mapping. However, multiple images may be used to provide different angles and allow the user to turn their head while the mask is applied. Mask generation may involve: extracting facial features from the image; mapping the facial features to the user's video; blending/recoloring of either or both of the image or the person's face; and applying the mask in real-time/on the fly.

Due to discrepancies in musical timing signatures, the invention assesses whether a recorded displacement, expressed in terms of beats and fractions, between exit and entry points for a potential musical splice or cut, corresponds to permit a seamless music splicing of different musical sections. Assessment is achieved by establishing a third time base of pulses having a length dependent upon a lowest common multiple of fractions within respective bars for different sections, with the bars of the respective sections then partitioned into an equal number of fixed length pulses. A coefficient aligns different time signatures; it is a ratio between pulses within the different sections. The coefficient identifies corresponding locations of a cut point, related to a suitable anacrusis, in terms of respectively an aligned bar, beat, quaver and fraction in differing time signatures. The coefficient ensures that the time anacrusis in one time signature is interchangeable with others.

For example, provided is an information processing device with which the entire performance can be felt really visually. The information processing device TB according to the present invention includes a processor performing a reception process of receiving input of performance information including pitch information, a first image output process of outputting a first image according to the received performance information, a performance determination process of determining at least any of tonality, a chord type, and a pitch name on the basis of the received performance information, and second image output process of outputting a second image according to a result determined in the performance determination process.

Systems and methods are provided for assembling and displaying a visual ensemble of musical performances that were created and uploaded from one or more locations that are remote from a host of the network, a director or other administrator reviewing submissions for selection and assembly, or perhaps merely remote from one or more other submissions received over a computer network. The assembled performances include a plurality of submissions, the submissions including performances created and uploaded at one or more locations remote from the location of the director for the assembly and display over the computer network. Systems and methods are also included for mapping one performance against another performance qualitatively, quantitatively, in real-time, or some combination thereof, enabling a musician, or a reviewer of performances, in the assessment of one performance relative to another performance.

A computer implemented media synchronization platform has a receiver that receives, from a user-operated computing device located at a live performance, a device media file. Furthermore, the computer implemented media synchronization platform receives, from a soundboard positioned at the live event, a soundboard audio file. The computer implemented media synchronization platform synchronizes, and receives, from the user-operated computing device via a synchronization interface, a realignment of a soundboard audio track. The device media file has a device video track and the device audio track. The soundboard audio file has the soundboard audio track. The computer implemented platform media synchronization platform has a processor that synchronizes the soundboard audio track with the video track, generates dual audio track A/V data based on the synchronization, generates the synchronization interface, and generates a single audio track A/V file based on the realignment. The processor is remotely positioned from the live performance.

In some implementations, a method includes: obtaining user movement information, wherein the user movement information characterizes real-world body pose and trajectory information of the user; generating, from real-world user movement information and a predetermined placement of the virtual instrument in the computer generated reality (CGR) environment, a predicted virtual instrument interaction time for a virtual instrument interaction prior to the virtual instrument interaction occurring; determining whether or not the predicted virtual instrument interaction time falls within an acceptable temporal range around one of a plurality of temporal sound markers; and in response to determining that the predicted virtual instrument interaction time falls within the acceptable temporal range around a particular temporal sound marker of the plurality of temporal sound markers, quantizing the virtual instrument interaction by presenting play of the virtual instrument to match the particular temporal sound marker of the plurality of temporal sound markers.