A musical composition in broken down by instrument or channels within the composition. The composition is further broken down by time period. Each distinct block of time by instrument is a discrete block within the musical composition for which a visualization is created based on the sound produced. These blocks are then scrambled, at least partially, by instrument and time and displayed on a grid where each block or instrument can be played by selecting same. The blocks are also movable with a goal of placing the blocks in order by time and instrument so that the entirety of the musical composition can again be played aurally.

An electronic device is provided. The electronic device includes a touch screen display, at least one of a speaker and a sound interface, a processor configured to electrically connect to the touch screen display, the speaker, and the sound interface, and a memory configured to electrically connect to the processor. The memory stores instructions for, when executed, causing the processor to display at least one item comprising a musical instrument shape on the touch screen display, receive a touch input through the touch screen display, load sound data corresponding to the at least one item based on the touch input, process the sound data based at least in part on information associated with the touch input, and output the processed sound data through the speaker or the sound interface.

Systems and methods for creating and presenting sensory stimulating content in a cyber reality environment. One aspect of the disclosure allows a composer to associate audio content with one or more virtual triggers, and to define behavior characteristics which control the functioning of each virtual trigger. Another aspect of the disclosure provides a variety of user interfaces through which a performer can cause content to be presented to an audience.

Embodiments include systems and methods for synthesizing, recording, performing and playing back visual music for virtual immersive video playback environments. Virtual music (VM) compositions can include 3D VM instruments, which can be controlled in a highly dynamic and expressive manner using human interface controllers. Some embodiments include novel techniques for controlling, synthesizing, and rendering VM instruments having complex particle system architectures. Some embodiments further provide VM compositional techniques built on four-dimensional modeling constructs that include path-adaptive coordinate systems that define a compositional space-time for modeling, and path-anchored object locators that place objects in the compositional space-time.

Embodiments include systems and methods for synthesizing, recording, performing and playing back visual music for virtual immersive video playback environments. Virtual music (VM) compositions can include 3D VM instruments, which can be controlled in a highly dynamic and expressive manner using human interface controllers. Some embodiments include novel techniques for controlling, synthesizing, and rendering VM instruments having complex particle system architectures. Some embodiments further provide VM compositional techniques built on four-dimensional modeling constructs that include path-adaptive coordinate systems that define a compositional space-time for modeling, and path-anchored object locators that place objects in the compositional space-time.

An electronic apparatus and/or a controlling method are provided. The electronic apparatus may include a camera for photographing an image, a microphone for receiving an input of a sound of a first channel, and a processor for generating sounds of a plurality of channels based on the input sound, wherein the processor is configured to identify an object and the location of the object from the photographed image, classify the input sound based on an audio source, and allot the sound to the corresponding identified object, copy the classified sound and generate sounds of two channels, adjust characteristics of the generated sounds of two channels based on the audio source allotted to the identified object and the location of the identified object, and mix the sounds of two channels wherein the characteristics were adjusted according to the audio source and generate a stereo sound of two channels.

An electronic device is provided. The electronic device includes a touch screen display, at least one of a speaker and a sound interface, a processor configured to electrically connect to the touch screen display, the speaker, and the sound interface, and a memory configured to electrically connect to the processor. The memory stores instructions for, when executed, causing the processor to display at least one item comprising a musical instrument shape on the touch screen display, receive a touch input through the touch screen display, load sound data corresponding to the at least one item based on the touch input, process the sound data based at least in part on information associated with the touch input, and output the processed sound data through the speaker or the sound interface.

Systems and methods for creating and presenting sensory stimulating content in a cyber reality environment. One aspect of the disclosure allows a composer to associate audio content with one or more virtual triggers, and to define behavior characteristics which control the functioning of each virtual trigger. Another aspect of the disclosure provides a variety of user interfaces through which a performer can cause content to be presented to an audience.

A method and a system for instrument separating and reproducing for a mixture audio source is provided. The method and/or the system includes inputting selected music into an instrument separation model for extracting features therefrom, determining audio source signals of multiple channels for the separation of all instruments, each channel containing sound of one instrument, and transmitting the signals of the different channels to multiple speakers placed at designated positions for playing, which can reproduce or recreate an immersive sound field listening experience for users.

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing Musical Instrument Digital Interface (MIDI) controls within a virtual conferencing system. The program and method provide, in association with designing a room for virtual conferencing, an interface for updating a property of an element within a room based on a received MIDI message; receiving, based on the interface, an indication of user input specifying to update the property of the element when the received MIDI message includes a predefined value; providing a virtual conference between plural participants within the room, the room including the element; receiving a MIDI message that includes the predefined value; and updating, in response to receiving the MIDI message, the property of the element.