Embodiments of the present technology are directed at features and functionalities of a VR/AR enabled digital audio workstation. The disclosed audio workstation can be configured to allow users to record, produce, mix, and edit audio in virtual 3D space based on detecting and manipulating human gestures in a virtual reality environment. The audio can relate to music, voice, background noise, speeches, background noise, one or more musical instruments, special effects music, electronic humming or noise from electrical/mechanical equipment, or any other type of audio.

The invention provides a system and a computer-implemented method for generating and editing a video including providing a mobile communication device comprising a camera, a display, a central processing unit (CPU), a video generating application and a memory. Next, starting the video generating application, and then opening the camera and providing camera tutorials. The camera tutorials comprise instructions for camera positioning, camera moving, and camera aligning while taking videos. Next, taking videos of a scene following the instructions for camera positioning, camera moving, and camera aligning while taking videos. Next, uploading the videos to the memory, editing the videos and producing a composite video for the scene. The camera tutorials include a “moving forward/backward” tutorial directing a user first to hold the camera still, to align a horizontal view line in the display with a marker line, and then to move the user's body forward or backward while taking a video of the scene. The editing of the videos includes slowing the videos down, and matching rhythm of music accompanying each video to transitions of consecutive videos. The slowing down of the videos includes removing every other frame.

The present invention is designed to allow easy synchronization of the movement of a computer graphics (CG) model with sound data. The data structure according to an embodiment of the present invention presents a data structure that relates to a computer graphics (CG) model, including first time-series information for designating the coordinates of the components of the CG model on a per beat basis, and the first time-series information is used on a computer to process the CG model.

A system for generating a signal includes a touchpad including touch cells and a touch detection device for detecting the location and intensity of a pressure exerted on the touchpad; a first computer generating a first instruction based on the location and intensity of the pressure; an optical detection device for detecting a movement and/or a position, including optics for capturing images; a second computer for determining a motion parameter based on the captured images and for generating a second instruction based on the parameter; and a signal generator for producing a second signal based on the first instruction or on a first signal extracted from the first instruction, to which there is applied a special effect extracted from the second instruction; or on the second instruction or on a first signal extracted from the second instruction, to which there is applied a special effect extracted from the first instruction.

A method and system for teaching oneself to learn to play a string instrument and master it by analyzing ones real-time hand/finger movement/technique/form, enabling oneself to progress and correct ones own mistakes simultaneously. In one aspect, a system for learning to play a string instrument is provided that includes a simulation instrument that includes a plurality of strings, wherein at least one of the strings includes at least one-touch sensing sensor thereon in communication with at least one processor to receive a signal from the at least one touch-sensing sensor and determine therefrom when and where on the at least one string a user applies pressure to the at least one string.

A reproduction control method, which is executed by a computer, includes determining, based on an image showing an object, whether a type of the object is a first type or a second type that is different from the first type, and reproducing a sound, triggered by a striking of an operation surface by the object, based on a result of the determining.

A trained model establishment method realized by a computer includes acquiring a plurality of datasets each of which is formed by a combination of first performance data of a first performance by a performer, second performance data of a second performance performed together with the first performance, and a satisfaction label indicating a degree of satisfaction of the performer, and executing machine learning of a satisfaction estimation model by using the plurality of datasets. In the machine learning, the satisfaction estimation model is trained such that, for each of the datasets, a result of estimating a degree of satisfaction the performer from the first performance data and the second performance data matches the degree of the satisfaction indicated by the satisfaction label.

A performance agent training method realized by at least one computer includes observing a first performance of a musical piece by a performer, generating, by a performance agent, performance data of a second performance to be performed in parallel with the first performance, outputting the performance data such that the second performance is performed in parallel with the first performance of the performer, acquiring a degree of satisfaction of the performer with respect to the second performance performed based on the output performance data, and training the performance agent by reinforcement learning, using the degree of satisfaction as a reward.

Some examples described in this disclosure are performed at a first electronic device (e.g., a computer system) that is in communication with a display and one or more input devices. In some examples, the first electronic device detects a change in a physical environment of the first electronic device due to movement of one or more physical objects in the physical environment indicative of a user input. In some examples, the first electronic device performs a first action at the first electronic device or at a second electronic device in communication with the first electronic device in accordance with the change in the physical environment due to movement of the one or more physical objects in the physical environment.

A computer-implemented method includes obtaining a trained model trained to store a relationship between first performance data and first evaluation data. The first performance data indicates a performance performed by a performer. The first evaluation data indicates a first evaluation of the performance. The first evaluation has been made by an audience who has received the performance. The method also includes obtaining second performance data. The method also includes processing the second performance data using the trained model to make an inference of a second evaluation of the second performance data. The method also includes outputting second evaluation data indicating the inference of the second evaluation.