Embodiments of the present technology are directed at features and functionalities of a VR/AR enabled digital audio workstation. The disclosed audio workstation can be configured to allow users to record, produce, mix, and edit audio in virtual 3D space based on detecting and manipulating human gestures in a virtual reality environment. The audio can relate to music, voice, background noise, speeches, background noise, one or more musical instruments, special effects music, electronic humming or noise from electrical/mechanical equipment, or any other type of audio.

Systems for a touchless musical instrument, where the touchless musical instrument includes: multiple sensors, where each sensor detects a presence of an object within a predetermined distance of the sensor; an audio speaker; a processor electrically connected to the sensors and the audio speaker; and executes instructions on the process to: receive, from the plurality of sensors, an object detection signal indicating a location of the object with respect to the plurality of sensors; generate an audio output signal based on the location of the object indicated by the object detection signal; and generate an audible output from the audio speaker by transmitting the audio output signal to the audio speaker.

The present disclosure relates to a virtual drum kit device. The virtual drum kit device includes a motion capture device and a drum sound processing device separated from each other; where, the motion capture device includes at least two drumsticks, the drumstick including a first cover body, as well as a first battery, a first control printed circuit board assembly (PCBA) board and a vibration motor which are arranged in the first cover body, the first control PCBA board being connected to the battery and the vibration motor separately, and being capable of transmitting an RF signal to the drum sound processing device according to a motion instruction captured by the drumstick, the drum sound processing device includes a second cover body, as well as a second battery and a second control PCBA board which are arranged in the second cover body.

A method for generating virtual and real composite image data includes: acquiring captured image data capturing an image of a real space as seen from a user's point of view; inputting the captured image data into a trained model, the training model outputting segmentation data segmenting the captured image data into a first region in which a target object is displayed, a second region in which at least a part of the user's body is displayed, and a third region that is other than the first and second regions; and compositing data of the first region and data of the second region with a virtual space image data based on the segmentation data.

Methods and systems are disclosed for performing operations comprising: receiving a monocular image that includes a depiction of a person wearing an article of clothing; generating a segmentation of the article of clothing worn by the person in the monocular image; obtaining one or more audio-track related augmented reality elements; and applying the one or more audio-track related augmented reality elements to the article of clothing worn by the person based on the segmentation of the article of clothing worn by the person.

Provided are an electric musical instrument system and a control method thereof. The electric musical instrument system includes a musical sound transmitting means and a musical sound receiving means having a first mode and a second mode that consumes less electric power than the first mode. The musical sound transmitting means has a detecting means for detecting rocking of a musical instrument and a transmitting means that transmits an audio signal emitted from the musical instrument to the musical sound receiving means in the first mode and transmits, when detecting rocking of the musical instruments, detected information to the musical sound receiving means in the second mode. The musical sound receiving means has a controlling means that processes the audio signal received in the first mode and cancels the second mode and changes the mode to the first mode when the detected information is received in the second mode.

Provided is a musical instrument controller capable of accurately controlling a musical sound parameter. This musical instrument controller includes: a reception means for receiving, from a musical performance device, a sound emission start signal transmitted on the basis of a musical performance operation; a sensor for detecting an amount of displacement from a reference position; and a control means for generating a control signal on the basis of the amount of displacement from the reference position and transmitting the control signal to a sound generation device. The control means sets the reference position on the basis of the sound emission start signal received from the musical performance device.

Provided are a method and device for processing a music file, a terminal and a storage medium. The method comprises: in response to a received sound effect adjustment instruction, acquiring a music file, the adjustment of which is indicated by the sound effect adjustment instruction; carrying out vocals and accompaniment separation on the music file to obtain vocal data and accompaniment data in the music file; carrying out first sound effect processing on the vocal data to obtain target vocal data, and carrying out second sound effect processing on the accompaniment data to obtain target accompaniment data; and synthesizing the target vocal data and the target accompaniment data to obtain a target music file.

An electronic musical instrument for generating a plurality of audio signals can include an attitude measuring unit that is configured to measure one or more changes in an attitude of the musical instrument with respect to a reference frame in a multi-dimensional space. The musical instrument can also include a processor that is configured to generate one or more signals based on the measured changes in the attitude of the musical instrument. The musical instrument can also include an audio synthesizer that is configured to generate the plurality of audio signals based on the generated signals.

Methods and apparatus to use predicted actions in VR environments are disclosed. An example method includes predicting a predicted time of a predicted virtual contact of a virtual reality controller with a virtual musical instrument, determining, based on at least one parameter of the predicted virtual contact, a characteristic of a virtual sound the musical instrument would make in response to the virtual contact, and initiating producing the sound before the predicted time of the virtual contact of the controller with the musical instrument.