A stick device that includes a base and a tip end, and a tip secured to the tip end of the stick, the stick tip including a sensor. The stick including the base thereof, and includes at least one control button, a communication element, and a processor in communication with the at least one control button, the stick tip and the communication element. The processor is configured to receive a signal from the stick tip and to generate output to the communication element. The output so generated includes a signal that specifies a sound file selected by operation of the at least one control button.

Provided is a musical instrument controller capable of accurately controlling a musical sound parameter. This musical instrument controller includes: a reception means for receiving, from a musical performance device, a sound emission start signal transmitted on the basis of a musical performance operation; a sensor for detecting an amount of displacement from a reference position; and a control means for generating a control signal on the basis of the amount of displacement from the reference position and transmitting the control signal to a sound generation device. The control means sets the reference position on the basis of the sound emission start signal received from the musical performance device.

A method for generating virtual and real composite image data includes: acquiring captured image data capturing an image of a real space as seen from a user's point of view; inputting the captured image data into a trained model, the training model outputting segmentation data segmenting the captured image data into a first region in which a target object is displayed, a second region in which at least a part of the user's body is displayed, and a third region that is other than the first and second regions; and compositing data of the first region and data of the second region with a virtual space image data based on the segmentation data.

Provided are an electric musical instrument system and a control method thereof. The electric musical instrument system includes a musical sound transmitting means having a first mode and a second mode that consumes less electric power than the first mode. The musical sound transmitting means has a detecting means for detecting rocking of a musical instrument and a transmitting means that transmits an audio signal emitted from the musical instrument to the musical sound receiving means in the first mode and transmits, when detecting rocking of the musical instruments, detected information to the musical sound receiving means in the second mode. The musical sound receiving means has a controlling means that processes the audio signal received in the first mode and cancels the second mode and changes the mode to the first mode when the detected information is received in the second mode.

Augmented reality is used to visualize musical notes or groups of musical notes for example using a display proximate a user's eyes, for example where images in the display appear to be in a space in front of or around the user, for example where the images in the display are for manipulation by an augmented reality controller, for example where the augmented reality controller is held by the user.

A percussion device includes a drumstick assembly. The drumstick assembly includes a drumstick having a base and a tip end, and a drumstick tip secured to the tip end of the drumstick, the drumstick tip including a sensor. The drumstick including the base thereof, and includes at least one control button, a communication element, and a processor in communication with the at least one control button, the drumstick tip and the communication element. The processor is configured to receive a signal from the drumstick tip and to generate output to the communication element. The output so generated includes a signal that specifies a sound file selected by operation of the at least one control button.

Systems for a touchless musical instrument, where the touchless musical instrument includes: multiple sensors, where each sensor detects a presence of an object within a predetermined distance of the sensor; an audio speaker; a processor electrically connected to the sensors and the audio speaker; and executes instructions on the process to: receive, from the plurality of sensors, an object detection signal indicating a location of the object with respect to the plurality of sensors; generate an audio output signal based on the location of the object indicated by the object detection signal; and generate an audible output from the audio speaker by transmitting the audio output signal to the audio speaker.

A role simulation method and a terminal apparatus in a VR scene are provided. The role simulation method in the VR scene includes: obtaining a role obtaining instruction triggered by a hand model in the VR scene, movement of the hand model in the VR scene being controlled by an interaction controller; determining a virtual role from the VR scene according to the role obtaining instruction; dynamically adjusting virtual props in the VR scene according to a music file played in the VR scene; displaying the virtual role in the VR scene through a VR display, and displaying the virtual props that are dynamically adjusted.

Gesture-controlled virtual reality systems and methods of controlling the same are disclosed herein. An example apparatus includes an on-body sensor to output first signals associated with at least one of movement of a body part of a user or a position of the body part relative to a virtual object and an off-body sensor to output second signals associated with at least one of the movement or the position relative to the virtual object. The apparatus also includes at least one processor to generate gesture data based on at least one of the first or second signals, generate position data based on at least one of the first or second signals, determine an intended action of the user relative to the virtual object based on the position data and the gesture data, and generate an output of the virtual object in response to the intended action.

Assemblies and techniques are described herein for use with a data glove. The data glove includes sensors configured to translate movement and forces associated with a human hand to the digital domain.