Systems, devices, media, and methods are described for presenting a tutorial in augmented reality on the display of a smart eyewear device. The system includes a marker registration utility for setting a marker on a musical instrument, a localization utility for locating the eyewear device relative to the marker location and the instrument, a virtual object rendering utility for presenting a series of virtual tutorial objects on the display near one or more actuators on the instrument, and a hand tracking utility for tracking the performer's finger locations in real time during playback of a song file. A high-definition video camera captures sequences of frames of video data. The series of virtual tutorial objects, in one example, includes graphical elements presented on a virtual scroll that appears to move toward the instrument at a speed correlated with the song tempo. The hand tracking utility calculates a set of expected fingertip coordinates based on a detected hand shape and a library of hand poses and landmarks.

Methods and systems are disclosed to facilitate creating the sensation of vibrotactile movement on the body of a user. Vibratory motors are used to generate a haptic language for music or other stimuli that is integrated into wearable technology. The disclosed system in certain embodiments enables the creation of a family of devices that allow people such as those with hearing impairments to experience sounds such as music or other input to the system. For example, a “sound vest” or other wearable array transforms musical input to haptic signals so that users can experience their favorite music in a unique way, and can also recognize auditory or other cues in the user's real or virtual reality environment and convey this information to the user using haptic signals.

Disclosed is a method of providing a music creation interface using a head-mounted device, including displaying first and second geometric loops fixed relative to a location in the real world, the first and second geometric loops each including a plurality of beat indicators. The second geometric loop is spaced apart from the first geometric loop. An interface comprising a plurality of sound or note icons is displayed, and in response to receiving user selection to move a selected sound or note icon to a particular beat indicator on one of the geometric loops, the selected sound or note icon is displayed at the particular beat indicator. In use, the geometric loops are rotated relative to at least one play indicator, and the selected sound or note icon is rendered when it reaches the at least one play indicator.

One or more accelerometers embedded with an earbud and/or a set of earphones are able to sense a moving pace of a user. Based on a moving pace of the user, a signal is sent to a remotely connected electronic device. The electronic device is able to separately increase and decrease a beat or rhythm of the audio from the electronic device based on a pace of the user. In some embodiments, an audio alert is sent to the user to inform the user of pace and whether the user has increased or decreased their pace. Additionally, in some embodiments, a program stored on the electronic device is used to compare the user's current progress and/or speed based on past runs and workouts.

Systems, devices, media, and methods are presented for playing audio sounds, such as music, on a portable electronic device using a digital color image of a note matrix on a map. A computer vision engine, in an example implementation, includes a mapping module, a color detection module, and a music playback module. The camera captures a color image of the map, including a marker and a note matrix. Based on the color image, the computer vision engine detects a token color value associated with each field. Each token color value is associated with a sound sample from a specific musical instrument. A global state map is stored in memory, including the token color value and location of each field in the note matrix. The music playback module, for each column, in order, plays the notes associated with one or more the rows, using the corresponding sound sample, according to the global state map.

An arrangement for the conversion of at least one detected force from the movement of a sensing unit into an auditory signal. The arrangement includes at least one sensor for generating a force signal from the at least one detected force. A processing unit is configured for converting the force signal into a digital auditory signal. An output unit for converting the digital auditory signal into an auditory signal is further included wherein the digital auditory signal includes in formation of acceleration, strength and duration of a single detected force. The present method is used for converting at least one detected force affecting an object into auditory signal, as well as the use of an arrangement according to the present invention for various entertainment and/or therapeutic purposes.

One or more accelerometers embedded with an earbud and/or a set of earphones are able to sense a moving pace of a user. Based on a moving pace of the user, a signal is sent to a remotely connected electronic device. The electronic device is able to separately increase and decrease a beat or rhythm of the audio from the electronic device based on a pace of the user. In some embodiments, an audio alert is sent to the user to inform the user of pace and whether the user has increased or decreased their pace. Additionally, in some embodiments, a program stored on the electronic device is used to compare the user's current progress and/or speed based on past runs and workouts.

A system of playing media content items determines transitions between pairs of media content items by determining desirable locations in which transitions across the pairs of media content items occur. The system uses a plurality of track features of media content items and determines such track features of each media content item associated with each of transition point candidates, such as beat positions, of that media content item. The system determines similarity in the plurality of track features between the transition point candidates of a first media content item and the transition point candidates for a second media content item being played subsequent to the first media content item. The transition points or portions of the first and second media content items are selected from the transition point candidates for the first and second media content items based on the similarity results.

Vibratory motors are used to generate a haptic language for music or other sound that is integrated into wearable technology. The disclosed system enables the creation of a family of devices that allow people with hearing impairments to experience sounds such as music or other auditory input to the system. For example, a “sound vest” or one or more straps comprising a set of motors transforms musical input to haptic signals so that users can experience their favorite music in a unique way, and can also recognize auditory cues in the user's everyday environment and convey this information to the user using haptic signals.

A toroidal actuator is responsive to a teleoperator for providing haptic feedback responsive to a curvature or articulated movement applied to a gripped object. The toroidal actuator surrounds an operator member such as a finger, and is responsive to pneumatic pressure for increasing telepresence force defined by resistance encounter against a teleoperated robotic claw. As the teleoperated claw grips an object, increased pneumatic pressure in the toroidal actuator tends to elongate the toroidal shape in a linear manner and oppose a curvature force applied by an inserted operator finger. Resistive force is based on soft sensing of the gripped object, thus the toroidal actuator applies an increasing resistive force to curvature as the robotic claw closes around a gripped object by solenoid regulated air pressure. A typical assembly includes at least 3 toroidal actuators for two digits and a thumb of an operator.