The interactive theater system is executed on an interactive gaming computer (IG PC) configured to execute an interactive multi-player video game. The IG PC outputs a video stream to a display system, such as a projector and screen, which allows the players to view the multi-player video game. Each player sits in a multi-sensory seat for providing multi-sensory feedback to each player during execution of the interactive multi-player video game. The player provides input using an input device having an infrared camera and motion sensors. The infrared camera is used to image an infrared light arrangement positioned above-the multi-sensory seats. Infrared camera data and motion sensor data is used by the IG PC to accurately determine the real time position of each input device. Players in a first interactive theater system can compete with players in a second interactive theater system in real time.

A method and device for pointing at an object in a Virtual Reality (VR) scene, and a VR apparatus are provided. The method includes sensing a movement of a gamepad from a first position to a second position in a reality scene through an Inertial Measurement Unit (IMU) orientation sensor. The IMU orientation sensor is mounted in the gamepad, and the gamepad is configured to control an object in a VR scene. The method also includes generating a first position vector based on the movement; acquiring a first absolute position coordinate of the second position in the reality scene according to a position coordinate of a starting point in the reality scene and the first position vector; converting the first absolute position coordinate into a first virtual position coordinate in the VR scene; and pointing at a target object located at the first virtual position coordinate in the VR scene.

A gameplay system for providing interactivity based on a piece of merchandise or other object held or worn by a player. The system includes an object identification assembly, which may be a modified computer vision system, that operates to sense, for a gameplay participant, the presence and identity of a particular augmenting object such as themed merchandise worn or held by a rider of a theme park ride. The sensing may be performed by detecting an on-and-off pattern of light emitted from the object. In response, the gameplay system selects or modifies the interactive features of gameplay associated with the participant based on the identified augmenting object. For example, the interactive features are selected from memory by the gameplay system controller to match a character associated with the augmenting object during any interactive gameplay. This allows the gameplay participant to choose their abilities or powers for gameplay.

A system for detecting an aim point on a display screen targeted by a user comprises a microcomputer that has at least a power source, a processor, a non-volatile memory, an I/O interface, and a display output. A hand held enclosure includes a camera having a field-of-view defined by an X-axis and a Y-axis, a user input, and a microcomputer interface to transmit camera data and user input data to the microcomputer. The aim point is a point located centrally in the field-of-view. The microcomputer is adapted to display two or more reference icons on the display screen, detect the reference icons using the camera, calculate the aim point on the display screen based on the position of the reference icons within the field-of-view of the camera, perform a function based on user input settings and the calculated value of the aim point, and repeat the process as necessary.

A configuration that analyzes reflection light of light emitted to a user face and executes both facial-expression analysis and biometric-signal analysis together is realized. The configuration has a light-receiving section that receives reflection light of light emitted to a user face, and a light-reception-signal analyzing section that analyzes a light-reception signal of the light-receiving section. The light-reception-signal analyzing section has a facial-expression analyzing section that analyzes user-skin-surface reflection light and generates facial-expression analysis information, and a biometric-signal analyzing section that analyzes subepidermal reflection light and generates bioanalysis information. The light-reception signal of the light-receiving section includes skin-surface reflection light and subepidermal-tissue reflection light, and the facial-expression analyzing section extracts a low frequency component from the light-reception signal, acquires the skin-surface reflection light, and executes facial-expression analysis. The biometric-signal analyzing section extracts a high frequency component from the light-reception signal, acquires the subepidermal-tissue reflection light, and executes a biometric-signal analysis process.

Methods and systems are provided for verifying an input provided at a controller including detecting a finger gesture on a surface of the controller. Responsive to detecting the finger gesture, multi-modal data is collected from a plurality of sensors and components tracking the finger gesture. The multi-modal data is used to generate an ensemble model using machine learning algorithm. The ensemble model is trained in accordance to training rules defined for different finger gestures. An output is identified from the ensemble model for the finger gesture. The output is interpreted to define an input for an interactive application selected for interaction.

A method to identify positions of fingers of a hand is described. The method includes capturing images of a first hand using a plurality of cameras that are part of a wearable device. The wearable device is attached to a wrist of a second hand and the plurality of cameras of the wearable device is disposed around the wearable device. The method includes repeating capturing of additional images of the first hand, the images and the additional images captured to produce a stream of captured image data during a session of presenting the virtual environment in a head mounted display (HMD). The method includes sending the stream of captured image data to a computing device that is interfaced with the HMD. The computing device is configured to process the captured image data to identify changes in positions of the fingers of the first hand.

A gaming system comprises a handheld user input device, a processor for receiving input data from the handheld user input device and a gaming application for receiving data from the processor based on the received input data from the handheld user input device. The handheld user input device emulates one of a plurality of different devices associated with the gaming application. Each of the plurality of different devices have a set of input commands associated therewith. The set of input commands associated with the one of a plurality of different devices includes a command based on one of motion and orientation of the handheld device.

An information processing device obtains information regarding the position of each fingertip of a user in a real space, and determines contact between a virtual object set within a virtual space and a finger of the user. The information processing device sets the virtual object in a partly deformed state such that a part of the virtual object, the part corresponding to the position of the finger determined to be in contact with the object among the fingers of the user, is located more to a far side from a user side than the finger, and displays the virtual object having the shape set thereto as an image in the virtual space on a display device.

A system and controller are provided. In one example, the system includes a controller for playing a video game. The controller includes input buttons for controlling interaction during play of the video game. The controller includes an inertial sensor for detecting translational or rotational movement of the controller and a vibrational device. A voice input device is associated with the controller. A wireless communication circuit of the controller is provided for communicating data between the controller and a local computing device. The local computing device is connectable to a network. The data includes control responsive to presses of said input buttons or control received from said voice input device.