A system is provided, including the following: an input device, the input device including, a light emitting diode (LED) array; an image capture device configured to capture images of an interactive environment in which the input device is disposed, the image capture device configured to generate captured image data that is processed to determine a movement of the input device in the interactive environment; wherein a gearing is established that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display, the gearing changing to different settings during the movement of the image as the movement of the image is rendered to the display.

Techniques for assigning a gesture dictionary in a gesture-based system to a user comprise capturing data representative of a user in a physical space. In a gesture-based system, gestures may control aspects of a computing environment or application, where the gestures may be derived from a user's position or movement in a physical space. In an example embodiment, the system may monitor a user's gestures and select a particular gesture dictionary in response to the manner in which the user performs the gestures. The gesture dictionary may be assigned in real time with respect to the capture of the data representative of a user's gesture. The system may generate calibration tests for assigning a gesture dictionary. The system may track the user during a set of short gesture calibration tests and assign the gesture dictionary based on a compilation of the data captured that represents the user's gestures.

A method for managing multi-player interactivity with an interactive application is provided, including the following operations: determining locations of a first controller, a second controller, a first user, and a second user; pairing the first controller to the first user based on the location of the first controller and the location of the first user; pairing the second controller to the second user based on the location of the second controller and the location of the second user; presenting a split-screen view of the interactive application, wherein presenting the split-screen view includes setting positions of a first view and a second view within the split-screen view based on one or more of the locations of the first user, the second user, the first controller, or the second controller.

A touch operation management unit detects, at prescribed time intervals, whether or not a touch operation has been performed on a touch operation input unit, and manages the touch operation. A state detection unit detects various kinds of states of a swipe operation performed by the player. A movement instruction acceptance unit accepts a movement instruction among various kinds of touch operations managed by the touch operation management unit. A skill instruction acceptance unit accepts a skill-related instruction among various kinds of touch operations detected by the touch operation management unit. In addition, the skill instruction acceptance unit accepts the designation of a prescribed direction in accordance with the movement direction of a touched position when the swipe transitions from the second state to the third state. A display image generation unit generates a display image for displaying various kinds of controllers. A display control unit executes control for displaying display images for various kinds of games on the display unit

Systems, apparatus and methods for enabling a mobile station to control a virtual world being executed on a console system are described. The method can include establishing a communication link between the mobile station and a console system, identifying user interface characteristics of the mobile station, and providing a controller application to the mobile station, the controller application being based on the identified user interface characteristics and configured to transform user interface inputs into controller commands. The method can further include receiving signals containing data representing the controller commands from the mobile station over the communication link and determining a sequence of events of the virtual world based on the received controller commands. In one aspect, the controller application can execute in the context of an application on the mobile station.

An example system and method is provided for detecting a moment of impact and/or strength of a swing based on moving a hand-held device including an accelerometer arrangement. A moment and a magnitude of simulated striking of the object are determined based on one or more accelerometer arrangement outputs resulting from the moving of the hand-held device. Using one or more of aural, visual and tactile outputs, the striking of the object is simulated in accordance with the determined moment of simulated striking and the determined magnitude of the simulated striking.

An apparatus for detecting user interaction or another input with a projected image includes a scanning mechanism. The scanning mechanism is capable of scanning the at least one input region for any objects therein. An input identification mechanism may identify the input identified by the object, movement of the object, or a change in the object's position. That information may be used to alter the projected display.

Computer implemented methods, apparatus and systems for enabling interactive interfacing with a computer game program are provided. One system includes an input device to accept and transmit user input to the computer game program. The system also includes a computing device for executing the computer game having an interactive component that is responsive to the user input. The computing devices is in communication with the input device for receiving and processing the user input. The computer device is also configured to apply a gearing parameter during processing of the user input, wherein the applied gearing parameter defines a scaling ratio between the user input and a rate at which the interactive component responds during execution of the computer game.

Embodiments of the present invention enable rich control input data to control video games that are remotely executed. Rich control input includes three-dimensional image data, color video, audio, device orientation data, and touch input. A remotely-executed video game is one executed on a server or other computing device that is networked to a client device receiving the rich control input. Rich control input includes more data than can be uploaded to a game server without degrading game performance. Embodiments of the present invention preprocess the rich control data on the client and into data that may be uploaded to the game server. The rich input stream may be processed in a general way or in a game-specific way.

A tracking system is disclosed. The system may comprise a processor and a non-transitory computer-readable storage medium coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform a method. The method may comprise training a detection model and an extraction model, capturing one or more images of at least a portion of an object, detecting the portion of the object in each of the one or more images through the trained detection model, tracking the detected portion of the object in real-time, obtaining 2D positions of one or more locations on the tracked portion of the object through the trained extraction model, and obtaining 3D positions of the one or more locations on the tracked portion of the object based at least in part on the obtained 2D positions.