In at least one general aspect, a method can include detecting an interaction with a first object in a virtual reality (VR) environment, producing first contextual information based on user interaction with the first object, determining a second object to display in the VR environment based on the first contextual information, and presenting the second object in the VR environment. The interaction of the user may include at least one of a movement, a speech, or a physiological trait of the user.

Provided is a method, computer program product, and system for predicting a location of an object using machine learning. A processor may monitor a data stream received from one or more observation devices. The processor may detect a gesture initiated by a user from the data stream, the gesture indicating that the user is searching for an object. The processor may identify the object by analyzing a set of contextual data associated with the user. The processor may predict, in response to identifying the object, a location of the object by analyzing historic data associated with the object from the data stream. The processor may output the predicted location of the object to the user.

A display case includes a transparent LCD panel for viewing display case contents therethrough.

A system comprising: a user device, comprising: sensors configured to sense data related to a physical environment of the user device, displays; hardware processors; and a non-transitory machine-readable storage medium encoded with instructions executable by the hardware processors to: place a virtual object in a 3D scene displayed by the second user device, determine a pose of the user device with respect to the physical location in the physical environment of the user device, and generate an image of virtual content based on the pose of the user device with respect to the placed virtual object, wherein the image of the virtual content is projected by the one or more displays of the user device in a predetermined location relative to the physical location in the physical environment of the user device.

A system or method for measuring human ocular performance can be implemented using an eye sensor, a head orientation sensor, an electronic circuit and a display that presents one of virtual reality information, augmented reality information, or synthetic computer-generated 3-dimensional information. The device is configured for measuring saccades, pursuit tracking during visual pursuit, nystagmus, vergence, eyelid closure, or focused position of the eyes. The eye sensor comprises a video camera that senses vertical movement and horizontal movement of at least one eye. The head orientation sensor senses pitch and yaw in the range of frequencies between 0.01 Hertz and 15 Hertz. The system uses a Fourier transform to generate a vertical gain signal and a horizontal gain signal.

A switching method and system of interactive modes of a head-mounted device is provided. The interactive modes include gamepad tracking interactive mode and bare hand tracking interactive mode. A standard deviation of position data, a standard deviation of attitude data and a standard deviation of accelerometer data among IMU data are acquired, respectively. Whether the standard deviation of the position data, the standard deviation of the attitude data and the standard deviation of the accelerometer data meet a first preset condition is determined. Moreover, the standard deviation of the accelerometer data within a second preset duration is acquired in real time, and whether the standard deviation of the accelerometer data meets a second preset condition is determined. In cases where the standard deviation of the accelerometer data meets the second preset condition, the bare hand tracking interactive mode is paused, and the gamepad tracking interactive mode is started.

An electronic device includes a communicator configured to perform communication with an external device, a display configured to display a UI (User Interface) element in a screen, and a processor. The processor receives through the communicator, touch panel information of the external device and first data according to a first input of a user detected on the touch panel of the external device, and changes a location of the UI element displayed on the screen based on the touch panel information and the first data.

An imaging method is provided for a modular mixed reality (MR) device having an MR calculation module, an MR optical path module and an MR posture module. The MR calculation module is configured to adjust display content according to data from the MR posture module. The MR optical path module comprises a virtual-image optical path and a mixed optical path. A semi-transparent semi-reflective mirror is provided in the mixed optical path. One surface of the mirror is a real-image introduction surface facing a real environment, while the other is a virtual-image introduction surface facing the virtual-image optical path. Virtual-image light is reflected by the virtual-image introduction surface onto an observation end and mixed with real environment light transmitted to the observation end by the real-image introduction surface to form a mixed reality image.

In one implementation, a non-transitory computer-readable storage medium stores program instructions computer-executable on a computer to perform operations. The operations include presenting, on a display of an electronic device, first content representing a standard view of a physical setting depicted in image data generated by an image sensor of the electronic device. While presenting the first content, an interaction with an input device of the electronic device is detected that is indicative of a request to present an enriched view of the physical setting. In accordance with detecting the interaction, second content is formed representing the enriched view of the physical setting by applying an enrichment effect that alters or supplements the image data generated by the image sensor. The second content representing the enriched view of the physical setting is presented on the display.

The disclosure relates to a method of delivering a video frame. One implementation may involve spatially partitioning a video frame into a plurality of blocks, encoding at least one of the plurality of blocks of the video frame, and transmitting the at least one of the plurality of blocks of the video frame.