The present invention discloses an interactive control method executed during instant video communication between a user and one or more other users. The method comprises: monitoring video information collected by a camera during the instant video communication between the user and the one or more other users; performing recognition on the video information after acquiring the video information, to acquire user behavior data inputted by the user in a preset manner; determining whether the user behavior data comprises preset trigger information; when it is determined that the user behavior data comprises the preset trigger information, further determining whether the user behavior data comprises a preset gesture action; and when it is determined that the user behavior data comprises the preset gesture action, determining an operation instruction corresponding to the preset gesture action in a preset operation instruction set, and performing an event corresponding to the operation instruction.

A near-eye display assembly presented herein includes an electronic display, an optical assembly, and scanning assembly. The electronic display has a first resolution. The optical assembly controls a field of view at an eye box and directs a plurality of light rays emitting from the electronic display toward the eye box. The scanning assembly shifts a direction of at least one of the light rays in accordance with emission instructions such that a virtual display is presented to the eye box, the virtual display having a second resolution greater than the first resolution. The display assembly can be implemented as a component of a head-mounted display of an artificial reality system.

Aspects of the present invention relate to methods and systems for the see-through computer display systems with adjustable-zoom cameras positioned such that their respective capture fields-of-view at least partially overlap at a target distance.

The invention relates to a device for the determination and analysis of the motor skill and the oculomotor skill of a person (100), with a headset comprising at least the following components: a display unit (9) for displaying an image to the eyes of a person (100), when the headset is mounted on the head of the person (100); an optical sensor system (3, 4, 6) for estimating the position and shape of an object in three-dimensional space and for estimating the position of the head set in three dimensional space, wherein the optical sensor system (3, 4, 6) is arranged and designed for the detection and registration of the hands and fingers of the person (100); an eye-tracking module (8) that is configured to determine a point of gaze of the person (100) wearing the device. The invention furthermore relates to various methods for using the device.

Methods, apparatus, systems, and computer-readable media are set forth for generating and/or utilizing image shortcuts that cause one or more corresponding computer actions to be performed in response to determining that one or more features are present in image(s) from a camera of a computing device of a user (e.g., present in a real-time image feed from the camera). An image shortcut can be generated in response to user interface input, such as a spoken command. For example, the user interface input can direct the automated assistant to perform one or more actions in response to object(s) having certain feature(s) being present in a field of view of the camera. Subsequently, when the user directs their camera at object(s) having such feature(s), the assistant application can cause the action(s) to be automatically performed. For example, the assistant application can cause data to be presented and/or can control a remote device in accordance with the image shortcut.

An electronic apparatus is provided. The electronic apparatus includes a camera; a memory configured to store at least one instruction; and at least one processor configured to execute the at least one instruction to: detect at least one object included in an image captured by the camera; identify information on an engagement of each of the at least one object with the electronic apparatus; obtain gesture information of each of the at least one object; obtain a target object from among the at least one object based on an operation status of the electronic apparatus, the information on the engagement of each of the at least one object, and the obtained gesture information of each of the at least one object; identify a function corresponding to gesture information of the target object; and execute the identified function.

One embodiment of the present invention relates to an electronic device for reproducing, in time sequence, first pen data in which handwriting strokes are digitized on a medium by an electronic pen. The electronic device comprises: a communication module; a control module; an output module for reproducing or outputting first handwriting information, first image or audio information photographed or recorded by the electronic device, and event information in time sequence; and a memory module for storing contents including the first handwriting information, the first image or audio information, and the event information, wherein, when the communication module further receives second pen data in a state where the first handwriting information is held on the output module after the contents are stored in the memory module, second handwriting information corresponding to the second pen data is added to the first handwriting information and then reproduced.

An apparatus for visualizing physical movements includes: a device for acquiring video image files; a data analysis system including processor and memory; a computer program operating in the processor to identify an area in the images where periodic motions associated with physical movement of an object may be detected and quantified, and compute a new image sequence in which the motions are visually amplified; and, a user interface that displays the motion-amplified video image of the mechanical component. An associated method for using the apparatus is also disclosed.

A waveguide apparatus includes a planar waveguide and at least one optical diffraction element (DOE) that provides a plurality of optical paths between an exterior and interior of the planar waveguide. A phase profile of the DOE may combine a linear diffraction grating with a circular lens, to shape a wave front and produce beams with desired focus. Waveguide apparati may be assembled to create multiple focal planes. The DOE may have a low diffraction efficiency, and planar waveguides may be transparent when viewed normally, allowing passage of light from an ambient environment (e.g., real world) useful in AR systems. Light may be returned for temporally sequentially passes through the planar waveguide. The DOE(s) may be fixed or may have dynamically adjustable characteristics. An optical coupler system may couple images to the waveguide apparatus from a projector, for instance a biaxially scanning cantilevered optical fiber tip.

While many augmented reality systems provide “see-through” transparent or translucent displays upon which to project virtual objects, many virtual reality systems instead employ opaque, enclosed screens. Indeed, eliminating the user's perception of the real-world may be integral to some successful virtual reality experiences. Thus, head mounted displays designed exclusively for virtual reality experiences may not be easily repurposed to capture significant portions of the augmented reality market. Various of the disclosed embodiments facilitate the repurposing of a virtual reality device for augmented reality use. Particularly, by anticipating user head motion, embodiments may facilitate scene renderings better aligned with user expectations than naïve renderings generated within the enclosed field of view. In some embodiments, the system may use procedural mapping methods to generate a virtual model of the environment. The system may then use this model to supplement the anticipatory rendering.