Developer and runtime environments supporting multi-modal input for computing systems are disclosed. The developer environment includes a gesture library of human body gestures (e.g., hand gestures) that a previously-trained, system-level gesture recognition machine is configured to recognize. The developer environment further includes a user interface for linking a gesture of the gesture library with a semantic descriptor that is assigned to a function of the application program. The application program is executable to implement the function responsive to receiving an indication of the gesture recognized by the gesture recognition machine within image data captured by a camera. The semantic descriptor may be additionally linked to a different input modality than the gesture, such as a natural language input.

A keyboard dock can include a keyboard coupling; a front surface; and a front spacer that extends outwardly from the front surface, where the front spacer and the front surface form a mobile device support.

In accordance with some embodiments, a method is performed at a first device with one or more processors, non-transitory memory, and a display. The method includes displaying, on the display, a device control transfer affordance while operating the first device based on user input from an input device that is in communication with the first device. The method includes receiving a device control transfer user input from the input device selecting the device control transfer affordance that is displayed on the display of the first device. In response to receiving the device control transfer user input, the method includes configuring a second device to be operated based on user input from the input device and ceasing to operate the first device based on user input from the input device.

An interface device includes an on-panel pad and a sensor controller. The interface device is used to perform an input to an electronic device in response to an operation by a user on a capacitive touch panel display of the electronic device. The on-panel pad causes sensor electrodes included in the touch panel display to generate a two-dimensional pattern of capacitance while the on-panel pad is on the touch panel display. When the two-dimensional pattern generated while the on-panel pad is on the touch panel display has been detected, the sensor controller generates or outputs data including operation information in response to a change in capacitance in a region that is associated with the two-dimensional pattern and that receives an operation of the on-panel pad, the operation information indicating an operation state of the on-panel pad.

Systems, methods, and non-transitory computer readable media for virtually extending a physical keyboard are disclosed. In one implementation, a non-transitory computer readable medium contains instructions that cause a processor to: receive image data representing a keyboard placed on a surface from an image sensor associated with a wearable extended reality appliance; determine that the keyboard is paired with the wearable extended reality appliance; receive an input for causing a display of a virtual controller with the keyboard; display, via the wearable extended reality appliance, the virtual controller in a first location on the surface, with original spatial orientation relative to the keyboard; detect a movement of the keyboard to a different location on the surface; and in response to the detected movement, present the virtual controller in a second location on the surface, with a subsequent spatial orientation relative to the keyboard that corresponds to the original spatial orientation.

A keyboard system comprises a housing and a key structure having a key body and key cap. The key body includes a transceiver circuit. The key cap includes a key cap transceiver and a memory structure storing a key cap ID data and/or a key function data corresponding to a key function. The transceiver circuit can poll the key cap memory via the key cap transceiver and retrieve the key cap ID data and/or the key function data. The keyboard system can determine a mapping of a location of the key cap on the keyboard by polling it and can generate and transmit a control signal to a host computing device communicatively coupled to the keyboard system based to the key cap ID and/or the key function data.

Disclosed herein is an apparatus for controlling an embedded device, through which the embedded device may be controlled in a host computer in real time. The apparatus for controlling an embedded device includes a host computer connected with a monitor, a mouse, and a keyboard; an embedded device; and a connection module for connecting the host computer with the embedded device such that a first task window for controlling the host computer and a second task window for controlling the embedded device are displayed on the monitor.

A computer system including an electronic device and an input device is disclosed. The electronic device includes a touch display area and a control unit. The input device includes a plurality of positioning structures and a grounding piece. The positioning structures are electrically connected to the grounding piece. When the input device is disposed on the touch display area of the electronic device, the control unit detects positions of the positioning structures on the touch display area, calculates a coverage area covered by the input device on the touch display area according to the positions, and determines a range of a display area of the touch display area according to the coverage area.

A display device configured to be connected to an information terminal, includes: a video reception unit that receives video information about a screen including a cursor from the information terminal; a touch panel that brings up the screen on display based on the video information received by the video reception unit and detects a touch position specified by a touch operation; and a transmission unit that transmits operation information in correspondence to the touch operation to the information terminal. The transmission unit transmits the operation information in a delay when the touch operation is performed continuously.

Embodiments of this application provide a method and a system for generating a note. The method includes: detecting, by an electronic device, to-be-stored content selected by a stylus on a touchscreen of the electronic device; controlling, by the electronic device, a target device to store the to-be-stored content, where the target device is the electronic device and/or the stylus; and outputting, by the target device, the to-be-stored content. In the embodiments of this application, a storage function of the electronic device and/or the stylus is used to output the to-be-stored content required by a user as a separate note, which facilitates convenience for the user to read key notes and improves user experience.