An imagery processing system determines pixel color values for pixels of captured imagery from volumetric data, providing alternative pixel color values. A main imagery capture device, such as a camera, captures main imagery such as still images and/or video sequences, of a live action scene. Alternative devices capture imagery of the live action scene, in some spectra and form, and capture information related to pixel color values for multiple depths of a scene, which can be processed to provide reconstruction.

A stereoscopic imager system, comprising: a sensor array comprising a first plurality of photosensors and a second plurality of photosensors spaced apart from the first plurality of photosensors by a gap, the first plurality of photosensors and the second plurality of photosensors being configured to detect ambient light in a scene; a moving component coupled to the sensor array and operable to move the sensor array between a first position and a second position within a full rotational image capturing cycle; and a system controller coupled to the sensor array and the moving component. The system controller can be configured to: move a field of view of a sensor array by instructing the moving component to capture a first image of an object in the scene with the first plurality of photosensors from a first perspective at the first position, and to capture a second image of the scene of the object in the scene with the second plurality of photosensors from a second perspective at the second position; and calculate, based on the first image and the second image, a distance to the object using an optical baseline defined by the gap.

Provided are a vision sensor, an image processing device including the same, and an operating method of the vision sensor. The vision sensor includes a pixel array including a plurality of pixels, a synchronization control module configured to receive a synchronization signal from an external image sensor and generate an internal trigger signal based on the synchronization signal, an event detection circuit configured to receive the internal trigger signal and detect whether an event has occurred in the plurality of pixels and generate event signals corresponding to pixels in which the event has occurred, and an interface circuit configured to receive the event signals and transmit to an external processor vision sensor data based on the trigger signal and at least one of the event signals, where the vision sensor data includes matching information for timing image frame information generated by the image sensor and the event signals generated by the vision sensor.

Apparatuses, systems, and techniques are presented to perform monocular view synthesis of a dynamic scene. Single and multi-view depth information can be determined for a collection of images of a dynamic scene, and a blender network can be used to combine image features for foreground, background, and missing image regions using fused depth maps inferred form the single and multi-view depth information.

A distance measuring camera 1 includes a first optical system for collecting light from a subject to form a first subject image, a second optical system for collecting the light from the subject to form a second subject image, an imaging part for imaging the first subject image formed by the first optical system and the second subject image formed by the second optical system, and a distance calculating part 4 for calculating a distance to the subject based on the first subject image and the second subject image imaged by the imaging part S. The first optical system and the second optical system are configured so that a change of a magnification of the first subject image according to the distance to the subject is different from a change of a magnification of the second subject image according to the distance to the subject.

This disclosure relates to techniques for generating robust depth estimations for captured images using semantic segmentation. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Such segmentations may be binary (e.g., a ‘person pixel’ or a ‘non-person pixel’) or multi-class (e.g., a pixel may be labelled as: ‘person,’ ‘dog,’ ‘cat,’ etc.). As semantic segmentation techniques grow in accuracy and adoption, it is becoming increasingly important to develop methods of utilizing such segmentations and developing flexible techniques for integrating segmentation information into existing computer vision applications, such as depth and/or disparity estimation, to yield improved results in a wide range of image capture scenarios. In some embodiments, an optimization framework may be employed to optimize a camera device's initial scene depth/disparity estimates that employs both semantic segmentation and color regularization in a robust fashion.

The present invention provides a user with a user interface for enabling the user to efficiently perform an operation for generating a virtual viewpoint image for each imaging target subject to be imaged by a plurality of imaging apparatuses. An event information acquisition unit acquires information about an event subjected to virtual viewpoint image generation, and transmits the acquired event information to a user interface (UI) determination unit. The UI determination unit determines a UI to be generated by a UI generation unit based on the event information transmitted from the event information acquisition unit. The UI generation unit generates the UI determined by the UI determination unit. The user performs an input operation for generating a virtual viewpoint image according to the UI generated by the UI generation unit.

The present invention provides a user with a user interface for enabling the user to efficiently perform an operation for generating a virtual viewpoint image for each imaging target subject to be imaged by a plurality of imaging apparatuses. An event information acquisition unit acquires information about an event subjected to virtual viewpoint image generation, and transmits the acquired event information to a user interface (UI) determination unit. The UI determination unit determines a UI to be generated by a UI generation unit based on the event information transmitted from the event information acquisition unit. The UI generation unit generates the UI determined by the UI determination unit. The user performs an input operation for generating a virtual viewpoint image according to the UI generated by the UI generation unit.

A stereoscopic imaging apparatus for use in a robotic surgery system is disclosed and includes an elongate sheath having a bore. First and second image sensors are adjacently mounted at the distal end to capture high definition images from different perspective viewpoints for generating three-dimensional image information. The image sensors produce an unprocessed digital data signal representing the captured images. A wired signal line transmits the unprocessed digital data signals along the sheath to a proximal end to processing circuitry. The processing circuitry is configured to perform processing operations on the unprocessed digital data signals to produce respective video signals suitable for transmission to a host system or for driving a 3D display. A secondary camera is also disclosed and includes an elongate strip of circuit substrate sized for insertion through a narrow conduit, the strip of circuit substrate connecting between an image sensor and a processing circuit substrate.

A system and method for determining a dimension of a target. The method includes: determining a camera parameter, the camera parameter including at least one of a focal length, a yaw angle, a roll angle, a pitch angle, or a height of one or more cameras; acquiring a first image and a second image of an target captured by the one or more cameras; generating a first corrected image and a second corrected image by correcting the first image and the second image; determining a parallax between a pixel in the first corrected image and a corresponding pixel in the second corrected image; determining an outline of the target; and determining a dimension of the target based at least in part on the camera parameter, the parallax, and the outline of the target.