A residual network is used to predict a set of residual motion vectors that provide additional motion data for portions of the frame for which motion vectors are not provided, such as animated textures, mirrored/reflected objects, and/or moving objects without motion information.

A description is given of a method for increasing resolution in microscopy, comprising providing at least one recorded sample image (22) which was generated by means of a microscope (2), providing a point spread function which characterizes an imaging behaviour of the microscope (2), and calculating a sample image with increased resolution from the recorded sample image (22), wherein the calculating is effected in an iteration process (S4) which repeatedly passes through an iteration loop (S4a; S4b) and which determines a correction image (24.0-24.n) from the recorded sample image (22) using the point spread function, wherein a difference between the correction image convolved with the point spread function and the recorded sample image (22) is minimized in the iteration process (S4), and wherein in the iteration process (S4) the passes through the iteration loop (S4a; S4b) are numbered with an ascending pass number (k) and each comprise a step size factor which is dependent on the pass number (k) of the respective pass and is determined without recourse to correction images.

A system for forming an image (110) of a substantially translucent specimen (102) has an illuminator (108) configured to variably illuminate the specimen from a plurality of angles of illumination such that (a) when each angle (495) at a given point on the specimen is mapped to a point (445) on a plane (420) perpendicular to an optical axis (490), the points on the plane have an increasing density (e.g. FIGS. 4, 11C, 11E, 12C, 12E, 13A, 14A, 14C, 14E, 15A, 15C, 15E) towards an axial position on the plane; or (b) the illumination angles are arranged with a substantially regular pattern in a polar coordinate system (FIG. 13A,13B) defined by a radial coordinate that depends on the magnitude of the distance from an optical axis and an angular coordinate corresponding to the orientation of the angle relative to the optical axis. A detector is configured to acquire a plurality of variably illuminated, relatively lower-resolution intensity images (104) of the specimen based on light emitted from the illuminator according to variable illumination and filtered by an optical element (109). A processor is arranged to computationally reconstruct a relatively higher-resolution image of the specimen by iteratively updating overlapping regions (1005) of the relatively higher-resolution image in Fourier space (FIG. 10B) with the variably-illuminated, lower-resolution intensity images.

A method for generating a super-resolution image and related device is provided. In one aspect, the method comprises: receiving a first low-resolution image and a second low-resolution image, the first low-resolution image and second low-resolution image have a first spatial resolution and having been captured simultaneously by a pair of pixel arrays of a common image sensor, wherein the pixel arrays of the image sensor are located as to be diagonally shifted from each other by a sub-pixel increment; adaptively enhancing the first low-resolution and the second low-resolution images to generate an enhanced first low-resolution image and an enhanced second low-resolution image, respectively; mapping (e.g., non-uniformly) pixels of each of the enhanced first and second low-resolution images to a super-resolution grid having a spatial resolution greater than the first spatial resolution to generate a first intermediate super-resolution image and a second intermediate super-resolution image, respectively; and combining the first intermediate super-resolution image and second intermediate super-resolution image to generate a composite super-resolution image.

An image quality enhancing apparatus which make a learning-type image quality enhancing method utilizing a sparse expression practical are provided. The apparatus calculates, from the feature quantity of an image, coefficients of low-image-quality base vectors expressing a feature quantity with a linear sum and generates the image with the image quality enhanced by calculating a linear sum of high-image-quality base vectors using the calculated coefficient. When calculating the coefficient, the number of base vectors with non-zero coefficients is determined, the determined number of base vectors is selected, and a solution of a coefficient matrix is calculated by assuming the coefficients of the base vectors other than the selected base vectors are zero. The amount of processes necessary for obtaining a sparse solution of a coefficient matrix can be reduced by adjusting the number of base vectors with non-zero coefficients, and a practical image quality enhancing apparatus can be realized.

A method and a system for processing an image and transform it into a high resolution and high-definition image using a computationally efficient image transformation procedure is provided. The transformation of the image comprises transforming a received intensity image into a layered distance field (DF) image including an ordered sequence of multiple layers. Each layer of the layered DF image includes a DF procedure defining DF values at all locations of the received intensity image and rules for mapping these DF values into intensity values of the layer. The intensity image is transformed iteratively until an error between the received intensity image and an intensity image reconstructed from the layered DF image by combining the intensities values of each level in their corresponding order is less than a threshold error value.

The invention relates to reconstructing a synthetic dense super-resolution image from at least one low-information-content image, for example from a sequence of diffraction-limited images acquired by single molecule localization microscopy. After having obtained such a sequence of diffraction-limited images, a sparse localization image is reconstructed from the obtained sequence of diffraction-limited images according to single molecule localization microscopy image processing. The reconstructed sparse localization image and/or a corresponding low-resolution wide-field image are input to an artificial neural network and a synthetic dense super-resolution image is obtained from the artificial neural network, the latter being trained with training data comprising triplets of sparse localization images, at least partially corresponding low-resolution wide-field images, and corresponding dense super-resolution images, as a function of a training objective function comparing dense super-resolution images and corresponding outputs of the artificial neural network.

A signal which is to be quality-improved often suffers from the quality degradation in the spatial high frequency region more than compared to the spatial low frequency region. Accordingly a quality improvement is performed efficiently by combining the signal to be quality improved with a high frequency portion extracted from a sparse approximation of the signal to be quality improved.

The present disclosure provides an image processing apparatus, and image processing method, and a display apparatus. The image processing apparatus includes a first image processor up-sampling an original low-resolution image on the basis of deep learning-based learning data to generate a first high-resolution image, a second image processor interpolating the original low-resolution image to generate a second high-resolution image, a third image processor generating a difference image between the first high-resolution image and the second high-resolution image, extracting a high frequency component from the difference image, and amplifying the extracted high frequency component, and a fourth image processor adding the amplified high frequency component to the first high-resolution image to generate a target high-resolution image.

In an image-processing method, a stack is provided for storing a predetermined number of frame portions. An image including a target object is obtained, the image being formed by an array of pixels. A frame portion is extracted from the image, the frame portion being at least a portion of the pixels forming the image, corresponding to a region of interest in the image, the region of interest comprising the target object. The frame portion is stored in the stack, the storing including discarding an oldest previously stored frame portion from the stack if the number of frame portions stored in the stack has reached the predetermined number. The steps of the method are repeated a plurality of times. Frame portions in the stack having a phase substantially equal to a given phase are averaged. A super-resolved image is calculated from the plurality of stored frame portions.