An electronic device includes at least one processor, and at least one memory storing instructions executable by the at least one processor and operatively connected to the at least one processor, where the at least one processor is configured to acquire voice data to be synthesized with at least one first image, generate a plurality of mouth shape candidates by using the voice data, select a mouth shape candidate among the plurality of mouth shape candidates, generate at least one second image based on the selected mouth shape candidate and at least a portion of each of the at least one first image, and generate at least one third image by applying at least one super-resolution model to the at least one second image.

Systems/techniques that facilitate self-supervised deblurring are provided. In various embodiments, a system can access an input image generated by an imaging device. In various aspects, the system can train, in a self-supervised manner based on a point spread function of the imaging device, a machine learning model to deblur the input image. More specifically, the system can append to the model one or more non-trainable convolution layers having a blur kernel that is based on the point spread function of the imaging device. In various aspects, the system can feed the input image to the model, the model can generate a first output image based on the input image, the one or more non-trainable convolution layers can generate a second output image by convolving the first output image with the blur kernel, and the system can update parameters of the model based on a difference between the input image and the second output image.

The present disclosure describes systems and techniques directed to optical image stabilization movement to create a super-resolution image of a scene. The systems and techniques include a user device (102) introducing (502), through an optical image stabilization system (114), movement to one or more components of a camera system (112) of the user device (102). The user device (102) then captures (504) respective and multiple frames (306) of an image of a scene, where the respective and multiple frames (306) of the image of the scene have respective, sub-pixel offsets of the image of the scene across the multiple frames (306) as a result of the introduced movement to the one or more components of the camera system (112). The user device (102) performs (506), based on the respective, sub-pixel offsets of the image of the scene across the respective, multiple frames (306), super-resolution computations and creates (508) the super-resolution image of the scene based on the super-resolution computations.

An eye tracking system for tracking one or more of a user's eyes includes an closed-eye detector operable to detect when a user has closed one or more of their eyes, an eye tracker operable to detect an eye orientation in dependence upon a measured deformation of an eyelid corresponding to the an eye that has been detected to be shut, and an image renderer operable to render a foveated image for display in response to the detected eye orientation.

The technology disclosed present systems and methods to produce an enhanced resolution image from images of a target using structured illumination microscopy (SIM). The method includes transforming at least three images of the target captured by a sensor in a spatial domain into a Fourier domain to produce at least three frequency domain matrices that each include first blocks of complex coefficients and redundant second blocks of complex coefficients that are conjugates to the first blocks. The method includes reducing computing resources required to produce the enhanced resolution image by using first blocks of complex coefficients to produce at least three phase-separated half-matrices in the Fourier domain. The method includes performing one or more intermediate transformation on the phase-separated half-matrices to produce realigned shifted half-matrices. The method includes calculating complex coefficients of second blocks in the Fourier domain to produce full matrices from half-matrices.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a synthesized signal. In some implementations, a computer-implemented system obtains generator input data including at least an input signal having one or more first characteristics, processes the generator input data to generate output data including a synthesized signal having one or more second characteristics using a generator neural network, and outputs the synthesized signal to a device. The generator neural network is trained, based on a plurality of training examples, with a discriminator neural network. The discriminator neural network is configured to process discriminator input data that combines a discriminator input signal having the one or more second characteristics with at least a portion of generator input data to generate a prediction of whether the discriminator input signal is a real signal provided in one of the plurality of training examples or a synthesized signal outputted by the generator neural network.

Disclosed are an image processing method and device using a line-wise operation. The image processing device, according to one embodiment, comprises: a receiver for receiving an image; a first convolution operator for generating a feature map by performing a convolution operation on the basis of the image; and a compressor for compressing the feature map into units of at least one line; and a decompressor for reconstructing the feature map compressed into units of lines.

A direct structured illumination microscopy (dSIM) reconstruction method is provided. First, a time domain modulation signal is extracted through a wavelet. Then, an incoherent signal is converted into a coherent signal. Next, an accumulation amount at each pixel is calculated. Finally, a super-resolution image is generated by using a correlation between signals at different spatial positions. An autocorrelation algorithm of dSIM is insensitive to an error of a reconstruction parameter. dSIM bypasses a complex frequency domain operation in structured illumination microscopy (SIM) image reconstruction, and prevents an artifact caused by the parameter error in the frequency domain operation. The dSIM algorithm has high adaptability and can be used in laboratory SIM, nonlinear SIM imaging systems, or commercial systems.

The current invention relates to the use of a neural network to improve the quality of images obtained from light scattered by an intermediate object that scatters light, such as tissue or a frosted screen. The invention relates to a method of imaging a human or animal bode using a nanocrystal array capable of fluorescing upon excitation from light from a near-infrared light source. This invention also relates to detection means and apparatus used in said methods, as well as to quantum dots useful in said use.

An image processing device includes a processor and a memory that is provided in or connected to the processor. The processor executes a region selection process of selecting a portion of a plurality of tomographic images, which indicate a plurality of tomographic planes of an object, respectively, and have a first resolution, as a target region to be set to a second resolution higher than the first resolution, a resolution enhancement process of increasing the resolution of the target region to the second resolution to generate a high-resolution partial image, and a composite two-dimensional image generation process of generating a high-resolution composite two-dimensional image having the second resolution, using the high-resolution partial image.