Methods, system, and computer storage media are provided for novel view synthesis. An input image depicting an object is received and utilized to generate, via a neural network, a target view image. In exemplary aspects, additional view images are also generated within the same pass of the neural network. A loss is determined based on the target view image and additional view images and is used to modify the neural network to reduce errors. In some aspects, a rotated view image is generated by warping a ground truth image from an initial angle to a rotated view angle that matches a view angle of an image synthesized via the neural network, such as a target view image. The rotated view image and the synthesized image matching the rotated view angle (e.g., a target view image) are utilized to compute a rotational loss.

The present application discloses a dynamic projection method for target tracking and a dynamic projection device. A dynamic projection method for target tracking includes: acquiring position information of a target; determining three-dimensional spatial coordinates of the target in a first coordinate system based on the position information of the target; determining three-dimensional spatial coordinates of the target in a second coordinate system based on the three-dimensional spatial coordinates of the target in the first coordinate system; determining a deflection angle of a projection screen based on the three-dimensional spatial coordinates of the target in the second coordinate system; determining a rotation angle of a dynamic control unit based on the deflection angle; controlling the dynamic control unit to rotate by the rotation angle; and controlling a projecting unit to project the projection screen. In this way, dynamic projection for target tracking is implemented.

A display control apparatus according to the present invention includes: an axis detection unit configured to detect, based on an image, a rotation axis about which a celestial body included in the image as an object rotates in response to the rotation of the earth; and a control unit configured to perform control such that a partial area of the image is displayed as a display area, and rotation display is performed by changing the display area while changing an angle corresponding to the display area around a position corresponding to the rotation axis detected by the axis detection unit.

An image processing method includes obtaining two-dimensional coordinate points of an input image, and according to a camera imaging model or a distortion correction model, performing a two-dimension to three-dimension conversion operation on the two-dimensional coordinate points to obtain a first processing result. The method also includes performing at least one of virtual reality processing or electronic image stabilization on the first processing result to obtain a second processing result, and mapping the second processing result to a two-dimensional image coordinate system to obtain an output image.

A display control apparatus according to the present invention includes: an axis detection unit configured to detect, based on an image, a rotation axis about which a celestial body included in the image as an object rotates in response to the rotation of the earth; and a control unit configured to perform control such that a partial area of the image is displayed as a display area, and rotation display is performed by changing the display area while changing an angle corresponding to the display area around a position corresponding to the rotation axis detected by the axis detection unit.

Methods, system, and computer storage media are provided for novel view synthesis. An input image depicting an object is received and utilized to generate, via a neural network, a target view image. In exemplary aspects, additional view images are also generated within the same pass of the neural network. A loss is determined based on the target view image and additional view images and is used to modify the neural network to reduce errors. In some aspects, a rotated view image is generated by warping a ground truth image from an initial angle to a rotated view angle that matches a view angle of an image synthesized via the neural network, such as a target view image. The rotated view image and the synthesized image matching the rotated view angle (e.g., a target view image) are utilized to compute a rotational loss.

Methods, system, and computer storage media are provided for novel view synthesis. An input image depicting an object is received and utilized to generate, via a neural network, a target view image. In exemplary aspects, additional view images are also generated within the same pass of the neural network. A loss is determined based on the target view image and additional view images and is used to modify the neural network to reduce errors. In some aspects, a rotated view image is generated by warping a ground truth image from an initial angle to a rotated view angle that matches a view angle of an image synthesized via the neural network, such as a target view image. The rotated view image and the synthesized image matching the rotated view angle (e.g., a target view image) are utilized to compute a rotational loss.

An apparatus includes an imaging unit for capturing a subject image; a setting unit setting initial posture of the imaging unit; a detector pirate for detecting posture information indicating a change in posture from the initial posture of the imaging unit; a corrector for correcting a posture of a preset virtual object arranged in a virtual three-dimensional space depending on the posture information; a projector for generating, based on a conversion function for projecting the virtual object on a device coordinate in a display screen of the captured image, an object image on the device coordinate, of the virtual object whose posture is corrected; a synthesizer for generating a synthetic image of the object and the captured images; a display for displaying the synthetic image; an image recording unit for recording the captured image; and a posture recording unit for recording the posture information in synchronization with the captured image.