Image Processing Method, Apparatus, and Device

Abstract

This application provides an image processing method, apparatus, a device, and the like. In the method, a special image filter is generated, and super-resolution is performed on an image based on the image filter, thereby improving an image super-resolution effect. The image filter includes filter parameters corresponding to each pixel in an image that requires super-resolution processing, and pixels with different texture features correspond to different filter parameters. The image super-resolution method, apparatus, device, and the like may be applied to various scenarios such as a video, a game, and photographing, to improve an image effect in these scenarios, and enhance user experience.

Claims

1. An image processing method, comprising: determining an image filter corresponding to a first image, wherein the image filter comprises filter parameters corresponding to each pixel in the first image, and pixels with different texture features correspond to different filter parameters; and performing super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

2. The method according to claim 1, wherein the determining an image filter corresponding to a first image comprises: obtaining a texture image of the first image; determining, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image, wherein textures in the local texture images have a same texture feature, C=f.sup.2×t, f is a preset amplification multiple, t is a quantity of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1; and determining the image filter based on the C local texture images and the weight value of each local texture image, wherein a quantity of channels of the image filter is C.

3. The method according to claim 2, wherein an (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image, wherein i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally comprised in the first image, N is a quantity of pixels vertically comprised in the first image, and M and N are both integers greater than 1.

4. The method according to claim 2, wherein the obtaining a texture image of the first image comprises: performing compression processing on the first image based on a preset size, wherein a size of a compressed first image is the preset size; and determining the texture image based on the compressed first image.

5. The method according to claim 1, wherein the determining an image filter corresponding to a first image comprises: processing the first image by using a recognition model, to obtain the image filter of the first image, wherein the recognition model is obtained by learning a plurality of groups of samples, each group of samples comprises a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

6. The method according to claim 5, wherein the processing the first image based on a recognition model comprises: performing compression processing on the first image based on a preset size, wherein a size of a compressed first image is the preset size; and processing the compressed first image by using the recognition model.

7. The method according to claim 1, wherein the performing super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image comprises: obtaining a gradient image of the first image, wherein a size of the gradient image is the same as that of the first image, the size of the gradient image is M*N, and M and N are both integers greater than 1; processing the gradient image by using the image filter, to obtain a second image, wherein a size of the second image is (f*M)*(f*N), and f is a preset amplification multiple; amplifying the first image by f times to obtain a third image, wherein a size of the third image is (f*M)*(f*N); and obtaining the super-resolution image based on the second image and the third image, wherein a size of the super-resolution image is (f*M)*(f*N).

8. The method according to claim 7, wherein the processing the gradient image by using the image filter, to obtain a second image comprises: processing the gradient image by using the image filter, to obtain f.sup.2 sub-images, wherein a size of each sub-image is (f*M)*(f*N); and determining the second image based on the f.sup.2 sub-images.

9. The method according to claim 8, wherein the processing the gradient image by using the image filter of the first image, to obtain f.sup.2 sub-images comprises: processing the gradient image by using parameters in a (k*t+1).sup.th channel to a (k+1)*t.sup.th channel in the image filter, to obtain a k.sup.th sub-image, wherein k is sequentially set to 0, 1, . . . , and f.sup.2−1.

10. The method according to claim 8, wherein the determining the second image based on the f.sup.2 sub-image comprises: separately performing splicing processing on pixels at a same pixel position in the f.sup.2 sub-images, to obtain M*N image blocks, wherein a size of each image block is f*f; and performing splice processing on the M*N image blocks based on pixel locations of pixels in each image block in the sub-image, to obtain the second image.

11. A computer system, comprising a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program, to cause the computer system to: determine an image filter corresponding to a first image, wherein the image filter comprises filter parameters corresponding to each pixel in the first image, and pixels with different texture features correspond to different filter parameters; and perform super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

12. The computer system according to claim 11, wherein the determining an image filter corresponding to a first image comprises: obtaining a texture image of the first image; determining, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image, wherein textures in the local texture images have a same texture feature, C=f.sup.2×t, f is a preset amplification multiple, t is a quantity of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1; and determining the image filter based on the C local texture images and the weight value of each local texture image, wherein a quantity of channels of the image filter is C.

13. The computer system according to claim 12, wherein an (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image, wherein i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally comprised in the first image, N is a quantity of pixels vertically comprised in the first image, and M and N are both integers greater than 1.

14. The computer system according to claim 12, wherein the obtaining a texture image of the first image comprises: performing compression processing on the first image based on a preset size, wherein a size of a compressed first image is the preset size; and determining the texture image based on the compressed first image.

15. The computer system according to claim 11, wherein the determining an image filter corresponding to a first image comprises: processing the first image by using a recognition model, to obtain the image filter of the first image, wherein the recognition model is obtained by learning a plurality of groups of samples, each group of samples comprises a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

16. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, cause the processor to: determine an image filter corresponding to a first image, wherein the image filter comprises filter parameters corresponding to each pixel in the first image, and pixels with different texture features correspond to different filter parameters; and perform super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

17. The non-transitory computer-readable storage medium according to claim 16, wherein the determining an image filter corresponding to a first image comprises: obtaining a texture image of the first image; determining, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image, wherein textures in the local texture images have a same texture feature, C=f.sup.2×t, f is a preset amplification multiple, t is a quantity of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1; and determining the image filter based on the C local texture images and the weight value of each local texture image, wherein a quantity of channels of the image filter is C.

18. The non-transitory computer-readable storage medium according to claim 17, wherein an (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image, wherein i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally comprised in the first image, N is a quantity of pixels vertically comprised in the first image, and M and N are both integers greater than 1.

19. The non-transitory computer-readable storage medium according to claim 17, wherein the obtaining a texture image of the first image comprises: performing compression processing on the first image based on a preset size, wherein a size of a compressed first image is the preset size; and determining the texture image based on the compressed first image.

20. The non-transitory computer-readable storage medium according to claim 16, wherein the determining an image filter corresponding to a first image comprises: processing the first image by using a recognition model, to obtain the image filter of the first image, wherein the recognition model is obtained by learning a plurality of groups of samples, each group of samples comprises a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0051] FIG. 1 is a schematic diagram of a super-resolution image according to an embodiment of this application;

[0052] FIG. 2 is a schematic diagram of a texture image according to an embodiment of this application;

[0053] FIG. 3 is a schematic diagram of a gradient image according to an embodiment of this application;

[0054] FIG. 4 is a schematic diagram of an image filter according to an embodiment of this application;

[0055] FIG. 5 is a schematic flowchart of a method for determining an image filter according to an embodiment of this application;

[0056] FIG. 6 is a schematic diagram of a local texture image according to an embodiment of this application;

[0057] FIG. 7 is a schematic diagram of another image filter according to an embodiment of this application;

[0058] FIG. 8 is a schematic diagram of an image according to an embodiment of this application;

[0059] FIG. 9 is a schematic flowchart of another method for determining an image filter according to an embodiment of this application;

[0060] FIG. 10 is a schematic diagram of a sample image set according to an embodiment of this application;

[0061] FIG. 11 is a schematic flowchart of a method for obtaining a sample image set according to an embodiment of this application;

[0062] FIG. 12 is a schematic flowchart of a method for performing super-resolution on a first image by an image filter according to an embodiment of this application;

[0063] FIG. 13 is a schematic diagram of processing a pixel by using filter parameters according to an embodiment of this application;

[0064] FIG. 14 is another schematic diagram of processing a pixel by using filter parameters according to an embodiment of this application;

[0065] FIG. 15 is a schematic diagram of a process of determining an image block according to an embodiment of this application;

[0066] FIG. 16 is a schematic diagram of a process of splicing image blocks according to an embodiment of this application;

[0067] FIG. 17 is a schematic flowchart of an image processing method according to an embodiment of this application;

[0068] FIG. 18 is a schematic diagram of a process of image processing according to an embodiment of this application;

[0069] FIG. 19 is a schematic diagram of an application scenario according to an embodiment of this application;

[0070] FIG. 20 is a schematic diagram of a process of generating and using a processing model according to an embodiment of this application;

[0071] FIG. 21 is a schematic structural diagram of an image processing apparatus according to an embodiment of this application; and

[0072] FIG. 22 is a schematic structural diagram of hardware of a computer system according to an embodiment of this application.

DESCRIPTION OF EMBODIMENTS

[0073] For ease of understanding, concepts in this application are first described.

[0074] Image super-resolution may also be referred to as image super-resolution, and refers to reconstruction of a low-resolution image to obtain a high-resolution image. The following describes the image super-resolution with reference to FIG. 1. FIG. 1 is a schematic diagram of a super-resolution image according to an embodiment of this application. Refer to FIG. 1. A raw image and a super-resolution image are included. Super-resolution processing is performed on the raw image to obtain the super-resolution image, and the raw image and the super-resolution image have same image content. For example, it is assumed that resolution of the raw image is a*b, and an amplification multiple is 3, a resolution of the super-resolution image is 3a*3b. In a resolution a*b of an image, a refers to a quantity of pixels horizontally included in the image, and b refers to a quantity of pixels vertically included in the image. The resolution of the image may also be referred to as a size of the image.

[0075] A texture may also be referred to as a pattern, and refers to a pattern or a pattern presented on a surface of an object.

[0076] A texture image refers to an image that includes a texture in the raw image. The following describes the texture image with reference to FIG. 2. FIG. 2 is a schematic diagram of a texture image according to an embodiment of this application. Refer to FIG. 2. A raw image and the texture image are included, and the texture image includes a texture in the raw image. It should be noted that FIG. 2 merely schematically shows a type of texture image in the raw image. For example, the type of the texture image may include a local binary patterns (local binary patterns, LBP) type, a Gaussian Markov random field (gaussian markov random field, GMRF) type, a grayscale co-occurrence matrix (gray-level co-occurrence matrix, GLCM) type, and the like.

[0077] A texture feature refers to a feature of a texture, and different objects in an image have different texture features. For example, in FIG. 2, a texture feature of a sky, a texture feature of a building, a texture feature of hair, a texture feature of a face, a texture feature of clothes, a texture feature of grass, and the like are all different.

[0078] Gradient image: If a raw image is considered as a two-dimensional discrete function, the gradient image is an image obtained by deriving the two-dimensional discrete function. The following describes the gradient image with reference to FIG. 3. FIG. 3 is a schematic diagram of a gradient image according to an embodiment of this application. Refer to FIG. 3. A raw image and a gradient image are included. The raw image usually includes three channels, and the gradient image usually includes one channel. Therefore, processing a gradient image of the raw image can reduce a computation amount. For example, a computation amount of the gradient image is one third of a computation amount of the raw image.

[0079] An image filter may process an image to improve resolution of the image. An image filter in this application is a three-dimensional filter, and the three dimensions may be denoted as H, W, and C respectively. For ease of understanding, the following describes the image filter with reference to FIG. 4. FIG. 4 is a schematic diagram of an image filter according to an embodiment of this application. Refer to FIG. 4. The image filter is a three-dimensional filter, a horizontal size of the image filter is W, a vertical size of the image filter is H, and a quantity of channels is C. Each grid in FIG. 4 represents one filter parameter, and the image filter includes H*W*C filter parameters. Processing an image by using the image filter is essentially processing the image by using filter parameters in the image filter.

[0080] Downsampling refers to compression processing. For example, performing downsampling on an image refers to performing compression processing on the image.

[0081] Upsampling refers to amplification processing. For example, performing upsampling on an image refers to performing amplification processing on the image.

[0082] In this application, when a super-resolution image of a first image needs to be obtained, an image filter of the first image may be first determined, and then super-resolution processing is performed on the first image by using the image filter, to obtain the super-resolution image of the first image. The image filter includes filter parameters corresponding to each pixel in the first image, and pixels with different texture features in the first image correspond to different filter parameters. In this way, when super-resolution processing is performed on the first image by using the image filter, different super-resolution processing may be performed on the pixels with different texture features in the first image, so that super-resolution processing performed on the pixels in the image is related to texture characteristics of the pixels, an effect of performing super-resolution processing on the image is better, and image processing reliability is improved.

[0083] Specific embodiments are used below to describe in detail the technical solutions of this application. It should be noted that the following embodiments may be independent or may be combined with each other, and same or similar content is not described repeatedly in different embodiments.

[0084] For ease of understanding, two manners of determining the image filter corresponding to the first image are first described. Embodiments shown in FIG. 5 to FIG. 8 are a manner of determining an image filter, and embodiments shown in FIG. 9 to FIG. 11 are another manner of determining an image filter.

[0085] FIG. 5 is a schematic flowchart of a method for determining an image filter according to an embodiment of this application. Refer to FIG. 5. The method may include the following steps.

[0086] S501: Obtain a texture image of a first image.

[0087] The first image is an image on which super-resolution processing is to be performed, and the first image is usually an image with relatively low resolution.

[0088] Optionally, the texture image of the first image may be obtained by using a convolutional neural network. For example, the texture image of the first image may be obtained by using any one of an LBP model, a GMRF model, or a GLCM model.

[0089] To improve efficiency of obtaining the texture image of the first image, the first image may be first compressed to a preset size (downsampling is performed on the first image to obtain an image of the preset size), and then a texture image of a compressed first image is obtained. Optionally, when a size of the first image is less than the preset size, compression processing may not be performed on the first image. For example, the preset size may be 256*256, 512*512, or the like. In an actual application process, the preset size may be set based on an actual requirement. It should be noted that a size of an image in this application is resolution of the image. For example, that the size of the first image is M*N means that the first image horizontally includes M pixels and vertically N pixels.

[0090] A size of the texture image of the compressed first image is the same as a size of the compressed first image. For example, if the size of the first image is M*N and the size of the compressed first image is M1*N1, the size of the texture image of the compressed first image is M1*N1. Because a quantity of pixels included in the compressed first image are reduced, the texture image of the compressed first image can be quickly obtained. Further, the texture image of the compressed first image also includes a relatively small quantity of pixels, so that a data volume for subsequently processing the texture image can be reduced.

[0091] S502: Determine, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image.

[0092] C=f.sup.2×t, f is a preset amplification multiple, and t is a quantity of filter parameters corresponding to each pixel in the first image. f is greater than 1, and t is an integer greater than or equal to 1.

[0093] Optionally, a value oft may be preset, t=a.sup.2, and a is usually an odd number greater than or equal to 1. For example, a may be 1, 3, 5, 7, or the like.

[0094] For example, if the first image needs to be amplified by f=3 times and a quantity of filter parameters corresponding to each pixel t=3.sup.2=9, C=81.

[0095] Optionally, each local texture image includes a part of textures in the texture image, and textures in one local texture image have a same texture feature. Optionally, one local texture image includes only face textures, and the face textures have a same texture feature. Alternatively, one local texture image includes only sky textures, and the sky textures have a same texture feature.

[0096] A size of the local texture image may be equal to the size of the texture image. If the first image is compressed to the preset size in S502, both the size of the texture image and the size of the local texture image are the preset size. If compression processing is not performed on the first image in S502, both the size of the texture image and the size of the local texture image are the same as the size of the first image.

[0097] The following describes a process of determining the local texture image with reference to FIG. 6.

[0098] FIG. 6 is a schematic diagram of a local texture image according to an embodiment of this application. Refer to FIG. 6. if C is 5, five local texture images may be determined in the texture image, which are separately denoted as a local texture image 1, a local texture image 2, a local texture image 3, a local texture image 4, and a local texture image 5.

[0099] Refer to FIG. 6. The local texture image 1 is a local texture image corresponding to a building, and the local texture image 1 includes a texture corresponding to the building. The local texture image 2 is a local texture image corresponding to a sky, and the local texture image 2 includes a texture corresponding to the sky. The local texture image 3 is a local texture image corresponding to grass, and the local texture image 3 includes a texture corresponding to the grass. The local texture image 4 is a local texture image corresponding to an auto, and the local texture image 4 includes a texture corresponding to the auto. The local texture image 5 is a local texture image corresponding to a cloud, and the local texture image 5 includes a texture corresponding to the cloud.

[0100] It should be noted that FIG. 6 shows, by way of example, only a part of local texture images that is determined in the texture image. Certainly, another local texture image may be further determined in the texture image. This is not specifically limited in this embodiment of this application.

[0101] One local texture image may correspond to one weight value. For example, a weight value of a local texture image may be a number between 0 and 1.

[0102] Optionally, the weight value of the local texture image may be determined based on an object type included in the local texture image. For example, a correspondence between the object type and the weight value may be preset. Correspondingly, the weight value of the local texture image may be determined based on the object type included in the local texture image and the correspondence. For example, an object type of an object in the local texture image may be identified, and the weight value of the local texture image may be determined based on the object type and the object correspondence.

[0103] Optionally, the weight value of the local texture image may be determined based on a texture feature of a texture in the local texture image. For example, a correspondence between the texture feature and the weight value may be preset. Correspondingly, the weight value of the local texture image may be determined based on the texture feature of the texture in the local texture image and the correspondence.

[0104] Optionally, a weight value of each local texture image may be further determined by using a preset model, and the preset model is obtained by learning a plurality of groups of sample images. Each group of sample images may include a sample low-resolution image and a corresponding sample high-resolution image. The plurality of groups of sample images may be input into the preset model, and the preset model may learn the plurality of groups of sample images, to obtain, through learning, a weight value of a sample local texture image corresponding to each sample low-resolution image when a sample low-resolution image is super-resolved into a sample high-resolution image. After the preset model learns the plurality of groups of sample images, the preset model has a function of determining the weight value of each local texture image. Therefore, the local texture image may be input into the preset model, so that the preset model outputs the weight value of the local texture image.

[0105] S503: Determine the image filter based on the C local texture images and the weight value of each local texture image, where the image filter is a three-dimensional image filter, and a quantity of channels of the image filter is C.

[0106] Optionally, each local texture image may be multiplied by a corresponding weight value to obtain an updated local texture image. Each updated local texture image corresponds to one channel of the image filter, and a pixel value in an updated local texture image is a filter parameter in one channel of the image filter. A horizontal size (W) of the image filter is the same as a horizontal size of the local texture image (a quantity of pixels included in each row of the local texture image), and a vertical size (H) of the image filter is the same as a vertical size of the local texture image (a quantity of pixels included in each column of the local texture image).

[0107] An (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image. i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally included in the first image, N is a quantity of pixels vertically included in the first image, and M and N are both integers greater than 1.

[0108] The following describes the image filter with reference to FIG. 7.

[0109] FIG. 7 is a schematic diagram of another image filter according to an embodiment of this application. Refer to FIG. 7. if C=5, the five local texture images are separately denoted as the local texture image 1, the local texture image 2, the local texture image 3, the local texture image 4, and the local texture image 5. After pixel values of the five local texture images are multiplied by corresponding weight values to obtain updated local texture images, pixel values of the five updated local texture images may be arranged in a manner shown in FIG. 7, and a pixel value of each updated local texture image is a filter parameter of one channel in the image filter.

[0110] It should be noted that FIG. 7 shows only a relationship between a local texture image and an image filter, and does not represent a real form of the image filter.

[0111] In the foregoing process, a weight value is set for each local texture image, so that a determined filter parameter of the image filter is related to the object type in the image. In this way, when the image is processed by using the image filter, smooth transition can be performed between different regions in a processed image, and an obvious segmentation line can be avoided between the different regions in the processed image.

[0112] The following describes an image processing effect obtained after the weigh values of the local texture image is used with reference to FIG. 8.

[0113] FIG. 8 is a schematic diagram of an image according to an embodiment of this application. Refer to FIG. 8. Super-resolution processing is performed on a raw image by using an image filter 1, to obtain a super-resolution image 1, and super-resolution processing is performed on the raw image by using a filter 2, to obtain a super-resolution image 2. In a process of determining the image filter 1, the weight value of the local texture image is not used, that is, the image filter 1 is directly determined based on the local texture image. In a process of determining the image filter 2, the weight value of the local texture image is used, that is, the image filter 2 is determined based on the local texture image and the weight value of the local texture image.

[0114] Refer to FIG. 8. In the super-resolution image 1, there is a relatively obvious segmentation line between a sky in an upper left corner and another part of the image. In other words, a transition between the sky in the upper left corner and the another part of the image is not smooth.

[0115] Refer to FIG. 8, in the super-resolution image 2, there is no relatively obvious segmentation line between a sky in an upper left corner and another part of the image. In other words, a transition between the sky in the upper left corner and the another part of the image is smooth.

[0116] In the embodiment shown in FIG. 5, the texture image of the first image is first obtained, the plurality of local texture images and the weight value of each local texture image in the texture image are determined, and the image filter is determined based on the local texture image and the weight value of the local texture image. Textures in one local texture image have a same texture feature. Therefore, in the image filter obtained by using the foregoing method, the filter parameters corresponding to each pixel in the first image are included, and the pixels with different texture features correspond to different filter parameters. When super-resolution processing is performed on the first image by using the image filter, different super-resolution processing may be performed on different pixels in the first image based on the image filter, thereby improving image processing reliability.

[0117] FIG. 9 is a schematic flowchart of another method for determining an image filter according to an embodiment of this application. Refer to FIG. 9. The method may include the following steps.

[0118] S901: Perform compression processing on a first image based on a preset size, where a size of a compressed first image is the preset size.

[0119] Optionally, when a size of the first image is greater than the preset size, compression processing may be performed on the first image. Optionally, when the size of the first image is less than the preset size, compression processing may not be performed on the first image.

[0120] For example, the preset size may be 256*256, 512*512, or the like. In an actual application process, the preset size may be set based on an actual requirement.

[0121] S902: Process the first image by using a recognition model, to obtain an image filter of the first image.

[0122] It should be noted that S901 may be an optional step. When S901 is performed, the compressed first image is processed by using the recognition model in S902. In this way, a data processing amount of the recognition model may be reduced, thereby improving efficiency of outputting an image filter by the recognition model. When S901 is not performed, the original first image is processed in S902.

[0123] The recognition model is obtained by learning a plurality of groups of samples, each group of samples includes a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

[0124] Optionally, the first sample image may be a high-resolution image, and the second sample image may be a low-resolution image. For example, the resolution of the first sample image is greater than or equal to a first threshold, and the resolution of the second sample image is less than or equal to a second threshold.

[0125] Before the embodiment shown in FIG. 9 is performed, the recognition model may be obtained through pre-learning. The following describes a process of obtaining the recognition model through learning.

[0126] Sample images with different texture features are first obtained to obtain a sample image set. For example, the sample image set may be shown in FIG. 10. FIG. 10 is a schematic diagram of a sample image set according to an embodiment of this application. Refer to FIG. 10. The sample image set includes a plurality of animal sample images, a plurality of face sample images, and a plurality of sky sample images.

[0127] The following describes, with reference to FIG. 11, a manner of obtaining the sample image set. FIG. 11 is a schematic flowchart of a method for obtaining a sample image set according to an embodiment of this application. Refer to FIG. 11. A super-resolution dataset, a dataset captured by using a search engine, and a self-collected dataset may be first obtained. Each dataset includes a plurality of initial sample images. The initial sample images in each dataset are input into a dataset processor, and the dataset processor processes the initial sample images in each dataset to obtain the sample image set.

[0128] Optionally, image segmentation (patch extraction) may be performed on the initial sample image in each data set, to extract required sample images from the initial sample images. For example, an initial sample image that includes a face, a sky, and a building is segmented to obtain sample image that includes only the face, a sample image that includes only the sky, and a sample image that includes only the building. Then, a data augmentation module is used to perform processing such as rotation, stretching, and scaling on the sample images obtained by image segmentation, to obtain the sample image set. Through the foregoing processing, the sample image set may include abundant sample images.

[0129] Optionally, the sample image set Dataset={Texture.sub.info|Texture∈AllData}.

[0130] Texture is a texture feature, and Data is a texture feature set. Texture.sub.info is an image with the texture feature Texture, for example, Texture.sub.info={Sky image|Face image|Animal image . . . }.

[0131] After the sample image set is obtained, a plurality of groups of sample images may be determined in the sample image set. For example, for a sample high-resolution image in the sample image set, compression processing is performed on the sample high-resolution image to obtain a corresponding sample low-resolution image, or for a sample low-resolution image in the sample image set, processing may be performed on the sample low-resolution image to obtain a corresponding sample high-resolution image.

[0132] After the plurality of groups of sample images are obtained, the plurality of groups of sample images may be input into the recognition model, and the recognition model may learn the plurality of groups of sample images. Because the plurality of groups of sample images include sample images with a plurality of texture characteristics, the recognition model may learn a sample image filter (or a filter parameter) corresponding to a sample low-resolution image in a process in which the sample low-resolution image (the second sample image) with various texture characteristics is super-resolved into a sample high-resolution image (the first sample image). After the recognition model learns the plurality of groups of sample images, the recognition model has a function of determining an image filter of an image. Only when the pixels with different texture features in the image filter correspond to different filter parameters, a high-resolution image obtained through super-resolution by using the image filter is a high-resolution image with a relatively good super-resolution effect. Because a sample high-resolution image in each group of sample images is a high-resolution image with a relatively good super-resolution effect, after the recognition model learns based on the foregoing plurality of groups of sample images, the recognition model has a function of output the image filter of the image, in addition, the output image filter has the following characteristics: The image filter includes filter parameters corresponding to each pixel in the image, and pixels with different texture features correspond to different filter parameters.

[0133] Optionally, data representing the first image, a preset amplification multiple, and a quantity t of filter parameters corresponding to each pixel may be input to the recognition model, and the recognition model outputs the image filter of the first image based on received data. The data representing the first image may be the first image, a grayscale image of the first image, or the like. For example, the recognition model may determine a quantity C of channels of the image filter based on a preset amplification multiple f and the quantity t of filter parameters corresponding to each pixel, and determine the image filter based on the quantity C of channels and the data representing the first image. The quantity of channels of the image filter output by the recognition model is C.

[0134] Optionally, a process shown in the embodiment in FIG. 9 may be described as follows: DTF.sub.(H,W,C)=Conv(resize(Input).sub.(H,W)), where Input is an input raw image, resize(Input).sub.(H,W) is to scale the input raw image to a size H*W, Cony represents the recognition model (a convolutional neural network), DTF.sub.(H,W,C) is the image filter output by the recognition model, H is a vertical size of the image filter, W is a horizontal size of the image filter, and C is the quantity of channels of the image filter.

[0135] In the embodiment shown in FIG. 9, the recognition model is obtained through pre-learning, and the image filter of the first image may be obtained by using the recognition model. The recognition model may learn filter parameters corresponding to images with various texture features from low-resolution images to high-resolution images by training the recognition model. Therefore, in the image filter output by the recognition model, the filter parameters of the image filter corresponding to the pixels with different texture features in the first image are different, so that different super-resolution processing may be performed on different pixels in the first image based on the image filter, thereby improving image processing reliability.

[0136] After the image filter of the first image is obtained by using any one of the foregoing methods, super-resolution processing may be performed on the first image by using the image filter to obtain a super-resolution image of the first image.

[0137] The following describes, by using an embodiment shown in FIG. 12 to FIG. 16, a processing of performing super-resolution on a first image by an image filter.

[0138] FIG. 12 is a schematic flowchart of a method for performing super-resolution on a first image by an image filter according to an embodiment of this application. Refer to FIG. 12. The method includes the following steps.

[0139] S1201: Obtain a gradient image of the first image.

[0140] A size of the gradient image is the same as that of the first image, the size of the gradient image is M*N, and M and N are both integers greater than 1.

[0141] Optionally, the gradient image of the first image may be obtained by using a convolutional neural network. For example, data representing the first image may be input into the convolutional neural network, and an output of the convolutional neural network is the gradient image of the first image.

[0142] Optionally, this step may be described as: GGF.sub.(H,W,1)=Conv(Input(H,W,3)), where Input.sub.(H,W,3) refers to the input first image, a quantity of channels of the first image is 3, the first image is usually an RGB image or a YUV image, and GGF.sub.(H,W,1) is to output a guided gradient image.

[0143] S1202: Process the gradient image by using the image filter, to obtain f.sup.2 sub-images.

[0144] A size of each sub-image is (f*M)*(f*N).

[0145] Optionally, it may be first determined whether H*W (W is a quantity of filter parameters horizontally included in a channel, and H is a quantity of filter parameters vertically included in a channel) of the image filter is consistent with the size of the gradient image (it is determined whether W is the same as a quantity of pixels included in the gradient image horizontally, and H is the same as a quantity of pixels included in the gradient image vertically). If H*W of the image filter is inconsistent with the size of the gradient image, in this case, H*W of the image filter is first adjusted to the size of the gradient image. W of an adjusted image filter is the same as the quantity of pixels included in the gradient image horizontally, and H of the adjusted image filter is the same as the quantity of pixels included in the gradient image vertically.

[0146] The gradient image is processed by using parameters in a (k*t+1).sup.th channel to a (k+1)*t.sup.th channel in the image filter, to obtain a k.sup.th sub-image, where k is sequentially set to 0, 1, . . . , and f.sup.2−1.

[0147] When each sub-image is obtained, a corresponding channel is first determined in the image filter, and the gradient image is processed (each pixel in the gradient image is processed) based on a filter parameter of the corresponding channel in the image filter, to obtain a sub-image. For example, the gradient image is processed by using parameters in a first channel to a k.sup.th channel in the image filter, to obtain a 0.sup.th sub-image. The gradient image is processed by using parameters in a (t+1).sup.th channel to a 20 channel in the image filter, to obtain a first sub-image. The gradient image is processed by using parameters in a (2t+1).sup.th channel to a 3t.sup.th channel in the image filter, to obtain a second sub-image. The rest may be deduced by analogy, until the f.sup.2 sub-images are obtained.

[0148] For any sub-image, in a process of obtaining the sub-image, a filter parameter corresponding to each pixel in the gradient image is first determined, and the pixel is processed according to the corresponding filter parameter, to obtain a pixel value of a corresponding pixel in the sub-image. Filter parameters corresponding to pixels in the gradient image may be determined in the following two feasible implementations.

[0149] In a feasible implementation, the H*W size of the image filter is the same as the size of the gradient image.

[0150] In this feasible implementation, coordinates of the pixels in the gradient image are the same as (h, w) coordinates of the corresponding filter parameters, where h is a positive integer between 0 and H−1, and w is a positive integer between 0 and W−1.

[0151] For example, h of a filter parameter corresponding to a pixel (0, 0) in the gradient image is 0, and w is 0. h of a filter parameter corresponding to a pixel (1, 2) in the gradient image is 1, and w is 2.

[0152] In the other feasible implementation, H*W of the image filter is different from the size of the gradient image.

[0153] In this feasible implementation, a correspondence between coordinates of pixels in the gradient image and (h, w) coordinates of filter parameters may be determined based on a ratio of H*W of the image filter to the size of the gradient image.

[0154] For example, if W is a half of a horizontal size of the gradient image, and H is a half of a vertical coordinate of the gradient image, h of filter parameters corresponding to pixels (0, 0), (0, 1), (1, 0), and (1, 1) in the gradient image is 0, and w is 0.

[0155] Each pixel in the gradient image corresponds to f filter parameters. There are a plurality of manners of processing one pixel by using the f filter parameters. The following describes, with reference to FIG. 13 and FIG. 14, a process of processing a pixel in a gradient image by using filter parameters.

[0156] FIG. 13 is a schematic diagram of processing a pixel by using filter parameters according to an embodiment of this application. FIG. 14 is another schematic diagram of processing a pixel by using filter parameters according to an embodiment of this application.

[0157] Refer to FIG. 13 and FIG. 14, if an amplification multiple f=3 and a quantity of filter parameters corresponding to one pixel is 9, a quantity C of channels of the image filter is 81 (FIG. 13 shows only some channels). When the 0.sup.th sub-image is obtained, the gradient image is processed by using parameters in a channel 1 to a channel 9 in the image filter.

[0158] Refer to FIG. 13. For the pixel (0, 0) in the gradient image, filter parameters corresponding to the pixel (0, 0) are nine filter parameters in the image filter whose h=0, w=0, and C=1 to 9. It is assumed that a pixel value of the pixel (0, 0) in the gradient image is 0.5, and a matrix formed by the nine filter parameters corresponding to the pixel (0, 0) is

[00001]$\left(\begin{array}{ccc}0.1& 0.1& 0.1\\ 0.1& 0.2& 0.1\\ 0.1& 0.1& 0.1\end{array}\right).$

Because there is no pixel around a part of the pixel (0, 0), a pixel 0 may be filled around the pixel (0, 0). A center (0.2) of the matrix is directly opposite to the pixel (0, 0), elements at corresponding positions are multiplied and then added, and an average value is obtained to obtain a pixel value on the pixel (0, 0).

[0159] For pixels located at edges in the gradient image, the pixels are processed in the manner shown in FIG. 13.

[0160] Refer to FIG. 14. For the pixel (1, 1) in the gradient image, filter parameters corresponding to the pixel (1, 1) are nine filter parameters in the image filter whose h=1, w=1, and C=1 to 9. It is assumed that a pixel value of the pixel (1, 1) in the gradient image is 0.5, and a matrix formed by the nine filter parameters corresponding to the pixel (1, 1) is

[00002]$\left(\begin{array}{ccc}0.1& 0.1& 0.1\\ 0.1& 0.1& 0.1\\ 0.1& 0.1& 0.2\end{array}\right).$

A center (0.1) of the matrix is directly opposite to the pixel (1, 1), elements at corresponding positions are multiplied and then added, and an average value is obtained to obtain a pixel value on the pixel (1, 1).

[0161] For pixels not located at the edges in the gradient image, the pixels are processed in the manner shown in FIG. 14.

[0162] It should be noted that FIG. 13 and FIG. 14 merely illustrate a process of processing a pixel by using filter parameters in a schematic form, but do not limit processing performed on a pixel by using filter parameters.

[0163] S1203: Separately perform splicing processing on pixels at a same pixel position in the f.sup.2 sub-images, to obtain M*N image blocks.

[0164] A size of each image block is f*f.

[0165] Optionally, f.sup.2 pixels whose coordinates are (0, 0) in the f.sup.2 sub-images are spliced, to obtain an image block corresponding to the coordinate (0, 0). f.sup.2 pixels whose coordinates are (0, 1) in the f.sup.2 sub-images are spliced, to obtain an image block corresponding to the coordinate (0, 1). The rest may be deduced by analogy, until the M*N image blocks are obtained.

[0166] The following describes a process of determining the image block with reference to FIG. 15.

[0167] FIG. 15 is a schematic diagram of a process of determining an image block according to an embodiment of this application. Refer to FIG. 15. If f=3, nine sub-images are determined, which are respectively denoted as a sub-image 1, a sub-image 2, . . . , and a sub-image 9. Nine pixels whose coordinates are (0, 0) in the nine sub-images are spliced to obtain an image block 1 corresponding to the coordinate (0, 0), nine pixels whose coordinates are (0, 1) in the nine sub-images are spliced to obtain an image block 2 corresponding to the coordinate (0, 1), . . . , and nine pixels whose coordinates are (2, 2) in the nine sub-images are spliced to obtain an image block 9 corresponding to the coordinate (2, 2).

[0168] It should be noted that, numbers in the sub-images and the image blocks shown in FIG. 15 are numbers of pixels. For example, “11” in the sub-image 1 is a number of a pixel (0, 0) in the sub-image 1, and “12” in the sub-image 1 is a number of a pixel (0, 1) in the sub-image 1. “11” in the sub-image 1 is a number of a pixel (0, 0) in the image block 1, and “21” in the sub-image 11 is a number of a pixel (0, 1) in the image block 1.

[0169] S1204: Perform splice processing on the M*N image blocks based on pixel locations of pixels in each image block in the sub-image, to obtain the second image.

[0170] A size of the second image is (f*M)*(f*N).

[0171] Optionally, a position of each image block may be determined based on a pixel position of a pixel in each image block in the sub-image, and splicing processing is performed on the M*N image blocks based on the placement position of each image block. The position of each image block corresponds to a pixel location of a pixel in the image block in the sub-image.

[0172] For example, if a pixel position of a pixel in the image block 1 in the sub-image is (0, 0), and a pixel position of a pixel in the image block 2 in the sub-image is (0, 1), a position of the image block 2 is located after the image block 1.

[0173] The following describes a process of splicing the image blocks with reference to FIG. 16.

[0174] FIG. 16 is a schematic diagram of a process of splicing image blocks according to an embodiment of this application. Refer to FIG. 16. it is assumed that nine image blocks are obtained, which are respectively denoted as the image block 1, the image block 2, . . . , and the image block 9. It is assumed that pixel locations of pixels in the nine image blocks are shown in FIG. 15. The second image shown in FIG. 16 is obtained by splicing the nine image blocks based on the pixel locations, in the sub-image, of pixels in an image block.

[0175] It should be noted that, numbers in the image blocks and the second image shown in FIG. 16 are numbers of pixels. For example, “11” in the image block 1 is a number of the pixel (0, 0) in the image block 1, and “21” in the image block 11 is a number of the pixel (0, 1) in the image block 1. “11” in the second image is a number of a pixel (0, 0) in the second image, and “21” in the second image is a number of a pixel (0, 1) in the second image.

[0176] Optionally, S1202 to S1204 may be described as: DTFS.sub.(H,W,C)=Conv(GGF.sub.(H,W,1), DTF.sub.(H,W,C))), where GGF.sub.(H,W,1) represents the gradient image, DTF.sub.(H,W,C)) represents the image filter, and DTFS.sub.(H,W,C) represents a feature map of fine texture adjustment after the gradient image is fused with the image filter.

[0177] According to the steps shown in S1202 to S1204, the second image may be determined based on the image filter and the gradient image. Certainly, the second image may be alternatively determined in another feasible implementation. This is not specifically limited in this embodiment of this application.

[0178] S1205: Amplify the first image by f times to obtain a third image.

[0179] A size of the third image is (f*M)*(f*N).

[0180] Optionally, the first image may be amplified by f times in a manner of bicubic interpolation amplification, to obtain the third image. Certainly, the first image may be alternatively amplified in another feasible implementation. This is not specifically limited in this embodiment of this application.

[0181] S1206: Obtain a super-resolution image based on the second image and the second image.

[0182] A size of the super-resolution image is (f*M)*(f*N).

[0183] Optionally, pixel values in the third image and pixel values in the second image may be correspondingly added to obtain the super-resolution image.

[0184] For example, a pixel value at a pixel position (0,0) in the third image and a pixel value at a pixel position (0,0) in the second image may be added together to serve as a pixel value at a pixel position (0,0) in the super-resolution image.

[0185] In the embodiment shown in FIG. 12, after the image filter is determined, the gradient image of the first image is first processed by using the image filter to obtain the second image, and then the super-resolution image is obtained based on the second image and the third image that is obtained by amplifying the first image. A data volume of the gradient image is less than a data volume of the first image. Therefore, the gradient image may be quickly processed by using the image filter, so that the super-resolution image may be quickly determined.

[0186] The following describes, with reference to FIG. 17, an image processing method.

[0187] FIG. 17 is a schematic flowchart of an image processing method according to an embodiment of this application. Referring to FIG. 17, the method may include the following steps.

[0188] S1701: Determine an image filter corresponding to a first image.

[0189] Pixels with different texture features in the first image correspond to different filter parameters of the image filter.

[0190] It should be noted that the image filter may be determined by using the methods shown in the embodiment in FIG. 5 to FIG. 8, or the image filter may be determined by using the methods shown in FIG. 9 to FIG. 11. Details are not described herein again.

[0191] S1702: Perform super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

[0192] It should be noted that the super-resolution processing may be performed on the first image by using the methods shown in the embodiment in FIG. 12 to FIG. 16 to obtain the super-resolution image of the first image. Details are not described herein again.

[0193] Optionally, this step may be represented as: HR.sub.(H,W)=Conv(DTFS.sub.(H,W,C), LR.sub.(H/f,W/f)), where HR.sub.(H,W) is the super-resolution image, DTFS.sub.(H,W,C) is the second image, LR.sub.(H/f,W/f) is the first image, H is a horizontal size of the first image, W is a vertical size of the first image, and f is an amplification multiple.

[0194] According to the image processing method provided in this embodiment of this application, when the super-resolution image of the first image needs to be obtained, the image filter of the first image may be first determined, and then super-resolution processing is performed on the first image by using the image filter, to obtain the super-resolution image of the first image. There is a correspondence between the filter parameters in the image filter and the pixels in the first image, and the pixels with different texture features in the first image correspond to different filter parameters of the image filter. In this way, when super-resolution processing is performed on the first image by using the image filter, different super-resolution processing may be performed on pixels with different texture features in the first image, so that super-resolution processing performed on the pixels in the image is related to the texture characteristics of the pixel, an effect of performing super-resolution processing on the image is better, and image processing reliability is improved.

[0195] The following describes a process of image processing with reference to FIG. 18.

[0196] FIG. 18 is a schematic diagram of a process of image processing according to an embodiment of this application. Refer to FIG. 18. When super-resolution processing is required to be performed on a raw image (a first image), downsampling is performed on the first image, and an image filter is determined by using the methods shown in the embodiment in FIG. 5 to FIG. 8, or the image filter is determined by using the methods shown in the embodiment in FIG. 9 to FIG. 11. After the image filter is determined, upsampling is performed on the image filter, so that an H*W size of the image filter obtained after upsampling is the same as a size of the first image.

[0197] A gradient image of the first image is further obtained, and the image filter and the gradient image are processed by using the methods shown in the embodiment in FIG. 12 to FIG. 16, to obtain a second image. Upsampling is performed on the first image to obtain an amplified image, and the second and the amplified image are processed by using the methods shown in the embodiment in FIG. 12 to FIG. 16, to obtain a super-resolution image.

[0198] In the embodiment shown in FIG. 18, the first image includes objects such as a sky, a cloud, a car, a building, and grass. In a process of performing super-resolution processing on the first image, different objects correspond to different filter coefficients. In this way, different super-resolution processing may be performed on different objects in the first image, so that problems such as sawtooth, white borders, and blurring that occur on some objects because same super-resolution processing is performed on all objects in the image can be avoided, and the super-resolution image is more real and natural. This improves image processing reliability.

[0199] The image processing method shown in the foregoing embodiments may be applied to a plurality of application scenarios. For example, the image processing method shown in the foregoing embodiment may be applied to display an image by a terminal device. To be specific, before the terminal device displays an image, super-resolution processing is performed on the image by using the image processing method shown in the foregoing embodiment, so that the image displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the foregoing embodiments may be further applied to a video call scenario of the terminal device. To be specific, in a process in which the terminal device performs video passing with another terminal device, the terminal device may receive a video frame from the another terminal device, and the terminal device may perform super-resolution processing on each frame of image in the video frame by using the method shown in the foregoing embodiment, so that a video picture displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the foregoing embodiments may be further applied to a scenario in which the terminal device plays a video. To be specific, in a process in which the terminal device plays the video, the terminal device may perform super-resolution processing on each frame of image in a video frame by using the method shown in the foregoing embodiment, so that a video picture displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the foregoing embodiments may be further applied to a game scenario. To be specific, before the terminal device displays a game picture, the terminal device may perform super-resolution processing on the game picture by using the method shown in the foregoing embodiment, so that the game picture displayed by the terminal device is clearer and more natural.

[0200] Optionally, the terminal device in this embodiment of this application may be a mobile phone, a computer, a television, an in-vehicle terminal (or an unmanned driving system), an augmented reality (augmented reality, AR) device, a virtual reality (virtual reality, VR) device, a hybrid reality device, a wearable device, a smart home device, an unmanned aerial vehicle terminal device, or the like.

[0201] Certainly, the image processing method shown in the foregoing embodiment may be further applied to another application scenario. This is not specifically limited in this embodiment of this application.

[0202] The following describes, with reference to FIG. 19, an application scenario of an image processing method.

[0203] FIG. 19 is a schematic diagram of an application scenario according to an embodiment of this application. Refer to FIG. 19. An application program is installed on a terminal device. For example, the application program may be an image display application program, a video playing application program, a video call application program, or the like. The application program may obtain a media stream (for example, a video stream). A decoder in the terminal device decodes the media stream to obtain a raw image. The terminal device performs super-resolution processing on the raw image by using the image processing method shown in the foregoing embodiments, to obtain a super-resolution image. The terminal device displays the super-resolution image.

[0204] Optionally, when the media stream is a video stream, when a video frame is processed by using the image processing method shown in this embodiment of this application, because content (or a scenario) in a video usually does not change in a short time, to improve processing efficiency, an image filter may be determined once every T frames. In other words, an image filter determined based on one image frame may be applied to T−1 image frames after the frame. For example, T may be any quantity ranging from 5 to 10.

[0205] Optionally, the image processing method shown in the foregoing embodiments may be implemented by using a processing model. For example, sample data may be obtained, and the sample data may be trained to obtain the processing model. The processing model may implement the foregoing image processing method. Optionally, in an actual application process, training may be performed on a personal computer (personal computer, PC) based on the sample data to obtain the foregoing processing model, the foregoing processing model is converted into an offline model on the computer, and the offline model may be moved to any other terminal device (for example, a mobile device such as a mobile phone or a tablet computer), so that the terminal device may perform image processing by using the processing model.

[0206] It should be noted that, for a training process of the processing model, refer to the training process of the recognition model in the foregoing embodiments. Details are not described herein again.

[0207] The following describes a process of generating and using the processing model with reference to FIG. 20.

[0208] FIG. 20 is a schematic diagram of a process of generating and using a processing model according to an embodiment of this application. Refer to FIG. 20. The processing model may be obtained through training on a PC based on sample data, and the foregoing processing model is converted into an offline model on a computer. The offline model is installed on a mobile terminal. For example, after an application program in the mobile terminal obtains a media stream (for example, a video stream), a decoder in a terminal device decodes the media stream to obtain a raw image. The terminal device performs super-resolution processing on the raw image by using the offline processing model, to obtain a super-resolution image. The terminal device displays the super-resolution image.

[0209] FIG. 21 is a schematic structural diagram of an image processing apparatus according to an embodiment of this application. Refer to FIG. 21. The image processing apparatus 10 may include a determining module 11 and a super-resolution module 12.

[0210] The determining module 11 is configured to determine an image filter corresponding to a first image, where the image filter includes filter parameters corresponding to each pixel in the first image, and pixels with different texture features correspond to different filter parameters.

[0211] The super-resolution module 12 is configured to perform super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

[0212] Optionally, the determining module 11 may perform S501 to S503 in the embodiment in FIGS. 5, S901 and S902 in the embodiment in FIG. 9, and S1701 in the embodiment in FIG. 17.

[0213] Optionally, the super-resolution module may perform S1201 to S1206 in the embodiment in FIG. 12, and S1702 in the embodiment in FIG. 17.

[0214] It should be noted that the image processing apparatus shown in the embodiments of this application may perform the technical solutions shown in the foregoing method embodiments. Implementation principles and beneficial effects of the image processing apparatus are similar to those of the technical solutions, and details are not described herein again.

[0215] In a possible implementation, the determining module 11 is specifically configured to: obtain a texture image of the first image; determine, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image, where textures in the local texture images have a same texture feature, C=f.sup.2×t, f is a preset amplification multiple, t is a quantity of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1; and determine the image filter based on the C local texture images and the weight value of each local texture image, where a quantity of channels of the image filter is C.

[0216] In a possible implementation, the determining module 11 is specifically configured to: perform compression processing on the first image based on a preset size, where a size of a compressed first image is the preset size; and determine the texture image based on the compressed first image.

[0217] In a possible implementation, an (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image. i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally included in the first image, N is a quantity of pixels vertically included in the first image, and M and N are both integers greater than 1.

[0218] In a possible implementation, the determining module 11 is specifically configured to: process the first image by using a recognition model, to obtain the image filter of the first image, where the recognition model is obtained by learning a plurality of groups of samples, each group of samples includes a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

[0219] In a possible implementation, the determining module 11 is specifically configured to: perform compression processing on the first image based on a preset size, where a size of a compressed first image is the preset size; and process the compressed first image by using the recognition model.

[0220] In a possible implementation, the super-resolution module 12 is specifically configured to: obtain a gradient image of the first image, where a size of the gradient image is the same as that of the first image, the size of the gradient image is M*N, and M and N are both integers greater than 1; process the gradient image by using the image filter, to obtain a second image, where a size of the second image is (f*M)*(f*N), and f is a preset amplification multiple; amplify the first image by f times to obtain a third image, where a size of the third image is (f*M)*(f*N); and obtain the super-resolution image based on the second image and the third image, where a size of the super-resolution image is (f*M)*(f*N).

[0221] In a possible implementation, the super-resolution module 12 is specifically configured to: process the gradient image by using the image filter, to obtain f.sup.2 sub-images, where a size of each sub-image is (f*M)*(f*N); and determine the second image based on the f.sup.2 sub-images.

[0222] In a possible implementation, the super-resolution module 12 is specifically configured to: process the gradient image by using parameters in a (k*t+1).sup.th channel to a (k+1)*t.sup.th channel in the image filter, to obtain a k.sup.th sub-image, where k is sequentially set to 0, 1, . . . , and f.sup.2−1.

[0223] In a possible implementation, the super-resolution module 12 is specifically configured to: separately perform splicing processing on pixels at a same pixel position in the f.sup.2 sub-images, to obtain M*N image blocks, where a size of each image block is f*f; and perform splice processing on the M*N image blocks based on pixel locations of pixels in each image block in the sub-image, to obtain the second image.

[0224] It should be noted that the image processing apparatus shown in the embodiments of this application may perform the technical solutions shown in the foregoing method embodiments. Implementation principles and beneficial effects of the image processing apparatus are similar to those of the technical solutions, and details are not described herein again.

[0225] FIG. 22 is a schematic structural diagram of hardware of a computer system according to an embodiment of this application. Refer to FIG. 22. The computer system 20 includes a memory 21 and a processor 22, where the memory 21 communicates with the processor 22. For example, the memory 21 and the processor 22 may communicate with each other through a communications bus 23. The memory 21 is configured to store a computer program, and the processor 22 executes the computer program to perform the following steps: determining an image filter corresponding to a first image, where the image filter includes filter parameters corresponding to each pixel in the first image, and pixels with different texture features correspond to different filter parameters; and performing super-resolution processing on the first image based on the image filter of the first image to obtain a super-resolution image of the first image.

[0226] Optionally, the processor 21 may implement functions of the determining module 11 and the super-resolution module 12 shown in the embodiment in FIG. 21.

[0227] It should be noted that the image processing apparatus shown in the embodiments of this application may perform the technical solutions shown in the foregoing method embodiments. Implementation principles and beneficial effects of the image processing apparatus are similar to those of the technical solutions, and details are not described herein again.

[0228] In a possible implementation, the processor 22 is specifically configured to: obtain a texture image of the first image; determine, in the texture image, C local texture images and a weight value of each local texture image based on texture features of pixels in the texture image, where textures in the local texture images have a same texture feature, C=f.sup.2×t, f is a preset amplification multiple, t is a quantity of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1; and determine the image filter based on the C local texture images and the weight value of each local texture image, where a quantity of channels of the image filter is C.

[0229] In a possible implementation, the processor 22 is specifically configured to: perform compression processing on the first image based on a preset size, where a size of a compressed first image is the preset size; and determine the texture image based on the compressed first image.

[0230] In a possible implementation, an (x, y).sup.th filter parameter in an i.sup.th channel of the image filter is a product of a pixel value of an (x, y).sup.th pixel in an i.sup.th local texture image and a weight value of the i.sup.th local texture image. i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is a quantity of pixels horizontally included in the first image, N is a quantity of pixels vertically included in the first image, and M and N are both integers greater than 1.

[0231] In a possible implementation, the processor 22 is specifically configured to: process the first image by using a recognition model, to obtain the image filter of the first image, where the recognition model is obtained by learning a plurality of groups of samples, each group of samples includes a first sample image and a second sample image, the first sample image and the second sample image have same image content, and resolution of the first sample image is higher than resolution of the second sample image.

[0232] In a possible implementation, the processor 22 is specifically configured to: perform compression processing on the first image based on a preset size, where a size of a compressed first image is the preset size; and process the compressed first image by using the recognition model.

[0233] In a possible implementation, the processor 22 is specifically configured to: obtain a gradient image of the first image, where a size of the gradient image is the same as that of the first image, the size of the gradient image is M*N, and M and N are both integers greater than 1; process the gradient image by using the image filter, to obtain a second image, where a size of the second image is (f*M)*(f*N), and f is a preset amplification multiple; amplify the first image by f times to obtain a third image, where a size of the third image is (f*M)*(f*N); and obtain the super-resolution image based on the second image and the third image, where a size of the super-resolution image is (f*M)*(f*N).

[0234] In a possible implementation, the processor 22 is specifically configured to: process the gradient image by using the image filter, to obtain f.sup.2 sub-images, where a size of each sub-image is (f*M)*(f*N); and determine the second image based on the f.sup.2 sub-images.

[0235] In a possible implementation, the processor 22 is specifically configured to: process the gradient image by using parameters in a (k*t+1).sup.th channel to a (k+1)*t.sup.th channel in the image filter, to obtain a k.sup.th sub-image, where k is sequentially set to 0, 1, . . . , and f.sup.2−1.

[0236] In a possible implementation, the processor 22 is specifically configured to: separately perform splicing processing on pixels at a same pixel position in the f.sup.2 sub-images, to obtain M*N image blocks, where a size of each image block is f*f; and perform splice processing on the M*N image blocks based on pixel locations of pixels in each image block in the sub-image, to obtain the second image.

[0237] It should be noted that the image processing apparatus shown in the embodiments of this application may perform the technical solutions shown in the foregoing method embodiments. Implementation principles and beneficial effects of the image processing apparatus are similar to those of the technical solutions, and details are not described herein again.

[0238] Optionally, the foregoing processor may be a CPU, or may be another general-purpose processor, a DSP, an ASIC, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor or the like. Steps of the authentication method embodiments disclosed with reference to this application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and a software module in a processor.

[0239] This application provides a computer-readable storage medium, where the computer-readable storage medium is configured to store a computer program, and the computer program is used to implement the image processing method according to the foregoing embodiments.

[0240] This application further provides a chip or an integrated circuit, including a memory and a processor.

[0241] The memory is configured to store program instructions, and is sometimes configured to store intermediate data.

[0242] The processor is configured to invoke the program instructions stored in the memory, to implement the foregoing image processing method.

[0243] Optionally, the memory may be independent, or may be integrated with the processor. In some implementations, the memory may alternatively be located outside the chip or the integrated circuit.

[0244] An embodiment of this application further provides a program product, where the program product includes a computer program, the computer program is stored in a storage medium, and the computer program is used to implement the foregoing image processing method.

[0245] All or some of the steps in the method embodiments may be implemented by hardware related to program instructions. The foregoing program may be stored in a computer-readable memory. When the program is executed, the steps of the methods in the embodiments are performed. The memory (storage medium) includes: a read-only memory (English: read-only memory, ROM for short), a RAM, a flash memory, a hard disk, a solid state drive, a magnetic tape (English: magnetic tape), a floppy disk (English: floppy disk), an optical disc (English: optical disc), and any combination thereof.

[0246] The embodiments of this application are described with reference to the flowcharts and/or block diagrams of the method, the device (system), and the computer program product according to the embodiments of this application. It should be understood that computer program instructions may be used to implement each process and/or each block in the flowcharts and/or the block diagrams and a combination of a process and/or a block in the flowcharts and/or the block diagrams. These computer program instructions may be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processing unit of any other programmable data processing device to generate a machine, so that the instructions executed by a computer or a processing unit of any other programmable data processing device generate an apparatus for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

[0247] These computer program instructions may be stored in a computer readable memory that can instruct the computer or any other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

[0248] These computer program instructions may be loaded onto a computer or another programmable data processing device, so that a series of operations and steps are performed on the computer or the another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or the another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.

[0249] Obviously, a person skilled in the art can make various modifications and variations to embodiments of this application without departing from the spirit and scope of this application. This application is intended to cover these modifications and variations provided that they fall within the scope of protection defined by the following claims and their equivalent technologies.

[0250] In this application, the term “including” and a variant thereof may refer to non-limitative inclusion; and the term “or” and a variant thereof may refer to “and/or”. In this application, the terms “first”, “second”, and the like are intended to distinguish between similar objects but do not necessarily indicate a specific order or sequence. “A plurality of” in this application refers to two or more than two. The term “and/or” describes an association relationship for describing associated objects and represents that three relationships may exist. For example, A and/or B may represent the following three cases: Only A exists, both A and B exist, and only B exists. The character “/” generally indicates an “or” relationship between the associated objects.