AI VIDEO PROCESSING METHOD AND APPARATUS

Abstract

The method comprises: connecting to a plurality of AI computing boards in an AI processing resource pool and a plurality of video encoding and decoding boards in a video processing resource pool by means of a unified high-speed interface; respectively allocating a specified number of AI computing boards and video encoding and decoding boards on account of resources and bandwidths required for completing a processing task to form a temporary cooperation relationship based on the processing task; in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, accessing more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; performing the processing task on account of the allocated AI computing boards or video encoding and decoding boards, and releasing the temporary cooperation relationship.

Claims

1. An artificial intelligence (AI) video processing method, comprising: performing, by a control device, following steps: connecting to a plurality of AI computing boards in an AI processing resource pool and a plurality of video encoding and decoding boards in a video processing resource pool by means of a unified express interface, so as to call AI processing resources and video processing resources; in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of the AI computing boards and the video encoding and decoding boards on account of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task; in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; and performing the processing task on account of the allocated AI computing boards or the allocated video encoding and decoding boards, and releasing the temporary cooperation relationship in response to completion of the processing task.

2. The method according to claim 1, wherein each of the AI computing boards is provided with a first number of AI computing chips of same model, and each of the video encoding and decoding boards is provided with a second number of video encoding and decoding chips of same model, wherein the first number and the second number are configured to be determined on account of bandwidths of the unified express interface and complexity of physical connecting lines between the AI computing boards and the video encoding and decoding boards.

3. The method according to claim 2, wherein video encoding and decoding supported by the video encoding and decoding chips comprises at least one of MPEG, H.264, H.265, AVS, or AVS+.

4. The method according to claim 1, wherein the connecting by means of the unified express interface comprises: establishing, by the control device, at least one of a direct connection by means of a peripheral component interconnect express (PCIE) physical interface on a main board, or an indirect connection via a switchboard having a PCIE switching chip.

5. The method according to claim 4, wherein the control device comprises a central processing unit (CPU) arranged on the main board, and at least one of a single chip micyoco (SCM) or a processor arranged on the switchboard.

6. An artificial intelligence (AI) video processing apparatus, comprising: an AI processing resource pool, comprising a plurality of AI computing boards for performing AI processing; a video processing resource pool, comprising a plurality of video encoding and decoding boards for performing video processing; a control device, connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards by means of a unified high-speed interface, and comprising a processor and a memory, wherein the memory stores computer instructions executable on the processor, and the computer instructions, when executed by the processor, implement following steps: in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of the AI computing boards and the video encoding and decoding boards on account of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task; in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; and calling the allocated AI computing boards or the allocated video encoding and decoding boards as AI processing resources and video processing resources for performing the processing task, and releasing the temporary cooperation relationship in response to completion of the processing task.

7. The apparatus according to claim 6, wherein each of the AI computing boards is provided with a first number of AI computing chips of same model, and each of the video encoding and decoding boards is provided with a second number of video encoding and decoding chips of same model, wherein the first number and the second number are configured to be determined on account of bandwidths of the unified high-speed interface and complexity of physical connecting lines between the AI computing boards and the video encoding and decoding boards.

8. The apparatus according to claim 7, wherein video encoding and decoding supported by the video encoding and decoding chips comprises at least one of MPEG, H.264, H.265, AVS, or AVS+.

9. The apparatus according to claim 6, wherein at least one of the control device is directly connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards by means of a peripheral component interconnect express (PCIE) physical interface on a main board; or the apparatus further comprises a switchboard having a PCIE switching chip, wherein the control device is indirectly connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards via the switchboard.

10. The apparatus according to claim 9, wherein the control device comprises a central processing unit (CPU) arranged on the main board, and at least one of a single chip micyoco (SCM) or a processor arranged on the switchboard.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to describe the technical solution in the embodiments of the present disclosure or the prior art more clearly, drawings required to be used in the description of the embodiments or the prior art will be briefly introduced below. Apparently, the drawings in the description below are only some embodiments of the present disclosure. Those ordinarily skilled in the art also can obtain other drawings according to the provided drawings without creative work.

[0031] FIG. 1 is a flowchart of an AI video processing method according to the present disclosure.

[0032] FIG. 2 is a schematic structural diagram of a direct connection form of an AI video processing apparatus according to the present disclosure.

[0033] FIG. 3 is a schematic structural diagram of an indirect connection form of an AI video processing apparatus according to the present disclosure.

DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the present disclosure clearer, embodiments of the present disclosure will be further described in detail below with reference to specific embodiments and the accompanying drawings.

[0035] It should be noted that all the expressions using “first” and “second” in the embodiments of the present disclosure are intended to distinguish two different entities with the same name or different parameters. It can be seen that “first” and “second” are merely for the convenience of expressions and should not be construed as limiting the embodiments of the present disclosure, and the subsequent embodiments will not make illustrations thereto one by one.

[0036] On the basis of the above object, in a first aspect of embodiments of the present disclosure, one embodiment of an AI video processing method capable of efficiently adapting to different application scenario algorithms is provided. FIG. 1 shows a flowchart of an AI video processing method according to the present disclosure.

[0037] The AI video processing method, as shown in FIG. 1, includes: performing, by a control device, the following steps:

[0038] Step S101: connecting to a plurality of AI computing boards in an AI processing resource pool and a plurality of video encoding and decoding boards in a video processing resource pool by means of a unified high-speed interface, so as to call AI processing resources and video processing resources.

[0039] Step S103: in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of AI computing boards and video encoding and decoding boards on the basis of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task.

[0040] Step S105: in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards.

[0041] Step S107: performing the processing task on the basis of the allocated AI computing boards or the allocated video encoding and decoding boards, and releasing the temporary cooperation relationship in response to completion of the processing task.

[0042] According to the requirements of general AI video processing acceleration, the present disclosure provides a general architecture and system form in the level of a board consisting of AI chips and video decoding chips. On the premise of completing an AI video processing acceleration function, the flexible expansibility of the AI processing capacity and the video encoding and decoding capacity is maintained by means of resource pooling, thereby realizing the use and upgrading under different application scenarios and algorithms.

[0043] It will be appreciated by those ordinarily skilled in the art that implementing all or part of the processes in the above embodiment methods may be completed by a computer program that instructs associated hardware. The program may be stored in a computer-readable storage medium. When executed, the program may include processes of the above method embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. The embodiments of the computer program may achieve the same or similar effects as any of the previously described method embodiments corresponding thereto.

[0044] In some implementations, each of the AI computing boards is provided with a first number of AI computing chips of the same model, and each of the video encoding and decoding boards is provided with a second number of video encoding and decoding chips of the same model. The first number and the second number are configured to be determined on the basis of bandwidths of the unified high-speed interface and complexity of physical connecting lines between the AI computing boards and the video encoding and decoding boards.

[0045] In some implementations, video encoding and decoding supported by the video encoding and decoding chips includes at least one of MPEG, H.264, H.265, AVS, and AVS+.

[0046] In some implementations, the connecting by means of the unified high-speed interface includes: establishing, by the control device, a direct connection by means of a PCIE physical interface on a main board, and/or an indirect connection via a switchboard having a PCIE switching chip.

[0047] In some implementations, the control device includes a CPU arranged on the main board, and a SCM and/or an ARM processor arranged on the switchboard.

[0048] The method disclosed according to the embodiments of the present disclosure may also be implemented as a computer program executed by a CPU. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by the CPU, the above functions defined in the method disclosed according to the embodiments of the present disclosure are performed. The above method steps and system units may also be implemented using a controller and a computer-readable storage medium for storing a computer program for causing the controller to perform the functions of the above steps or units.

[0049] Specific implementations of the present disclosure are further illustrated below according to specific embodiments.

[0050] Firstly, for selected AI chips and video encoding and decoding chips, a uniform high-speed interface is required. In the present disclosure, considering the compatibility, the current most mainstream PCIE3.0 interface is selected. Except for low-power chips applied on a device side, most chips in the current market support a PCIE interface. Moreover, the PCIE interface has the characteristic of forward compatibility, and even if PCIE4.0 becomes the mainstream interface on the market, existing chips can be compatible for use. If the chip is not compatible with PCIE3.0, an interface conversion module may be added in the board-level design.

[0051] In order to maximally maintain the generality, video encoding and decoding chips should support as many video standards as possible, including MPEG, H.264, H.265, AVS, AVS+, etc. Some of the products in the prior art do not support certain standards, and video standards beyond the primary application scenarios are abandoned only for power consumption and area considerations. However, the present disclosure is less sensitive to the power consumption area of a single chip.

[0052] The AI chips and the video encoding and decoding chips are separately placed on different daughter boards, and the board-level design is performed independently. On the one hand, the number of chips placed on a single board may be evaluated according to the board-level power consumption and the complexity of a physical connecting line. On the other hand, the amount of data transmission between the AI chips and the video encoding and decoding chips needs to be considered. If too many chips are placed, the interface bandwidth may become the bottleneck of overall performance In the present disclosure, the daughter board is connected to a host side through PCIE3.0/4.0, and two or four chips are generally placed, which specification is half of the mainstream PCIE cards in height and length. Compared with the board-level design of heterogeneous multi-core chips and heterogeneous multi-chips, the scheme of only placing the chips of the same type on the same daughter board is more prone to layout and more stable design.

[0053] AI chip daughter cards and video encoding and decoding daughter cards do not have a one-to-one corresponding relationship, but respectively construct a resource pool. That is to say, there may be a plurality of daughter cards, and if the number of daughter cards is small, a PCIE interface of a main board may be directly used for connection. If the number is large, a switch card with a PCIE switching chip needs to be added for connection.

[0054] A controller is required for data transmission between the AI processing resource pool and the video encoding and decoding resource pool. In a system with small resource pools, it may be directly controlled by a CPU, the two resource pools communicate with the CPU in an interrupted manner, and the CPU sends a control signal to complete transmission according to rules. In a system with large resource pools, in order to ensure the efficiency and reduce the occupation of CPU time, a micro control unit (which may be an embedded SCM or an ARM processor) may be added to a switchboard of the PCIE switching chip for managing the data transmission of the resource pools. The two cases are shown in FIGS. 2 and 3, respectively.

[0055] There is no longer a fixed corresponding relationship between a single AI chip and one or more video encoding and decoding chips, but an interaction between the two resource pools. Therefore, for the matching of processing capacities, it is only necessary to consider the processing capacity of an overall resource pool and the limitation of data transmission bandwidth (if there are too many daughter cards connected to a single switchboard and the communication between the daughter cards is too frequent, data congestion may be caused; in this case, a more complex bus structure needs to be adopted, but it is generally impossible to place such a large-scale resource pool on a single server for the design of the overall system). When the switching of application scenarios and algorithms results in transformation of a proportional relationship between processing capacity requirements of resource pool, this problem can be solved by removing or adding daughter cards.

[0056] It can be seen from the above embodiments that, in the AI video processing method provided by the embodiments of the present disclosure, the technical solution includes: connecting to a plurality of AI computing boards in an AI processing resource pool and a plurality of video encoding and decoding boards in a video processing resource pool by means of a unified high-speed interface, so as to call AI processing resources and video processing resources; in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of AI computing boards and video encoding and decoding boards on the basis of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task; in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; and performing the processing task on the basis of the allocated AI computing boards or the allocated video encoding and decoding boards, and releasing the temporary cooperation relationship in response to completion of the processing task. The AI processing capacity and the video encoding and decoding capacity can be flexibly distributed and expanded on demand, thereby efficiently adapting to different application scenario algorithms

[0057] It should be particularly noted that the various steps in the various embodiments of the above AI video processing method may be crossed with each other, substituted, added, or deleted. Therefore, these reasonable permutations, combinations and transformations to the AI video processing method should also fall within the protection scope of the present disclosure and should not limit the protection scope of the present disclosure to the embodiments.

[0058] On the basis of the above object, in a second aspect of embodiments of the present disclosure, one embodiment of an AI video processing apparatus capable of quickly checking for efficient non-default options in BIOS is provided. The AI video processing apparatus includes:

[0059] an AI processing resource pool, including a plurality of AI computing boards for performing AI processing;

[0060] a video processing resource pool, including a plurality of video encoding and decoding boards for performing video processing; and

[0061] a control device, connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards by means of a unified high-speed interface, and including a processor and a memory, wherein the memory stores computer instructions executable on the processor, and the instructions, when executed by the processor, implement the following steps:

[0062] in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of AI computing boards and video encoding and decoding boards on the basis of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task;

[0063] in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; and

[0064] calling the allocated AI computing boards or the allocated video encoding and decoding boards as AI processing resources and video processing resources for performing the processing task, and releasing the temporary cooperation relationship in response to completion of the processing task.

[0065] In some implementations, each of the AI computing boards is provided with a first number of AI computing chips of the same model, and each of the video encoding and decoding boards is provided with a second number of video encoding and decoding chips of the same model. The first number and the second number are determined on the basis of bandwidths of the unified high-speed interface and complexity of physical connecting lines between the AI computing boards and the video encoding and decoding boards.

[0066] In some implementations, video encoding and decoding supported by the video encoding and decoding chips includes at least one of MPEG, H.264, H.265, AVS, and AVS+.

[0067] In some implementations, the control device is directly connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards by means of a PCIE physical interface on a main board. And/or the apparatus further includes a switchboard having a PCIE switching chip. The control device is indirectly connected to the plurality of AI computing boards and the plurality of video encoding and decoding boards via the switchboard.

[0068] In some implementations, the control device includes a CPU arranged on the main board, and a SCM and/or an ARM processor arranged on the switchboard.

[0069] It can be seen from the above embodiments that, in the AI video processing apparatus provided by the embodiments of the present disclosure, the technical solution includes: connecting to a plurality of AI computing boards in an AI processing resource pool and a plurality of video encoding and decoding boards in a video processing resource pool by means of a unified high-speed interface, so as to call AI processing resources and video processing resources; in response to reception of a processing task, respectively allocating, from the AI processing resource pool and the video processing resource pool, a specified number of AI computing boards and video encoding and decoding boards on the basis of resources and bandwidths required for completing the processing task to form a temporary cooperation relationship based on the processing task; in response to resource overflow or insufficiency in the AI processing resource pool or the video processing resource pool caused by a processing task change, guiding the AI processing resource pool or the video processing resource pool to access more AI computing boards or video encoding and decoding boards or stopping using redundant AI computing boards or video encoding and decoding boards; and performing the processing task on the basis of the allocated AI computing boards or the allocated video encoding and decoding boards, and releasing the temporary cooperation relationship in response to completion of the processing task. The AI processing capacity and the video encoding and decoding capacity can be flexibly distributed and expanded on demand, thereby efficiently adapting to different application scenario algorithms

[0070] It should be particularly noted that in the above embodiments of the AI video processing apparatus, the embodiments of the AI video processing method are used to specifically describe the operation process of each module. It would have readily occurred to those skilled in the art to apply these modules to other embodiments of the AI video processing method. Of course, since the various steps in the embodiments of the AI video processing method may be crossed with each other, substituted, added, or deleted. Therefore, these reasonable permutations, combinations and transformations to the AI video processing apparatus should also fall within the protection scope of the present disclosure and should not limit the protection scope of the present disclosure to the embodiments.

[0071] While the above is directed to the exemplary embodiments of the present disclosure, it should be noted that various changes and modifications can be made without departing from the scope of the disclosed embodiments of the present disclosure as defined by the appended claims. It is not necessary to perform the functions, steps, and/or actions of the method claims according to the disclosed embodiments described herein in any particular order. In addition, although elements disclosed in the embodiments of the present disclosure may be described or claimed in a single form, a plurality of elements may be contemplated unless limitation to a singular element is explicitly stated.

[0072] Those ordinarily skilled in the art will appreciate that the above discussion of any embodiments is intended to be exemplary only, and is not intended to suggest that the scope of disclosure of the embodiments of the present disclosure (including the claims) is limited to these examples. Combinations of technical features in the above embodiments or in different embodiments are also possible under the idea of the embodiments of the present disclosure, and many other variations of different aspects of the embodiments of the present disclosure as described above are possible, which are not provided in detail for the sake of concision. Therefore, any omissions, modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included within the protection scope of the embodiments of the present disclosure.