AUTOMATIC VIDEO EDITING METHOD AND PORTABLE TERMINAL

Abstract

Provided are an automatic video editing method and a portable terminal, the method comprising: obtaining a video to be edited; extracting key frames of the video to be edited; inputting the key frames to a pre-trained scene classification method and a pre-trained target detection method, and respectively obtaining scene type markers and target object markers of the key frames; selecting multiple video segments from the video to be edited which satisfy preset editing criteria; respectively calculating average scores of the multiple video segments by means of a pre-trained image quality scoring method; respectively obtaining the video segment having the highest average score of each shot type for splicing.

Claims

1. An automatic video editing method, comprising: acquiring a video to be edited; extracting a key frame of the video to be edited; inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object; screening out a plurality of video segments meeting a preset editing rule from the video to be edited; respectively calculating an average score of the plurality of video segments through a pre-trained image quality scoring method; acquiring and stitching video segments with a highest average score corresponding to each lens type.

2. The automatic video editing method according to claim 1, wherein a number of videos to be edited is equal to n, and n≥5.

3. The automatic video editing method of claim 1, wherein the inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object comprises: there being p scene types in the pre-trained scene categorization method, p corresponding to a sequence of preset scene types, with p≥100, inputting the i-th key frame into the pre-trained scene categorization method to determine the scene type of the key frame, obtaining the marker S.sub.i of the scene type, with S.sub.i∈[0, p]; there being k preset target objects, k corresponding to a sequence of preset target objects, with k≥50, inputting the i-th key frame into the pre-trained target detection method to detect the target object in the key frame, and obtaining the marker D.sub.i of the target object, with D.sub.i∈[0, k].

4. The video automatic editing method according to claim 1, wherein the preset editing rule comprises a preset lens type and a number of key frames of a video segment to be edited corresponding to the preset lens type; the screening out the plurality of video segments meeting the preset editing rule from the video to be edited comprises: detecting each lens type of the video segment to be edited sequentially, a number of preset lens types in the preset editing rule being j, with j≥3; determining whether a lens type of the key frame is a currently detected lens type, wherein the determining whether a lens type of the key frame is a currently detected lens type comprising: determining whether a lens type of the i-th key frame is the j-th lens type through a function f.sub.j(i), $\begin{array}{cc}{f}_j\left(i\right)=\{\begin{array}{cc}1& \mathrm{if}{S}_i\in {\mathrm{SSet}}_j.Math.D_i\in {\mathrm{DSet}}_j\\ 0& \mathrm{else}\end{array}& \left(1\right)\end{array}$ wherein in the formula (1), SSet.sub.j and DSet.sub.j are respectively a set of scene types and a set of target objects for the j-th preset lens type in the preset editing rule, and S.sub.i and D.sub.i are respectively the scene type and the target object of the i-th key frame; when f.sub.j(i)=1, the lens type of the i-th key frame is the currently detected lens type; when f.sub.j(i)=0, the lens type of the i-th key frame is not the currently detected lens type; determining whether the lens type of the video segment to be edited meets the currently detected lens type of the key frame, wherein the determining whether the lens type of the video segment to be edited meets the currently detected lens type of the key frame comprises: presetting a number of consecutive key frames of the video segment to be edited corresponding to each lens type to t, with t≥3, determining whether the edited video segment meets the currently detected lens type through a function m.sub.j(i,t), $\begin{array}{cc}{m}_j\left(i,t\right)=\{\begin{array}{cc}1& \mathrm{if}{f}_j\left(i\right).Math.f_j\left(i+1\right).Math.\mathrm{.Math.}.Math.f_j\left(i+t-1\right)=1\\ 0& \mathrm{else}\end{array}& \left(2\right)\end{array}$ wherein, in the formula (2), m.sub.j(i,t)=1 indicates that the lens type of the edited video segment meets the currently detected lens type, and m.sub.j(i,t)=0 indicates that the lens type of the edited video segment does not meet the currently detected lens type; acquiring video segments to be edited, lens types of which meeting the currently detected lens types of the key frames, wherein the acquiring video segments to be edited, lens types of which meeting the currently detected lens types of the key frames comprises: when detecting the j-th lens type, denoting video segments corresponding to the i-th key frame as mask.sub.j(i,t), mask.sub.j(i,t) representing video segments between beginning time of displaying the i-th key frame and beginning time of displaying the (i+t)-key frame in the video.

5. The automatic video editing method according to claim 1, wherein the respectively calculating the average score of each video segment through the pre-trained image quality scoring method comprises: providing a formula for calculating the average score M.sub.j(i,t) of the video segments mask.sub.j(i,t) as: $\begin{array}{cc}{M}_j\left(i,t\right)=\frac{l_i+l_{i+1}+\mathrm{.Math.}+l_{i+t-1}}{t}& \left(3\right)\end{array}$ wherein, in the formula (3), l.sub.i is an image quality score obtained by inputting the i-th key frame into the pre-trained image quality scoring method, and l.sub.i+t−1 is an image quality score obtained by inputting the (i+t−1)-th key frame into the pre-trained image quality scoring method, t is a number of consecutive key frames of the video segments mask.sub.j(i,t).

6. The automatic video editing method according to claim 1, further comprising: acquiring and stitching video segments with a highest average score corresponding to each lens type, wherein the lens type is determined by a combination of the scene type and the target object.

7. The automatic video editing method according to claim 1, further comprising: adding preset background music after completing the stitching.

8. (canceled)

9. A portable terminal, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, such that the processors, when executing the computer programs, perform operations comprising: acquiring a video to be edited; extracting a key frame of the video to be edited; inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object; screening out a plurality of video segments meeting a preset editing rule from the video to be edited; respectively calculating an average score of the plurality of video segments through a pre-trained image quality scoring method; acquiring and stitching video segments with a highest average score corresponding to each lens type.

10. The portable terminal according to claim 9, wherein the inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object comprises: there being p scene types in the pre-trained scene categorization method, p corresponding to a sequence of preset scene types, with p≥100, inputting the i-th key frame into the pre-trained scene categorization method to determine the scene type of the key frame, obtaining the marker S.sub.i of the scene type, with S.sub.i∈[0, p]; there being k preset target objects, k corresponding to a sequence of preset target objects, with k≥50, inputting the i-th key frame into the pre-trained target detection method to detect the target object in the key frame, and obtaining the marker D.sub.i of the target object, with D.sub.i∈[0, k].

11. The portable terminal according to claim 9, wherein the preset editing rule comprises a preset lens type and a number of key frames of a video segment to be edited corresponding to the preset lens type; the screening out the plurality of video segments meeting the preset editing rule from the video to be edited comprises: detecting each lens type of the video segment to be edited sequentially, a number of preset lens types in the preset editing rule being j, with j≥3; determining whether a lens type of the key frame is a currently detected lens type, wherein the determining whether a lens type of the key frame is a currently detected lens type comprising: determining whether a lens type of the i-th key frame is the j-th lens type through a function f.sub.j(i), $\begin{array}{cc}{f}_j\left(i\right)=\{\begin{array}{cc}1& \mathrm{if}{S}_i\in {\mathrm{SSet}}_j.Math.D_i\in {\mathrm{DSet}}_j\\ 0& \mathrm{else}\end{array}& \left(1\right)\end{array}$ wherein in the formula (1), SSet.sub.j and DSet.sub.j are respectively a set of scene types and a set of target objects for the j-th preset lens type in the preset editing rule, and S.sub.i and D.sub.i are respectively the scene type and the target object of the i-th key frame; when f.sub.j(i)=1, the lens type of the i-th key frame is the currently detected lens type; when f.sub.j(i)=0, the lens type of the i-th key frame is not the currently detected lens type; determining whether the lens type of the video segment to be edited meets the currently detected lens type of the key frame, wherein the determining whether the lens type of the video segment to be edited meets the currently detected lens type comprises: presetting a number of consecutive key frames of the video segment to be edited corresponding to each lens type to t, with t≥3, determining whether the edited video segment meets the currently detected lens type through a function m.sub.j(i,t), $\begin{array}{cc}{m}_j\left(i,t\right)=\{\begin{array}{cc}1& \mathrm{if}{f}_j\left(i\right).Math.f_j\left(i+1\right).Math.\mathrm{.Math.}.Math.f_j\left(i+t-1\right)=1\\ 0& \mathrm{else}\end{array}& \left(2\right)\end{array}$ wherein, in the formula (2), m.sub.j(i,t)=1 indicates that the lens type of the edited video segment meets the currently detected lens type, and m.sub.j(i,t)=0 indicates that the lens type of the edited video segment does not meet the currently detected lens type; acquiring video segments to be edited, lens types of which meeting the currently detected lens type, wherein the acquiring video segments to be edited, lens types of which meeting the lens types of the key frames comprises: when detecting the j-th lens type, denoting video segments corresponding to the i-th key frame as mask.sub.j(i,t), mask.sub.j(i,t) representing video segments between beginning time of displaying the i-th key frame and beginning time of displaying the (i+t)-key frame in the video.

12. The portable terminal according to claim 9, wherein the respectively calculating the average score of each video segment through the pre-trained image quality scoring method comprises: providing a formula for calculating the average score M.sub.j(i,t) of the video segments mask.sub.j(i,t) as: $\begin{array}{cc}{M}_j\left(i,t\right)=\frac{l_i+l_{i+1}+\mathrm{.Math.}+l_{i+t-1}}{t}& \left(3\right)\end{array}$ wherein, in the formula (3), l.sub.i is an image quality score obtained by inputting the i-th key frame into the pre-trained image quality scoring method, and l.sub.i+t−1 is an image quality score obtained by inputting the (i+t−1)-th key frame into the pre-trained image quality scoring method, t is a number of consecutive key frames of the video segments mask.sub.j(i,t).

13. The portable terminal according to claim 9, wherein the operation further comprise: acquiring and stitching video segments with a highest average score corresponding to each lens type, wherein the lens type is determined by a combination of the scene type and the target object.

14. The portable terminal according to claim 9, wherein the operation further comprise: adding preset background music after completing the stitching.

15. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, the computer program, when executed by a processor, causes the processor to perform operations, comprising: acquiring a video to be edited; extracting a key frame of the video to be edited; inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object; screening out a plurality of video segments meeting a preset editing rule from the video to be edited; respectively calculating an average score of the plurality of video segments through a pre-trained image quality scoring method; acquiring and stitching video segments with a highest average score corresponding to each lens type.

16. The non-transitory computer-readable storage medium according to claim 15, wherein the inputting the key frame into a pre-trained scene categorization method and a pre-trained target detection method to respectively obtain a marker of a scene type and a marker of a target object comprises: there being p scene types in the pre-trained scene categorization method, p corresponding to a sequence of preset scene types, with p≥100, inputting the i-th key frame into the pre-trained scene categorization method to determine the scene type of the key frame, obtaining the marker S.sub.i of the scene type, with S.sub.i∈[0, p]; there being k preset target objects, k corresponding to a sequence of preset target objects, with k≥50, inputting the i-th key frame into the pre-trained target detection method to detect the target object in the key frame, and obtaining the marker D.sub.i of the target object, with D.sub.i∈[0, k].

17. The non-transitory computer-readable storage medium according to claim 15, wherein the preset editing rule comprises a preset lens type and a number of key frames of a video segment to be edited corresponding to the preset lens type; the screening out the plurality of video segments meeting the preset editing rule from the video to be edited comprises: detecting each lens type of the video segment to be edited sequentially, a number of preset lens types in the preset editing rule being j, with j≥3; determining whether a lens type of the key frame is a currently detected lens type, wherein the determining whether a lens type of the key frame is a currently detected lens type comprising: determining whether a lens type of the i-th key frame is the j-th lens type through a function f.sub.j(i), $\begin{array}{cc}{f}_j\left(i\right)=\{\begin{array}{cc}1& \mathrm{if}{S}_i\in {\mathrm{SSet}}_j.Math.D_i\in {\mathrm{DSet}}_j\\ 0& \mathrm{else}\end{array}& \left(1\right)\end{array}$ wherein in the formula (1), SSet.sub.j and DSet.sub.j are respectively a set of scene types and a set of target objects for the j-th preset lens type in the preset editing rule, and S.sub.i and D.sub.i are respectively the scene type and the target object of the i-th key frame; when f.sub.j(i)=1, the lens type of the i-th key frame is the currently detected lens type; when f.sub.j(i)=0, the lens type of the i-th key frame is not the currently detected lens type; determining whether the lens type of the video segment to be edited meets the currently detected lens type of the key frame, wherein the determining whether the lens type of the video segment to be edited meets the currently detected lens type comprises: presetting a number of consecutive key frames of the video segment to be edited corresponding to each lens type to t, with t≥3, determining whether the edited video segment meets the currently detected lens type through a function m.sub.j(i,t), $\begin{array}{cc}{m}_j\left(i,t\right)=\{\begin{array}{cc}1& \mathrm{if}{f}_j\left(i\right).Math.f_j\left(i+1\right).Math.\mathrm{.Math.}.Math.f_j\left(i+t-1\right)=1\\ 0& \mathrm{else}\end{array}& \left(2\right)\end{array}$ wherein, in the formula (2), m.sub.j(i,t)=1 indicates that the lens type of the edited video segment meets the currently detected lens type, and m.sub.j(i,t)=0 indicates that the lens type of the edited video segment does not meet the currently detected lens type; acquiring video segments to be edited, lens types of which meeting the currently detected lens type, wherein the acquiring video segments to be edited, lens types of which meeting the lens types of the key frames comprises: when detecting the j-th lens type, denoting video segments corresponding to the i-th key frame as mask.sub.j(i,t), mask.sub.j(i,t) representing video segments between beginning time of displaying the i-th key frame and beginning time of displaying the (i+t)-key frame in the video.

18. The non-transitory computer-readable storage medium according to claim 15, wherein the respectively calculating the average score of each video segment through the pre-trained image quality scoring method comprises: providing a formula for calculating the average score M.sub.j(i,t) of the video segments mask.sub.j(i,t) as: $\begin{array}{cc}{M}_j\left(i,t\right)=\frac{l_i+l_{i+1}+\mathrm{.Math.}+l_{i+t-1}}{t}& \left(3\right)\end{array}$ wherein, in the formula (3), l.sub.i is an image quality score obtained by inputting the i-th key frame into the pre-trained image quality scoring method, and l.sub.i+t−1 is an image quality score obtained by inputting the (i+t−1)-th key frame into the pre-trained image quality scoring method, t is a number of consecutive key frames of the video segments mask.sub.j(i,t).

19. The non-transitory computer-readable storage medium according to claim 15, wherein the operation further comprise: acquiring and stitching video segments with a highest average score corresponding to each lens type, wherein the lens type is determined by a combination of the scene type and the target object.

20. The non-transitory computer-readable storage medium according to claim 15, wherein the operation further comprise: adding preset background music after completing the stitching.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG. 1 is a flow chart showing an automatic video editing method according to an embodiment I of the present disclosure.

[0020] FIG. 2 is a flow chart of screening video segments that meet a preset editing rule according to the embodiment I of the present disclosure.

[0021] FIG. 3 is a schematic structure diagram of a portable terminal according to an embodiment III of the present disclosure.

DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solution, and advantages of the present disclosure clearer, the present disclosure will be described in detail with reference to the accompanying drawings and embodiments. It should be appreciated that the specific embodiments described here are only used for explaining the present disclosure, rather than limiting the present disclosure.

[0023] In order to illustrate the technical solution of the present disclosure, specific embodiments are used for description below.

Embodiment I

[0024] Referring to FIG. 1, an automatic video editing method provided by the embodiment I of the present disclosure includes the following steps.

[0025] S101: a video to be edited is acquired.

[0026] The number of videos to be edited is n, and n≥5

[0027] It should be noted that the video to be edited in the embodiments of this disclosure can be a video file such as a movie, a TV series, or a short film shot by the user, or can also be a video file of other types or sources or a video file of a common format, such as mp4, insv, etc., which is not limited herein.

[0028] S102: a key frame of the video to be edited is extracted.

[0029] The key frame of the video to be edited can be extracted, but not limited to, by using a third-party library “ffmpeg”.

[0030] It should be noted that the key frame describes information of an image background and a moving subject. By reading the key frame image data, the image content of the video can be acquired. The packaging format of the video file is marked with a key frame list, such as a video in mp4 format. A “stss box” indicates information of the key frame list, which contains a video moment of each key frame and an address deviation of a video file. By reading the “stss box” information, an image of each key frame of the video to be edited can be acquired.

[0031] S103: for the key frame, a pre-trained scene categorization method and a pre-trained target detection method are adopted to extract a marker of a scene type and a marker of a target object respectively.

[0032] In the pre-trained scene categorization method, there are p scene types, which include but are not limited to streets, offices, subway stations, airport, shops, beaches, etc. The p corresponds to a sequence of the preset scene types, with p≥100. The i-th key frame is inputted into the pre-trained scene categorization method to determine the scene type of the key frame, the marker of the scene type is obtained and is denoted as S.sub.i, with S.sub.i∈[0,p].

[0033] There are k preset target objects, which can include but are not limited to people, vehicles, food, balls, electronic equipment, animals and plants, etc. The k corresponds to a sequence of the preset target objects, and k≥50. The i-th key frame is inputted into the pre-trained target detection method to detect the target object in the key frame, and the marker D.sub.i of the target object is obtained, with D.sub.i∈[0, k].

[0034] It should be noted that the pre-trained scene classification method is a pre-trained scene categorization algorithm based on a deep convolutional network, which can determine the scene type of the image. The pre-trained target detection method is a pre-trained target detection algorithm based on the deep convolutional network, which can detect whether there exists a preset target object in the image.

[0035] S104: multiple video segments meeting a preset editing rule are screened out from the videos to be edited.

[0036] The preset editing rule includes preset lens types and the number of key frames of a video segment to be edited corresponding to each lens type.

[0037] The preset lens type in the preset editing rule is determined by a combination of the scene type and the target object, that is, one preset lens type is determined by one preset scene type and one or more target objects. The preset lens type can include but is not limited to work, travel, shopping, scenery, sports and other daily life scenes. For example, the scene type corresponding to a preset lens type of work can be an office or a conference room, and/or the target object can be a computer, a desk, a keyboard and/or a mouse, etc.

[0038] Referring to FIG. 2, the step of screening out the video segments meeting the preset editing rule includes the following steps.

[0039] S1041: each lens type of the video segment to be edited is detected sequentially.

[0040] The number of preset lens types in the preset editing rule is equal to j, and j≥3.

[0041] S1042: it is determined sequentially whether the lens type of each key frame is a lens type currently detected.

[0042] A function f.sub.j(i) is adopted to sequentially determine whether the lens type of the i-th key frame is the j-th lens type:

[00001]$\begin{array}{cc}{f}_j\left(i\right)=\{\begin{array}{cc}1& \mathrm{if}{S}_i\in {\mathrm{SSet}}_j.Math.D_i\in {\mathrm{DSet}}_j\\ 0& \mathrm{else}\end{array}& \left(1\right)\end{array}$

[0043] In the formula (1), SSet.sub.j and DSet.sub.j are respectively a set of scene types and a set of target objects for the j-th preset lens type in the preset editing rule; and S.sub.i and D.sub.i are respectively the scene type and the target object of the i-th key frame obtained in the step S103. When f.sub.j(i)=1, it is indicated that the lens type of the i-th key frame is the currently detected lens type; when f.sub.j(i)=0, it is indicated that the lens type of the i-th key frame is not the currently detected lens type.

[0044] S1043: a video segment is edited, and it is sequentially determined whether the lens type of the edited video segment meets the lens type of the key frame.

[0045] In the preset editing rule, the number of consecutive key frames of the video segment to be edited corresponding to each lens type is preset to t, and t≥3; it is determined whether the edited video segment meets the currently detected lens type of the key frame through a function m.sub.j(i,t):

[00002]$\begin{array}{cc}{m}_j\left(i,t\right)=\{\begin{array}{cc}1& \mathrm{if}{f}_j\left(i\right).Math.f_j\left(i+1\right).Math.\mathrm{.Math.}.Math.f_j\left(i+t-1\right)=1\\ 0& \mathrm{else}\end{array}& \left(2\right)\end{array}$

[0046] In the formula (2), m.sub.j(i,t)=1 indicates that the lens type of the edited video segment meets the currently detected lens type of the key frame; and m.sub.j(i,t)=0 indicates that the lens type of the edited video segment does not meet the currently detected lens type of the key frame.

[0047] S1044: video segments to be edited, lens types of which meet the currently detected lens types of the key frames, are respectively acquired.

[0048] When the j-th lens type is detected, the video segments corresponding to the i-th key frame are recorded as mask.sub.j(i,t); mask.sub.j(i,t) represents video segments between beginning time of displaying the i-th key frame in the video and beginning time of displaying the (i+t)-key frame in the video.

[0049] S105: an average score of the multiple video segments is calculated through a pre-trained image quality scoring method.

[0050] The step of calculating the average score of the multiple video segments that meet the preset editing rule specifically includes:

[0051] a formula for calculating the average score M.sub.j(i,t) of the video segments mask.sub.j(i,t) is:

[00003]$\begin{array}{cc}{M}_j\left(i,t\right)=\frac{l_i+l_{i+1}+\mathrm{.Math.}+l_{i+t-1}}{t}& \left(3\right)\end{array}$

[0052] In the formula (3), l.sub.i is an image quality score obtained by inputting the i-th key frame into the pre-trained image quality scoring method; and l.sub.i+t−1 is an image quality score obtained by inputting the (i+t−1)-th key frame into the pre-trained image quality scoring method, t is the number of consecutive key frames of the video segments mask.sub.j(i,t).

[0053] It should be noted that the pre-trained image quality scoring method is a pre-trained image quality scoring algorithm based on the deep convolutional network, which can output a corresponding score l for the quality of the inputted image, with l∈(0, 1); and the greater the value l, the better the image quality.

[0054] S106: video segments with a highest average score corresponding to each lens type are respectively acquired and stitched.

[0055] The step of acquiring and stitching the video segments with the highest average score corresponding to each lens type to generate a new video file specifically includes following steps.

[0056] When the j-th lens type is detected, the video segments max(M.sub.j(i)) with the highest average score corresponding to the lens type are acquired, and finally j video segments can be acquired; and the j video segments are stitched according to a preset sequence of scenes to generate a new video file.

[0057] After the stitching is completed, a preset background music can be added. There may be multiple preset background music. The background music includes but is not limited to ordinary music and audio files.

[0058] In the present disclosure, through the pre-trained scene categorization method and the image quality scoring method, the editing rule and procedures are generated, and an effect of automatic video editing is achieved. The technical scheme of the present disclosure can avoid the problems of low editing efficiency and low accuracy caused by human factors, and improve the efficiency of video editing.

Embodiment II

[0059] The second embodiment of the present disclosure provides a computer-readable storage medium which stores a computer program, and the computer program, when executed by a processor, performs the steps of the automatic video editing method as provided in the embodiment I of the present disclosure. The computer-readable storage medium can be a non-transitory computer-readable storage medium.

Embodiment III

[0060] FIG. 3 shows a specific structure block diagram of a portable terminal provided in embodiment III of the present disclosure. The portable terminal 100 includes: one or more processors 101, a memory 102, and one or more computer programs; the processor 101 is connected to the memory 102 by a bus; the one or more computer programs are stored in the memory 102, and are configured to be executed by the one or more processors 101; and the processor 101, when executing the computer programs, implements the steps of the automatic video editing method provided in the embodiment I of the present disclosure.

[0061] In the embodiments of the present disclosure, those of ordinary skill in the art can understand that all or part of the steps in the methods of the above-mentioned embodiments can be implemented by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. The storage medium mentioned is, such as ROM/RAM, a magnetic disk, an optical disk, etc.

[0062] The above are merely the preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure shall be regarded as the protection scope of the present disclosure.