METHOD FOR ENABLING SEAMLESS AND BIDIRECTIONAL PLAYBACK OF VIDEO

Abstract

Method for enabling seamless bidirectional and multiple speed rate playback of video, performing the steps of a pre-processing step having steps of analyzing a video to be played by unwrapping video containers; checking if the video is in a pre-defined normalized format and transcoding the video if not normalized; analyzing the video for bidirectional encoding by extracting and generating metadata, and generating general and bidirectional conversion instructions; an encoding step encoding and generating multiple video streams based on the generated conversion instructions for videos and for bidirectional playback on different devices and playback modes; a post-processing step synchronizing bidirectional video streams and metadata; and a step of extracting metadata from each generated video stream, and a step of distribution by streaming bidirectional video to a target device in a requested format together with accompanying metadata.

Claims

1. A method enabling seamless bidirectional and multiple speed rate playback of video, performing the following steps when executed on a computer: processing a source video by: analyzing the video by unwrapping video containers; checking if the video is in a pre-defined normalized format and transcoding the video if not normalized; analyzing the video for bidirectional encoding by extracting metadata; generating general and bidirectional conversion instructions based on said analysis; encoding and generating multiple video streams, for bidirectional playback on different devices and playback modes, based on the generated conversion instructions for videos; synchronizing bidirectional video streams and metadata; extracting metadata from each generated video stream; and streaming bidirectional video from the computer to a target device in a requested format together with accompanying metadata.

2. The method according to claim 1, by gathering metadata from the source video corresponding to bitrate, resolution and framerate.

3. The method according to claim 1, by including the conversion instructions in a matrix defining different video formats.

4. The method according to claim 1, by packing the generated multiple video streams in a video stream package.

5. The method according to claim 1, by gathering metadata from each generated video stream corresponding to one or more of NAL units, picture parameters, video parameters, video blocks, video samples, motion vectors and audio information.

6. The method according to claim 1, by synchronizing bidirectional video streams and metadata by time-syncing all bidirectional video streams by generating new key-frames with matching positions for forward and reverse playback streams.

7. The method according to claim 1, by using a content delivery network, CDN, or an end point as the target device.

8. A data processing system comprising a server computer and a client computer, where the server computer is configured for carrying out the method according to claim 1, and where the client computer is configured for receiving and playing bidirectional streamed video from the server computer by: receiving user input defining properties of playback; loading metadata from the streamed video, and predicting memory buffering requirements from the metadata and the properties of playback for controlling seamless bidirectional playback based in user input and loaded metadata.

9. A computer program stored on a non-transitory computer readable medium comprising instructions which when executed by a computing device or system cause the computing device or system to perform the method according to claims 1.

10. A non-transitory computer readable medium having stored thereon instructions which when executed by a computing device or system perform the method according to claim 1.

11. A data stream stored on a non-transitory computer readable medium which is representative of a computer program having instructions which when executed by a computing device or system cause the computing device or system to perform the method according to claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] One or more embodiments of the invention will now be described in detail with reference to the drawings where:

[0022] FIG. 1 shows an overview of modules comprised in a backend service according to one or more embodiments of the invention;

[0023] FIG. 2 shows the workflow in pre-processing of video in accordance with one or more embodiments of the present invention;

[0024] FIG. 3 shows the workflow for encoding in accordance with one or more embodiments of the present invention;

[0025] FIG. 4 shows the workflow for post-processing in accordance with one or more embodiments of the present invention;

[0026] FIG. 5 illustrates the result of the post-processing by synchronizing bidirectional videos in accordance with one or more embodiments of the present invention;

[0027] FIG. 6 shows the workflow for extracting metadata in accordance with one or more embodiments of the present invention, and

[0028] FIG. 7 shows the different steps in the backend distribution in accordance with one or more embodiments of the present invention.

DETAILED DESCRIPTION

[0029] One or more embodiments of the invention introduce a novel method for enabling real-time requested seamless bidirectional video playback by pre- and post-processing video by extracting and generating meta-information from native video formats.

[0030] One or more embodiments of the inventive method is executed on a computer comprising a CPU and memory.

[0031] FIG. 1 shows an overview of a backend service for processing and preparing video for seamless bidirectional playback. The figure indicates three different parts, an upper A), middle B), and lower part C), where the lower part C) is where processing of video according to the inventive method is performed before being distributed via a network, e.g. Internet.

[0032] The processing part receives input data, controlled via an interface, i.e. API (Application Programming Interface), via the network. Input video data may be provided by, e.g. a CDN (Content Delivery Network) or a streaming endpoint, and output data may be produced and made available to the same via the network. The input and output dataflow to and from the processing part is illustrated by middle part B) in FIG. 1. The upper part A) shows client backend processing performed locally on a user device connected to the network.

[0033] The lower part C) illustrates the modules for processing and preparing of video for seamless bidirectional playback. These modules will be described in detail below with reference to the figures. The modules are named pre-processing 1, encoding 2, post-processing 3, meta-information generation and extraction 4 and distribution 5.

[0034] The procedure for performing the method according to one or more embodiments of the invention is initiated by an end user selecting a video to be streamed to his or her end point device. This may be a smart phone or tablet. User inputs are entered via an input interface on the phone or tablet, and a selected video file and device specific information are sent to the backend service, illustrated in part C) of FIG. 1, via said API. User inputs provide data acting as stage triggers that will initiate and provide input to the backend pre-processing illustrated in FIG. 2.

[0035] FIG. 2 shows a first stage of a workflow comprising pre-processing of video for enabling seamless bidirectional playback of the video. This stage will control how video is processed in the next encoding stage. The pre-processing step will analyze input video and data, and generate instructions for further handling and processing of video.

[0036] When source data is entering the pre-processing step, the video source file is analyzed by checking if the file is wrapped in a container. This is most likely the case. A container specifies the file format, e.g. MKV, MOV, AVI, WMW, MP4 etc., and contains the actual video file. A popular container is the MPEG-4 (MP4) format offering advanced features. Inside the container there will be video and audio data as well as metadata defining the structure and properties of the video file. Metadata is placed in the header of the file.

[0037] If the video file is wrapped in a container, the content of the file is unwrapped and analyzed by checking the video properties and extracting metadata. The properties may for instance be type of codec used, structure of file, bit rate, frame rate, resolution, etc.

[0038] If the source video does not have a pre-defined format, it will be normalized by transcoding it to a pre-defined format. Even if the source data file has a pre-defined format, for instance MP4, it might not have a complete set of metadata. If this is the case, the metadata will be corrected to a complete set of metadata in the transcoding process of a source video to a master video. The master video file with a complete set of metadata will then be processed and analyzed further.

[0039] When the master file comprising normalized video has been established the file will be analyzed further. Properties of the sound and video will be detected. Sound may for instance be coded in a DD 5.1 format and the video may have a variable bit rate according to required bandwidth at any time driven by video content, i.e. movements and changes in the picture over time. It may also be that the video has a fixed bit rate which it is desirable to convert to a variable bit rate in order to require less bandwidth when streaming.

[0040] The video content is then analyzed with regards to bidirectional encoding by generating and extracting metadata. Metadata with bidirectional streaming and decoding instructions for a specific client is generated. These metadata are streamed to a specific device together with the video they describe.

[0041] Based on said analysis, general conversion instructions will be generated as well as bidirectional conversion instructions for several different video files having different properties. This also includes conversion instructions for sound, e.g. converting existing sound form multi-channel audio, e.g. Dolby Digital 5.1 to client capable audio, e.g. Stereo 2.0.

[0042] All figures indicate that there might be storage of data after all steps. Storage of all extracted and generated data may be perfoimed for utilizing of these data in all steps and for deriving the different formats enabling the seamless playback.

[0043] The output from the pre-processing stage is the detailed bidirectional conversion instructions which are input to the next encoding stage. In one or more embodiments the conversion instructions are comprised in a matrix defining the different video formats to be encoded.

[0044] FIG. 3 shows the workflow in the second stage which is the encoding stage. Encoding of a video file is performed according to the conversion instructions. The conversion instructions may define instructions for encoding several video streams for all supported platforms, bitrates, framerates and playback modes. These video files are encoded into files defining relevant video- and sound formats.

[0045] Multiple video streams are then generated based on the encoded video files prepared for bidirectional playback on different devices and playback modes.

[0046] FIG. 4 shows the workflow for post-processing of the different encoded video files generated in the previous encoding stage. The different video files are packed as a video stream matrix together with a package of metadata defining the properties of all the video streams in the package.

[0047] The next step is to synchronize bidirectional videos. This is performed in order to be able to start bidirectional playback at any speed from any point in a video without time lag when switching from normal playback to reverse playback.

[0048] Bidirectional videos streams and metadata are synchronized by time-syncing all bidirectional video streams by generating new key-frames with matching positions for forward and reverse playback streams.

[0049] FIG. 5 illustrates the result of the post-processing step by showing forward video blocks A.sub.n, A.sub.n+1 . . . A.sub.n+x and the corresponding reverse video blocks B.sub.n, B.sub.n+1 . . . B.sub.n+x that are normalized and synchronized. In this way a video file is prepared for being streamed and played back seamlessly in both directions. Playback of video will start at the correct position irrespective of direction and speed of playback.

[0050] The output data from the post-processing stage is a file comprising a matrix with several different bidirectional video files.

[0051] FIG. 6 illustrates the next stage performing the next steps of the method according to one or more embodiments of the invention. This stage comprises extracting metadata from each generated video stream in the bidirectional video file matrix.

[0052] In the case of a MP4 stream this may comprise MP4 specific data such as NAL units (Network Abstraction Layer), picture and video parameters, MP4 blocks and samples, motion vectors and audio information. This extracting stage makes it possible to start playing from anywhere in a video without first having to read metadata in the header of the file. Motion vector data can be used for calculation required streaming bandwidth and pre-buffering streams by prediction.

[0053] FIG. 7 illustrates a stage comprising the steps for distributing the video file. In this stage, type of distribution to be performed is checked. The video stream may be published to a streaming endpoint. The video stream may also be published to a CDN as a target device.

[0054] When the steps described above have been completed, bidirectional streaming of video to a target device in a requested format together with accompanying metadata is performed. Successful distribution of a streamed video file will be verified.

[0055] A device receiving this streamed video file will have a backend client installed. When receiving the streamed video the backend client will have all information available in the received streamed video for perfoiming bidirectional and multiple speed playback of video according to inputs selected by a user.

[0056] Although the disclosure has been described with respect to only a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that various other embodiments may be devised without departing from the scope of the present invention. Accordingly, the scope of the invention should be limited only by the attached claims.