MEDIA STREAMS
20230029446 · 2023-01-26
Inventors
Cpc classification
H04N21/438
ELECTRICITY
H04N21/4402
ELECTRICITY
H04N21/26275
ELECTRICITY
H04N21/8455
ELECTRICITY
H04N21/4384
ELECTRICITY
H04N21/6118
ELECTRICITY
H04N21/8456
ELECTRICITY
H04N21/2387
ELECTRICITY
H04N21/26258
ELECTRICITY
H04N21/6587
ELECTRICITY
International classification
H04N21/4402
ELECTRICITY
H04N21/438
ELECTRICITY
Abstract
In one embodiment, a system for modifying a video stream includes a video player receiving a video stream encoded as a series of non-predictive video frames and predictive video frames. The video player decodes the frames and is repositioned at a playback time from a current time to a different time in the video stream. In one embodiment, the video player uses a time reference table to identify a position in a variant of the video stream corresponding to the different time that is encoded as a non-predictive video frame. The video player receives the variant of the video stream in response to identifying the position. In another embodiment, a system for modifying a video stream includes a video player receiving a video stream encoded as a series of non-predictive video frames and predictive video frames. The video player decodes the frame of the video stream and provides the decoded video stream to a display. The video player repositions a playback time from a current time to a different time in the video stream. The different time is provided to a video encoder of the video stream which encodes the video stream based upon the different time, where a frame corresponding to the different time is encoded as a non-predictive video frame.
Claims
1. A method for modifying a video stream comprising: (a) a video player receiving a video stream encoded as a series of non-predictive video frames and predictive video frames; (b) said video player decoding said frames of said video stream and providing said decoded video stream to a display; (c) said video player repositioning a playback time from a current time to a different time in said video stream; (d) said video player using a time reference table to identify a position in a variant of said video stream corresponding to said different time, where the frame at said position is encoded as a non-predictive video frame; (e) said video player receiving said variant of said video stream in response to said identifying said position.
2. The method of claim 1 further comprising said player receiving said video stream over a cable network.
3. The method of claim 1 wherein said non-predictive video frames include at least one of an I frame and an instantaneous decoder refresh frame.
4. The method of claim 1 wherein said predictive video frames includes at least one of a P frame and a B frame.
5. The method of claim 1 wherein said video stream is provided as a HTTP live streaming video stream.
6. The method of claim 1 wherein said video stream is provided as a dynamic adaptive streaming over HTTP video stream.
7. The method of claim 1 wherein said variant includes non-predictive video frames and predictive video frames.
8. The method of claim 1 wherein said variant includes only non-predictive video frames.
9. The method of claim 8 wherein said video player selects among a plurality of variants, each of which includes only non-predictive video frames.
10. The method of claim 1 wherein said video player selects among a plurality of variants, where each of said variant includes non-predictive video frames and predictive video frames.
11. The method of claim 10 wherein said video player selects one of said variants having a non-predictive video frame at said position.
12. The method of claim 1 further comprising said video player automatically switching back to said video stream.
13. The method of claim 12 wherein said switching occurs prior to 3 seconds
14. A method for modifying a video stream comprising: (a) a video player receiving a video stream encoded as a series of non-predictive video frames and predictive video frames; (b) said video player decoding said frames of said video stream and providing said decoded video stream to a display; (c) said video player repositioning a playback time from a current time to a different time in said video stream; (d) said video player identifying a position in a variant of said video stream corresponding to said different time, where said variant includes only non-predictive video frames; (e) said video player receiving said variant of said video stream in response to identify said position; (f) said video player automatically switching back to said video stream.
15. The method of claim 14 wherein said video player selects among a plurality of variants, each of which includes only non-predictive video frames.
16. The method of claim 14 wherein said switching occurs prior to 3 seconds.
17. A method for modifying a video stream comprising the steps of: (a) a video player receiving a video stream encoded as a series of non-predictive video frames and predictive video frames; (b) said video player decoding said frames of said video stream and providing said decoded video stream to a display; (c) by said video player repositioning a playback time from a current time to a different time in said video stream; (d) providing said different time to a video encoder of said video stream; (e) encoding said video stream including a plurality of frames proximate said different time, wherein a frame corresponding to said different time is encoded as a non-predictive video frame, based upon said different time.
18. The method of claim 17 further comprising said player receiving said video stream over a cable network.
19. The method of claim 17 wherein said non-predictive video frames include at least one of an I frame and an instantaneous decoder refresh frame.
20. The method of claim 17 wherein said predictive video frames includes at least one of a P frame and a B frame.
21. The method of claim 17 wherein said video stream is provided as a HTTP live streaming video stream.
22. The method of claim 17 wherein said video stream is provided as a dynamic adaptive streaming over HTTP video stream.
23. The method of claim 17 further comprising a plurality of said video players receiving said video stream each of which repositioning a respectively playback time from a respective current time to a respective different time in said video stream.
24. The method of claim 23 further comprising providing each of said different times to said video encoder.
25. The method of claim 24 further comprising encoding said video stream including plurality of frames proximate said respective different times, wherein said different times is encoded as a non-predictive video frame.
26. The method of claim 25 wherein said encoded as a non-predictive video frame is based upon a plurality of said different times being having the same time.
27. The method of claim 17 wherein said providing said different time is based upon a presentation time stamp.
28. The method of claim 17 further comprising said video player providing a thumbnail along with said decoded video stream to said display.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0005] For a better understanding of the invention, and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings, in which:
[0006]
[0007]
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
DETAILED DESCRIPTION
[0023] Referring to
[0024] Referring to
[0025] The player monitors changing bandwidth conditions over time to the player. If the change in bandwidth conditions indicates that the stream should be changed to a different bit rate, the player checks the master manifest file for the location of additional streams having different bit rates. Using a stream specific manifest file for a selected different stream, the URL of the next chuck of video data is requested. In general, the switching between video streams by the player is seamless to the viewer.
[0026] A master playlist (e.g., manifest file) describes all of the available variants for the content. Each variant is a version of the stream at a particular bit rate and is contained in a separate variant playlist (e.g., manifest file). The client switches to the most appropriate variant based on the measured network bit rate to the player. The master playlist isn't typically re-read. Once the player has read the master playlist, it assumes the set of variants isn't changing. The stream ends as soon as the client sees the EXT-X-ENDLIST tag on one of the individual variant playlists.
[0027] For example, the master playlist may include a set of three variant playlists. A low index playlist, having a relatively low bit rate, may reference a set of respective chunk files. A medium index playlist, having a medium bit rate, may reference a set of respective chunk files. A high index playlist, having a relatively high bit rate, may reference a set of respective chunk files.
[0028] Referring to
[0029] EXTM3U: Indicates that the playlist is an extended M3U file. This type of file is distinguished from a basic M3U file by changing the tag on the first line to EXTM3U. All HLS playlists start with this tag.
[0030] EXT-X-STREAM-INF: Indicates that the next URL in the playlist file identifies another playlist file. The EXT-X-STREAM-INF tag has the following parameters.
[0031] AVERAGE-BANDWIDTH: An integer that represents the average bit rate for the variant stream.
[0032] BANDWIDTH: An integer that is the upper bound of the overall bitrate for each media file, in bits per second. The upper bound value is calculated to include any container overhead that appears or will appear in the playlist.
[0033] FRAME-RATE: A floating-point value that describes the maximum frame rate in a variant stream.
[0034] HDCP-LEVEL: Indicates the type of encryption used. Valid values are TYPE-0 and NONE. Use TYPE-0 if the stream may not play unless the output is protected by HDCP.
[0035] RESOLUTION: The optional display size, in pixels, at which to display all of the video in the playlist. This parameter should be included for any stream that includes video.
[0036] VIDEO-RANGE: A string with valid values of SDR or PQ. If transfer characteristic codes 1, 16, or 18 aren't specified, then this parameter must be omitted.
[0037] CODECS: (Optional, but recommended) A quoted string containing a comma-separated list of formats, where each format specifies a media sample type that's present in a media segment in the playlist file. Valid format identifiers are those in the ISO file format name space defined by RFC 6381 [RFC6381].
[0038] Referring to
[0039] Exemplary tags used in the VOD playlist may include one or more of the following.
[0040] EXTM3U: Indicates that the playlist is an extended M3U file. This type of file is distinguished from a basic M3U file by changing the tag on the first line to EXTM3U. All HLS playlists start with this tag.
[0041] EXT-X-PLAYLIST-TYPE: Provides mutability information that applies to the entire playlist file. This tag may contain a value of either EVENT or VOD. If the tag is present and has a value of EVENT, the server must not change or delete any part of the playlist file (although it may append lines to it). If the tag is present and has a value of VOD, the playlist file must not change.
[0042] EXT-X-TARGETDURATION: Specifies the maximum media-file duration.
[0043] EXT-X-VERSION: Indicates the compatibility version of the playlist file. The playlist media and its server must comply with all provisions of the most recent version of the IETF Internet-Draft of the HTTP Live Streaming specification that defines that protocol version.
[0044] EXT-X-MEDIA-SEQUENCE: Indicates the sequence number of the first URL that appears in a playlist file. Each media file URL in a playlist has a unique integer sequence number. The sequence number of a URL is higher by 1 than the sequence number of the URL that preceded it. The media sequence numbers have no relation to the names of the files.
[0045] EXTINF: A record marker that describes the media file identified by the URL that follows it. Each media file URL must be preceded by an EXTINF tag. This tag contains a duration attribute that's an integer or floating-point number in decimal positional notation that specifies the duration of the media segment in seconds. This value must be less than or equal to the target duration.
[0046] EXT-X-ENDLIST: Indicates that no more media files will be added to the playlist file.
[0047] The VOD playlist example in
[0048] Referring to
[0049] Exemplary tags used in the EVENT playlist may include one or more of the following.
[0050] EXTM3U: Indicates that the playlist is an extended M3U file. This type of file is distinguished from a basic M3U file by changing the tag on the first line to EXTM3U. All HLS playlists start with this tag.
[0051] EXT-X-PLAYLIST-TYPE: Provides mutability information that applies to the entire playlist file. This tag may contain a value of either EVENT or VOD. If the tag is present and has a value of EVENT, the server must not change or delete any part of the playlist file (although it may append lines to it). If the tag is present and has a value of VOD, the playlist file must not change.
[0052] EXT-X-TARGETDURATION: Specifies the maximum media-file duration.
[0053] EXT-X-VERSION: Indicates the compatibility version of the playlist file. The playlist media and its server must comply with all provisions of the most recent version of the IETF Internet-Draft of the HTTP Live Streaming specification that defines that protocol version.
[0054] EXT-X-MEDIA-SEQUENCE: Indicates the sequence number of the first URL that appears in a playlist file. Each media file URL in a playlist has a unique integer sequence number. The sequence number of a URL is higher by 1 than the sequence number of the URL that preceded it. The media sequence numbers have no relation to the names of the files.
[0055] EXTINF: A record marker that describes the media file identified by the URL that follows it. Each media file URL must be preceded by an EXTINF tag. This tag contains a duration attribute that's an integer or floating-point number in decimal positional notation that specifies the duration of the media segment in seconds. This value must be less than or equal to the target duration.
[0056] Items are not removed from the playlist when using the EVENT tag; rather new segments are appended to the end of the file. New segments are added to the end of the file until the event has concluded, at which time the EXT-X-ENDLIST tag may be appended. Referring to
[0057] Referring to
[0058] Exemplary tags used in the live playlist may include one or more of the following.
[0059] EXTM3U: Indicates that the playlist is an extended M3U file. This type of file is distinguished from a basic M3U file by changing the tag on the first line to EXTM3U. All HLS playlists must start with this tag.
[0060] EXT-X-TARGETDURATION: Specifies the maximum media-file duration.
[0061] EXT-X-VERSION: Indicates the compatibility version of the playlist file. The playlist media and its server must comply with all provisions of the most recent version of the IETF Internet-Draft of the HTTP Live Streaming specification that defines that protocol version.
[0062] EXT-X-MEDIA-SEQUENCE: Indicates the sequence number of the first URL that appears in a playlist file. Each media file URL in a playlist has a unique integer sequence number. The sequence number of a URL is higher by 1 than the sequence number of the URL that preceded it. The media sequence numbers have no relation to the names of the files.
[0063] EXTINF: A record marker that describes the media file identified by the URL that follows it. Each media file URL must be preceded by an EXTINF tag. This tag contains a duration attribute that's an integer or floating-point number in decimal positional notation that specifies the duration of the media segment in seconds. This value must be less than or equal to the target duration. In addition, the live playlist can use an EXT-X-ENDLIST tag to signal the end of the content. Also, the live playlist preferably does not include the EXT-X-PLAYLIST-TYPE type.
[0064] Referring to
[0065] Referring to
[0066] Another adaptive streaming technology is referred to as Dynamic Adaptive Streaming over HTTP (DASH), also generally referred to as MPEG-DASH, that enables streaming of media content over the Internet delivered from conventional HTTP web servers. MPEG-DASH employs content broken into a sequence of small HTTP-based file segments, where each segment contains a short interval of playback time of content. The content is made available at a variety of different bit rates. While the content is being played back at an MPEG-DASH enabled player, the player uses a bit rate adaptation technique to automatically select the segment with the highest bit rate that can be downloaded in time for playback without causing stalls or re-buffering events in the playback. In this manner, a MPEG-DASH enabled video player can adapt to changing network conditions and provide high quality playback with fewer stalls or re-buffering events. DASH is described in ISO/IEC 23009-1:2014 “Information technology—Dynamic adaptive streaming over HTTP (DASH)—Part 1: Media presentation description and segment formats”, incorporated by reference herein in its entirety.
[0067] In many video streaming technologies, including MPEG-2, the video frames are encoded as a series of frames to achieve data compression and typically provided using a transport stream. Each of the frames of the video are typically compressed using either a prediction-based technique and a non-prediction based technique. An I frame is a frame that has been compressed in a manner that does not require other video frames to decode it. A P frame is a frame that has been compressed in a manner that uses data from a previous frame(s) to decode it. In general, a P frame is more highly compressed than an I frame. A B frame is a frame that has been compressed in a manner that uses data from both previous and forward frames to decode it. In general, a B frame is more highly compressed than a P frame. The video stream is therefore composed of a series of I, P, and B frames. MPEG-2 is described in ISO/IEC 13818-2:2013 “Information technology—Generic coding of moving pictures and associated audio information—Part 2: Video” incorporated by reference herein in its entirety. In some encoding technologies, including H.264, an IDR (instantaneous decoder refresh) frame is made up an intra code picture that also clears the reference picture buffer. However, for purposes of discussion the I frame and the IDR frame will be referred to interchangeably. In some encoding technologies, the granularity of the prediction types may be brought down to a slice level, which is a spatially distinct region of a frame that is encoded separately from any other regions in the same frame. The slices may be encoded as I-slices, P-slices, and B-slices in a manner akin to I frames, P-frames, and B-frames. However, for purposes of discussion I frame, P frame, and B frame are also intended to include I-slice, P-slice, and B-slice, respectively. In addition, the video may be encoded as a frame or a field, where the frame is a complete image and a field is a set of odd numbered or even numbered scan lines composing a partial image. However, for purposes of discussion both “frames” and “pictures” and “fields” are referred to herein as “frames”. H.264 is described in ITU-T (2019) “SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Infrastructure of audiovisual services—Coding of moving video”, incorporated by reference herein in its entirety.
[0068] When a user is watching a decoded video stream on a display, the user often wants reposition the playback of the video stream to a different position in the video content, such as a previous location of the video stream to display some video content again or a future location of the video stream to skip over the current content being displayed. In either case, the user selects a different location in the video stream, and as a result the video player repositions the location that is to be played either using video content in the buffer of the video player or otherwise requesting an appropriate segment of the video content from a video server, and in either case starting to decode and render a video stream based upon the position selected.
[0069] Referring to
[0070] Referring to
[0071] In another embodiment, the system may include a plurality of I frame variant files, each of which have a different bitrate. In this manner, the video player may select the I frame variant file, from the plurality of I frame variant files, that most closely matches the current variant file being decoded and rendered.
[0072] Referring to
[0073] Referring to
[0074] In another embodiment, the player may select the repositioning position. Either the video player and/or the content server may correlate the repositioning position with available I frames within each of the variant files or otherwise an I frame variant file. If the repositioning position does not match an I frame within any of the variant files, then the content server and/or the video player may select a variant file with the closest I frame to the repositioning position. The content server then provides to the video player the selected variant for a limited duration, such as 1-3 seconds. The video player switches to the selected variant and decodes and renders corresponding I frame(s) starting at the repositioning position. The video player then decodes and renders the selected variant file for a limited duration, such as 1-3 seconds. After decoding and rendering the selected variant file for the limited duration, the content server and/or player automatically switches back to the previous variant file to continue decoding and rendering the video stream at the previous bitrate.
[0075] Referring to
[0076] Referring to
[0077] Referring to
[0078] Moreover, each functional block or various features in each of the aforementioned embodiments may be implemented or executed by a circuitry, which is typically an integrated circuit or a plurality of integrated circuits. The circuitry designed to execute the functions described in the present specification may comprise a general-purpose processor, a digital signal processor (DSP), an application specific or general application integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic, or a discrete hardware component, or a combination thereof. The general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, a controller, a microcontroller or a state machine. The general-purpose processor or each circuit described above may be configured by a digital circuit or may be configured by an analogue circuit. Further, when a technology of making into an integrated circuit superseding integrated circuits at the present time appears due to advancement of a semiconductor technology, the integrated circuit by this technology is also able to be used.
[0079] It will be appreciated that the invention is not restricted to the particular embodiment that has been described, and that variations may be made therein without departing from the scope of the invention as defined in the appended claims, as interpreted in accordance with principles of prevailing law, including the doctrine of equivalents or any other principle that enlarges the enforceable scope of a claim beyond its literal scope. Unless the context indicates otherwise, a reference in a claim to the number of instances of an element, be it a reference to one instance or more than one instance, requires at least the stated number of instances of the element but is not intended to exclude from the scope of the claim a structure or method having more instances of that element than stated. The word “comprise” or a derivative thereof, when used in a claim, is used in a nonexclusive sense that is not intended to exclude the presence of other elements or steps in a claimed structure or method.