SIMULTANEOUSLY VIEWING MULTIPLE CAMERA ANGLES
20170359612 · 2017-12-14
Assignee
Inventors
Cpc classification
H04N21/43079
ELECTRICITY
H04N21/21805
ELECTRICITY
H04N21/4305
ELECTRICITY
H04N21/4622
ELECTRICITY
H04N21/43
ELECTRICITY
H04N21/266
ELECTRICITY
International classification
H04N21/43
ELECTRICITY
Abstract
Various devices, systems, products and methods for synchronizing video displayed on multiple display devices are presented. The systems, methods, devices and products described herein optionally utilize one or more techniques for synchronizing two or more displays of video of an event filmed from two or more cameras. Synchronization signals are optionally used to allow the video displays to be matched precisely to account for differences in display time, transmission time, delivery time and processing time.
Claims
1. A method for real-time synchronization of video of an event from multiple cameras, the method comprising: receiving a first video stream corresponding to video of the event from a first camera angle; receiving a second video stream corresponding to video of the event from a second camera angle; transmitting the first video stream to the first display device; receiving a request to display the second video stream on a second display device; transmitting the second video stream to the second display device; and adjusting displays of at least one of the first video stream and the second video stream in real-time to synchronize displays of the first video stream and the second video stream.
2. The method of claim 1, wherein adjusting displays of at least one of the first video stream and the second video stream includes: identifying a first timestamp in the first video stream; identifying a second timestamp in the second video stream; determining a time difference between the first timestamp and the second timestamp; and changing a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the time difference.
3. The method of claim 1, wherein the first video stream includes a synchronization signal and wherein adjusting displays of at least one of the first video stream and the second video stream includes: monitoring a display of the first video stream; and beginning displaying the second video stream upon detecting the synchronization signal in the first video stream.
4. The method of claim 1, wherein adjusting displays of at least one of the first video stream and the second video stream includes: identifying a delay associated with displaying at least one of the first video stream and the second video stream; and changing a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the delay.
5. The method of claim 1, wherein the first video stream is transmitted wirelessly to the first display device or the second video stream is transmitted wirelessly to the second display device.
6. The method of claim 1, wherein adjusting displays of at least one of the first video stream and the second video stream includes: delaying a transmission or display of a first portion of the first video stream or a second portion of the second video stream; skipping displaying or transmitting the first portion of the first video stream or the second portion of the second video stream; or changing a frame rate of the first portion of the first video stream or the second portion of the second video stream.
7. The method of claim 1, further comprising: transmitting a display signal to display the second video stream on the second display device.
8. The method of claim 1, wherein the first video stream and the second video stream are each independently received via antenna, satellite, cable or data network.
9. A television receiver comprising: one or more processors; and a memory communicatively coupled with and readable by the one or more processors and having stored therein processor-readable instructions that, when executed by the one or more processors, cause the one or more processors to: receive a first video stream corresponding to video of the event from a first camera angle; receive a second video stream corresponding to video of the event from a second camera angle; transmit the first video stream to the first display device; receiving a request to display the second video stream on a second display device; transmit the second video stream to the second display device; adjust displays of at least one of the first video stream and the second video stream in real-time to synchronize displays of the first video stream and the second video stream.
10. The television receiver of claim 9, further comprising: one or more receiver circuits for receiving at least one of the first video stream and the second video stream; one or more data output connections for transmitting at least one of the first video stream and the second video stream.
11. The television receiver of claim 9, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: identify a first timestamp in the first video stream; identify a second timestamp in the second video stream; determine a time difference between the first timestamp and the second timestamp; and change a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the time difference.
12. The television receiver of claim 9, wherein the first video stream includes a synchronization signal and wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: monitor a display of the first video stream; and begin displaying the second video stream upon detecting the synchronization signal in the first video stream.
13. The television receiver of claim 9, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: identify a delay associated with displaying at least one of the first video stream and the second video stream; and change a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the delay.
14. The television receiver of claim 9, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: delay or skip transmission or display of at least a first portion of the first video stream or a second portion of the second video stream; or to change a frame rate of the first portion of the first video stream or the second portion of the second video stream.
15. The television receiver of claim 9, further comprising: a synchronization signal sensor for monitoring a display of a synchronization signal from the first display device or the second display device; and/or a synchronization signal generator for creating synchronization signals.
16. A non-transitory processor-readable medium for real-time synchronization of video from multiple cameras, the medium comprising processor-readable instructions configured to cause one or more processors to: receive a first video stream corresponding to video of the event from a first camera angle; receive a second video stream corresponding to video of the event from a second camera angle; transmit the first video stream to the first display device; transmit the second video stream to the second display device; transmit a display signal to display the second video stream on the second display device; adjust displays of at least one of the first video stream and the second video stream in real-time to synchronize displays of the first video stream and the second video stream.
17. The non-transitory processor-readable medium of claim 16, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: identify a first timestamp in the first video stream; identify a second timestamp in the second video stream; determine a time difference between the first timestamp and the second timestamp; and change a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the time difference.
18. The non-transitory processor-readable medium of claim 16, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: monitor a display of the first video stream; and begin displaying the second video stream upon detecting a synchronization signal in the first video stream.
19. The non-transitory processor-readable medium of claim 16, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: identify a delay associated with displaying at least one of the first video stream and the second video stream; and change a timing of transmitting or displaying at least one of the first video stream and the second video stream to accommodate the delay.
20. The non-transitory processor-readable medium of claim 16, wherein the processor-readable instructions that cause the one or more processors to adjust displays of at least one of the first video stream and the second video stream cause the one or more processors to: delay or skip transmission or display of at least a first portion of the first video stream or a second portion of the second video stream; or to change a frame rate of the first portion of the first video stream or the second portion of the second video stream.
21. The non-transitory processor-readable medium of claim 16, further comprising processor-readable instructions that cause the one or more processors to: receive a request to display the second video stream on a second display device.
22. The non-transitory processor-readable medium of claim 16, further comprising processor-readable instructions that cause the one or more processors to: transmit one or more synchronization signals; and/or receive one or more synchronization signals.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] A further understanding of the nature and advantages of various embodiments may be realized by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
DETAILED DESCRIPTION
[0028] Various factors can affect whether two video streams are displayed in synch. For example, a slight offset in display rendering times of two displays can impact what would be otherwise perfect synchronicity. Processing the video for display may also result in a delay of displayed video. Depending on the video delivery method, a video stream may be delayed, slowed or otherwise stalled, which can also impact whether synchronized displays are even possible. For example, for video delivered over a data network, such as the internet, network congestion, video bitrate and available bandwidth can change the need to buffer video before it is displayed.
[0029] To compensate for these and other issues which impact synchronicity of video displays, embodied methods, devices, systems and products of the invention implement techniques that allow for delays in video displays to be accommodated to enable synchronized video of different camera angles to be presented on two different displays.
[0030] Although the embodiments described herein may frequently be described with reference to a live video broadcast, the disclosed methods, systems, devices and products are applicable to recorded video, such as video recorded by a digital video recorder (DVR). For example, the video displayed by one or more display devices is optionally recorded video. In some embodiments, the videos captured by cameras at various camera angles are optionally provided as separate broadcasts, such as on different channels/frequencies or different digital subchannels. For example, in one embodiment a first video stream from a first camera angle is provided on a first channel or subchannel and a second video stream from a second camera angle is provided on a second channel or subchannel. Using the techniques described herein, synchronicity of the two video streams displayed on two different display devices can be achieved.
[0031] Further, although embodiments described herein may be described with reference to two video streams having displays synchronized on two display devices, the skilled artisan will understand that the devices, methods, systems and products of the invention are applicable to the synchronization of any number of displays of video streams across any number of display devices. For example, embodiments are contemplated where three video streams are displayed in synchronicity on three display devices, or where ten video streams are displayed in synchronicity on ten display devices.
[0032] Additionally, depending upon the specific video streams, portions of the video streams may be identical, in part, such that the same exact video, taken from a single camera, may be displayed simultaneously on multiple displays. This configuration would typically result from the choice of a director, producer or other production staff to show video from the same camera on two separate video streams for a short period of time when, otherwise, the two video streams include video from two different camera angles.
[0033]
[0034] Receiver 120 is optionally connected to display device 110 using a wired analog or digital connection, such as by way of a coaxial or HDMI cable. In some embodiments, display device 110 connects to receiver 120 wirelessly, such as via WiDi, WirelessHD or another wireless display technique. In some embodiments, receiver 120 may be a small adapter that connects directly to a video input port on display device 110, draws power from the video input port or another power source, such as a USB power source, and receives video using one or more wireless protocols, including the 802.11 family of wireless technologies.
[0035] In
[0036]
[0037] In
[0038] Various other techniques can be used to synchronize the display of videos on different display devices. For example, an embodiment is shown in
[0039] As described herein, a synchronize signal refers to a detectable signal included with a video stream that is used to synchronize the display of two video streams. In some embodiments, a synchronize signal is included as an audible signal in an audio portion of a video stream. Optionally, the synchronize signal included as an audible signal is a sequence or series of pops, beeps, etc. that are detectable by a display device, receiver, system or product of an embodiment of the invention but may be imperceptible or subtle to the human ear, so as not to interrupt or degrade from the video viewing experience. In some embodiments a synchronize signal is included as a viewable signal in a video portion of a video stream. In some embodiments, a synchronize signal is a watermark included in audio or video portions of a video stream. Optionally, the synchronize signal included as an audible signal is a sequence or series of images or patterns that are detectable by a display device, receiver, system or product of an embodiment of the invention but may be imperceptible or subtle to the human ear, so as not to interrupt or degrade from the video viewing experience. For example, in one embodiment the synchronize signal is included at an edge of a video or image displayed on a display device. In another embodiment, the synchronize signal is one or more video frames of a single color (e.g., white) or video frames that include images or patterns, such as a barcode. Other embodiments are contemplated, such as where the synchronize signal is a radio frequency or optical signal and hardware is included in a display device or receiver for generating and/or detecting the radio frequency or optical signal. For example, in one embodiment, the optical signal is provided by way of an infrared emitter, such as an infrared light emitting diode and is detected by an infrared detector, such as an optical camera or an infrared photodiode. Use of an infrared signal is beneficial as infrared light is not perceivable by the human eye but is readily detectable using electronic systems configured to do so. For example, infrared receivers and emitters are routinely incorporated into a display device and its remote control.
[0040] In various embodiments, the synchronize signal includes information about video to be synchronized, such as a time stamp, which allows the second display device to begin displaying video matching the time stamp in order that the two displays are synchronized. Optionally, synchronize signals are periodically or aperiodically included in the video stream to allow for multiple step synchronization, synchronization verification or re-synchronization of video that becomes out of sync.
[0041]
[0042]
[0043]
[0044] As illustrated in
[0045] As used herein, the phrase “real-time” refers to a timing of events such that one event occurs within a short time period of another event, such as a time period of 10 seconds or less or 5 seconds or less or 1 second or less or 0.5 seconds or less. Real-time, as used herein, contrasts with processes that take significant time and effort to perform by a single person, such as a time period of minutes to hours or more.
[0046] As used herein, the terms synchronous, sync, synchronized, synchronization and the like refer to a matching of two or more activities such that they occur at the same time or with a nearly imperceptible time difference between them, such as a time difference of less than 0.1 seconds, less than 0.05 seconds or less than 0.01 seconds. In embodiments, two video streams are synchronized when the events that they depict are matched in time such that the same event appears on two displays at the same time or with a nearly imperceptible time difference between them, such as a time difference of 0.1 seconds, less than 0.05 seconds or less than 0.01 seconds. For example, in one embodiment, two video streams are synchronized when an event that is captured in one video stream is displayed at the same time, or within a nearly imperceptible time, as a display of the same event that is captured in a second video stream. Displays of video are considered to be synchronized, in embodiments, when events that are depicted in the displays that occurred at the same time are displayed at the same time. As will be understood by the present disclosure, even though different video can appear on each display, such as videos captured from different cameras and from different vantage points, the display of the videos can synchronized.
[0047]
[0048] In
[0049]
[0050] In
[0051] In
[0052] A computer system as illustrated in
[0053] The computer system 700 is shown comprising hardware elements that can be electrically coupled via a bus 705 (or may otherwise be in communication, as appropriate). The hardware elements may include one or more processors 710, including without limitation one or more general-purpose processors and/or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, video decoders, and/or the like); one or more input devices 715, which can include without limitation a mouse, a keyboard, remote control, and/or the like; and one or more output devices 720, which can include without limitation a display device, a printer, and/or the like.
[0054] The computer system 700 may further include (and/or be in communication with) one or more non-transitory storage devices 725, which can comprise, without limitation, local and/or network accessible storage, and/or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device, such as a random access memory (“RAM”), and/or a read-only memory (“ROM”), which can be programmable, flash-updateable and/or the like. Such storage devices may be configured to implement any appropriate data stores, including without limitation, various file systems, database structures, and/or the like.
[0055] The computer system 700 might also include a communications subsystem 730, which can include without limitation a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and/or a chipset (such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, cellular communication device, etc.), and/or the like. The communications subsystem 730 may permit data to be exchanged with a network (such as the network described below, to name one example), other computer systems, and/or any other devices described herein. In many embodiments, the computer system 700 will further comprise a working memory 735, which can include a RAM or ROM device, as described above.
[0056] The computer system 700 also can comprise software elements, shown as being currently located within the working memory 735, including an operating system 740, device drivers, executable libraries, and/or other code, such as one or more application programs 745, which may comprise computer programs provided by various embodiments, and/or may be designed to implement methods, and/or configure systems, provided by other embodiments, as described herein. Merely by way of example, one or more procedures described with respect to the method(s) discussed above might be implemented as code and/or instructions executable by a computer (and/or a processor within a computer); in an aspect, then, such code and/or instructions can be used to configure and/or adapt a general purpose computer (or other device) to perform one or more operations in accordance with the described methods.
[0057] A set of these instructions and/or code might be stored on a non-transitory computer-readable storage medium, such as the non-transitory storage device(s) 725 described above. In some cases, the storage medium might be incorporated within a computer system, such as computer system 700. In other embodiments, the storage medium might be separate from a computer system (e.g., a removable medium, such as a compact disc), and/or provided in an installation package, such that the storage medium can be used to program, configure, and/or adapt a general purpose computer with the instructions/code stored thereon. These instructions might take the form of executable code, which is executable by the computer system 700 and/or might take the form of source and/or installable code, which, upon compilation and/or installation on the computer system 700 (e.g., using any of a variety of generally available compilers, installation programs, compression/decompression utilities, etc.), then takes the form of executable code.
[0058] It will be apparent to those skilled in the art that substantial variations may be made in accordance with specific requirements. For example, customized hardware might also be used, and/or particular elements might be implemented in hardware, software (including portable software, such as applets, etc.), or both. Further, connection to other computing devices such as network input/output devices may be employed.
[0059] As mentioned above, in one aspect, some embodiments may employ a computer system (such as the computer system 700) to perform methods in accordance with various embodiments of the invention. According to a set of embodiments, some or all of the procedures of such methods are performed by the computer system 700 in response to processor 710 executing one or more sequences of one or more instructions (which might be incorporated into the operating system 740 and/or other code, such as an application program 745) contained in the working memory 735. Such instructions may be read into the working memory 735 from another computer-readable medium, such as one or more of the non-transitory storage device(s) 725. Merely by way of example, execution of the sequences of instructions contained in the working memory 735 might cause the processor(s) 710 to perform one or more procedures of the methods described herein.
[0060] The terms “machine-readable medium,” “computer-readable storage medium” and “computer-readable medium,” as used herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. These mediums may be non-transitory. In an embodiment implemented using the computer system 700, various computer-readable media might be involved in providing instructions/code to processor(s) 710 for execution and/or might be used to store and/or carry such instructions/code. In many implementations, a computer-readable medium is a physical and/or tangible storage medium. Such a medium may take the form of a non-volatile media or volatile media. Non-volatile media include, for example, optical and/or magnetic disks, such as the non-transitory storage device(s) 725. Volatile media include, without limitation, dynamic memory, such as the working memory 735.
[0061] Common forms of physical and/or tangible computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, any other physical medium with patterns of marks, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and/or code.
[0062] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor(s) 710 for execution. Merely by way of example, the instructions may initially be carried on a magnetic disk and/or optical disc of a remote computer. A remote computer might load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and/or executed by the computer system 700.
[0063] The communications subsystem 730 (and/or components thereof) generally will receive signals, and the bus 705 then might carry the signals (and/or the data, instructions, etc. carried by the signals) to the working memory 735, from which the processor(s) 710 retrieves and executes the instructions. The instructions received by the working memory 735 may optionally be stored on a non-transitory storage device 725 either before or after execution by the processor(s) 710.
[0064] It should further be understood that the components of computer system 700 can be distributed across a network. For example, some processing may be performed in one location using a first processor while other processing may be performed by another processor remote from the first processor. Other components of computer system 700 may be similarly distributed. As such, computer system 700 may be interpreted as a distributed computing system that performs processing in multiple locations. In some instances, computer system 700 may be interpreted as a single computing device, such as a distinct laptop, desktop computer, or the like, depending on the context.
[0065] The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components as appropriate. For instance, in alternative configurations, the methods may be performed in an order different from that described, and/or various stages may be added, omitted, and/or combined. Also, features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves and, thus, many of the elements are examples and do not limit the scope of the disclosure or claims.
[0066] Specific details are given in the description to provide a thorough understanding of example configurations (including implementations). However, configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configurations of the claims. Rather, the preceding description of the configurations will provide those skilled in the art with an enabling description for implementing described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.
[0067] Also, configurations may be described as a process which is depicted as a flow diagram or block diagram. Although each may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may have additional steps not included in the figure. Furthermore, examples of the methods may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium such as a storage medium. Processors may perform the described tasks.
[0068] Having described several example configurations, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over or otherwise modify the application of the invention. Also, a number of steps may be undertaken before, during, or after the above elements are considered.