Device and method for synchronizing different parts of a digital service

Abstract

The invention relates to a reproduction device (21), with a device (20) acting as a source of digital services. It also relates to a method of synchronizing two parts of a digital service in a system including a source device according to the invention and at least one reproduction device according to the invention. According to the invention, the reproduction device (21) includes means for receiving the data forming at least a part of a digital service originating from a digital service source device (20), means for processing (210) at least some of the data received, means (211) for reproducing an output of at least a part of the digital service, the time for processing and reproducing the data introducing a delay in the output of the reproduced data. This device also includes communication means (213) for informing the source device of the delay introduced.

Claims

1. Device for reproducing at least one part of data corresponding to at least one digital service originating from a digital service source device comprising a receiver for receiving first data forming a first part of the digital service and second data forming a second part of the digital service, a first processing unit for processing said first received data, a first reproduction unit for reproducing an output of said first processed data, the time for processing and/or reproducing the first received data introducing a first delay-in the output of the first reproduced data, wherein said reproduction device further comprises a second processing unit for processing said second received data, a second reproduction unit for reproducing an output of said second processed data, the time for processing and/or reproducing the second data introducing a second delay in the output of the second reproduced data and wherein the device also comprises a communication unit for informing the digital service source device of said first delay according to the format of said first data or of said second delay according to the format of said second data.

2. Device according to claim 1, wherein said first data are video data and said second data are audio data.

3. Method for reproducing at least one part of data corresponding to at least one digital service originating from a digital service source device comprising: receiving first data forming a first part of a digital service; processing said first received data; reproducing an output of said first processed data, the time for processing and/or reproducing the first received data introducing a first delay in the output of the first reproduced data, receiving second data forming a second part of the digital service; processing said second received data; reproducing an output of said second processed data, the time for processing and/or reproducing the second received data introducing a second delay in the output of the second reproduced data, informing the digital service source device of said first delay according to the format of said first data or of said second delay according to the format of said second data.

4. Method according to claim 3, wherein said first data are video data and said second data are audio data.

5. Device acting as a digital service source, comprising a unit for outputting first data forming a first part of a digital service, a unit for outputting second data forming a second part of the digital service, wherein it also comprises a unit for recovering, according to the format of the first data, a first delay indication relating to a delay introduced when processing and/or reproducing said first data or for recovering, according to the format of the second data, a second delay indication relating to a delay introduced when processing and/or reproducing said second data, a delay unit for applying a programmable delay to the output data forming the second part of the digital service according to said first and second delay indications recovered.

6. Device according to claim 5, wherein said first data are video data and said second data are audio data.

7. Method for transmitting data corresponding to at least one digital service, comprising: outputting first data forming a first part of the digital service; outputting second data forming a second part of the digital service; recovering, according to the format of the first data, a first delay indication relating to a delay introduced when processing and/or reproducing said first data or recovering, according to the format of the second data, a second delay indication relating to a delay introduced when processing and/or reproducing said second data; and applying a programmable delay to the output data forming the second part of the digital service according to said first and second delay indications recovered.

8. Method according to claim 7, wherein said first data are video data and said second data are audio data.

Description

4. DESCRIPTION OF THE DRAWINGS

(1) The invention will be better understood and illustrated by means of advantageous exemplary embodiments, by no means limiting, with reference to the appended figures in which:

(2) FIG. 1, already described, shows a diagram of a flat screen television according to the state of the art;

(3) FIG. 2 represents a device for receiving and reproducing an audio/video digital service according to the invention which uses an internal audio reproduction device;

(4) FIG. 3 represents a device for receiving and reproducing an audio/video digital service according to the invention which uses an external audio reproduction device;

(5) FIG. 4 represents a device for receiving and reproducing an audio/video digital service according to the invention in which the reproduction device estimates a delay compensated in the source;

(6) FIG. 5 represents a device for receiving and reproducing an audio/video digital service according to the invention in which the reception device estimates and compensates for a delay;

(7) FIG. 6 represents a device for receiving and reproducing an audio/video digital service according to the invention in which the reproduction device estimates and compensates for a delay;

(8) FIG. 7 illustrates a method of estimating delay for liquid crystal screens according to the invention;

(9) FIG. 8 illustrates a chart representing the response times of liquid crystals for different grey transition levels;

(10) FIG. 9 illustrates a manual delay selection method according to the invention;

(11) FIG. 10 illustrates a method of selecting delays for different video formats;

(12) FIG. 11 illustrates a manual delay estimation device according to the invention;

(13) FIG. 12 illustrates a semi-automatic delay estimation device according to the invention;

(14) FIG. 13 illustrates a semi-automatic delay estimation method according to the invention;

(15) FIG. 14 represents a device for receiving and reproducing an audio/video digital service according to the invention which uses an external audio reproduction device, the reception device including 2 delay modules; and

(16) FIG. 15 represents a device for receiving and reproducing an audio/video digital service according to the invention, the reception device being connected to two video reproduction devices.

5. DETAILED DESCRIPTION OF THE INVENTION

(17) The embodiments will be described with particular reference to an audiovisual digital service. The A/V source is likened to a decoder but can be any other type of A/V source (for example, a DVD player). The audiovisual reproduction device is likened to a television including a screen and an audio output (i.e. built-in speaker) but can also be any other type of audiovisual reproduction device (for example a computer). The audio reproduction device can be external to the television and likened to a device including an amplifier linked to one or more speakers (for example, an audio amplifier of a home cinema device) but can also be any other type of audio reproduction device.

(18) Some deinterlacing circuits have compensating audio inputs to which is applied the same delay as to the video to remain in phase. However, in the case where the user chooses to use the sound from an external audio reproduction device (for example of home cinema type), no delay compensation is applied. It therefore seems natural to place the A/V synchronization module in the digital decoder, the latter being the source of the A/V signals and moreover necessarily compatible with A/V equipment already on the market. One of the principles of the invention is to provide automatic means to the television so that the latter can make known to the decoder the value of the delay between the video at the input of the television and the video displayed on the screen.

(19) FIGS. 2 and 3 represent two variants of a device for receiving and displaying an audiovisual digital service according to the invention. Only the main elements of the device are shown. FIG. 2 represents an audiovisual service device which includes a digital decoder 20 linked to a television 21, in particular by links 220, 221 and 222. The decoder 20 receives at its input an encoded audiovisual stream 22 (for example in MPEG coding). This A/V stream 22 is demultiplexed by a demultiplexer 204 into at least one audio signal and one video signal. The video signal is then decoded by a video decoder 200. As for the audio signal, this is decoded by an audio decoder 201. The two streams are synchronized using the A/V synchronization module 202 which communicates with the two decoders 200 and 201. The A/V synchronization module 202 is, moreover, linked to a processing unit 203. The video decoder 200 is linked to the television 21 via a DVI/HDMI link 220 (DVI/HDMI standing for Digital Video Interface/High Definition Multimedia Interface). More specifically, the video decoder 200 is linked to a video processing module 210 of the television 21. This processing module is in turn linked to a screen 211. As for the audio decoder, this is linked to an audio reproduction device of the television 212 via a link 222. The synchronization module 202 is linked via the unit 203 to a non-volatile memory 213 (for example EDID EPROM, in which EDID stands for Extended Display Identification Data) of the television 21 via an I2C bus 221 using, for example, the DDC communication protocol (Display Data Channel protocol for recovering screen-related data).

(20) FIG. 3 represents a similar device in which the audio stream is reproduced by an external audio reproduction device 31 (for example an amplifier of a home cinema device). This device 31 includes an audio amplifier 310 linked to speakers 33. The decoder 30 includes elements (in particular, audio decoder, video decoder, demultiplexer, A/V synchronization module) similar to those of the decoder 20, elements which are given the same reference numbers. It also includes a programmable audio delay module 300, an HDMI interface 302 and a video format management module 301. It is linked to a television 32 including a video processing module 320 which is in turn linked to a screen 321. The decoder is linked to the external audio reproduction device 31 via a link 340, for example via an SPDIF (Sony/Philips Digital Interface) interface and to the television 32 via a DVI/HDMI link. The objective of the two proposed solutions (FIGS. 2 and 3) is to compensate for a delay D.sub.dc, in which D.sub.dc=D.sub.d+D.sub.c, induced by video processes (for example deinterlacing, format conversion, image quality enhancement, etc) 320 210 and/or a delay D.sub.e due to the screen (for example, the response time of the liquid crystals) 211. The delay D.sub.e can itself be the sum of a number of delays such as those defined in the table above. In the rest of the document, the overall delay, i.e. D=D.sub.dc+D.sub.e, is denoted D. D.sub.dc and D.sub.e can be fixed or variable in time, as is the case of the delay D.sub.e for liquid crystal screens for which the response time is variable from frame to frame. In the case where the screen introduces no delay, D=D.sub.dc. Similarly, D.sub.dc can be zero.

(21) According to the invention, at least a programmable delay D is applied to the audio signal by storing it either in its compressed form or in its decoded form. The delay D is applied to the audio signal in the module 300. According to a variant of the invention, the delay is directly applied to the audio signal in the A/V synchronization module 202 present in the decoder. The decoder is responsible for applying the appropriate delay value D in the audio delay module 300 or in the A/V synchronization module of the decoder 202 to compensate for the delay induced by the video processes and/or the delay due to the type of screen 210 320.

(22) According to the invention, the delay D induced by the television 21 or 32 can vary according to the format of the input video, the management of which is handled by the module 301. Thus, a new delay value D can be programmed in the audio delay module 300 or in the A/V synchronization module of the decoder 202 each time the video format changes if this delay depends on the video format. This delay is denoted D.sub.x, where x is the video format with which the delay is associated. The programmed delay value can also be an overall value D operating independently of the input video format.

(23) The proposed solution therefore consists in enhancing the HDMI/DVI control protocols with parameters that indicate the delays D.sub.x, for example, for different video formats, or even the overall delay D independently of the input video format. These protocols enable the screens to share with the decoder information concerning their characteristics and capabilities. According to these protocols, the video source uses the DDC channel 221 to read a non-volatile memory 213 placed in the television 21 or 32 to ascertain, for example, the resolution, the polarities of the synchronization signals and colorimetric data. This data is represented using EDID descriptors which are defined in document EIA/CEA-861B. They can be supplied by the screen manufacturers and programmed in the EPROM EDID memory 213.

(24) The invention therefore consists in adding EDID information descriptors other than those already standardized in order to store in the television information characteristics of the delays introduced either by the digital video processing of the television (D.sub.dc), or by the response time of the screen (D.sub.e), or by both (D or D.sub.x). The delay information for each television 21 or 32 equipped according to the invention is stored in a non-volatile memory of the television 213. This information can include the 4 delays D.sub.x corresponding to the 4 video formats described previously (50 Hz interlaced input, 50 Hz progressive input, 60 Hz interlaced input, 60 Hz progressive input). Storing the delays relating to other video formats can also be envisaged.

(25) According to the invention, the decoder recovers these values in order for the A/V synchronization module 202 or the delay module 300 to synchronize the audio and video streams. The information concerning the delays D.sub.dc and D.sub.e can be supplied by the manufacturer of the television 21 or 32 and can be transmitted by the television 21 32 to the decoder 20 in electronic form. The overall delay information D or D.sub.x must then be transferred on switching on the decoder. This information can also, optionally, be transferred on a change of channel if necessary or on request to the decoder.

(26) An alternative solution to the use of the DDC channel is to use the CEC (Consumer Electronics Control) interactive interchange protocol specified in HDMI.

(27) FIG. 4 illustrates a particular embodiment of the invention. The decoder 20 is the same as the one described in FIG. 2 and will not be described further. The television 41 includes elements similar to the television 21 which are given the same references and will not therefore be described further in detail. The television also includes a module 410 for estimating a delay D.sub.e. In practice, for televisions which introduce a delay that is variable in time according to the screen type, this delay must be estimated. This estimated delay D.sub.e is then added to the delay D.sub.dc induced by the various video processes (for example, deinterlacing). The value of the overall delay D is stored in an EPROM EDID memory 411 to be used by the A/V synchronization module 202 or the delay module 300 placed in the decoder 20. This delay is therefore recovered by the decoder via the DDC link 221 in order to synchronize the audio and video streams. Storing the different delays (D.sub.dc and D.sub.e) separately in the EPROM EDID memory can also be envisaged, these different delays then being recovered by the decoder via the DDC link 221. To estimate the variable delay D.sub.e due to the screen type, the estimation method described later in the document can be used for liquid crystal display screens.

(28) In another embodiment represented in FIG. 5, the delay linked to the screen D.sub.e is estimated in an estimation module 500 located in the processing unit of the decoder 50. The elements similar to the preceding figures are given the same references and will not be described further. According to the invention, it is therefore proposed to define descriptors for storing, in the EPROM EDID memory of the screen, the data needed to estimate the delay (for example, charts as supplied by the liquid crystal suppliers and illustrated in FIG. 8). This data is then recovered via the link 221 by the decoder which itself estimates the average delay in the module 500. For this, it can use the estimation method described later in the document.

(29) One advantage of the solutions described is to synchronize the audio and video streams when using a television and an external audio output device 31 (for example, HiFi system, home cinema device).

(30) In another embodiment illustrated in FIG. 6, the delay D.sub.e linked to the screen is also estimated in the block 410 of the television 61 as previously. However, the synchronization is carried out directly in the television. In this case, a memory 610 in the television can be used to buffer the audio data and restore it to the user according to the average delay. A method of estimating the average delay is described below.

(31) FIG. 7 illustrates a device according to the invention for estimating 7 the delay D.sub.e for liquid crystal screens. The device includes a frame memory 71 and a delay computation module 70. The frame memory 71 is used to delay the input video (video t.sub.N) by one frame (video t.sub.N-1). The device 7 can use the frame memory 71 of the enhancement device. The delay computation module 70 computes, for each pixel, the difference in grey levels between two successive frames. This module then uses the charts (FIG. 8) supplied by the liquid crystal manufacturers. These charts give the response times for different grey level transitions. They can thus be used to estimate, for each pixel, the response time between two frames. By constructing a histogram on all the pixels, an average delay D.sub.e is therefore estimated (for example, by calculating a weighted average which takes into account the number of pixels having a given response time) which is used by the A/V synchronization device. In the case where the estimation of the delay D.sub.e is performed in the decoder (FIG. 5), these charts can be stored in the EPROM EDID memory and recovered by the decoder via the DDC link 221. In this case, the delay estimation device 7 then recovers the EDID data via the link 72.

(32) The following solutions propose other embodiments for manually or semi-automatically making known to the video source the delay parameters D induced both by the video processes and by the screen. These solutions are in particular used when there is no HDMI link between the A/V source and the video reproduction device.

(33) FIG. 9 illustrates a manual tuning method enabling the user to select delay parameters empirically using a menu. The selection device is shown in FIG. 11. This method is useful when no delay information is supplied by the manufacturer. In this case, the decoder 110 generates an A/V sequence enabling the user to synchronize the delay D manually and finely. According to the invention, in a step 91, the decoder 110 switches to the appropriate video format (denoted X, for example 50 Hz interlaced). In a step 92, the user 115 then selects a delay value D.sub.x from the menu for this format. The value is programmed in the audio delay module of the decoder 300 or in the A/V synchronization module 202. In a step 93, the decoder then sends the synchronized A/V sequence 111 using the selected delay value. The video is then displayed on the screen 112. The synchronized audio signal is amplified by the audio amplifier 113 and the sound is reproduced using speakers 114. The user can judge the quality of synchronization by watching the screen 112 and listening to the sound produced by the speakers 114. In a step 94, the user informs the decoder using the menu as to whether the synchronization is accurate enough for him. If not, the process is then repeated with a new delay value D. If the synchronization is satisfactory, the selection is ended and the value of D.sub.x is stored in the programmable delay module 300 or in the A/V synchronization module 202.

(34) This operation can be repeated for any video format in order to determine, for each, the delay that should be applied to the audio stream. This method is illustrated in FIG. 10. The delays D.sub.25i (relating to the 50 Hz interlaced video format), D.sub.50p (relating to the 50 Hz progressive video format), D.sub.30i (relating to the 60 Hz interlaced video format) and D.sub.60p (relating to the 60 Hz progressive video format) determined, in the steps 101, 102, 103 and 104 respectively, are selected in turn according to the method described in FIG. 9.

(35) According to another embodiment, since the delay is known because it is, for example, supplied by the manufacturer, the user can use a menu to manually enter the delay values D.sub.x to be applied in the delay module 300 or in the A/V synchronization module 202 of the decoder for example for different video formats. These values may be, for example, entered on installing the digital service device.

(36) According to another device illustrated in FIG. 12, a probe 122 is used, fixed to the screen in order to detect characteristics of what is displayed on the screen and return this information to the decoder. Such a semi-automatic method of estimating the delay D is illustrated in FIG. 13.

(37) According to this method, the decoder 120 generates a series of black images 130 (i.e. of low grey levels), then a single white image 131 (i.e. of high grey levels) and a series of black images 132 again, which it sends to the television 121. The first series of black images is then displayed 133 on the screen, followed by the white image 134 and finally the second series of black images 135. The probe 122 is capable of detecting a white image on the screen and of sending an instantaneous message to the decoder 120 to inform it of the display 134 of this white image. The decoder 120 computes the time elapsed between the moment 138 at which the white image was sent by the decoder and the moment 139 at which it was displayed on the screen of the television 121. The probe 122 is typically a device that is sensitive to light intensity and that can be positioned against the screen, for example in the top left corner of the screen or even in the middle of the screen. It is, moreover, capable of instantaneously evaluating the light intensity over a limited area of the screen. The probe 122 has 2 logic states. It is in a first state 136 when the light intensity level detected is below a certain threshold (i.e. on displaying the first series of black images) and in a second state 137 when the level is above the threshold (i.e. on displaying the first series of white images). The threshold is determined such that, when a black image is displayed, the probe is in the first state and, when a white image is displayed, the probe is in the second state. The black image can be replaced with an image of low light intensity and the white image with an image of high intensity. All that is needed is for the probe to be capable of detecting the transition from one to the other. Its logic state can be translated into an electrical signal having 2 values. This signal can then be recovered by the decoder. The decoder stores the time 138 from which the white image starts being sent to the screen and stores the time 139 at which the probe detects the transition from the first state to the second state. The difference D 140 between these 2 time markers represents the delay induced by the video processes and the screen. This operation can be repeated for various video formats in order to obtain a set of delays D.sub.x for all the video formats. This method is semi-automatic since, even if no menu is used, the user must connect the probe to the decoder, apply it to the screen and begin the process manually.

(38) It is also possible to envisage having a blue screen with a black square displayed somewhere on this screen. With the probe 122 positioned on the black square, white is sent to the black square in order for the probe to detect the change of light intensity. Knowing the position of the probe on the screen means the delay can be measured more precisely for screens that do not reproduce all the pixels simultaneously (for example, for scanning screens).

(39) Another application of the present invention is disclosed in FIG. 14. The elements common to FIGS. 2 and 3 are given the same references and will not be described further. The decoder 140 includes an additional programmable audio delay module 141. This second module is used to synchronize the audio and video streams that will be reproduced by the same reproduction device 142 (for example a television). This television 142 includes a video processing module 320 which induces a delay D.sub.dc on the video. It also includes an audio processing module 144 which induces a delay D.sub.ta on the audio stream. This module is in turn connected to speakers 151 built into the television. In order for the audio and video streams reproduced by the built-in speakers 151 and the screen 321 to be synchronized, the second programmable audio delay module 141 applies to the audio stream output from the decoder a delay (D.sub.dcD.sub.ta). In this example, it is proposed to add to the EDID table 4 parameters relating to the video delays D.sub.dc introduced by the reproduction device which depend on the video format: 50 Hz interlaced input 50 Hz progressive input 60 Hz interlaced input 60 Hz progressive input
4 parameters relating to the delay D.sub.ta introduced by the audio processing system for the above 4 video formats are also added. The delay values can be encoded on 1 byte in order to represent delay values varying from 0 to 255 milliseconds. Thus, in the case where the audio stream is reproduced by an external device 31 (for example SPDIF), the source applies a delay D.sub.dc (module 300) to the audio stream at the output of the decoder 201. In the case where the audio stream is reproduced by the HDMI reproduction device (for example a television) 142, the source 140 applies to it a delay (D.sub.dcD.sub.ta), via the module 141, which takes account of the video and audio delays introduced by the different processing systems of said HDMI device 142.

(40) A different application is illustrated in FIG. 15. The elements common to FIGS. 2 and 3 are given the same references and will not be described further. The decoder 150 is connected to two televisions 152 and 155 located in two different rooms. The television 152 is linked to a home cinema type device 31 for audio output (for example, via an SPDIF interface). The television 155 which mainly includes a screen 157 and which is located in another room, receives the audio and video streams together via a link 162 (for example scart or analogue RF). The television 152 includes a video processing module 153 introducing a delay D.sub.dc and a screen 154. As for the television 155, it maintains the synchronization of the audio and video streams that are transmitted together to it. According to the invention, a device is proposed that has a single audio output which is synchronized with any one of the video outputs. The solution according to the invention consists in synchronizing, in the decoder, the audio stream with the first video output 158 as already described with reference to FIG. 3 (that is, by applying a compensating delay to the audio). A second programmable delay module 161 is added which will apply to the second video output 159 the same delay D=D.sub.dc as to the audio output in order for the second video output 159 to be synchronized with the audio output 160. This solution in particular makes it possible to use the second television 155 in another room with the audio output of this same television and to have only one and the same audio output 160 operating with both video outputs 158 and 159.

(41) The invention described in the context of the DVI and HDMI communication protocols could be extended to any control protocol developed in the future, provided that it allows for the interchanging of such delay data or data for computing delays in the decoder (for example, charts).

(42) The invention is disclosed in the context of the synchronization of audio and video streams of a digital service, in the case where the screen introduces a delay whereas the audio part of the digital service is associated with an instantaneous process. It can be generally applied to any type of device for reproducing any digital service, said service being separated into different parts processed by different reproduction devices, each of them applying specific delays to the part of the service that it handles. In this case, the capability of communicating the specific delay of the reproduction device to the source device enables the latter to synchronize all the parts of the digital service between themselves for good reproduction of the complete service.