METHOD FOR ADAPTING A DATA STREAM TO BE TRANSFERRED TO A RESOURCE CONSUMPTION

Abstract

In a method for adjusting data streams during a video conference the audio-based and video-based real-time communication to be transmitted to various receiver units with respective resource loads that vary over time, a server unit receives an incoming data stream from each of a multiple number of transmitter units. The server unit generates from such incoming data streams a multiple number of outgoing data streams, each having a different compressed data rate. The server unit receives from the specified receiver units, their resource loads (varying over time) in chronological intervals and accordingly subjects the data rates of the compressed outgoing data streams to a controlling adjustment to the resource loads of the respective specified receiver units. The outgoing data streams so adjusted are sent to the specified receiver units.

Claims

1-24. (canceled)

25. A method for adjusting data streams to respective resource loads of a plurality of first receiver units in audio-based and video-based real-time communication, said data streams to be transmitted during a video conference to the plurality of first receiver units, each respective resource load of each respective first receiver being variable over time, the method comprising the following steps: a server unit receives an incoming data stream from each of a plurality of transmitter units, each respective transmitter unit being integrated with a respective first receiver unit in a respective participant device that is participating in the video conference; the server unit generates from such incoming data streams a plurality of outgoing data streams, wherein each of the plurality of outgoing data streams has a compressed data rate; from a first one of the plurality of first receiver units, the server unit receives information about the existing resource load of the respective receiver unit at a first point in time; based on the information received by the server unit from the first one of the plurality of first receiver units about the existing resource load of the respective first receiver unit, the server unit performs on a first one of the plurality of outgoing data streams an adjustment of the compressed data rate of the first one of the outgoing data streams; and the server unit sends through transmission channels to the respective first one of the plurality of first receiver units the first one of the outgoing data streams with the compressed data rate adjusted based on the information about the existing resource load of the respective first receiver unit at the first point in time.

26. A method according to claim 25, further comprising: from the first one of the plurality of first receiver units, the server unit receives information about the existing resource load of the respective receiver unit at a second point in time after the first point in time; for the first one of the plurality of first receiver units, and based on the information received by the server unit from the respective first receiver unit about the existing resource load of the respective first receiver unit at the second point in time, the server unit performs an adjustment of the data rate of the compressed outgoing data stream for the respective first receiver unit; and the server unit sends through transmission channels to the respective first one of the plurality of first receiver units the outgoing data stream with the compressed data rate adjusted based on the information about the existing resource load of the respective first receiver unit at the second point in time.

27. A method according to claim 26, wherein the server unit calculates an expected resource load for the first one of the plurality of first receiver units on the basis of the information about the existing resource load received by the server unit from the respective first receiver unit during at least the first point in time and the second point in time, and accordingly subjects the data rate of the compressed outgoing data stream to a controlling adjustment that is based on the respective expected resource load of the respective first receiver unit.

28. A method according to claim 27, wherein the calculation of the expected resource load is carried out using a successive approximation method.

29. A method according to claim 27, wherein the calculation of the expected resource load is carried out using an exponential smoothing method.

30. A method according to claim 25, wherein each of the plurality of outgoing data streams has an adjusted data rate; wherein the server unit controls the adjustment of the data rate of the compressed outgoing data stream to the one of the plurality of first receiver units on the basis of the information received by the server unit from the respective first receiver unit about the existing resource load of the respective first receiver unit by selecting the outgoing data stream having the unique compressed data rate that is closest to the existing resource load of the respective first receiver unit; and the server unit sends through transmission channels to the respective one of the plurality of first receiver units the outgoing data stream with the compressed data rate that is closest to the existing resource load of the respective first receiver unit.

31. A method according to claim 25, wherein each of at least two incoming data streams contains a video signal and originates from a different transmitter unit, and wherein said at least two incoming data streams are combined by the server unit into a combined data stream; whereas from this combined data stream the server unit generates a plurality of outgoing data streams, and each of the plurality of outgoing data streams has an adjusted compressed data rate; and each of the plurality of outgoing data streams with adjusted data rate is transmitted to the respective first receiver units.

32. A method according to claim 31, wherein the server unit transmits to the first one of the plurality of first receiver units an outgoing data stream with a compressed data rate that is closest to the existing resource load of the respective receiver unit at the first point in time.

33. A method according to claim 25, wherein the compressed data rate of each of the plurality of outgoing data streams is given a predefined value prior to the commencement of the adjustment, wherein the predefined value of the compressed data rate of each of the plurality of outgoing data streams differs from the predefined value of the compressed data rate of each other of the plurality of outgoing data streams, and that the value of the compressed data rate of the outgoing data stream sent to the first one of the plurality of first receiver units is modified successively in the course of successive adjustments, each modification being on the basis of the information about the resource load of the respective first receiver unit at successively later points in time from the first point in time.

34. A method according to claim 33, wherein, after the end of a transmission of an outgoing data stream to the first one of the plurality of first receiver units, the data rate of the outgoing data stream returns again to the respective predefined value prior to the commencement of the adjustment.

35. A method according to claim 25, wherein, upon a change to the resource load of the first one of the plurality of first receiver units, the respective first receiver unit sends to the server unit the corresponding information about the change to the resource load of the first one of the plurality of first receiver units.

36. A method according to claim 39, wherein the first one of the plurality of first receiver units sends to the server unit at each of successive points in time, the resource load of the first one of the plurality of first receiver units at each such respective successive point in time.

37. A method according to claim 36, wherein each of the successive points in time is separated by a regular interval of time from the immediately preceding successive point of time.

38. A method according to claim 36, wherein each of the successive points in time is separated by a varying interval of time from the immediately preceding successive point of time.

39. A method according to claim 38, wherein the interval of time falls within the range of from 0.5 seconds to 20 seconds, inclusive.

40. A method according to claim 25, further comprising: from a first one of a plurality of second receiver units, wherein each of the plurality of second receiver units is assigned to a participant of the video conference and each such participant is at least momentarily passive, and wherein each of the second receiver units at least momentarily does not transmit data streams to the server unit, the server unit receiving information about the existing resource load of the respective second receiver unit at a first point in time; based on the information received by the server unit from the first one of the plurality of second receiver units about the existing resource load of the respective second receiver unit, the server unit sends through transmission channels to the respective first one of the plurality of second receiver units the outgoing data stream with the compressed data rate that is least remote from the data rate of the available resource load of the first one of the plurality of second receiver units.

41. A method according to claim 40, wherein the outgoing data stream for the first one of the plurality of second receiver units is transmitted to the respective second receiver unit without being adjusted.

42. A method according to claim 45, wherein the server unit uses an additional algorithm to determine an approximated optimum value for the data rate of the first one of the plurality of second receiver units and adjusts the outgoing data stream that is transmitted to the first one of the plurality of second receiver units on the basis of the determined approximated optimum value for the data rate of the first one of the plurality of second receiver units.

43. A method according to claim 25, wherein the server unit performs the adjustment of the compressed data rate of the first one of the outgoing data streams by adjusting at least one of the resolution, color depth, image repetition rate and compression codec of the first one of the outgoing data streams.

44. A method according to claim 25, for use in a video conference with a plurality of active and passive participants, wherein there is a maximum number P of active participants, and the number N of active participants with first receiver units that are to be provided with data streams is less than the maximum number P, the method includes the following first functionality: the server unit generates at least N outgoing data streams, with corresponding N transmission channels; from the N outgoing data streams the server unit performs a 1:1 allocation between the outgoing data streams and the participant devices of the N participants; the server unit subjects each of the N outgoing data streams to a controlling adjustment to the resource load of the respective first receiver unit to which the respective N outgoing data stream is allocated; and the server unit sends through transmission channels to each respective first receiver unit the allocated outgoing data stream with the compressed data rate adjusted based on the information about the existing resource load of the respective first receiver unit at the first point in time; and wherein passive participants in the video conference are connected via a plurality of second receiver units; and the server unit allocates to each of the plurality of second receiver units the data stream that is closest to the resource load of the respective second receiver unit.

45. A method according to claim 25, wherein, for use in a video conference with a plurality of active participants there is a maximum number P of active participants, and the number K of active participants with first receiver units that are to be provided with data streams is greater than the maximum number P, the method includes the following second functionality: the server unit generates up to M outgoing data streams, with corresponding M transmission channels, wherein N is smaller than or equal to K, and wherein the server unit sets the compressed data rate of each of the M outgoing data streams at a predefined value, wherein the predefined value of the compressed data rate of each of the M outgoing data streams differs from the predefined value of the compressed data rate of each other of the Am outgoing data streams: the first receiver unit of each of the K active participants is allocated the outgoing data stream with the value of the compressed data rate that is closest to the current resource load of the respective first receiver unit of each of the K active participants; the server unit subjects each of the selected M outgoing data streams to a controlling adjustment to the resource load of the respective first receiver unit to which the respective selected M data stream is allocated; the server unit sends through transmission channels to each respective one of the K first receiver units the allocated outgoing data stream with the compressed data rate adjusted based on the information about the existing resource load of the respective one of the K first receiver units at the first point in time; and wherein the compressed data rates of other outgoing, non-assigned data streams are adjusted to at least one constant target value that is representative for all second receiver units of the passive participants.

46. A system for an audio-based and video-based real-time communication within the framework of a video conference, the system comprising: a server unit including a processing unit and with at least one memory unit; at least one interface for receiving and transmitting data, the at least one interface being connected for communication with the server unit; a plurality of first receiver units, each receiver unit being connected for communication with the interface; a plurality of transmitter units being connected for communication with the interface; wherein the server unit is configured with software for carrying out a method for adjusting data streams to respective resource loads of a plurality of first receiver units in the case of audio-based and video-based real-time communication to be transmitted during a video conference to the plurality of first receiver units, each first receiver unit having a resource load that can vary over time, the method comprising the following steps: the server unit receives an incoming data stream from each of the plurality of transmitter units, each respective transmitter unit being integrated with a respective first receiver unit in a respective participant device that is participating in the video conference; the server unit generates from such incoming data streams a plurality of outgoing data streams, wherein each of the plurality of outgoing data streams has a compressed data rate; from at least a first one of the plurality of first receiver units, the server unit receives information about the existing resource load of the respective receiver unit at a first point in time; for the at least first one of the plurality of first receiver units, and based on the information received by the server unit from the respective first receiver unit about the existing resource load of the respective first receiver unit, the server unit performs an adjustment of the data rate of the compressed outgoing data stream for the respective first receiver unit; and the server unit sends through transmission channels to the respective first one of the plurality of first receiver units the outgoing data stream with the compressed data rate adjusted based on the information about the existing resource load received by the server unit from the respective first receiver unit.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Further advantages of the invention are described in the following embodiments. The following is shown:

[0081] FIG. 1 a schematic representation of a method for adjusting a data stream to be transmitted with four different compressed outgoing data streams,

[0082] FIG. 2 a data rate/time diagram of available resources of four first receiver units,

[0083] FIG. 3 a data rate/time diagram for representing outgoing data streams adjusted with time, and

[0084] FIG. 4 a data rate/time diagram with four outgoing data streams and one available bandwidth of a receiver unit.

DETAILED DESCRIPTION

[0085] FIG. 1 shows a schematic sequence of a software-assisted method for adjusting a data stream of computer program product to be transmitted, with four different compressed outgoing data streams 4a-d. Each of the four transmitter units 3a-d transmits an incoming data stream 2a-d with video signals, and preferably audio signals, to a server unit 1. In the server unit 1, the incoming data streams 2a-d are combined to form one mosaic image 7, which is subsequently compressed in the server unit 1 into four different outgoing data streams 4a-d with different data rates. The compressed outgoing data streams 4a, 4b, 4c and 4d are sent to the first receiver units 5a, 5b, 5c and 5d, respectively. Thus, the compressed outgoing data streams 4a-d and the first receiver units 5a-d each form a 1:1 allocation. This 1:1 allocation is suitable for a relatively small video conference with up to approx. ten active participants.

[0086] Each of the first receiver units has a computer program that calculates the resource load of the respective first receiver unit. Through data connections, preferably Internet-assisted data connections, each of the first receiver units 5a-d transmits information 8a-d regarding its resource load back to the server unit 1, after its calculation with the aid of the corresponding computer program in the respective first receiver unit 5a-d. From such information 8a-d, the data rate for each of the compressed outgoing data streams 4a-d is subject to a controlling adjustment by the server unit 1 for transmitting to the respective specified first receiver units 5a-d, preferably through the software-assisted precalculation of the resource load of the respective first receiver units 5a-d on the part of the server unit 1. For this purpose, the use of the exponential smoothing process described above is particularly preferred.

[0087] The dashed lines 13a-d indicate that the compressed data streams 4a-d (that is, the mosaic image 7 that is combined by the server unit 1) are also returned to the transmitter units 3a-d allocated to the receiver units 5a-d. Each of the transmitter units 3a-d is a part of a participant device (here, for the sake of clarity, not shown in a common housing or the like), in which the respective transmitter unit 3a, 3b, 3c and 3d and the respective first receiver unit 5a, 5b, 5c and 5d are contained. In the case of a video conference, the information sent by the other transmitter units 3a-d or, in the case of a video conference, the participants and/or their presentations and the like can also be viewed, for example, on a screen of the respective participant device of the active participants. The participant associated with this participant device can then respond to the other participants in a known manner by means of his or her transmitter unit 3a, 3b, 3c and 3d, for example if a question is asked. It is particularly preferred that all participants view the same image on the output devices, for example the screens, of their participant devices, irrespective of whether they are active or passive participantsbut with at least partially varying qualities, which depend on the data rates of the respectively received outgoing data streams 4a-d.

[0088] It is also shown that an additional, second receiver unit 6 participates in the video conference. This second receiver unit 6, which is exemplary of one of the possible many passive participants in a video conference, is assigned with a compressed outgoing data stream 4d. Thereby, the data rate of the compressed outgoing data stream 4d is either not adjusted or adjusted to the resource load of the second receiver unit 6 in accordance with a method described above.

[0089] FIG. 2 shows a diagram of available resource loads 11a-d from four first receiver units 5a, 5b, 5c and 5d. The data rate 9 with a value range of 0 kbit/s to 1400 kbit/s is plotted on the vertical axis. A time interval 10 is indicated on the horizontal axis. In this example, the time interval of 0 to 21 is plotted. For example, the time interval represents seconds (of course, other time divisions are also possible). In this case, the diagram would show the available resources 11a-d of the first receiver units in a period of 21 seconds.

[0090] The four first receiver units 5a, 5b, 5c and 5d feature, as shown in FIG. 2, the highest available resource load 11a or 11b or 11c or 11d, as the case may be, at any time in the diagram. The available resource load 11a, in particular its bandwidth, is approximately in the range of 700 kbit/s to 900 kbit/s. The respective other first receiver units feature less available resource loads 11b-d, which fluctuate with time.

[0091] FIG. 3 shows a diagram of the data rates of the adjusted outgoing data streams. The same value ranges and variables as in FIG. 2 are assigned to the vertical axis and the horizontal axis. In addition, four outgoing data streams 12a-d are shown. Such four outgoing data streams 12a-d form a 1:1 allocation with the four first receiver units from FIG. 2the outgoing data stream 12a is allocated to the first receiver unit with its resource load 11a, the outgoing data stream 12b is allocated to the first receiver unit 5b with its resource load 11b, etc.

[0092] It can also be seen in FIG. 3 that, at time one, a predefined value is allocated to the respective outgoing data streams 12a-d. As can be seen, the predefined values for the outgoing data stream 12a are 1500 kbit/s, for the outgoing data stream 12b are 1100 kbit/s, for the outgoing data stream 12c are 700 kbit/s, and for the outgoing data stream 12d are 300 kbit/s. Thus, the outgoing data stream 12d features the worst quality. This is apparent from the fact that, for example, the resolution or the color depth (in the case of a video conference) is lower than with the other data streams.

[0093] In accordance with FIG. 2, an iterative adjustment of the respective outgoing data streams 12a-d to the available resource loads 11a-d of the first receiver units 5a, 5b, 5c and 5d from FIG. 2 is carried out. This method is explained using the example of the outgoing data stream 12a and the resource load 11a of the corresponding first receiver unit 5a. For the other outgoing data streams 12b-d and the resource loads 11b-d of the other first receiver units 5b-d, the method is preferably analogous.

[0094] As already mentioned, the outgoing data stream 12a commences with a predefined value for the bandwidth of 1500 kbit/s. However, at this point in time (point in time 1 in FIG. 2), the resource load 11a of the corresponding first receiver unit offers only one bandwidth (the bandwidth is not to be restricted here, but should only be shown as an example) of 700 kbit/s. On the basis of such information 8a, in several control steps, the server unit 1 iteratively adjusts the data rate of the outgoing data stream 12a downward to the available resource load 11a. This adjustment is carried out, for example, with the method of exponential smoothing described above, with which the server unit 1 precalculates the expected resource load 11a of the corresponding first receiver unit on the basis of the information 8a and, accordingly, subjects the data rate of the outgoing data streams 12a to a controlling adjustment to such first receiver units 5a. At the beginning, this occurs more rapidly or with larger value adjustments than at a later point in time, as the data rates have already been approximated more closely. An adjustment of the data rate of the outgoing data stream 12a takes place successively in a plurality of control steps on the basis of the information 8a, provided continuously or discontinuously by the first receiver unit 5a with the resource load 11a of the server unit 1, regarding the bandwidth currently available for this first receiver unit 5a. This control of the data rate adjustment takes place, for example, at each point in time marked in FIGS. 2 and 3 (in the present exemplary case, at 21 points in time).

[0095] Towards the end of the time interval shown in FIGS. 2 and 3, finally, both bandwidths 11a-d and 12a-d then feature approximately the same value. For example, the available resource load 11a or bandwidth, as the case may be, of the specified first receiver unit 5a is approximately 850 kbit/s, and the data rate of the outgoing data stream 12a is approximately 830 kbit/s (see respective points in time 21 in FIGS. 2 and 3).

[0096] FIG. 4 shows a diagram with four outgoing data streams 12a-d and an available resource load 11 or bandwidth of a first receiver unit. The vertical and horizontal axes are again assigned with the same variables as in FIGS. 2 and 3. No receiver unit is assigned to the two outgoing data streams 12c and 12d with a predefined value of 900 kbit/s or 700 kbit/s, as the case may be. Such data streams 12c and 12d are therefore not adjusted, but remain constant at the respective predefined value. The outgoing data stream 12a features a data rate of 500 kbit/s and the outgoing data stream 12b features a data rate of 300 kbit/s. The specified first receiver unit has an available resource load 11 or bandwidth (once again only as an example of a resource) of 400 kbit/s, which initially increases slightly and thus features a fluctuating resource load. The server unit 1 calculates the differences of the currently available resource loads 11 or bandwidths of each first receiver unit at each outgoing data stream 12a-d. The outgoing data stream 12a-d that features the smallest difference from the available resource load 11 or bandwidth of the corresponding first receiver unit is then selected by the server unit 1. In other words, this means that the outgoing data stream 12a-d that is closest to the available resource load 11 or bandwidth of the respective first receiver unit 1 is selected. In the diagram of FIG. 4, the outgoing data stream 12a is initially closest to the currently available resource load 11 or bandwidth, as the case may be, of the first receiver unit. As such, the server unit 1 adjusts the outgoing data stream 12a downward. However, from the point in time 8, the outgoing data stream 12b is closer to the available resource load 11 or bandwidth, as the case may be, of the first receiver unit. From this point in time, the outgoing data stream 12b is adjusted to the available resource load 11 of this first receiver unit. The outgoing data stream 12a thereupon adjusts once again to its predefined value of 500 kbit/s.

[0097] By means of the invention, it is possible to provide a significantly better video quality to both active and passive participants in a video conference.

[0098] This invention is not limited to the illustrated and described embodiments. Variations within the scope of the claims, just as the combination of characteristics, are possible, even if they are illustrated and described in different embodiments.

LIST OF REFERENCE SIGNS

[0099] 1 Server unit

[0100] 2a-d Incoming data stream

[0101] 3a-d Transmitter unit

[0102] 4a-d Outgoing data stream

[0103] 5a-d First receiver unit

[0104] 6 Additional, second receiver unit

[0105] 7 Mosaic image

[0106] 8a-d Information regarding resource load

[0107] 9 Data rates

[0108] 10 Time interval

[0109] 11a-d Available resource load from a first receiver unit

[0110] 12a-d Data rate of the outgoing data stream

[0111] 13a-d Information sent back