Method computer program and driver unit for streaming gaze data packets

Abstract

Data packets containing gaze data are streamed from an eyetracker to a client via a driver unit by receiving, repeatedly, gaze data packets in a first interface; and, providing, repeatedly, via a second interface, gaze data packets. The client sends a request message to the driver unit. The request message defines a delivery point in time in a first time frame structure at which delivery point in time in each frame of the first time frame structure the gaze data packet shall be provided to the client via the second interface. An offset is calculated between a reception point in time and the delivery point in time. The reception point in time indicates when a gaze data packet is received from the eyetracker relative to the first time structure. An adjusted data acquisition instance is assigned based on the offset. The adjusted data acquisition instance represents a modified point in time in a second time frame structure when at least one future gaze data packet shall be produced by the eyetracker. The driver unit sends a control message to the eyetracker. The control message is adapted to cause the eyetracker to produce the at least one future gaze data packet at such an adjusted acquisition instance in the second time structure that the reception point in time for the at least one future gaze data packet is expected to lie within a margin prior to the delivery point in time.

Claims

1. A method performed in a driver unit for streaming gaze data packets containing gaze data from an eyetracker to a client, the method comprising: receiving, repeatedly, in a first interface, gaze data packets from the eyetracker, and providing, repeatedly, via a second interface, the gaze data packets to the client, wherein providing the gaze data packets to the client comprises: receiving a request message from the client, which request message defines a delivery point in time at which a respective one of the gaze data packets shall be provided, in each frame of a first time frame structure, to the client via the second interface; calculating an offset, in the first time frame structure, between (i) a reception point in time indicating when a gaze data packet is received from the eyetracker relative to the first time frame structure and (ii) the delivery point in time at which the gaze data packet shall be provided to the client; assigning an adjusted data acquisition instance based on the offset, the adjusted data acquisition instance representing a modified point in time in a second time frame structure when at least one future gaze data packet shall be produced by the eyetracker; and sending a control message to the eyetracker, which control message is adapted to cause the eyetracker to produce the at least one future gaze data packet at the adjusted acquisition instance in the second time frame structure, such that the reception point in time for the at least one future gaze data packet is expected to lie within a margin prior to the delivery point in time; and synchronizing the second time frame structure to the first time frame structure such that the first time frame structure and the second time frame structure share a common time base reference, wherein the adjusted acquisition instance in the second time frame structure when the at least one future gaze packet shall be produced by the eyetracker is such that an expected reception point in time indicating when the at least one future gaze data packet is received from the eyetracker is at least a safety time prior to the delivery point in time.

2. The method of claim 1, wherein the synchronizing of the second time frame structure to the first time frame structure comprises adjusting an interval between consecutive data acquisition instances to match a period of the first time frame structure.

3. The method of claim 1, wherein the first time frame structure is synchronized to a first time base reference different from a second time base reference to which the second time frame structure is synchronized, and the method further comprises: calculating, repeatedly, an updated offset between the reception point in time and the delivery point in time; and if the updated offset does not lie within the margin prior to the delivery point in time, sending an updated control message to the eyetracker, wherein the updated control message is adapted to cause the eyetracker to produce the at least one future gaze data packet at the adjusted acquisition instance in the second time frame structure, such that the reception point in time for the at least one future gaze data packet is expected to lie within the margin prior to the delivery point in time.

4. The method of claim 1, wherein assigning the adjusted data acquisition instance is further based on a latency in the eyetracker and a transmission delay between the eyetracker and the driver unit.

5. A non-transitory computer-readable storage medium comprising computer-executable instructions which when executed by a processor cause the processor to perform operations comprising: receiving, repeatedly, in a first interface, gaze data packets from an eyetracker, and providing, repeatedly, via a second interface, the gaze data packets to a client, wherein providing the gaze data packets to the client comprises: receiving a request message from the client, which request message defines a delivery point in time at which a respective one of the gaze data packets shall be provided, in each frame of a first time frame structure, to the client via the second interface; calculating an offset, in the first time frame structure, between aa reception point in time indicating when a gaze data packet is received from the eyetracker relative to the first time frame structure and (ii) the delivery point in time at which the gaze data packet shall be provided to the client; assigning an adjusted data acquisition instance based on the offset, the adjusted data acquisition instance representing a modified point in time in a second time frame structure when at least one future gaze data packet shall be produced by the eyetracker; and sending a control message to the eyetracker, which control message is adapted to cause the eyetracker to produce the at least one future gaze data packet at the adjusted acquisition instance in the second time structure, such that the reception point in time for the at least one future gaze data packet is expected to lie within a margin prior to the delivery point in time; and synchronizing the second time frame structure to the first time frame structure such that the first time frame structure and the second time frame structure share a common time base reference, wherein the adjusted acquisition instance in the second time frame structure when the at least one future gaze packet shall be produced by the eyetracker is such that an expected reception point in time indicating when the at least one gaze data packet is received from the eyetracker is at least a safety time prior to the delivery point in time.

6. A driver unit adapted to stream gaze data packets containing gaze data from an eyetracker to a client, the driver unit comprising: a first interface configured to receive, repeatedly, gaze data packets from the eyetracker, a second interface configured to provide, repeatedly, the gaze data packets to the client; and processing circuitry configured to control a functioning of the driver unit, wherein the driver unit is configured to: receive a request message from the client, which request message defines a delivery point in time at which a respective one of the gaze data packets shall be provided, in each frame of a first time frame structure, to the client via the second interface; calculate an offset, in the first time frame structure, between (i) a reception point in time indicating when a gaze data packet is received from the eyetracker relative to the first time frame structure and (ii) the delivery point in time at which the gaze data packet shall be provided to the client; assign an adjusted data acquisition instance based on the offset, the adjusted data acquisition instance representing a point in time in a second time frame structure when at least one future gaze data packet shall be produced by the eyetracker; and send a control message, via a control interface, to the eyetracker, which control message is adapted to cause the eyetracker to produce the at least one future gaze data packet at the adjusted acquisition instance in the second time frame structure such that the reception point in time for the at least one future gaze data packet is expected to lie within a margin prior to the delivery point in time, wherein the driver unit and the eyetracker are connected to a common clock generator configured to provide a common time base reference for the first and second frame structures, and wherein the adjusted acquisition instance in the second time frame structure when the at least one future gaze packet shall be produced by the eyetracker is such that an expected reception point in time indicating when the at least one gaze data packet is received from the eyetracker is at least a safety time prior to the delivery point in time.

7. The driver unit of claim 6, wherein the driver unit is connected to a first clock generator configured to provide a first time base reference for the first frame structure and the eyetracker is connected to a second clock generator configured to provide a second time base reference for the second frame structure, and the processing circuitry is further configured to: calculate, repeatedly, an updated offset between the reception point in time and the delivery point in time; and if the updated offset does not lie within the margin prior to the delivery point in time, send an updated control message to the eyetracker, wherein the updated control message is adapted to cause the eyetracker to produce at least one future gaze data packet at the adjusted acquisition instance in the second time structure such that the reception point in time for the at least one future gaze data packet is expected to lie within the margin prior to the delivery point in time.

8. The driver unit of claim 6, wherein the processing circuitry is configured to assign the adjusted data acquisition instance further based on of a latency in the eyetracker and a transmission delay between the eyetracker and the driver unit.

9. The non-transitory computer-readable storage medium of claim 5, wherein the synchronizing of the second time frame structure to the first time frame structure comprises adjusting an interval between consecutive data acquisition instances to match a period of the first time frame structure.

10. The non-transitory computer-readable storage medium of claim 5, wherein the first time frame structure is synchronized to a first time base reference different from a second time base reference to which the second time frame structure is synchronized, wherein the operations further comprise: calculating, repeatedly, an updated offset between the reception point in time and the delivery point in time; and if the updated offset does not lie within the margin prior to the delivery point in time, sending an updated control message to the eyetracker, wherein the updated control message is adapted to cause the eyetracker to produce at least one future gaze data packet at the adjusted acquisition instance in the second time frame structure such that the reception point in time for the at least one future gaze data packet is expected to lie within the margin prior to the delivery point in time.

11. The non-transitory computer-readable storage medium of claim 5, wherein assigning the adjusted data acquisition instance is further based on a latency in the eyetracker and a transmission delay between the eyetracker and the driver unit.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention is now to be explained more closely by means of preferred embodiments, which are disclosed as examples, and with reference to the attached drawings.

(2) FIG. 1 shows a block diagram over a computer system including an eyetracker, a driver unit and a client according to a first embodiment of the invention;

(3) FIG. 2 illustrates, by an example, how gaze data packets and a control message are exchanged between the eyetracker, the driver unit and the client;

(4) FIG. 3 shows one example of a time sequence according to the first embodiment of the invention;

(5) FIG. 4 shows a block diagram over a computer system including an eyetracker, a driver unit and a client according to a second embodiment of the invention;

(6) FIG. 5 shows one example of a time sequence according to the second embodiment of the invention; and

(7) FIG. 6 illustrates, by means of a flow diagram, the general method according to the invention.

DETAILED DESCRIPTION

(8) FIG. 1 shows a block diagram over a computer system according to a first embodiment of the invention, which computer system is arranged to implement at least one gaze-data based service. The computer system includes an eyetracker 120, a driver unit 110 and a client 130.

(9) The driver unit 110, in turn, contains a processing circuitry 116, which for example may include one or more general-purpose processors. The processing circuitry 116 is further preferably communicatively connected to a data carrier 114 in the form computer-readable storage medium, such as a Random Access Memory (RAM), a Flash memory, or the like. The data carrier 114 contains computer-executable instructions, i.e. a computer program 115, for causing the processing circuitry 116 to perform in accordance with the embodiments of the invention as described herein, when the computer-executable instructions are executed in the processing circuitry 116.

(10) The driver unit 110 is adapted to stream gaze data packets D.sub.GT(i) from the eyetracker 120 to the client 130. The gaze data packets D.sub.GT(i) contain gaze data, i.e. typically information relating to how at least one eye of a user is moved relative to the user's head and/or how a gaze point of a user is moved across a particular surface, e.g. a display.

(11) Referring now also to FIG. 2, we see an example how gaze data packets D.sub.GT(1), D.sub.GT(2) and D.sub.GT(3) respectively are received in the driver unit 110 and forwarded to the client 130. Specifically, in the driver unit 110, a first interface 111 is configured to receive, repeatedly, the gaze data packets D.sub.GT(1), D.sub.GT(2) and D.sub.GT(3) from the eyetracker 120; and a second interface 112 is configured to provide, repeatedly, the gaze data packets D.sub.GT(1), D.sub.GT(2) and D.sub.GT(3) to the client 130.

(12) The processing circuitry 116 is configured to control the overall functioning of the driver unit 110. Inter alia, this means that the processing circuitry 116 controls the driver unit 110 to receive a request message m.sub.req from the client 130. The request message m.sub.req defines a delivery point in time t.sub.D1 in a first time frame structure FS1. The delivery point in time designates t.sub.D1 when, in each frame of the first time frame structure FS1 the gaze data packet shall be provided to the client 130 via the second interface 112 in order to best suit a functionality being implemented in the client 130, such as for example an input device for human-computer interaction in the form of a head-mounted display (HUD).

(13) Additionally, the processing circuitry 116 is configured to calculate an offset t.sub.L1 between a reception point in time t.sub.R and the delivery point in time t.sub.D1. The reception point in time t.sub.R indicates when a gaze data packet is received from the eyetracker 120 relative to the first time structure FS1. Based on the offset t.sub.L1, the processing circuitry 116 is configured to assign an adjusted data acquisition instance t.sub.DAcq′, which represents a point in time in a second time frame structure FS2 when at least one future gaze data packet. D.sub.GT(2) and D.sub.GT(3) in FIG. 2, shall be produced by the eyetracker 120.

(14) Via a control interface 118, the driver unit 110 is configured to send a control message m.sub.ctrl to the eyetracker 120. The control message m.sub.ctrl is adapted to cause the eyetracker 120 to produce the at least one future gaze data packet D.sub.GT(2) and D.sub.GT(3) at such an adjusted acquisition instance t.sub.DAcq′ in the second time structure FS2 that the reception point in time t.sub.R for each of the at least one future gaze data packet D.sub.GT(2) and D.sub.GT(3) is expected to lie within a margin t.sub.m prior to the delivery point in time t.sub.D1. Thus, the eyetracker 120 regards the control message m.sub.ctrl as an instruction for setting its timing of the gaze data recording occasion. This adjustments enables use of the freshest possible gaze data to be delivered in the client 130 when such data are requested.

(15) To safeguard that the future gaze data packets D.sub.GT(2) and D.sub.GT(3) do not arrive with too small a margin to the delivery point in time t.sub.D1—and thus enable the client 130 to handle the gaze data being carried in these data packets D.sub.GT(2) and D.sub.GT(3)—the adjusted acquisition instance t.sub.DAcq′ in the second time structure FS2 is preferably set such that the expected reception point in time t.sub.R is at least a safety time t.sub.SAFE prior to the delivery point in time t.sub.D1.

(16) In FIG. 3, a graph illustrates one example of a time sequence according to the first embodiment of the invention (shown in FIG. 1), where the driver unit 110 and the eyetracker 120 are connected to one and the same clock generator 140 that is configured to provide a common time base reference CLK for the first and second frame structures FS1 and FS2 respectively. Naturally, this is advantageous in terms of maintaining a stable temporal relationship between the first and second frame structures FS1 and FS2. In FIG. 3, this is reflected by the fact that once the adjusted acquisition instance t.sub.DAcq′ has been set, the updated offset t.sub.L1 may remain essentially constant over time t, and within the margin t.sub.m.

(17) FIG. 4 shows a block diagram over a computer system including an eyetracker 120, a driver unit 110 and a client 130 according to a second embodiment of the invention. Here, all units, signals and data packets bearing reference signs that are also present in FIG. 1 designate the same units, signals and data packets as described above with reference to FIGS. 1 and 2.

(18) Thus, in contrast to the first embodiment of the invention, in FIG. 4, the driver unit 110 is connected to a first clock generator 141 and the eyetracker 120 is connected to a second clock generator 142, The first clock generator 141 is configured to provide a first time base reference CLK1 for the first frame structure FS1, and the second clock generator 142 is configured to provide a second time base reference CLK2 for the second frame structure FS2.

(19) Here, since a stable temporal relationship between the first and second frame structures FS1 and FS2 neither can be guaranteed nor be expected, the processing circuitry 116 is further configured to execute the following procedure. An updating of the offset t.sub.L1 between the reception point in time t.sub.R and the delivery point in time t.sub.D1 is calculated repeatedly. If the updated offset t.sub.L1 does not lie within the margin t.sub.m prior to the delivery point in time t.sub.D1, the processing circuitry 116 is configured to send an updated control message m.sub.ctrl.sup.+ to the eyetracker 120. The updated control message m.sub.ctrl.sup.+ is adapted to cause the eyetracker 120 to produce at least one future gaze data packet at such an adjusted acquisition instance t.sub.DAcq′ in the second time structure FS2 that the reception point in time t.sub.R for the at least one future gaze data packet is expected to lie within the margin t.sub.m prior to the delivery point in time t.sub.D1.

(20) FIG. 5 shows one example of a time sequence according to the second embodiment of the invention. Here, a linear relative drift between the first and second time base references CLK1 and CLK2 respectively is presumed. As a result, if unadjusted, the offset t.sub.L1 would gradually increase from zero up to a maximum value equal to a frame period T.sub.FS1 of the first frame structure FS1. After that the offset t.sub.L1 had attained this maximum value, it would again had become zero, and so on. However, the repeated updating of the offset t.sub.L1 between the reception point in time t.sub.R and the delivery point in time t.sub.D1 constrains the variation in the offset t.sub.L1 between zero and the margin t.sub.m, or between the safety time t.sub.SAFE and the margin t.sub.m depending on the control principle being applied.

(21) According to one embodiment of the invention, the processing circuitry 116 is configured to assign the adjusted data acquisition instance t.sub.DAcq′ on the further basis of a latency t.sub.L2 in the eyetracker 120 and a transmission delay between the eyetracker 120 and the driver unit 110, i.e. the time difference between the reception point in time t.sub.R and a point in time t.sub.D2 in the second frame structure FS2 when the gaze data packets D.sub.GT(2) and D.sub.GT(3) are sent out from the eyetracker 120.

(22) Consequently, the data path between the eyetracker 120 and the driver unit 110 is modeled adequately, and the timing of the gaze data in the client 130 can be made even more accurate with respect to the client's 130 requested delivery point in time t.sub.D1 in a first time frame structure FS1 as expressed in the request message m.sub.req.

(23) In order to sum up, and with reference to the flow diagram in FIG. 6, we will now describe the general method according to the invention for streaming gaze data from an eyetracker to a client via a driver unit. The dashed boxes at the bottom illustrate one embodiment of the invention, which is preferable to apply if the eyetracker and the client use different time base references.

(24) A first step 610, checks if a request message has been received from the client. If so, a step 620 follows; and otherwise, the procedure loops back and stays in step 610. The request message defines a delivery point in time in a first time frame structure at which delivery point in time in each frame of the first time frame structure the gaze data packet shall be provided to the client.

(25) In step 620, an offset is calculated between a reception point in time and the delivery point in time. Then, in a step 630, an adjusted data acquisition instance is assigned based on the offset. The adjusted data acquisition instance represents a modified point in time in a second time frame structure when at least one future gaze data packet shall be produced by the eyetracker.

(26) Subsequently, in a step 640, a control message is sent to the eyetracker. The control message is adapted to cause the eyetracker to produce at least one future gaze data packet at such an adjusted acquisition instance in the second time structure that the reception point in time for the at least one future gaze data packet is expected to lie within a margin prior to the delivery point in time.

(27) Thereafter, according to the general method of the invention, the procedure loops back to step 610. However, if the eyetracker and the client use different time base references, a step 650 may follow.

(28) In step 650, an updated offset is calculated between the reception point in time and the delivery point in time. Then, a step 660, checks if the offset lies within the margin prior to the delivery point in time. If so, the procedure loops back to step 610. Otherwise, a step 670 follows in which an updated control message is sent to eyetracker for adjusting its data acquisition instance to lie within the margin prior to the delivery point in time.

(29) Subsequently, the procedure loops back to step 610.

(30) All of the process steps, as well as any sub-sequence of steps, described with reference to FIG. 6 above may be controlled by means of at least one programmed processor. Moreover, although the embodiments of the invention described above with reference to the drawings comprise processor and processes performed in at least one processor, the invention thus also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other form suitable for use in the implementation of the process according to the invention. The program may either be a part of an operating system, or be a separate application. The carrier may be any entity or device capable of carrying the program. For example, the carrier may comprise a storage medium, such as a Flash memory, a ROM (Read Only Memory), for example a DVD (Digital Video/Versatile Disk), a CD (Compact Disc) or a semiconductor ROM, an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic recording medium, for example a floppy disc or hard disc. Further, the carrier may be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable or by radio or by other means. When the program is embodied in a signal which may be conveyed directly by a cable or other device or means, the carrier may be constituted by such cable or device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing, or for use in the performance of, the relevant processes.

(31) The term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components. However, the term does not preclude the presence or addition of one or more additional features, integers, steps or components or groups thereof.

(32) The invention is not restricted to the described embodiments in the figures, but may be varied freely within the scope of the claims.