Navigation device with fast frame rate upshift
10613650 ยท 2020-04-07
Assignee
Inventors
Cpc classification
G06F3/0312
PHYSICS
International classification
G06F3/038
PHYSICS
G06F3/03
PHYSICS
Abstract
A navigation device including an image sensor, a processor and a memory is provided. The memory stores a lookup table of a plurality of moving speeds each corresponding to one frame period. The image sensor captures image frames successively. The processor calculates a current speed according to a current image frame and a previous image frame, reads a frame period from the lookup table according to the calculated current speed, wherein the read frame period is multiplied by a ratio, which is smaller than 1, when an acceleration is confirmed by the processor according to the captured image frames.
Claims
1. A navigation device, configured to detect displacement with respect to a surface, the navigation device comprising: a memory configured to store a predetermined relationship between a plurality of moving speeds and a plurality of frame periods; an image sensor configured to successively capture a second previous image frame, a previous image frame and a current image frame; and a processor electrically coupled to the memory and the image sensor, and configured to calculate a previous displacement according to the second previous image frame and the previous image frame, calculate a current displacement and a current speed according to the previous image frame and the current image frame, read a frame period from the memory according to the calculated current speed, determine an acceleration according to the current displacement and the previous displacement, set a next frame period as the read frame period when the acceleration is smaller than an acceleration threshold, and set the next frame period to be shorter than the read frame period when the acceleration is larger than the acceleration threshold, wherein the previous displacement includes a first direction previous displacement and a second direction previous displacement; the current displacement includes a first direction current displacement and a second direction current displacement; the calculated current speed includes a first direction speed and a second direction speed, and the read frame period is determined according to a bigger one of the first direction speed and a second direction speed; and the acceleration is a larger absolute value of (i) a first direction difference between the first direction previous displacement and the first direction current displacement, and (ii) a second direction difference between the second direction previous displacement and the second direction current displacement.
2. The navigation device as claimed in claim 1, wherein the first direction speed is calculated by dividing the first direction current displacement by a current frame period between the current image frame and the previous image frame; and the second direction speed is calculated by dividing the second direction current displacement by the current frame period.
3. The navigation device as claimed in claim 2, wherein the current frame period is another read frame period accessed from the memory or a frame period shorter than the read frame period.
4. The navigation device as claimed in claim 1, wherein when the acceleration is larger than the acceleration threshold, the next frame period is set according to an acceleration ratio, when the first direction difference is larger than the second direction difference, the acceleration ratio is determined according to a ratio between the first direction current displacement and the first direction previous displacement; and when the second direction difference is larger than the first direction difference, the acceleration ratio is determined according to a ratio between the second direction current displacement and the second direction previous displacement.
5. The navigation device as claimed in claim 1, further comprising a register electrically coupled to the processor and configured to store a flag indicating whether the acceleration is smaller or larger than the acceleration threshold.
6. The navigation device as claimed in claim 1, further comprising a light source configured to illuminate the surface, wherein the processor is further configured to adjust an emission period of the light source corresponding to the next frame period.
7. A navigation device, configured to detect displacement with respect to a surface, the navigation device comprising: a light source; an image sensor configured to successively capture a second previous image frame, a previous image frame and a current image frame; and a processor electrically coupled to the light source and the image sensor, and configured to calculate a previous displacement according to the second previous image frame and the previous image frame, calculate a current displacement and a current speed according to the previous image frame and the current image frame, determine an emission period of the light source corresponding to the calculated current speed, determine an acceleration according to the current displacement and the previous displacement, set a next emission period as the determined emission period when the acceleration is smaller than an acceleration threshold, and set the next emission period to be shorter than the determined emission period when the acceleration is larger than the acceleration threshold, wherein the previous displacement includes a first direction previous displacement and a second direction previous displacement; the current displacement includes a first direction current displacement and a second direction current displacement; the calculated current speed includes a first direction speed and a second direction speed, and the emission period is determined corresponding to a bigger one of the first direction speed and a second direction speed; and the acceleration is a larger absolute value of (i) a first direction difference between the first direction previous displacement and the first direction current displacement, and (ii) a second direction difference between the second direction previous displacement and the second direction current displacement.
8. The navigation device as claimed in claim 7, wherein the first direction speed is calculated by dividing the first direction current displacement by a current frame period between the current image frame and the previous image frame; and the second direction speed is calculated by dividing the second direction current displacement by the current frame period.
9. The navigation device as claimed in claim 7, wherein when the acceleration is larger than the acceleration threshold, the next emission period is set corresponding to an acceleration ratio, when the first direction difference is larger than the second direction difference, the acceleration ratio is determined according to a ratio between the first direction current displacement and the first direction previous displacement; and when the second direction difference is larger than the first direction difference, the acceleration ratio is determined according to a ratio between the second direction current displacement and the second direction previous displacement.
10. The navigation device as claimed in claim 7, wherein the emission period is determined corresponding to a predetermined relationship between a plurality of moving speeds and a plurality of frame periods.
11. The navigation device as claimed in claim 7, further comprising a register electrically coupled to the processor and configured to store a flag indicating whether the acceleration is smaller or larger than the acceleration threshold.
12. The navigation device as claimed in claim 7, wherein the image sensor is further configured to capture a next image frame using a frame period corresponding to the set emission period.
13. A navigation device, configured to detect displacement with respect to a surface, the navigation device comprising: a memory configured to store a predetermined relationship between a plurality of moving speeds and a plurality of frame rates; an image sensor configured to successively capture a second previous image frame, a previous image frame and a current image frame; and a processor electrically coupled to the memory and the image sensor, and configured to calculate a previous displacement according to the second previous image frame and the previous image frame, calculate a current displacement and a current speed according to the previous image frame and the current image frame, read a frame rate from the memory according to the calculated current speed, determine an acceleration according to the current displacement and the previous displacement, set a next frame rate as the read frame rate when the acceleration is not detected, and set the next frame rate to be faster than the read frame rate when the acceleration is detected, wherein the previous displacement includes a first direction previous displacement and a second direction previous displacement; the current displacement includes a first direction current displacement and a second direction current displacement; the calculated current speed includes a first direction speed and a second direction speed, and the read frame rate is determined according to a bigger one of the first direction speed and a second direction speed; and the acceleration is an absolute value of (i) a first direction difference between the first direction previous displacement and the first direction current displacement, or (ii) a second direction difference between the second direction previous displacement and the second direction current displacement.
14. The navigation device as claimed in claim 13, wherein the first direction speed is calculated by dividing the first direction current displacement by a current frame period between the current image frame and the previous image frame; and the second direction speed is calculated by dividing the second direction current displacement by the current frame period.
15. The navigation device as claimed in claim 13, wherein when the acceleration is detected, the next frame rate is set according to an acceleration ratio, when the first direction difference is larger than the second direction difference, the acceleration ratio is determined according to a ratio between the first direction current displacement and the first direction previous displacement; and when the second direction difference is larger than the first direction difference, the acceleration ratio is determined according to a ratio between the second direction current displacement and the second direction previous displacement.
16. The navigation device as claimed in claim 13, further comprising a light source configured to illuminate the surface, wherein the processor is further configured to adjust an emission period of the light source corresponding to the next frame rate.
17. The navigation device as claimed in claim 13, further comprising a register electrically coupled to the processor and configured to store a flag indicating whether the acceleration occurs.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Other objects, advantages, and novel features of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
(2)
(3)
(4)
(5)
(6)
DETAILED DESCRIPTION OF THE EMBODIMENT
(7) It should be noted that, wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
(8) Referring to
(9) The navigation device 10 includes a light source 11, an image sensor 13, a processor 15, a memory 17 and a register 19 electrically coupled to each other, e.g., through an electric board, signal line and/or semiconductor circuit. In some embodiments, these elements included in the navigation device 10 are arranged inside a housing 100 of the navigation device 10. The housing 100 preferably has an opening through which the navigation device 10 detects the relative displacement with respect to the surface S.
(10) The light source 11 emits light of an identifiable spectrum to illuminate the surface S. Generally, said identifiable spectrum is arranged within an invisible spectrum such that the emitted light does not affect the user. For example, the light source 11 is at least one red light emitting diode (LED), at least one red laser diode, at least one infrared LED or at least one infrared laser diode, but not limited thereto.
(11) The image sensor 13 receives light reflected from the surface S at a frame rate to output image frames. The image sensor 13 is a CMOS image sensor, a CCD image sensor or other optical sensors for generating raw data according to the received light from the surface S.
(12) In some embodiments, the navigation device 10 further includes the light guiding element(s) for guiding the light emitted from the light source 11 and/or received by the image sensor 13. It is possible to arrange the light source 11 and the image sensor 13 as the so-called bright field arrangement or dark field arrangement according to different applications.
(13) Referring to
(14) The light source 11 is controlled by the processor 15 to illuminate light every an emission period Pe which is adjusted corresponding to the frame periods, e.g., P.sub.P, P.sub.C and P.sub.N, of the image sensor 13. The image sensor 13 is controlled by the processor 15 to capture image frames with an adjustable frame period (described later). For example,
(15) The memory 17 includes a volatile and/or a nonvolatile memory device for storing a predetermined relationship between a plurality of moving speeds and a plurality of frame periods. Other parameters or algorithms for the operation are also stored in the memory 17. For example, referring to
(16) Preferably, the lookup table or the predetermined relationship is previously stored in the memory 17 before the shipment of the navigation device 10. It can be seen from
frame period=frame rate/FPC clock
(17) In
(18) The processor 15 is a central processing unit (CPU), a microcontroller (MCU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or the like. The processor 15 is electrically coupled to the memory 15 to access the lookup table and electrically coupled to the image sensor 13 to receive the image frames, e.g., F.sub.P2, F.sub.P, F.sub.C and F.sub.N, from the image sensor 13 for post-processing. The above operation performed by the processor 15 is implemented by software codes and/or hardware codes without particular limitations. In the present disclosure, the processor 15 adjusts the frame period (or frame rate) based on
(19) In one embodiment, the processor 15 calculates a previous displacement according to the second previous image frame and the previous image frame, calculates a current displacement and a current speed according to the previous image frame and the current image frame, reads a frame period from the memory according to the calculated current speed, determines a speed change according to the current displacement and the previous displacement, sets a next frame period as the read frame period when the speed change is smaller than an acceleration threshold, and sets the next frame period as the read frame period multiplied by an acceleration ratio, which is smaller than 1 and calculated according to the previous displacement and the current displacement, when the speed change is larger than the acceleration threshold.
(20) Referring to
(21) The processor 15 controls the image sensor 13 to successively capture a second previous image frame F.sub.P2, a previous image frame F.sub.P and a current image frame F.sub.C. The processor 15 calculates a previous displacement D.sub.P by comparing the second previous image frame F.sub.P2 and the previous image frame F.sub.P, e.g., calculating the correlation between the second previous image frame F.sub.P2 and the previous image frame F.sub.P, but not limited thereto. The previous displacement D.sub.P includes a first direction previous displacement D.sub.PX and a second direction previous displacement D.sub.PY, wherein the first direction X is perpendicular to the second direction Y in the present disclosure. The previous displacement D.sub.P is divided into the first direction previous displacement D.sub.PX and the second direction previous displacement D.sub.PY using triangulation.
(22) In a similar way, the processor 15 calculates a current displacement D.sub.C by comparing the previous image frame F.sub.P and the current image frame F.sub.C. The current displacement D.sub.C also includes a first direction current displacement D.sub.CX and a second direction current displacement D.sub.CY.
(23) Furthermore, the processor 15 calculates a current speed Vc using the current displacement D.sub.C and a current frame period P.sub.C between the current image frame F.sub.C and the previous image frame F.sub.P. Similarly, the current speed Vc includes a first direction speed and a second direction speed. The first direction speed is calculated by dividing the first direction current displacement D.sub.CX by the current frame period P.sub.C, and the second direction speed is calculated by dividing the second direction current displacement D.sub.CY by the current frame period P.sub.C.
(24) Then the processor 15 reads a frame period from the memory 17 according to the calculated current speed Vc. For example, the processor 15 compares the first direction speed and the second direction speed at first, and then selects the frame rate in
(25) Finally, the processor 15 identifies whether an acceleration or de-acceleration (i.e. a negative acceleration) is detected so as to determine the next frame period P.sub.N for capturing a next image frame F.sub.N. In one embodiment, the acceleration occurs if anyone of (i) a first direction difference, which is an absolute value, between the first direction previous displacement D.sub.PX and the first direction current displacement D.sub.CX and (2) a second direction difference, which is an absolute value, between the second direction previous displacement D.sub.PY and the second direction current displacement D.sub.CY is larger than an acceleration threshold, wherein the acceleration threshold is arranged as a half pixel size, one pixel size or others depending on different applications. The first direction difference (i.e. |D.sub.PXD.sub.CX|) and the second direction difference (i.e. |D.sub.PYD.sub.CY|) are referred to the speed change or the acceleration between the previous frame period P.sub.P and the current frame period P.sub.C herein.
(26) Regarding the way of comparison, the processor 15 respectively compares the first direction difference (i.e. |D.sub.PXD.sub.CX|) and the second direction difference (i.e. |D.sub.PYD.sub.CY|) with the acceleration threshold, or compares a larger one of the first direction difference (i.e. |D.sub.PXD.sub.CX|) and the second direction difference (i.e. |D.sub.PYD.sub.CY|) with the acceleration threshold to confirm whether an enough acceleration occurs. In the present disclosure, the acceleration threshold is a fixed value or adaptively changeable.
(27) If the acceleration is not detected (i.e. both the first and second direction differences smaller than the acceleration threshold), the next frame period P.sub.N is set as the read frame period Pr, which is read from the memory 17 according to the calculated current speed Vc within the current frame period P.sub.C.
(28) If the acceleration is detected (i.e. any one the first direction difference and second direction difference larger than the acceleration threshold), the next frame period P.sub.N is set as the read frame period Pr multiplied by an acceleration ratio. The acceleration ratio is smaller than 1 and calculated according to the previous displacement D.sub.P and the current displacement D.sub.C. For example, when the first direction difference (i.e. |D.sub.PXD.sub.CX|) is larger than the second direction difference (i.e. |D.sub.PYD.sub.CY|), a larger acceleration in the first direction, the acceleration ratio is set as a smaller one of (i) the first direction current displacement D.sub.CX divided by the first direction previous displacement D.sub.PX, and (ii) the first direction previous displacement D.sub.PX divided by the first direction current displacement D.sub.CX, i.e. the one of (i) and (ii) which is smaller than 1. However, when the second direction difference (i.e. |D.sub.PYD.sub.CY|) is larger than the first direction difference (i.e. |D.sub.PXD.sub.CX|), a larger acceleration in the second direction, the acceleration ratio is a smaller one of (i) the second direction current displacement D.sub.CY divided by the second direction previous displacement D.sub.PY, and (ii) the second direction previous displacement D.sub.PY divided by the second direction current displacement D.sub.CY, i.e. the one of (i) and (ii) which is smaller than 1. Accordingly, the next frame period P.sub.N is shorter than the read frame period Pr to avoid the runaway happens due to the existence of the acceleration.
(29) In this way, the processor 15 sets the next frame period P.sub.N as the read frame period Pr or being shorter than the read frame period Pr. Using the similar process, the current frame period P.sub.C is another read frame period P.sub.r2 (i.e. determined according to a previous moving speed Vp) accessed from the memory 17 or the another read frame period P.sub.r2 multiplied by another acceleration ratio calculated within the previous frame period P.sub.P.
(30) In some embodiments, the navigation device 10 further includes a register 19 electrically coupled to the processor 15. The register 19 stores a flag bit indicating whether the acceleration is smaller or larger than the acceleration threshold. For example, the register 19 stores a digital value 0 or 1 if the acceleration is not detected (e.g., speed change smaller than an acceleration threshold) by the processor 15, and the digital value is changed to 1 or 0 if the acceleration is detected (e.g., speed change bigger than an acceleration threshold). In other embodiments, the register 19 is included in the memory 17 or the processor 15, but not limited to that shown in
(31) Referring to
(32) Referring to
(33) Step S41: The processor 15 controls the image sensor 13 to successively capture a second previous image frame F.sub.P2 and a previous image frame F.sub.P separated by a previous frame period P.sub.P, and captures the previous image frame F.sub.P and a current image frame F.sub.C separated by a current frame period P.sub.C. Meanwhile, the processor 15 controls the light source 11 to illuminate corresponding to the image capturing of the image sensor 13.
(34) Step S42: Next, the processor 15 calculates a previous displacement D.sub.P by comparing the second previous image frame F.sub.P2 and the previous image frame F.sub.P using any suitable way. The previous displacement D.sub.P includes a first direction previous displacement D.sub.PX and a second direction previous displacement D.sub.PY. It is possible that the previous displacement D.sub.P is calculated and temporarily stored, e.g., in the memory 17, before the current image frame F.sub.C is captured.
(35) Step S43: The processor 15 calculates a current displacement D.sub.C by comparing the previous image frame F.sub.P and the current image frame F.sub.C using any suitable way. The current displacement D.sub.C also includes a first direction current displacement D.sub.CX and a second direction current displacement D.sub.CY. Meanwhile, the processor 15 calculates a current speed Vc using the current displacement D.sub.C and a current frame period P.sub.C. The current speed Vc is possible to be divided into two elements in a first direction (e.g., X direction) and a second direction (e.g., Y direction). The calculation of the first direction speed and the second direction speed has been illustrated above, and thus details thereof are not repeated herein.
(36) Step S44: The processor 15 reads a frame period from the memory 17 according to a larger one of the first direction speed and the second direction speed.
(37) Step S45: The processor 15 then confirms whether an acceleration is detected according to a speed change between the current displacement D.sub.C and the previous displacement D.sub.P. As mentioned above, if any one of the first direction difference (i.e. |D.sub.PXD.sub.CX|) and the second direction difference (i.e. |D.sub.PYD.sub.CY|) is larger than an acceleration threshold, which is previously set based on pixel size, the processor 15 confirms the acceleration occurs.
(38) Step S46: If the acceleration does not occur, e.g., by accessing the register 19, the processor 15 captures a next image frame F.sub.N after a next frame period P.sub.N which is directly set as a read frame period Pr corresponding to the calculated current speed Vc (e.g., the first or second direction speed) by accessing the lookup table stored in the memory 17. The calculation of the current speed Vc has been illustrated above and thus details thereof are not repeated herein.
(39) Step S46: If the acceleration is detected, e.g., by accessing the register 19, the processor 15 sets a next frame period P.sub.N as a read frame period Pr multiplied by an acceleration ratio. As mentioned above, the processor 15 calculates the acceleration ratio using a larger speed change among the first and second directions. If the larger speed change is in the first direction, the processor 15 calculates the acceleration ratio by using a smaller one among the first direction current displacement D.sub.CX and the first direction previous displacement D.sub.PX to be divided by a larger one among the first direction current displacement D.sub.CX and the first direction previous displacement D.sub.PX. However, if the larger speed change is in the second direction, the processor 15 calculates the acceleration ratio by using a smaller one among the second direction current displacement D.sub.CY and the second direction previous displacement D.sub.PY to be divided by a larger one among the second direction current displacement D.sub.CY and the second direction previous displacement D.sub.PY.
(40) In other embodiments, if the lookup table stored in the memory 17 includes a plurality of moving speed each corresponding to one frame rate as shown in the second and third columns in
(41) Referring to
(42) The steps S41-S45 in
(43) Step S56: The processor 15 identifies whether an acceleration or de-acceleration occurs by comparing the acceleration, which is referred herein as a speed change between the previous frame period P.sub.P and the current frame period P.sub.C, with an acceleration threshold. In some embodiments, the acceleration is calculated by (|D.sub.PX/P.sub.PD.sub.CX/P.sub.C|) or (|D.sub.PY/P.sub.PD.sub.CY/P.sub.C|) only the acceleration threshold is arranged as a different value.
(44) The next frame period P.sub.N is set as the frame period, which is read in the Step S44, multiplied by an acceleration ratio. More specifically, in this embodiment, the processor 15 determines the acceleration ratio according to whether the acceleration is detected or not. If the acceleration is detected, the acceleration ratio is set as 1. If the acceleration is not detected, the acceleration ratio is set as being smaller than 1 to shorten the frame period. The calculation of the acceleration ratio is identical to that illustrated in
(45) It should be mentioned that in the above embodiments the displacement and moving speed are separated into two elements in two orthogonal directions (e.g., X direction and Y direction), but it is only intended to illustrate but not to limit the present disclosure. In other embodiments, the acceleration threshold is compared with |D.sub.PD.sub.C| and the read frame period Pr is determined according to the current displacement Vc instead of the sub-elements in two orthogonal directions. It is appreciated that in this case, values of the acceleration threshold and in the lookup table are set to be different from the above embodiments.
(46) The operating method mentioned above is also applicable to a touchpad or a touch screen without being limited to optical displacement detection devices. In this case, the device employing the operating method of the present disclosure does not include a light source.
(47) As mentioned above, as the so-called runaway or speed break often happens when an optical mouse operates in a very low speed if an image capture frame rate of the optical mouse is adjusted with the moving speed. Therefore, the present disclosure further provides a navigation device (
(48) Although the disclosure has been explained in relation to its preferred embodiment, it is not used to limit the disclosure. It is to be understood that many other possible modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the disclosure as hereinafter claimed.