SYSTEM FOR THREE-DIMENSIONAL IMAGE CAPTURE WHILE MOVING

Abstract

A system for three-dimensional image capture while moving. The system includes: an image capture device including at least two digital image sensors placed to capture a stereoscopic image; a processor for obtaining disparity information, associated with the digital images obtained by the image capture device and movement speed of elements in the digital images; a transmitter for sending the digital images to enable control of movement based on the digital images and the associated disparity information; and a controller for controlling the image capture device.

Claims

1. A system for capturing three-dimensional images from an object capable of movement, the system comprising: an image capture device including at least two digital image sensors, synchronized with each other, producing digital images, and arranged for capture a stereoscopic image, processing means to obtain disparity information associated with the digital images obtained by the image capture device and movement speed of elements in the digital images, transmission means for controlling movement of the object, based on the digital images and the disparity information associated with the digital images, and control means to command the image capture device, by modifying, to facilitate controlling the movement, choice of dimensions of fields of view of the digital image sensors, and frequency of stereoscope image capture, taking into account maximum throughput tolerated by the transmission means and the movement speed.

2. The system for three-dimensional image capture according to claim 1, wherein the movement speed is assessed by a contrast gradient calculation on the digital images.

3. The system for three-dimensional image capture according to claim 1, wherein the transmission means comprises one of a USB 3.0 connection, a Giga Ethernet connection, and a Thunderbolt connection.

4. The system for three-dimensional image capture according to claim 1, wherein the processing means comprises a computer programme to be run on a microcomputer.

5. The system for three-dimensional image capture according to claim 1, wherein the processing means comprises a chipset on a printed circuit substrate.

6. The system for three-dimensional image capture according to claim 1, wherein the dimensions of the fields of view of the digital image sensors and the frequency of stereoscopic image capture can be set by a user.

7. The system for three-dimensional image capture according to claim 1, wherein the two digital image sensors are master and slave, respectively, for synchronization purposes.

8. The system for three-dimensional image capture according to claim 1, wherein the image capture device comprises a digital image processing controller and a printed circuit substrate, and the controller and the two digital image sensors are located on a printed circuit substrate of a shared substrate.

9. The system for three-dimensional image capture according claim 1, wherein the transmission means comprises means for transmitting the digital images between the image capture device and the processing means.

10. The system for three-dimensional image capture according to claim 1, wherein the transmission means comprises means for transmitting the digital images and the disparity information associated with the digital images to means for controlling the movement.

11. The system for three-dimensional image capture according to claim 1, wherein, if the elements that are observed are moving relatively quickly, the frequency of stereoscopic image capture is increased, and the field of view is retracted, and, if the elements that are observed are moving relatively slowly, the frequency of stereoscopic image capture is decreased, and the field of view is expanded.

12. An object capable of moving and comprising an image capture system according to claim 1, wherein the object comprises means for sending a human operator or an electronic module the disparity information associated with the digital images produced by the system for three-dimensional image capture, in order to control the movement of the object.

13. The object according to claim 12, comprising means for moving in autonomous navigation by using the disparity information associated with the digital images obtained by the system for three-dimensional image capture.

14. A method for controlling movement of an object including movement means and having an on board system for three-dimensional image capture, the method comprising: capturing an image using at least two digital image sensors synchronized with each other and arranged to capture a stereoscopic image, obtaining movement speed of elements in the digital images, obtaining disparity information associated with pairs of synchronized digital images, modifying, to facilitate control of the movement of the object, dimensions of fields of view of the sensors and frequency of capture of the digital images, taking into account a maximum throughput tolerated by transmission means and taking into account speed of the movement, sending digital images and the disparity information associated with the digital images to a navigation unit via the transmission means, controlling the movement based on the digital images and the disparity information associated with the pairs of synchronized digital images, with the navigation unit.

15. The movement control method according to claim 14, wherein the navigation unit is an autonomous navigation unit.

Description

BRIEF DESCRIPTION OF DRAWING FIGURES

[0028] The invention will be better understood, and other aims, features, details and advantages thereof will appear more clearly, in the following explanatory description done in reference to the appended drawings, provided solely as an example illustrating one embodiment of the invention, and in which:

[0029] FIG. 1 is a general view in space of one embodiment of an image capture device according to the invention,

[0030] FIG. 2 is a diagram showing the operation of the elements of one embodiment of an image capture device according to the invention,

[0031] FIG. 3 is one aspect of the operation of an image capture device according to the invention,

[0032] FIG. 4 shows an outside view of one embodiment of an image capture device according to the invention,

[0033] FIGS. 5 and 6 show example embodiments of the image capture device according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0034] FIG. 1 provides a schematic view of an electronic board of an image capture device 10 according to one embodiment of the invention. The board 10 is formed by a single printed circuit substrate supporting electronic components. The image capture device board 10 comprises a first digital image sensor 100 and a second digital image sensor 110, which can be CCD (charge-coupled device) sensors or sensors using another technology, such as the CMOS technology, and receive light through an optical lens focusing system (not shown).

[0035] The selected sensors 100 and 110 have an active surface that can be controlled by a command outside the sensor, and which can be used for different viewing angle dimensions, or different resolutions. They may also be controlled to capture images at different frequencies.

[0036] The sensors 100 and 110 are both positioned on a printed circuit substrate 120. The substrate 120 can be an elongated rectangle, the large dimension of which is placed along a direction X, and the two sensors are then placed substantially at both ends of the rectangle in the direction X. The two sensors 100 and 110 are positioned with their sensitive surfaces oriented opposite the substrate 120, turned substantially in a direction Z perpendicular to the direction X, and normal to the plane of the printed circuit substrate 120.

[0037] On the printed circuit substrate 120 is a controller 130, in the form of an integrated circuit electrically connected to the two sensors 100 and 110. The controller 130 receives commands from third-party components and provides operating parameters to the sensors 100 and 110. It furthermore verifies that these commands have indeed been received and understood by the sensors 100 and 110.

[0038] Additionally, the printed circuit substrate 120 bears a connector 140, making it possible to connect the image capture device 10 to a third-party device.

[0039] FIG. 2 functionally shows the image capture device board 10, and its interaction with an outside processing and control unit 20. The first sensor 100 and the second sensor 110 are again shown, as well as the controller 130 and the connector 140. The controller 130 runs several programs constituting several modules 131, 132, 133 at 134 that will be outlined later. The controller 130, the sensors 100 and 110 and the connector 140 communicate via tracks of conductive material (copper) of the board 10.

[0040] Each of the first and second sensors 100 and 110 captures digital images at an identical frequency for both sensors, and set by the controller 130, which executes a program forming an image sensor processing module 133. This module 133 sends control commands of the first sensor C1 and control commands of the second center C2 to both sensors, in particular to define the capture frequency of the images.

[0041] The first sensor 100 regularly sends a synchronization message M1 to the second sensor 110, to allow the latter to align the capture moments of its images with the capture moments of the images by the first sensor 100. This involves a master-slave mode synchronization, the first sensor 100 acting as master, imposing the capture moments, and the second sensor 110 acting as slave, applying the instruction received to implement the image capture.

[0042] The sensors 100 and 110 send the images they capture to the controller 130, via messages I1 and I2 comprising the color and brightness information of the pixels. The captured and transmitted images are rectangular, the rectangles of the two sensors being identical (same width, same height, same orientation).

[0043] A first program constitutes a receiving and resynchronization module 131, responsible for receiving the digital images captured by the sensors 100 and 110 and re-synchronizing them such that the following programs implemented by the controller 130 are able to recognize the pairs of images sent by the first and second sensors 100 and 110.

[0044] A second program constitutes a merging module 132 responsible for merging the two images taken at the same time by the two sensors, and transmitted by the module 131. The two images are merged into a single digital image, as will be described in connection with FIG. 3. The single merged image is sent to the following module.

[0045] The following module, already described, constitutes an image sensor processing (ISP) module 133. In some embodiments, it may perform parallel calculations to specify the color and light of each pixel. Once its processing is complete, it sends the image to the following module.

[0046] The next program defines a fourth module, which is a conversion module 134, responsible for converting the data making up the digital images, and potentially other information, into a format compatible with the transmission via a wired connector 150 connected to the connector 140, and conversely, extracting the information received by the wired connector 150 and the connector 140 to send it to the image sensor processing module 133.

[0047] The wired module 150, which for example is flexible, connects the connector 140 to a connector 201 of the outside processing and control unit 20. The wired connector 150 can perform an electrical or optical transmission.

[0048] The outside processing and control unit 20 comprises a connector 201, and a processor 200 that runs a processing and control program 202. It also comprises a conversion module 204 for the reception and sending of data by the connector 201, which may be run by the processor 200 or a dedicated controller.

[0049] In one embodiment, the transmission by the wired connector 150 is done according to standard USB 3.0. The conversion module 134 and the wired connector 150, as well as the connectors 140 and 201, are configured to implement standard USB 3.0. Other standards can be used, in particular the Gigethernet and Thunderbolt standards.

[0050] The processor 200 implements the processing and control program 202, which uses the digital images transmitted by the image sensor processing module 133. In particular, the disparity or depth information is calculated for each pair of synchronized images.

[0051] Furthermore, a contrast gradient calculation makes it possible to assess the presence of motion blurring in the images.

[0052] The processing and control program 202 is also able to send commands to the controller 130 of the image capture device 10, for example via the wired connector 150, or by another wired or wireless means.

[0053] Depending on the result of the contrast gradient calculation, the processing and control program 202 is able to command automatically, or with agreement from a user to whom the command is suggested by the program, a change in the choice of the dimensions of the field of view of the sensors 100 and 110 and the image capture frequency by the sensors. This choice is framed by the relationship

width×height×16×frequency max throughput

[0054] wherein the width and height are expressed in pixels and the maximum throughput in bits/s. The maximum throughput is that of the connection between the image capture device 10 and the outside processing and control unit 20, which is for example the maximum throughput of a USB 3.0 connection.

[0055] If motion blurring is detected, it is then chosen to decrease the field of view and increase the image capture frequency, to improve the vision of the objects in relative movement around the image capture device 10.

[0056] If little or no motion blurring is observed, it is chosen to increase the field of view and decrease the image capture frequency, to improve the perception of the environment, including on the sides.

[0057] The outside processing and control unit 20 includes a second connector 203, as well as a second conversion module 205 for the reception and transmission of data by the connector 203, which can be done by the processor 200 or a dedicated controller.

[0058] The outside processing and control unit 20 can assume the form of a multifunctional computer or a unit dedicated to controlling the image capture device 10. In the second case, it may for example be built in the form of a single printed circuit board bearing the connectors 201 and 203, as well as a processor 200, which can then run both the program 202 and the modules 204 and 205.

[0059] The module 205 allows the outside processing and control unit 20 to communicate with a third-party device, via a second wired connection means 250. The second wired connector 250 can perform an electrical or optical transmission.

[0060] This is for example a USB 3.0 cable, or a Giga Ethernet or Thunderbolt connection.

[0061] The processing and control unit 20 sends the stereoscopic digital images and the disparity or depth information associated therewith via the wired connector 250. Thus, a three-dimensional depiction of the image captured by the image capture device 10 is provided, either to a human user, or to a software program capable of exploiting it.

[0062] FIG. 3 shows the merging process implemented by the merging module 132.

[0063] Two images 1000 and 1001 have been captured by the sensors 100 and 110 (FIG. 1). Their content is referenced G and D in the figure, to indicate that these are images captured by the left and right sensors, respectively, of the image capture device 10. The two images are each a rectangular set of pixels with the same heights and the same lengths. They are subject to merging by the merging module 132, which creates a single image 2000 made up of a rectangular set of pixels, of the same height as the two images 1000 and 1001 and twice the length relative to the length of the images 1000 and 1001. The left side of the right image D is placed next to the right side of the left image G.

[0064] FIG. 4 shows a three-dimensional view of one embodiment of the image capture device. It may for example assume the form of a box 10′ generally in the shape of a rectangular rhomb or the like, in which the electronic components of the device are contained, and which has two openings to the light on one face, through which the first and second sensors 100 and 110 expose their respective sensitive surfaces to capture digital images. The wired connector 250 allowing the extraction of the digital data is not visible.

[0065] FIG. 5 shows a use of an image acquisition device box 10′ in a flying object 5000 of the drone type capable of moving without human intervention, or remotely controlled by a human. The flying object 5000 carries an image acquisition device box 10′, as well as an outside processing and control unit 20, connected by a wired connection means 150. It also comprises an autonomous navigation module 5100, interacting with the outside processing and control unit 20, via a wired connection means 250. The autonomous navigation unit 5100 uses the stereoscopic images provided by the image capture device and the associated disparity or depth information, pixel by pixel, to make navigation decisions. The navigation decisions are of better quality if the dimensions of the field of view of the sensors and the image capture frequency have been chosen carefully and are readjusted regularly, reactively based on the environment and objects observed in the images and their relative speed with respect to the flying object 5000.

[0066] Likewise, in FIG. 6, the same principle is presented, this time with a rolling object 6000 of the autonomous car type capable of moving without human intervention in a complex environment, for example a road or urban environment. The vehicle can also be a construction vehicle or a wagon moving in a warehouse, for example.

[0067] For the objects 5000 and 6000, the navigation can be autonomous or computer-assisted navigation, the decision-making in this case always being done by a human, but the latter having additional information obtained owing to the image capture device and the associated system.

[0068] In one alternative, the box 10′ of the image capture device can incorporate the control unit 20 inside it, and be on board a vehicle such that the user simply connects the box 10′, which is then the only one, to the autonomous navigation module 5100 or 6100 using the wired connection means 250. The wired connection means 150 is then inside the box 10′.

[0069] Within the single box 10′, in one alternative, the processor 200 can also be mounted on the board 10, and communicate via a connection using conductive metal tracks (copper), with the module 133 of the controller 130, in place of the communication via the wired connection means 150, the connectors 140 and 201 and the conversion module 134.

[0070] The maximum throughput value used to command a change in the choice of dimensions of the field of view of the sensors 100 and 110 and the image capture frequency by the sensors is in this case that of the wired connection means 250.

[0071] In another alternative, the control unit 20 and the autonomous navigation unit 5100 or 6100 are embodied by a same piece of computer equipment, having a processor that runs both the autonomous navigation program and the processing and control program 202. The connection 250, the connector 203 and the conversion module 205 are then absent.

[0072] In general, the maximum throughput value used to command a change in the choice of the dimensions of the field of view of the sensors 100 and 110 and the image capture frequency by the sensors is that of the most limiting connection for the transmission of the digital images to obtain disparity information and to send digital images to the navigation module.

[0073] The invention is not limited to the described embodiment, but extends to all alternatives within the scope of the claims.