CAMERA SYSTEM AND OPERATING METHODS THEREFOR
20180167550 ยท 2018-06-14
Assignee
Inventors
Cpc classification
H04N23/66
ELECTRICITY
H04N1/00172
ELECTRICITY
H04N1/32448
ELECTRICITY
H04N23/00
ELECTRICITY
International classification
Abstract
The invention relates to a camera system with an image sensor for generating primary image data and at least one interface unit for the data exchange with an external unit, wherein the camera system is designed to store at least a portion of the primary image data generated by the image sensor at least temporarily in a storage unit associated with the camera system.
Claims
1. A camera system with an image sensor for generating primary image data and at least one interface unit for the data exchange with an external unit, wherein the camera system is designed to store at least a portion of the primary image data generated by the image sensor at least temporarily in a storage unit associated with the camera system, in order to obtain stored primary image data, to subject at least one of a portion of the primary image data generated by the image sensor and of the stored primary image data to a data reduction by means of a data reduction unit, in order to obtain secondary image data, and to output at least a portion of the secondary image data via the at least one interface unit to the external unit.
2. The camera system according to claim 1, wherein the camera system is designed to receive at least one control command containing control information from an external unit, wherein, in particular, the camera system is designed to influence an operation of the camera system depending on the control information.
3. The camera system according to claim 2, wherein the camera system is designed: to output, depending on the control information, at least a portion of the stored primary image data to the external unit, and/or to control, depending on the control information, an operation of the data reduction unit.
4. The camera system according to claim 1, wherein the data reduction unit is designed to carry out with regard to at least one of the generated primary image data and the stored primary image data, at least one of the following procedures: reducing a spatial resolution of the image data, reducing a spectral resolution of the image data, reducing an amplitude resolution of the image data, reducing an image rate, and providing the image data with an identification feature.
5. The camera system according to claim 1, wherein the control unit comprises at least one circular buffer.
6. An image processing system comprising: at least one camera system comprising an image sensor for generating primary image data and at least one interface unit for the data exchange with an external unit, wherein the camera system is designed to store at least a portion of the primary image data generated by the image sensor at least temporarily in a storage unit associated with the camera system, in order to obtain stored primary image data, to subject at least a portion of the primary image data generated by the image sensor and/or of the stored primary image data to a data reduction by means of a data reduction unit, in order to obtain secondary image data, and in order to output at least a portion of the secondary image data via the at least one interface unit to the external unit; at least one computer system, wherein the computer system is designed to receive the secondary image data output by the camera system via the at least one interface unit, to analyze at least a portion of the received secondary image data, and to output, depending on a result of the analysis, at least one control command containing control information to the camera system.
7. The image processing system according to claim 6, wherein the computer system is designed to cause, by means of the control command the camera system to output at least a portion of the stored primary image data as requested image data to the computer system, and to receive the requested image data from the camera system, wherein, in particular, the computer system analyzes the received requested image data.
8. A method for operating a camera system with an image sensor for generating primary image data and with an interface unit for the data exchange with an external unit, wherein the method comprises the following steps: at least temporary storing of at least a portion of the primary image data generated by the image sensor in a storage unit associated with the camera system, whereby stored primary image data is obtained, subjecting at least a portion of the primary image data generated by the image sensor and/or of the stored primary image data to a data reduction, whereby secondary image data is obtained, outputting at least a portion of the secondary image data via an interface unit to an external unit.
9. The method according to claim 8, wherein the camera system receives at least one control command containing control information from an external unit, wherein the camera system performs at least one of the following: depending on the control information, outputs at least a portion of the stored primary image data to the external unit, and, depending on the control information, controls an operation of the data reduction unit.
10. A method for operating an image processing system with at least one camera system comprising an image sensor for generating primary image data and at least one interface unit for the data exchange with an external unit, wherein the camera system is designed to store at least a portion of the primary image data generated by the image sensor at least temporarily in a storage unit associated with the camera system, in order to obtain stored primary image data, to subject at least a portion of the primary image data generated by the image sensor and/or of the stored primary image data to a data reduction by means of a data reduction unit, in order to obtain secondary image data, and in order to output at least a portion of the secondary image data via the at least one interface unit to the external unit, and with at least one computer system, comprising the steps of receiving in the computer system the secondary image data output by the camera system via the at least one interface unit, analyzing at least a portion of the received secondary image data, and outputting, depending on a result of the analysis, at least one control command containing control information to the camera system.
11. The method according to claim 10, wherein the computer system, by means of the control command, causes the camera system to output at least a portion of the stored primary image data as requested image data to the computer system and receives the requested image data from the camera system, wherein, in particular, the computer system analyzes the received requested image data.
Description
[0040] Below, exemplary embodiments of the invention are explained in reference to the drawing.
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050] The camera system 100 comprises an image sensor, not shown in
[0051] While the camera 100 generates primary image data with the image sensor thereof, for example, in the context of industrial image processing, process automation or the like, the computer system 200 performs tasks that are known per se such as, for example, image analysis or image processing. The type and extent of the image analysis and/or image processing can vary considerably depending on the field of application. Via the data connection 10, the computer system 200 is supplied with image data, which the camera system 100 detects by means of the image sensor thereof.
[0052]
[0053] The camera system 100a according to
[0054] In particular, primary image data BD1 generated by the image sensor 110 can be represented, for example, by a series of digital images. These digital images have, for example, a width which corresponds to a number of pixels or sensor elements of the image sensor 110 along a width coordinate, at a height which corresponds to the number of pixels or sensor elements of the image sensor 110 along a height coordinate orthogonal to the width coordinate.
[0055] Moreover, the primary image data BD1 generated by the image sensor 110 can be characterized by a spectral resolution, that is to say, for example, by the presence of one or more color channels, for example, RGB channels or other color channels. It is also conceivable that, in other embodiments, the image sensor 110 is designed as a black-and-white image sensor or works as a gray scale sensor.
[0056] Another characteristic of the image sensor 110 is an amplitude resolution which defines how many different brightness values are available for a digital representation of the individual pixels of the primary image data BD1 obtained by the image sensor 110. For example, in an 8-bit amplitude resolution, 2?8=256 different brightness values or intensity values can be represented; accordingly, in the case of a 16-bit resolution, 2?16=65536 different brightness or intensity values can be represented, and the like.
[0057] Another characteristic of the image sensor 110 is an image rate which, for example, can be indicated in images that can be delivered by the image sensor 110 per second.
[0058] In a manner known per se, the above-mentioned criteria or characteristics have an effect on a data quantity which is needed in order to represent the primary image data BD1 generated by the image sensor 110. For many applications, it can be undesirable if the image data BD1, that is to say the image data which represents the best image quality or the highest possible information content of the information delivered by the image sensor 110, is transmitted at the maximum possible image rate to the computer system 200 (
[0059] Therefore, it is proposed according to the invention that the camera system 100a is designed to store at least a portion of the primary image data BD1 generated by the image sensor 100 at least temporarily in a storage unit 130 associated with the camera system 100a. This is indicated in
[0060] In addition, the camera system 100a is designed according to the invention to subject at least a portion of the primary image data BD1 generated by the image sensor 110 to a data reduction, which is carried out by the data reduction unit indicated in
[0061] By the data reduction, the data reduction unit 140 generates secondary image data BD2 which corresponds to a lower data quantity or requires a smaller data volume for the representation thereof.
[0062] In another embodiment, it is also possible to provide that the data reduction unit 140 uses stored primary image data BD1 as input data for the data reduction.
[0063] According to the invention, the secondary image data BD2 obtained at the output of the data reduction unit 140 or at least a portion of the secondary image data BD2 is output via an interface unit 120 to an external unit 200, for example, to the computer system 200 of the image processing system 1000 (
[0064] For the case in which the computer system 200 requires, instead of or in addition to the data-reduced secondary image data BD2, the primary image BD1 generated originally by the image sensor 110, which as a rule has a greater information content than the data-reduced secondary image data BD2, said primary image data can be advantageously used, since, however, the primary image data BD1 in the form of the stored primary image data BD1 is kept available in the storage unit 130 at least temporarily, for example, for a certain calculated time period for the generation by the image sensor 110. For example, in such cases, the computer system 200 can request from the camera system 100a the transmission of certain stored primary image data BD1. Such a transmission can occur, for example, directly from the storage unit 130 via the data interface 120 and then the data connection 10 (
[0065] Alternatively, as indicated in
[0066] However, in general, the best possible available quality of the primary image data BD1 available from the storage unit 130 is preferably requested by the computer system 200. In this case, such a quantity of primary image data or stored primary image data can be removed from the storage unit 130 and transmitted, for example, via the data interface 120 and the computer system 200 directly, that is to say without any possible data reduction.
[0067] The data transmitted belatedly upon request of the computer system 200 to the computer system 200, that is to say, for example, the requested portions of the primary stored image data BD1, is marked in Figure two by the reference numeral BD2.
[0068] In a preferred embodiment, it is possible to provide that, at least temporarily, but also possibly continuously, the secondary image data BD2 formed by the data reduction unit 140 is buffered, in particular for a presettable buffer time. For this purpose, a buffer 141 associated with the data reduction unit 140 can be provided, which, in a preferred embodiment, can be designed similarly to the storage unit 130. Alternatively or additionally, it is possible to provide that the secondary image data BD2 to be buffered is buffered in temporary storage in a storage area of the storage unit 130 provided for this purpose. By means of the above-described optional buffering of the secondary image data BD2, it is ensured that said secondary image data can also be transmitted with temporally offset to the computer system 200 for the generation thereof by the data reduction unit 140, which is advantageous, for example, if the data connection 10 (
[0069] In addition to the components already described above,
[0070]
[0071] In the embodiment 100b shown in
[0072] The burden on the computer system 200 receiving the secondary image data BD2 is also considerably lightened, since a smaller data quantity per time unit has to be processed. Thus, the computer system 200 can analyze, for example, a scene represented by the secondary image data BD2, which the camera system 100b has recorded, and, for example, determine the position of an object to be examined in further detail or of a code, for example, of a one- or two-dimensional barcode.
[0073] To the extent that no object of interest is found by the computer system 200 in a certain digital image of the secondary image data BD2, the computer system 200 waits, for example, for the next digital image of the secondary image data BD2 received thereby and performs a new evaluation in this regard.
[0074] To the extent that the computer system 200 finds an image area of interest for the present image processing task in a digital image of the secondary image data BD2, the computer system 200 can send a control command sb to the camera system 100b in order to cause the camera system 100b to output at least a portion of the stored primary image data BD1 (
[0075] As can be seen in
[0076] By means of the above-described sequence according to a preferred embodiment of the present invention it is advantageously ensured that, if needed, particularly if areas of interest that need to be analyzed in further detail are identified in the secondary image data BD2, the computer system 200 can request, from the camera system 100b, usually high-resolution primary image data BD1 comprising a maximum information content or stored images b2 which can then be transmitted to the computer system 200.
[0077] In another preferred embodiment, a control command sb can contain, for example, control information which denotes a certain digital image b 1 of the primary image data BD1 stored in the storage unit 130, so that it can be read out in a targeted manner from the circular buffer 132 or the storage unit 130 and transmitted to the computer system 200.
[0078] Furthermore, according to other embodiments, such a control command can contain additional control information which characterizes, for example, an image area of interest (AOI, area of interest), typically a rectangular image area, of a digital image. In this case, it is then not necessary, for example, to transmit the complete stored digital image b2, instead it is sufficient if the section of interest of the stored digital image b2 is transmitted to the computer system 200, whereby the band width requirements imposed on the data connection 10 (
[0079] Particularly advantageously, in an embodiment, using an image number and optionally using coordinate values which characterize at least one image area, the camera system 100b can localize the stored primary image data of interest to the computer system 200 in the storage unit 130 or in the circular buffer 132 thereof and then transmit it to the computer system 200.
[0080] In another advantageous embodiment, it is provided that a special identification can be associated with the image data, for example, integrated in the image data and/or integrated in associated metadata, wherein this special identification enables the receiving computer system 200, for example, to recognize that a subsequently requested image or subsequently requested stored primary image data BD1 is involved.
[0081] Furthermore, such a special identification can be designed so that it thus enables an association showing to which digital image or to which portion of the previously received secondary image data BD2 the special identification or the image identified thereby belongs. For example, in other embodiments, at least one time stamp can be used as identification in the above-mentioned sense, in order to enable a subsequent targeted requesting of stored primary image data of interest to the computer system 200. For example, it is conceivable that the secondary image data BD2, for example, individual images thereof, particularly preferably all the images thereof, is provided in each case with a time stamp formed, for example, by the camera system 100. After the reception of the corresponding secondary image data, based on a time stamp in an image of the received secondary image data BD2, the computer system 200 can thereby request in a targeted manner corresponding primary image data from the camera system 100, for example, by transmission of a value characterizing the time stamp.
[0082] In other embodiments, it is possible that the secondary image data is not provided directly with the time stamp as described above (for example, as associated data value or integrated in associated metadata of corresponding images). Instead, the time stamp can also be transmitted through a separate communication channel from the camera system 100 to the computer system 200. The separate communication channel can be implemented, for example, using a virtual connection via the interfaces 10, 120 already described above, or also by a separate physical data connection (wired or wireless, not shown) between the systems 100, 200.
[0083] In another embodiment, it is also conceivable that the time stamps are generated by a separate timer unit (not shown) and are provided to both systems 100, 200, for example, via at least one of the above-mentioned interfaces or a separate wired or wireless data connection.
[0084] The method according to the invention advantageously allows a considerable saving of transmission bandwidth in the area of the data connection 10 (
[0085] In
[0086] The image analysis functionality 202a is used, for example, to analyze the secondary image data BD2 received in the computer system 200, in particular with a view to areas of interest that possibly have to be analyzed further with inclusion of primary image data. Furthermore, this functionality 202a can be designed to transmit one or more of the above-mentioned control commands sb to the camera system 100b.
[0087] The image processing functionality 202b can be provided, for example, to further analyze a digital image which is requested by the computer system 200 and which characterizes a portion of the primary image data BD1 or of the stored primary image data BD1, bar for example according to a presettable image processing task such as an object recognition, a decoding of codes contained in the image data and the like.
[0088]
[0089] Subsequently, in step 320, at least a portion of the primary image data BD1 generated by the image sensor 110 is subjected to a data reduction by the data reduction unit 140 (
[0090] In some embodiments it is also conceivable not to output at least temporarily secondary image data BD2 via the interface unit 120 to the external unit 200. In a preferred embodiment, this can be controlled by corresponding control commands from the external unit 200 to the camera system.
[0091] Alternatively to the procedure described above in reference to step 320, at least portions of the primary image data BD stored in the storage unit 130 can also be subjected to a data reduction, in order to obtain the secondary image data BD2.
[0092] In other embodiments, it is furthermore conceivable to provide several storage devices in the storage unit 130, in order to fulfill the different storage purposes, for example, a first storage device for storing the primary image data for later retrieval by the external unit 200, and a second storage device for the at least temporary storage of the primary image data BD1 before the data reduction thereof by the data reduction unit 140.
[0093]
[0094]
[0095]
[0096] The optional step 430 represents the reception of the requested image data b2 (
[0097] In a particularly preferred embodiment of the method according to the invention, the data reduction unit 140 (
[0098] Reducing a local resolution of the image data (in at least one dimension or preferably in two dimensions),
[0099] reducing a spectral resolution of the image data (for example, by reducing the number of color channels),
[0100] reducing an amplitude resolution of the image data (for example, by an adaptation of the quantization steps used for the digital representation of the amplitude values),
[0101] reducing an image rate,
[0102] providing the image data with an identification feature (for example, image number, time stamp).
[0103] Combinations of two or more of the above-mentioned procedures, in particular also of the procedures for data reduction, are also conceivable. In other embodiments, it is moreover conceivable that the computer system 200 parametrizes the desired type of data reduction by the data reduction unit 140, that is to say, by corresponding control commands sb or associated control information, it presets, for the camera system 100b, the manner in which or the parameters with which or the techniques by which the data reduction for presettable primary image data BD1 is to be carried out.
[0104]
[0105] Block 144 symbolizes a data reduction as can be used, for example, by the above-described data reduction unit 140. The data reduction 144 leads to a data-reduced secondary digital image b1 which is supplied via the data connection 10 (
[0106]
[0107]
[0108] The present invention advantageously enables a reduction of the required transmission bandwidth between the camera system 100 (
[0109] In a particularly preferable variant, advantageously relatively small thumbnails (which thus have, for example, a comparatively low local resolution or generally a lower data quantity than the primary image data) can be transmitted as secondary image data BD2 to the computer system 200, and software present in the computer system 200, for example, an image processing system or an image analysis system, optionally determines areas of interest of the thumbnails and, if needed, requests the areas of interest in the form of at least temporarily buffered primary image data from the camera system.
[0110] In another advantageous embodiment, thumbnails which have no content of interest can be completely ignored. In other words, these thumbnails can be rejected by the computer system 200, and, in particular, no primary image data corresponding to these thumbnails to be rejected needs to be requested in the camera system by the computer system.
[0111] Subsequently, the image data of interest, which, for example, also represents only sections of image data stored with high resolution in the camera system, can advantageously be transmitted to the computer system and supplied there to an analysis.
[0112] In particular, for the transmission of the secondary image data to the computer system, the principle according to the invention enables relatively high image repetition rates. Advantageously, for the implementation of the data interfaces 120, 122, standardized data interfaces can also be used, which should make possible a largely lower maximum data rate than would be necessary for direct transmission of the primary image data BD1.
[0113] Particularly advantageously, in preferred embodiments, the computer system 200 can determine which portions of the image data should be transmitted in a comparatively greater or maximum possible quality from the camera system 100 to the computer system 200.
[0114] In other advantageous embodiments, the computer system 200 can be programmed by application developers with, for example, algorithms for image processing that are known per se. In particular, for the implementation of an image processing in the computer system 200, no specialized knowledge is required concerning an internal design of the camera system 100, 100a, 100b or the components thereof.
[0115] Also advantageously, the use of the principle according to the invention allows short development times for image processing systems and a high flexibility.
[0116] In some embodiments, for the data interface ten, for example, a GB (Gigabit) Ethernet interface can be used, which provides a potential transmission rate of approximately 125 MB (megabytes) per second. Alternatively or additionally, one or more of the following data interface types can also be used: 10 GB Ethernet, USB3, CameraLink HS, CoaxPress.