SYSTEM AND METHOD FOR CONTROLLING AUTOMATIC INSPECTION OF ARTICLES
20230016639 · 2023-01-19
Assignee
Inventors
Cpc classification
Y02P90/02
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
Y02P90/80
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
G05B2219/32186
PHYSICS
International classification
Abstract
Techniques for inspection of articles having multiple features of one or more types are disclosed. Input data indicative of one or more selected features of interest is used for inspection by a given inspection system characterized by associated imaging configuration data. The input data is analyzed to extract information regarding one or more inspection tasks, and an inspection plan data is generated, to be used as a recipe data for operation of the given inspection system to provide measured data in accordance with the one or more inspection tasks. Selected inspection mode data corresponding to the inspection task data may be retrieved from a database system and utilized to generate the inspection plan data.
Claims
1. A control system for use in managing inspection of articles having multiple features of one or more types, the control system comprising: a data input utility for receiving input data indicative of one or more selected features of interest to be inspected by a given inspection system characterized by associated imaging configuration data; a data processor configured and operable to analyze the input data to extract information regarding one or more inspection tasks and generate inspection plan data to be used as a recipe data for operation of said given inspection system to provide measured data in accordance with said one or more inspection tasks, said data processor comprises: an identifier utility configured and operable to utilize the input data to define inspection task data indicative of said one or more inspection tasks, the inspection task data comprising data indicative of the input data, data indicative of said one or more selected features, and a measurement type corresponding to said one or more inspection tasks; an analyzer utility configured and operable to analyze the inspection task data and determine the recipe data by generating a selected group of attributes, which is selected from a predetermined attributes' set comprising geometry related attributes and material related attributes, and corresponds to the inspection task data; and a planning module configured and operable for analyzing the inspection task and selected inspection mode data corresponding to said selected group of attributes, and generating inspection plan data to be performed by said given inspection system with regard to said one or more selected features of interest.
2. The control system according to claim 1, wherein said data processor is configured and operable for communication with a database system to request and receive, from said database system, selected inspection mode data corresponding to the inspection task data and assigned to a group of attributes including at least one of geometrical and material-relating attributes in association with one or more imaging configurations for inspection of features corresponding to said attributes, and utilize the selected inspection mode data to generate the inspection plan data.
3-5. (canceled)
6. The control system according to claim 1, wherein said planning module is configured and operable to carry out the following: generate request data to a database system comprising data indicative of said selected group of attributes to request said selected inspection mode data assigned to said selected group of attributes in association with the given inspection system; and analyzing the selected inspection mode data, based on the inspection task data, and generate said inspection plan data.
7. The control system according to claim 1, wherein said inspection mode data comprises data indicative of one or more of the following conditions with respect to a region of interest to be imaged in one or more inspection sessions performed by said given inspection system: selected radiation patterns to be projected onto the region of interest; illumination intensity; illumination spectral data; orientation of a scan path with respect to the region of interest; scan density.
8. The control system according to claim 7, wherein the inspection plan data comprises data indicative of at least one of the following: a sequence of inspection modes during the one or more inspection sessions; optimized configuration of one or more selected radiation patterns; a relative orientation of at least one radiating channel and at least one detection channel during the one or more inspection sessions; an alignment of radiating and detection channels with the region of interest; a number of the inspection sessions; a data readout mode for collecting detection data in association with the region of interest.
9. The control system according to claim 1, wherein said predetermined attributes' set comprises a plurality of basic geometrical shapes and a plurality of radiation response properties of various surfaces.
10-12. (canceled)
13. The control system according to claim 1, further comprising at least one of the following: a monitor configured and operable for receiving and analyzing measured data obtained by the inspection system in one or more inspection sessions performed utilizing said inspection plan data and being indicative of one or more parameters associated with said one or more selected features, and generating output data indicative of inspection results; an operational controller configured and operable for controlling operation of the given inspection system to perform one or more inspection sessions according to said inspection plan data; a storage utility for storing said database; a communication module configured and operable for performing data communication of said data processor and said database system located in a remote storage system; an alignment module of said operational controller configured and operable for monitoring a preliminary alignment condition between the article being inspected and input location data about one or more regions of interest on said article associated with said one or more selected features of interest.
14. The control system according to claim 13, wherein said data indicative of the inspection results comprise one or more of the following: an updated inspection task data; update for optimizing contents of the database containing predetermined inspection mode data pieces assigned to corresponding groups of attributes in association with the inspection systems.
15. The control system according to claim 13, wherein said monitor is configured and operable to communicate with a remote central system for communicating said output data indicative of the inspection results to said central system, thereby enabling to use said inspection results data for at least one of the following: updating inspection task data; optimizing contents of the database containing predetermined inspection mode data pieces assigned to corresponding groups of attributes in association with inspection systems
16. The control system according to claim 1, wherein the input data comprises one or more of the following: CAD model data indicative of said one or more features of interest; 3D scan of at least a part of the article and corresponding metadata indicative of one or more measurement types to be performed; and location data about one or more regions of interest on said article associated with said one or more selected features of interest.
17. The control system according to claim 16, wherein the location data comprises data about at least one of the following: relative position of the features of interest with respect to an alignment location; and relative orientation of the features of interest with respect to an alignment location.
18. The control system according to claim 1, wherein said data indicative of the inspection task comprises one or more of the following: (i) per each of said one or more selected features, verification of presence of said selected feature in one or more predetermined regions of interest; (ii) per each of said one or more selected features, measurement of one or more parameters of said feature; (iii) per each pair of features from said one or more selected features, measurement of at least one distance between them and their relative orientation, wherein said features of the pair are located in the same or different regions of interest; (iv) determining whether a surface roughness of a surface portion within a region of interest satisfies a predetermined condition, wherein said surface portion includes one of the following: a surface of the selected feature; or a surface of the article between the selected features; (v) determining a relation between one or more parameters of said one or more selected features of interest and corresponding input data relating to said one or more selected features, and generating data indicative of said relation.
19. The control system according to claim 9, wherein said radiation response properties related attributes comprise one or more of the following: color, hyperspectral response, reflectivity, transparency and diffusivity.
20. (canceled)
21. The control system according to claim 1, wherein the imaging configuration data comprises data indicative of one or more of the following: a number of radiating channels for projecting one or more patterns onto a region of interest, a number of detection channels for collecting image data from at least a portion of an irradiated region of interest, locations of the radiating and detection channels with respect to an inspection plane, relative orientations between the radiating and detection channels, and properties of a radiation source and detector of the inspection system.
22. (canceled)
23. An inspection system for inspecting articles having multiple features of one or more types, the inspection system comprising: an imaging system comprising: one or more illuminators defining one or more radiating channels for projecting patterns on one or more regions of interest being irradiated; one or more detectors defining one or more detection channels for detecting radiation response of at least a portion of each of said one or more regions of interest being irradiated and generating corresponding image data; said imaging system being configured and operable for executing inspection according to various inspection plans using various relative orientations between the radiating and detection channels and various properties of radiation and detection; and the control system of claim 1.
24. The inspection system according to claim 23, wherein said imaging system is an optical imaging system configured to define at least one pair of illumination-detection channels formed by at least one illuminator and at least one detector.
25. The inspection system according to claim 24, wherein the at least one illuminator comprises at least one 2D projector having at least one fast axis for projecting light patterns.
26. (canceled)
27. The inspection system according to claim 25, wherein the 2D projector has one of the following configurations: (i) comprises a resonant 2D MEMS scanning mirror said fast axis of the dynamic scan mode being one of mechanical axes of the MEMS scanning mirror; (ii) comprises a raster MEMS scanning mirror, said fast axis of the dynamic scan mode being a resonant axis of the MEMS; (iii) comprises a 2D MEMS structure, said fast axis of the dynamic scan mode being an axis corresponding to a sequence of MEMS positions providing a light pattern in the form of a substantially straight line; (iv) said 2D projector is configured and operable to perform said projection of the light patterns in a dynamic scan mode having at least one fast axis.
28. The inspection system according to claim 24, wherein the at least one detector comprises a camera with multiple dynamically repositioned regions of interest (MROIs).
29. The inspection system according to claim 24, wherein said optical imaging system comprises multiple illuminator-detector pairs defined by at least one of the following configurations: (a) said multiple illuminator-detector pairs sharing at least one common unit being illuminator or detector thereby defining multiple pairs of the illumination-detection channels; (b) said multiple illuminator-detector pairs comprise multiple detector units associated with a common 2D illumination projector unit (c) said multiple illuminator-detector pairs comprise multiple 2D illumination projectors associated with a common detector unit (d) each illuminator-detector pair defines a base line vector, base line vectors of the illumination-detector pairs having the common unit defining a predetermined orientation of the base line vectors with respect to one another; and (e) the base line vectors of the illumination-detector pairs have the common unit satisfying a condition of substantial perpendicularity of the base line vectors to one another.
30-34. (canceled)
35. An optical inspection system for inspecting articles having multiple features of one or more types, the optical inspection system comprising an imaging system comprising: one or more illuminators defining one or more illumination channels for projecting light patterns on one or more regions of interest being irradiated, and one or more detectors defining one or more detection channels for detecting response of at least a portion of each of said one or more regions of interest to said illumination and generating corresponding image data, thereby defining at least one pair of illumination-detection channels formed by at least one illuminator and at least one detector, wherein said at least one illuminator comprises a 2D illumination projector of the light patterns, the system being characterized by at least one of the following: (i) the 2D projector is configured and operable to perform said projection in a dynamic scan mode having at least one fast axis; and (ii) the imaging system comprises multiple pairs of the illumination-detection channels formed by multiple illuminator-detector pairs sharing at least one common unit being the 2D illumination or detector, wherein base line vectors defined by the illumination-detector pairs have the common unit defining a predetermined orientation of the base line vectors with respect to one another a control system providing inspection plan data to be executed by the imaging system in one or more inspection sessions to measure one or more parameters of one or more features of interest, said control system comprising: a data input utility for receiving input data indicative of one or more selected features of interest to be inspected by a given inspection system characterized by associated imaging configuration data; and a data processor configured and operable to analyze the input data to extract information regarding one or more inspection tasks and generate inspection plan data to be used as a recipe data for operation the inspection system to provide measured data in accordance with said one or more inspection tasks.
36. The inspection system according to claim 35, wherein the base line vectors of the illumination-detector pairs have the common unit satisfying a condition of substantial perpendicularity of the base line vectors to one another.
37. The inspection system according to claim 35, wherein the 2D projector has one of the following configurations: (i) comprises a resonant 2D MEMS scanning mirror, said fast axis of the dynamic scan mode being one of mechanical axes of the MEMS scanning mirror; (ii) comprises a raster MEMS scanning mirror, said fast axis of the dynamic scan mode being a resonant axis of the MEMS; and (iii) comprises a 2D MEMS structure, said fast axis of the dynamic scan mode being an axis corresponding to a sequence of MEMS positions providing a light pattern in the form of a substantially straight line.
38. The optical inspection system according to claim 35, wherein said one or more illuminators comprises at least one laser source.
39. The optical inspection system according to claim 35, wherein the imaging system comprises at least one detector which is associated with at least one of the following: at least first and second 2D illumination projectors operable to perform said projection in the dynamic scan mode having at least one fast axis, wherein a scan direction of at least one first projector is 90 degrees rotated with respect to a scan directions of at least one second projector, such that the fast scanning axis of the first projector is perpendicular to the fast scanning axis of the second projector; and an array of the 2D illumination projectors operable to perform said projection in the dynamic scan mode having at least one fast axis, wherein said 2D illumination projectors and the camera are oriented such that the fast axis of each projector is substantially perpendicular to a base line vector defined by said projector and the detector.
40. (canceled)
41. The inspection system according to claim 35, wherein the at least one detector comprises a camera with multiple dynamically repositioned regions of interest (MROIs).
42. (canceled)
43. The inspection system according to claim 35, further comprising an operational controller configured and operable for controlling execution of one or more inspection sessions according to said inspection plan data.
44. The inspection system according to claim 43, wherein said operational controller comprises an alignment module configured and operable for monitoring a preliminary alignment condition between the article being inspected and input location data about one or more regions of interest on said article associated with said one or more selected features of interest.
45-47. (canceled)
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0074] In order to better understand the subject matter that is disclosed herein and to exemplify how it may be carried out in practice, embodiments will now be described, by way of non-limiting examples only, with reference to the accompanying drawings, in which:
[0075]
[0076]
[0077]
[0078]
[0079]
[0080]
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095]
DETAILED DESCRIPTION OF EMBODIMENTS
[0096] Referring to
[0097] As also indicated above, although the technique of the present invention is exemplified and described herein below in relation to optical inspection techniques, the principles of the invention are not limited to this specific application and can be used with any known radiation excitation based inspection (e.g. LiDAR, MRI, CT, X-rays based inspection).
[0098] In the present not limiting example illustrated in
[0099] An article to be inspected is of the kind having multiple features/elements of the same or different types. It should be noted that the article may or may not be a functional device, but may be a substrate carrying one or more functional devices/elements (being active or passive elements), each constituting a feature or a region of interest with multiple features. Some specific but not limiting examples of the articles will be described further below with reference to
[0100] The control system 10 is further associated with (e.g. includes or is connectable to) a storage system 30 containing and managing a database 32, the construction of which will be described further below with reference to
[0101] The control system 10 is generally a computer system having inter alia such main functional utilities (software and/or hardware) as data input and output utilities 14, 16, memory 18 and a data processor 20. The data input utility 14 is configured and operable to be responsive to input (which may be user's input and/or electronic device input) to provide corresponding input data D.sub.in to the data processor 20. The data processor 20 is configured and operable to utilizes the input data D.sub.in to determine inspection plan data IPD.sub.ij with respect to n selected (n≥1) features feature(s) of interest, e.g. j-th feature(s), assigned for inspection by the given i-th optical inspection system OIS.sub.i. The data processor 20 includes an identifier 20A, an analyzer 20B, and a planning module 20C.
[0102] The input data D.sub.in may include data indicative of a CAD model containing object data and required measurements; and/or 3D scan of an object together with corresponding metadata identifying which measurements are to be performed; and/or location data about the region/elements of interest. This will be described and exemplified more specifically further below.
[0103] The identifier utility/module 20A is responsive to the input data D.sub.in and configured and operable to extract information regarding inspection task(s) and define corresponding inspection task data ITD.sub.ij with regard feature(s) Fj to be inspected by the respective optical inspection system OIS.sub.i. As will be described more specifically further below, the input are data D.sub.in provided by user and/or by image or CAD data may include various reference mark(s) which allow identification of a parameter(s)/condition(s) to be determined and provide some prior knowledge (e.g. location information) about the feature, allowing to properly define the inspection task data.
[0104] As a result, the inspection task data ITD.sub.ij actually contains information about the input data itself, D.sub.in, region(s) of interest and feature(s) therein on which measurement/inspection session(s) is/to be performed, and a required measurement/inspection type.
[0105] In one possible example, aimed at inspecting one element, the inspection task data ITD.sub.j may include: (i) data indicative of the input data D.sub.in including a CAD model of a specific article being a printed circuit board (PCB), (ii) data indicative of an object/feature Fj of interest being a resistor R17 on the PCB, and (iii) a required measurement being a length of the resistor R17. In this example, the identifier 20A defines the inspection task data based on the analysis of the CAD model.
[0106] In another possible example, the inspection task data ITD.sub.j may include: (i) data indicative of the input data D.sub.in including a point cloud scan of a PCB, (ii) data indicative of an object/feature Fj of interest being associated with two edges A,B on the cloud, and (iii) a required measurement being a distance between the edges A,B. In this non-limiting example the identifier 20A analyzes the input data and provides a user with a corresponding GUI enabling the user to select two points on the edges and also to indicate that these are indeed the edges, and completes the inspection task data ITD.sub.j based on the user input.
[0107] The analyzer utility 20B is configured and operable to analyze the inspection task data ITD.sub.ij to extract/identify a selected group of attributes GA.sub.j, from a predetermined attributes' set PAS, corresponding to the inspection task data. The predetermined attributes' set PAS comprises geometry related attributes (physical parameters), and preferably also includes material related attributes defining radiation-response related attributes/parameters, e.g. optical properties related attributes. In the description below, such radiation response related attributes are at times referred to also as “optical properties related” and “material related”. More specifically, the predetermined attributes' set PAS includes M attributes (A.sub.1, . . . A.sub.m) comprising K geometry related attributes (A.sub.1 . . . A.sub.k) and L optical properties related attributes (A.sub.k+1 . . . A.sub.m). Examples and details of the geometry related and optical properties related attributes will be described further below.
[0108] The analyzer utility 20B translates/converts feature-related and measurement type related data embedded in the inspection task data ITD.sub.ij, into a selected group of attributes GA.sub.j including one or more of at least the geometry related attributes. Thus, the selected group of attributes GA.sub.j is a breakdown of the inspection task data ITD.sub.ij in relation to the feature(s) of interest and the measurement type, where the feature(s) of interest are presented by geometrical attribute(s) and possibly also optical attribute(s) (depending on the measurement type data).
[0109] In one possible example, the selected group of attributes GA.sub.j may include the following: (a) geometrical attributes corresponding to a 3D box with a flat surface (e.g. resistor R17 of the PCB) including the box location, and its size and orientation; (b) optical attributes corresponding to a smooth and white element/surface; and (c) the required measurement type, which is a length of the rectangle. In this example, the analyzer 20B utilizes the respective data from the CAD and the predetermined attributes' set PAS.
[0110] In another possible example, the selected group of attributes GA.sub.j may include the following: (a) geometrical attributes including a list of two walls (e.g. edges A,B in the point cloud scan of PCB); (b) indication N/A (not available or no answer) with regard to optical attributes, since no user input in this regard has been provided during a previous step (the user could have used the GUI provided by the identifier 20A to input the optical properties related data); and (c) the required measurement type being a distance between the edges.
[0111] The planning module 20C is configured and operable for analyzing the inspection task data ITD.sub.ij and predetermined inspection-mode data IMD.sub.ij assigned to the selected group of attributes GA.sub.j, and determining inspection plan data IPD.sub.ij to be performed by the i-th optical inspection system OIS.sub.i, characterized by optical configuration data OCD.sub.i, to serve the inspection task(s) with regard to the selected one or more features F.sub.j. In other words, the planning module 20C utilizes the data indicative of the selected group of attributes GA.sub.j and the optical data OCD.sub.i to create inspection plan data IPD.sub.ij.
[0112] It should be understood that the optical configuration data OCD.sub.i includes data indicative of the structure and model of the optical system. The inspection plan data IPD.sub.ij includes instruction to the optical system as to how to perform inspection session(s) to provide measured data in accordance with the inspection task(s).
[0113] To this end, the data processor 20 (the planning module 20C) communicates with the storage system 30 managing the database 32 to send request data RD.sub.ij comprising data indicative of the selected group of attributes GA.sub.j and data indicative of the optical configuration data OCD.sub.i and receive from the storage system 30 corresponding inspection-mode data IMD.sub.ij that matches (is assigned to) the selected group of attributes GA.sub.j in association with the optical configuration data OCD.sub.i. It should be understood that the selection of the inspection-mode data IMD.sub.ij might require data indicative of the optical data configuration OCD.sub.i of the respective inspection system OIS.sub.i. Therefore, the request data RD.sub.ij embeds either such optical configuration data OCD.sub.i or identification code/data ID.sub.i of the respective inspection system.
[0114] It should be noted that the entire inspection goal may include more than one tasks with respect to the same feature(s). In this case, the request data is configured accordingly, i.e. includes data indicative of the corresponding selected groups of attributes, which are in turn based on the inspection tasks and the measurement types. The storage system 30 thus provides corresponding more than one inspection mode data pieces IMDs. Once all such data pieces IMDs are received, the planning module 20C analyzes and optimizes them all together (“compile” them) to create the optimal inspection plan data.
[0115] Considering the above two examples, the combined inspection mode data may include a requirement for illumination with a light pattern of four (4) straight white lines and determination of locations of the line breaks. In this case, the optimal inspection plan data may include instructions to split this inspection mode into a sequence of four (4) frames, each containing a single line.
[0116] Referring to
[0117] More specifically, the inspection mode data piece stored in the database may include data indicative of one or more illumination conditions (e.g. light pattern(s) assigned to measurements/inspection of various primitive shapes/geometries).
[0118] It should be understood that the storage system 30 managing the database 32 may be part of the control system 10. Alternatively, as exemplified in
[0119] As also schematically shown in
[0120] The planning module 20C of the control system 10 is responsive to the inspection mode data IMD.sub.ij received from the storage system 30 in reply to the request data RD.sub.ij generated at the control system 10, and analyses the inspection mode data taking into account the inspection task data and the initial article-related data, to generate inspection plan data IPD.sub.ij for the inspection system OIS.sub.i.
[0121] As will be described more specifically further below, the optical inspection system OIS.sub.i includes one or more illumination units (one or more light sources and possibly associated light directing optics) defining one or more illumination channels and being configured for projecting one or more light patterns onto at least a portion of a region of interest; and one or more light detectors defining one or more light detection channels.
[0122] The inspection mode data comprises one or more illumination conditions. In a simple example, this may include light patterns for use in at least one illumination channel being applied to one or more regions of interest.
[0123] The inspection plan data, which is to be used as recipe data for the inspection system, includes one or more of the following parameters/conditions of inspection session(s) performed with said one or more light patterns: one or more sequences of light patterns for use in at least one illumination channel being applied to one or more regions of interest; light intensity; scan path orientation; scan density; a relative orientation of at least one illumination channel and at least one detection channel; a data readout mode for collecting detection data in association with a region of interest; scan mode parameters. The configuration and operational principles of the optical inspection system, as well as example of the inspection plans, will be described more specifically further below.
[0124] In this connection, it should be noted that light pattern sequence of the inspection plan data is not necessarily a sequence of different light patterns which are to be sequentially applied for the purposes of the same inspection task. More specifically, the inspection plan may include a light pattern sequence, where for a given inspection task the same light pattern is used for a local “scan” of a region of interest (e.g. a part of element/feature); or for the combined performance of different inspection tasks (based on a decision of the planning module) such sequence may include different light patterns
[0125] Turning back to
[0126] Further, the control system 10 may be associated with (e.g. may include as a part of its data processor 20) a monitor 26 which receives and analyzes measured data MD.sub.i from the optical inspection system OIS.sub.i, and generates output data indicative of inspection results IR. It should be understood that, alternatively, such monitor 26 may be part of the optical inspection system OIS.sub.i; or functional utilities of the monitor 26 may be distributed between the control system 10 and the optical inspection system OIS.sub.i, as the case may be.
[0127] Generally, the inspection results IR may include various types of data. For example, the monitor 26 is configured and operable for receiving and analyzing measured data MD and generating output data indicative of one or more parameters/conditions of the one or more selected features.
[0128] Alternatively or additionally, analysis of the measured data may be used for selectively generating updated inspection task data, in which case the control system 10 operates as described above to provide one or more updated inspection plans. For example, the updating of the inspection task data may be associated with a need to verify the input data (including article- and feature-related data, e.g. CAD information and/or article specification data).
[0129] Further, the monitor 26 may be configured and operable to utilize data indicative of the inspection results to generate guiding/navigation data for one or more robotic procedures to be performed on the article.
[0130] According to some other examples, the analysis of the measured data/inspection results may alternatively or additionally be used together with the analysis of the data indicative of the corresponding one or more inspection tasks and data indicative of one or more of the corresponding inspection plans to optimize/update data in the database 32.
[0131] It should be noted that, generally, selected feature(s) to be inspected, as well as inspection task(s), may be related to various parameters/conditions of the article's structure. For example, the selected feature(s) may be associated with the active element(s) and/or their arrangement on the support substrate. More specifically, the inspection task may include verification of presence of one or more selected feature (e.g. active element) in predetermined region(s) of interest. Alternatively or additionally, the inspection task may include measurement/verification of dimension(s) and/or surface relief (e.g. surface flatness/roughness) of a selected surface portion of the article. It should be understood that the surface portion may be a surface of the selected element; or a surface of the article between the selected elements. In yet another example, the inspection task may include measurement/verification of distance(s) between the selected elements and/or relative orientation of the elements.
[0132] It should also be noted that the input data, enabling to define the inspection task data with regard to selected feature(s) of the article, utilizes (is based on) some initial article-related data, which may be image data (e.g. 2D or 3D map) and/or CAD data. The inspection task may be aimed at the verification of such input data with regard to any parameter(s)/condition(s) relating to the features and their arrangement on the article. Hence, the inspection results may be aimed at the verification of the initial article-related data.
[0133] Thus, monitor 26 analyses the measured data MD to determine a relation between one or more parameters of said one or more selected features of interest measured/inspected by the optical inspection system and the corresponding initial article-related data (e.g. CAD data) relating to said one or more selected features, and generates the corresponding inspection results IR. This enables, if needed, correction/update of the initial article-related data.
[0134] Reference is now made to
[0135]
[0136]
[0137] As described above, the initial article-related data (input data) required to define the inspection task data may be in the form of image data (e.g. 2D or 3D map) and/or CAD data. For example,
[0138] For example,
[0139] Position of each of these features F.sub.1 and F.sub.2 in the same article may be predefined and fixed within certain mechanical tolerances, which might vary from article to article due to the manufacturing (assembling) process. Also, the elements/features of the same type (e.g. all connectors of the same type) should be of the same geometry-related parameters (dimensions and shape) with predefined/allowed tolerances, and be made of the same material (i.e. have the same optical properties). Thus, the inspection tasks may be aimed at monitoring the geometry-related parameters (e.g. actual tolerances to identify whether they satisfy a predetermined condition for the purposes of assembling process control), and the material-relating conditions.
[0140]
[0141] Reference is made to
[0142] Input data including initial article-related data (prior knowledge) is provided (step 102), being image or CAD data, comprising respective indication about feature(s) of interest F.sub.j. This input data is used to extract/define inspection task data ITD.sub.ji associated with the optical inspection system data OCD.sub.i (step 104). For example, the inspection task data includes: verification of presence of a specific feature/element (e.g. bolt) in a region of interest, defining an inspection task; measurement of the element's dimensions (said bolt's dimensions), which might form a separate inspection task; measurement of a distance between similar bolts that are connected to the article within the same region of interest forming an inspection task relating to the same region of interest; and measurement of a distance between two similar bolts connected to different regions of interest, defining an inspection task relating to the different regions of interest.
[0143] Generally, the inspection task data ITD.sub.ij defines at least one inspection task and at least one feature of interest on at least one region of interest. It should be noted that the inspection task may be associated with (related to) multiple features of interest (for example, measuring a distance between two features of interest); or more than one inspection tasks may be associated with respect to the same feature of interest (for example, measurement of a diameter of a hole and inspection/verification of the shape of the hole). In case the inspection task relates to multiple features of interest belonging to different regions of interest (inspection parts), an alignment procedure is to be performed with regard to each of the different regions of interest separately, and the inspection task data takes into account relative region-to-region displacement.
[0144] In some embodiments of the present invention, the inspection task data is defined by a user from a CAD model. The CAD model of the full article (or at least full region of interest) is loaded into the control system 10 and is analyzed, together with additional user's input, by the identifier module 20A. For example, the identifier module 20A is configured as an API providing the user with a predefined list of various relevant task procedures allowing the user to select features of interest on the CAD model and select one or more tasks from said predefined list defining the type(s) of measured data that the optical inspection system needs to provide.
[0145] This is exemplified in
[0146] Also, in this example, features of interest F.sub.1 and F.sub.2 are constituted by parts/fragments of an element 80 on the article. It should be understood that in some other examples, the feature may be constituted by the entire element.
[0147] In some other embodiments of the present invention, the inspection task is defined by the user from a reference image (i.e. 2D or 3D map/image data). Such reference image may be previously prepared and stored to be used by the user to define the inspection task(s); or may be acquired in preliminary inspection stage. For example, the user can select reference points on the actual region of interest in the article being illuminated by the optical inspection system, and then select one or more tasks from the predefined list of tasks (as described above) defining the type of measured data that the optical inspection system needs to provide.
[0148] As indicated above, the selection of the reference points on the actual region of interest may be implemented in several ways. One of such possible ways utilizes an initial (preliminary) 3D imaging of the actual region of interest (e.g. using a single-exposure imaging with structured light illumination) and the user's selection of the reference point(s) on that 3D image. For example,
[0149] In some other examples, the initial 3D image/map of the actual region of interest is obtained by the inspection system itself performing a preliminary imaging session to create or update the initial article-related data, and this 3D image/map is then used as at least a part of the input data D.sub.in for the determination of the inspection task data ITD. For example, such a preliminary inspection session of the course-stage inspection may utilize a scan-mode imaging, (e.g. scan of the entire field-of-view of a camera with a single-line pattern).
[0150] Irrespective of the imaging mode of obtaining the 3D map, the user can select reference points on that 3D map, and then specify inspection task(s) from the predefined list of such tasks. In yet further examples, the user may be provided with a live 2D image of the article/region of interest via a user's interface of the control system, and is allowed to perform the selection(s). The user's selection of reference point(s) may be assisted by the optical inspection system during the course-stage imaging procedure. For example, this can be implemented in a scan-mode imaging using a single dot pattern as a pointer to select reference points. As will be described more specifically further below with reference to
[0151] As indicated above, in some embodiments of the invention, the control system operates to automatically define/identify the inspection task(s) from the CAD model without additional user's input. In a non-limiting example, the CAD data includes all critical dimensions (e.g. specified by a mechanical engineer who has created a CAD model). In this case, the identifier module 20A selects relevant features of interest, and defines required inspection task(s).
[0152] Turning back to
[0153] It should be noted that in some other examples, the selected group of geometry-related attributes GA.sub.j may include edge/cliff direction and gradient of an element or a fragment thereof. If for example, the inspection task includes also verification/inspection of the surface flatness/roughness, and/or a difference between such properties of similar elements, the selected group of attributes GA.sub.j may also include optical properties related attributes, such as reflectivity of the surface portion.
[0154] Thus, in order to attend to selection of optimized inspection plan(s), the selected features of interest are analyzed and broken down/converted into geometric primitives (such as holes, pins, balls, boxes, walls, edges, grating structures, etc.). Considering automatic inspection (e.g. inspection of articles progressing on a production line), such conversion of the features of interest into the group of attributes, as well as determination of inspection plans per features/regions of interest, may be performed once (as a part of recipe or during the application setup phase). For example, when a CAD model is used to select feature(s) of interest (with user's assistance or fully automatically, as described above), the group of attributes, determined once, is then included in the updated CAD model for further automatic inspection procedure, to select the same or different inspection plans to serve the same or different inspection tasks. Generally, this procedure of conversion is either performed once or performed each time based on the initial 3D image or the height map.
[0155] As for the optical properties related group of attributes (reflectivity/transparency related parameters) of each feature (element) of interest or a fragment thereof, this can also be estimated from the initial 3D image/map, for example by analyzing a relation between the intensity of detected reflected light and expected intensity (i.e. based on the initial article-related data) or from the definition of materials/surface finishing in the CAD model.
[0156] The data indicative of the selected group of attributes GA.sub.j (possibly together with the optical data characterizing a given optical inspection system OIS.sub.i, as the case may) is then used to create a request data RD.sub.ij to the database system (step 108). The request data may be directly communicated to the storage system 30 (step 110) managing the database 32, as described above with reference to
[0157] The manager utility 34 at the storage system 30 operates to automatically select at least one inspection mode data IMD.sub.ij (which is prepared/formatted for communication with the control system) to be received by the planning module 20C of the control system 10 (step 112). The inspection mode data IMD.sub.ij may include data indicative of one or more light parameters (illumination pattern(s), illumination spot shape, illumination intensity and/or spectrum) to be used during the inspection session(s), and/or scan density and/or scan axis orientation. This inspection mode data is analyzed, based on the inspection task data, and the optimal inspection plan data IPD.sub.ij is generated (step 114) to be executed by the given optical inspection system.
[0158] The inspection plan data IPD.sub.ij, including sequence(s) of the selected light patterns, and possibly also variable orientations of the light patterns, may be then used (by the operational controller 28) to manage/control the implementation of the inspection plan by the inspection system with regard to the selected region(s) of interest, while taking into considerations all the inspection tasks and all the features of interest simultaneously.
[0159] Determination of the optimal inspection plan data is aimed at minimizing acquisition time and avoiding interference between different patterns. For example, in some cases two or more of the selected light patterns can be projected simultaneously, if they are projected onto different parts of the field of view of the inspection system. In other cases, there might be a need to perform different scans using different light patterns for the inspection of the same feature of interest, if it is required by different inspection tasks. The inspection plan data IPD.sub.ij can be stored in the memory of the control system and/or that of the associated optical inspection system in relation to/association with a coordinate system of the respective region(s) of interest.
[0160] Reference is made to
[0161] Also provided in the optical inspection system is a control unit 78 which has a processor (image processor) 78A configured and operable to process the detected light response DLR, based on the inspection plan data IPD, and generate measured data MD indicative of one or more parameters/conditions defined by the inspection plan data IPD (e.g. analyzing a sequence of reflections of projected patterns to obtain 3D information on the inspected part). The optical inspection system OIS is configured and operable to perform inspection sessions using structured light. Accordingly, the illuminator(s) 74 is/are configured as projector(s) for projecting light patterns on one or more regions of interest being illuminated.
[0162] As exemplified in the figure, the optical inspection system OIS may include or may be in data communication with the above described control system 10. As mentioned above, the inspection sessions performed by the optical inspection system OIS are aimed at executing the inspection plan(s) provided as described above. To this end, an operational controller 28 is used (being part of the optical inspection system OIS and/or control system 10) for controlling execution of the inspection plans(s) in accordance with the inspection plan data IPD, which in turn is based on the optical configuration data of the optical inspection system. The operational controller 28 includes a pattern generator module 28A (or scan controller) which is configured and operable, by a main task controller 28B, to generate light pattern(s) in accordance with the inspection plan data (optimized inspection plan data).
[0163] As also mentioned above and shown in the figure, the optical inspection system may be associated with a monitor 26 analyzing the measured data MD provided by the control unit 78 and generating output data indicative of inspection results IR. The latter can then be further analyzed by the control system 10 for the purposes of updating the inspection task data and/or updating/optimizing CAD data and/or updating/optimizing the database.
[0164] Reference is made to
[0165] In the example of
[0166] In the example of
[0167] In both of the above non-limiting examples, the database 32 is maintained at a remote storage system, being accessible by the control unit via a webserver. As also shown in the figures, the control unit 10 provides measured data or measured data analysis (inspection results) back to a central system managing the database for updating/optimizing the database via machine learning procedure. It should, however, be understood that the invention is not limited to such an example of a need to communicate with a remote database system. The entire database or at least a part thereof (e.g. inspection mode data pieces associated with geometry-features attributes) may be stored and managed by an internal memory of the control unit 10, and the data processor properly communicates with such internal memory requesting and receiving therefrom the inspection mode.
[0168] As described above, the optical configuration data of a given optical inspection system OIS is defined by a number of illumination channels IC (i.e. number of light pattern projectors); a number of the optical detection channels DC; locations of the illumination and detection channels with respect to an inspection plane; possible relative orientations between the illumination and detection channels; as well as various properties of the illuminator(s) and detector(s) of the optical inspection system.
[0169] Generally, an imaging system suitable for implementing the principles of the present invention described above may include at least one projector/illuminator and at least one imager/camera, and preferably includes at least two projectors and/or at least two cameras. The projector is preferably a 2D projector (i.e., can direct it output light on a 2D surface). Such a 2D projector may utilize a Spatial Light Modulator (SLM), Digital Light Processor (DLP) or a scanning mirror (e.g., MEMS, Galvo, etc.).
[0170] The present invention in its other aspect provides a novel approach for configuration and operation of an imaging system, which can advantageously be used in optical inspection system implementing the principles of the above-described aspect of the invention (i.e. adaptive inspection planning).
[0171] In some embodiments of the imaging system of the present invention it includes one or more 2D projectors, each being associated with two or more cameras; or one or more cameras each being associated with two or more projectors. Preferably, the camera and projectors (or the projector and cameras) are arranged in a triangular configuration. Fields of view (FOVs) of multiple projectors are preferably overlapping (at least partially) on a region in an inspection plane, where a region of interest is located, when the system is in operation. Distances from the camera to multiple projectors, as well as distances between the projector to multiple cameras, may or may not be the same.
[0172] In such embodiments of multiple projectors and/or multiple cameras, i.e. multiple illuminator-detector pairs sharing at least one common unit being illuminator or detector, multiple pairs of the illumination-detection channels are provided. Each illuminator-detector pair defines a base line vector, and the arrangement of the illuminators and detectors is such that the base line vectors of the illumination-detector pairs having the common unit define a predetermined orientation of the base line vectors with respect to one another.
[0173] In some embodiments, the arrangement of the projector(s) and camera(s) may be such that their base line vectors are approximately/substantially perpendicular. More specifically, a line connecting a projector to one camera (i.e. connecting operational centers thereof) is approximately/substantially perpendicular to a line connecting said projector to another camera, and the same with regard to the connection of the same camera to different projectors. In other words, each pair of illumination-detection channels defines a vector between the centers of the illumination and detection channels which is approximately/substantially perpendicular with respect to vectors defined by other illumination-detection channels sharing at least one common element/unit.
[0174] Such a condition of approximately/substantially perpendicular base line vectors is associated with the following:
[0175] Assuming the projector does not operate in a scan mode (i.e. a light (laser) beam is not moved and is “stuck” on a single position), a light beam illuminates a single dot on a target surface and the illuminated dot is imaged as a single dot on the camera. When the target surface changes its height (z-position), i.e. there is a surface relief, the image of the illuminated dot moves on the camera along a straight line (epipolar line). This is in accordance with the principles of the epipolar geometry (which are generally known and need not be specifically described).
[0176] Considering the use of a 2D projector, its output might not be a single dot but a straight line. For each dot on the line being illuminated, there is an epipolar line on a camera. If these epipolar lines are the same, it will be difficult to detect and locate the target height changes, because the camera will “see” the same line. If two cameras are used with the common 2D projector and their arrangement meets the condition of “approximate/substantial perpendicularity” of the base line vectors such problem is eliminated, because each line created/illuminated by the 2D projector provides height sensitivity for at least one of the at least two camera. Thus, such configuration of the imaging system of the present invention optimizes the system ability to extract 3D information of the region being inspected.
[0177] Reference is made to
[0178] In the example of
[0179] The above condition of “approximate/substantial perpendicularity” of the base line vectors provides that, in case a 2D light pattern being projected by the projector 74A is parallel to base line vector V.sub.(76-74A), and hence 3D information is hard to extract from the detected light response of the illuminated pattern, then relevant 3D information can be extracted from the detected light response of the pattern illuminated by the projector 74B. Similarly, if the illuminated pattern of the projector 74B is parallel to V.sub.(76-74B), the 3D information can be extracted from that of projector 74A. If each of the patterns of projectors 74A and 74B is not parallel to any of base line vector V.sub.(76-74A), and V.sub.(76-74B), then 3D information can be extracted from a combination of both patterns.
[0180]
[0181] In the example of
[0182]
[0183]
[0184] This configuration might allow an optimal combination of the projector (74A or 74B) and the camera (76A or 76B) that solves both base line vectors perpendicularity constraints and shading minimization (for shading coming from 3D shapes on the inspected part). Also, with this configuration, when fields of view of projectors 74A and 74B are not overlapping, projectors with narrow scanning angle can be used.
[0185]
[0186] The projector(s) (illuminator(s) configured for projecting light patterns) may be of any known suitable configuration. Considering the use of more than one projector in the imaging system, they may generally be of similar or different configurations/types.
[0187] In any of the above-exemplified configurations of the imaging system including at least one projector and at least one camera, the present invention further advantageously provides for using a 2D projector based on “dynamic” projection of a 2D pattern for example by means of a MEMS or the like having or operable with at least one fast axis.
[0188] For example, the 2D projector may include a resonant or raster 2D MEMS scanning mirror, and MEMS control board, associated with at least one laser source, laser driver ICs and power management ICs. Typically, 3-4 laser sources may be used (RGB and IR). Laser beam(s) from the light source(s) are directed onto the 2D projector, i.e., a scanning mirror, which reflects them to the region being inspected. The scanning mirror moves fast allowing a creation of a light pattern on the inspected region. To allow high speed inspection, as an example, a resonant MEMS-based mirror can be used similar to a pico projector. One axis of the resonant 2D MEMS-based mirror is a fast axis (resonant) with typical frequencies of >10 kHz, the perpendicular axis is a slow raster-scanning axis with typical frequencies of ˜1 kHz. Hence, scanning sequence with lines along the fast axis is significantly faster compared to scanning sequence with lines along the slow axis of the MEMS scanner.
[0189] The imager/detector may be of any known suitable type. In some embodiments, it is preferable to use camera(s) with multiple dynamically repositioned regions of interest (MROIs). This allows significantly faster readout and data transfer (as compared to the readout of the full frame). Some CMOS cameras allow changing the direction of readout from rows to columns. The combination of multiple ROIs readout with the capability to switch readout direction can increase significantly (˜10×) a typical frame rate for optimized regions of interest.
[0190] Generally, cameras may be configured as RGB, monochromatic, NIR, IR and hyperspectral. For example, CMOS cameras with static multiple regions of interest may be used, but in some cases it may slow down insignificantly the performance of the sensor. According to yet another example, CMOS cameras without multiple regions of interest or CCD cameras may be used (although this slows down the performance of the sensor).
[0191] In some embodiments, the present invention utilizes MEMS-based projector(s) and camera(s) with MROIs. This allows for imaging a selected portion of the illuminated pattern.
[0192] For example, the resonant or raster 2D MEMS-type projector(s) can be used in the above-described imaging systems 72 and 172 of
[0193] The principles of the above approach are schematically illustrated in
[0194] The above configuration can significantly increase the speed of the inspection session. In conventional scanners, when scanning the full field of view, the scanner's operation is typically limited by the frequency of the slow axis of the projector (˜+FPS). However, when the scanning sequence is optimized in accordance with the inspection task, the direction of the light pattern is taken into account when optimizing the scanning sequence. In this connection, it should be understood that a scanning sequence is a sequence of successively applied different patterns and readout modes for the cameras. The projector(s) with the optimal orientation of the fast scanning axis are selected based on the direction of pattern. Depending on the ability to sync the scanned pattern with the different projectors fast axis orientations, the scanning speed may increase by up to 100×. In such case the CMOS camera becomes a limiting factor for the overall sensor scanning speed. But if the CMOS camera(s) with multiple ROIs and variable readout direction (rows/columns) is/are used, the overall speed can increase by ˜10× (depending on the ROI optimization).
[0195] It should be noted that the principles of the present invention are not limited to the above exemplified “condition of perpendicularity”, as well as not limited to the 2D projector having any fast axis.
[0196]
[0197] As described above, the inspection mode related data IMD.sub.ij provided by the database' manager is selected to match the request data indicative of the selected group of attributes GA.sub.j and the optical configuration data OCD.sub.i of the given inspection system OIS.sub.i. The selected group of attributes GA.sub.j is in turn selected based on the feature-related data defined by the inspection task data.
[0198] As described above, the planning module analyzes the inspection mode related data and also the inspection task data regarding one or more features, and generates optimal inspection plan data. For example, the optimization is in that the inspection plan includes inspection of multiple features within the same inspection session, e.g. measuring some parameters of one or two features and also a distance between these two features of interest. If the inspection plan is to be performed on multiple features of interest belonging to different regions of interest, and the relative position between said regions of interest may change from one inspection to another, an alignment procedure is performed for each such region separately, and the inspection plan data includes data relating to the region-to-region displacement. Also, the inspection plan may be determined to enable measurement/inspection of more than parameters of the same feature (e.g., measure the diameter of the hole and inspect the shape of the hole). Also, the inspection plan data utilizes the configuration of the selected light pattern(s) provided by the database, and also takes into account the imaging configuration of the inspection system. For example, the initial light pattern is optimized for the alignment procedure. For example, for a region of interest having smooth surfaces, an initial fringe pattern can be used, while for regions with sharp edges—a chess line pattern can be used. According to the invention, light pattern parameters, including, but not limited to pattern frequency and distance between different light patterns can be adapted automatically based on height estimation performed during the setup phase (from CAD or reference image).
[0199] The control system can analyze the detected light response (reflected image of sequentially projected light pattern) in order to localize the region of interest in 6 dimensions: X, Y, Z and rotation on all three axes.
[0200] For example,
[0201] It should be noted, that in order to solve epi-polar constraints for any edge in the field of view, a configuration with multiple projectors (
[0202] The following is the description of some specific not limiting examples of the determination of the inspection plan data by a given configuration of the optical inspection system.
[0203]
[0204] The control system of the invention (configured as described above) operates to analyze the inspection task data about feature F and creates corresponding recipe to be further used by the given optical inspection system. To this end, the control system identifies the feature-related data and converts it to a selected group of attributes, i.e. primitive shapes. In this specific example, the primitive shape description is a rectangular surface parallel to Z axis. The primitive shape data, together with the optical inspection system relating data (optical configuration data or system's ID assigned to the respective configuration data), is used to generate request data to the database' manager, which selects from the database a respective light pattern data defining inspection mode related data. In this specific example, the selected light pattern data includes a single frame pattern sequence containing a grid G.sub.1 of several dots D to be projected onto the top surface of the pad (
[0205] Reference is made to
[0206] The control system of the invention (configured as described above) operates to analyze the inspection task data about feature F and creates corresponding recipe to be further used by the given optical inspection system. For the purpose of the recipe creation the control system generates a selected group of attributes, i.e., primitive shapes, describing the feature F based on the inspection task data. In this specific example, the primitive shape description is a flat surface. The primitive shape data, together with the optical inspection system relating data (optical configuration data or system's ID assigned to the respective configuration data), is used to generate request data to the database' manager, which utilizes this request data to select from the database respective light pattern data defining inspection mode related data. In this specific example, the selected light pattern data includes a grid G.sub.2 of spaced-apart parallel lines L. The control system (planning module) analyses the light pattern data and the inspection task data and determines the corresponding inspection plan data to be included in the recipe data, defining the optimal light pattern application sequence. In this example, this is a sequence of two frames, shown in
[0207] Reference is made to
[0208] The control system of the invention (configured as described above) operates to analyze the inspection task data and creates corresponding recipe including the inspection plan data to be further used by the given optical inspection system. For the purpose of the recipe creation the control system generates a selected group of attributes, i.e. primitive shapes, describing the feature F based on the inspection task data. In this specific example, the primitive shape description is a flat surface having polygonal geometry. The primitive shape data, together with the optical inspection system relating data (optical configuration data or system's ID assigned to the respective configuration data), is used to generate request data to the database' manager, which utilizes the request data to select from the database respective light pattern data defining inspection mode related data. In this specific example, the selected light pattern data includes a single line L. The control system (planning module) analyses the light pattern data and the inspection task data and determines the corresponding recipe data, defining the optimal light pattern application sequence. In this example, this is a multi-frame sequence—three such frames R.sub.1, R.sub.2, R.sub.3 being exemplified in
[0209] Reference is made to
[0210] The control system of the invention analyzes the inspection task data and the initial data (prior knowledge) and creates corresponding recipe to be further used by the given optical inspection system. For the purpose of the recipe creation the control system generates a selected group of attributes, i.e., primitive shape(s), describing the feature F based on the inspection task data. In this specific example, the primitive shape description is a pair of spaced parallel walls. The control system communicates with the database' manager and receives therefrom data indicative of matching light pattern, which in this example is in the form of a grid G.sub.3 of parallel lines L extending along the X-axis and being spaced-apart along the Y-axis (
[0211] Reference is made to
[0212] The control system of the invention analyzes the inspection task data and the initial data (prior knowledge) and creates corresponding recipe to be further used by the given optical inspection system. For the purpose of the recipe creation the control system generates a selected group of attributes, i.e., primitive shape(s), describing the feature F based on the inspection task data. In this specific example, the primitive shape description is a pair of spaced parallel surfaces. The control system communicates with the database' manager and receives therefrom data indicative of matching light pattern(s), which in this example is in the form of a single line L pattern. The planning module of the control system analyzes the light pattern data and generates the recipe including the inspection plan data defining a pattern sequence, which in this example is a multi-frames sequence, where in each frame the patterns includes exactly one short line “scanning” the area of the approximate location (per the prior knowledge), and the line is perpendicular to the walls' orientation. It should be understood with the short wall feature, where a region around the wall is empty, location tolerance does not allow exact projector line positioning on the wall. Therefore, exact locations of the left and right sides of the wall are to be found. Image data collected during such a multi-frame inspection session (four-frame session in this example) is shown in
[0213] Reference is made to
[0214] The control system (10) analyzes the inspection task data and the initial data (prior knowledge) and creates corresponding recipe to be further used by the given optical inspection system. For the purpose of the recipe creation the control system generates a selected group of attributes, i.e., primitive shape(s), describing the feature F based on the inspection task data. In this specific example, the primitive shape(s) description comprises a pair of spaced parallel surfaces. The control system communicates with the database' manager and receives therefrom data indicative of matching light pattern(s), which in this example comprises a single illumination line pattern L, which is perpendicular to the orientation of the sides/facets S.sub.1 and S.sub.2. The illumination line pattern L is also broken into several segments, some are solid continues illumination lines, and others are separate illumination dots. The planning module (20C) of the control system analyzes the light pattern data and generates the recipe including the inspection plan data defining an illumination pattern sequence, which in this example is a multi-frames sequence, where in each frame the pattern includes either a continues illumination line or a single illumination dot along an imaginary continuity of the line L. Such sequence of frames is a time-based coding scheme which allows separation between different segments (segment can be a dot or a continuous line) along the scanning line L. Using this coding scheme allows for tighter constrains when solving the 3D position of the scanned target, which in turn results in better scan resolution.
[0215] Therefore, exact locations of the left and right sides S.sub.1,S.sub.2 of the wall PS are to be found. Image data collected during such a multi-frame inspection session (12 frames in this specific example) is shown in
[0216]
[0217]
[0218]
[0219]
[0220]
[0221] Feature F1 was scanned with lines along axis S1 since the object width is to be measured. It should be noted that only part of the object was scanned since width measurements can be averaged over part of the object.
[0222] Features F2 and F3 were first scanned with lines along axis S1 and then along axis S2 (scans were combined) in order the get good resolution for both width and length measurements.
[0223] Features F1 and F3 were scanned with higher illumination power than feature F2 (not represented in the figures) because they are located at the edges of the field of view and thus are at larger distance from the imaging device.
[0224] In some other examples, which are not specifically illustrated, the inspection task may be aimed at determining the presence of at least one 3D bump on a surface portion, which may be surface of a region of interest on the article's substrate or a top surface of an active element on the article (e.g. pad-like element). For this purpose, the initial article-related data includes data indicative of the boundary location of said surface portion. In this case, the selected group of primitives includes a polygonal flat surface, and the selected light pattern data received from the database includes a fringe-like pattern characterized by its phase. The control system analyses the light pattern data taking into account the inspection task data and creates a corresponding recipe. For example, the recipe defines a pattern sequence in the form of at least three frame patterns, where each pattern is the fringe with a phase different from that of the other frames. Image data can then be processed to build a height map from the fringes, and identify whether said height map corresponds to the existence of one or more 3D bumps.
[0225] In another example, the article inspection task may be aimed at identifying whether a region of interest contains any feature/element on its surface. The initial data is indicative of reflectivity of said surface in a certain wavelength (or wavelength range). In this case, the control system converts the feature (reflective surface) to a group of attributes associated with optical-properties relating primitives (e.g. illumination wavelength for which said surface is maximally reflective in order to maximize a received signal/light response). As for the pattern, any pattern may or may not be used.
[0226] It should, however, be noted that the same region of interest, as well as the same feature/element, might be associated with more than one inspection task, and the recipes should thus be prepared accordingly. In case multiple recipes do not relate to the same field of view (i.e. to the same region of interest being imaged), they can be combined into a single recipe structure containing multiple recipes each operating within its relevant field of view.
[0227] Upon proper creation and storage of the recipe(s), the inspection system can execute the inspection sessions. During the run-time execution, the position of the region of interest relative to the imaging system may change from one execution cycle (inspection session) to another. Hence, part alignment-localization of the region of interest in the coordinate system of the imaging system is to be performed.
[0228]
[0229] As described above, the monitor 26 (being part of the control system and/or optical inspection system) may further be used to analyze the measured data (data indicative of the sequence of reflections of projected patterns) to provide inspection results matching the inspection task and generate corresponding output data (e.g. one or more parameters/conditions of the one or more selected features) — step 516. The analysis of the inspection results may be used to decide as to whether to attend to define a further inspection task—step 518.
[0230] In addition to the above described examples of the inspection results, the inspection results may also include the following types: Local Point Cloud or Local Height Map; Height profiles in multiple directions; vector representation of a 3D primitive (such as holes, pins, balls, boxes, grating structures etc.); feature of interest location (XYZ) and/or orientation (X,Y,Z,Rx,Ry,Rz); properties of features of interest (size, circle radius, corner radios, area, average/max height etc.); distance between features of interest; angle between planes
[0231] The analysis of the measured data depends on the type of the inspection results and on the projected pattern.
[0232] The following Table exemplifies various recipe structures and inspection plan schemes provided by the technique of the present invention, based on the input data about inspection task and associated feature(s) and data about the type of light pattern received from the database system.
TABLE-US-00001 Light patterns Single dot Fringe scan Single line scan Multi line scan Coded lines Coded frame pattern Required 1 Local Point find X, Y, Z Measure local Count lines, decode and decode and compute data Cloud point by shift, compute Assign lines, triangulate triangulate phase 2 Local Height triangulation X, Y, Z of each Measure local shift Map line by shift, compute 3 Height triangulation X, Y, Z of each profiles in (combine line by multiple calculations triangulation directions from two (combine cameras or two calculations projectors) from two cameras or two projectors) 4 Vector Format Compute location of line breaks of 3D on edges. Primitive (e.g. Count and assign. holes, pins, Find XYZ coordinates of breaks balls, boxes, and interpolate grating structures, etc.) 5 Feature of Use information from items 1-4 and post-process it interest location (e.g. coordinates X, Y, Z, Rx, Ry, Rz) 6 Properties of Use information from items 1-4 and post-process it features of interest (e.g. Size, Circle Radius, Corner Radios, Area, Average/Max Height etc.) 7 Distance Use information from items 1-4 and post-process it between features of interest 8 Angle Identify Planar surfaces of interest. Compute between planar angles using lines direction. Compute angle planes between planes
[0233] As described above, the database containing data indicative of various light patterns in association with/assigned to groups of attributes and imaging configurations is a generic database, and can be accessed by multiple control systems, which generates data indicative of the group of attributes and respective request data to the database' manager. More specifically, the database matches the best light pattern(s) to 3D primitives and to inspection plan to be executed by the given imaging system configuration. Such 3D primitives and inspection tasks and plans often repeat themselves, for example because machine vision in the industrial automation analyzes from thousands to millions of identical parts; and/or because different parts (even from different customers/production lines) have similar primitives as they are all modeled using CAD software.
[0234] Hence, the inspection results obtained by each inspection system may be used for updating/optimizing the database. This can be performed as follows: The database' manager/controller collects information from multiple imaging systems deployed in the field, running on various primitives and performing various inspection plans to serve various inspection tasks. This information and inspection results are uploaded to such central database, and the manager runs optimization algorithms in order to improve inspection plans for certain primitives and inspection tasks, thus enabling access to the periodically improved database.