CENTRALIZED ANALYTICS OF MULTIPLE VISUAL INSPECTION APPLIANCES

Abstract

A visual inspection data collection and analysis system comprising: a plurality of visual inspection appliances (VTA) configured to inspect and acquire visual inspection data relating to inspected items; and a data collection and analytics server (DCAS) configured to receive information comprising the visual inspection data from the multiple VIAs and to analyze the received information to form a big data analysis. The VIAs are adapted for detecting defects or gating or counting the inspected items without the involvement of the DCAS.

Claims

1-17. (canceled)

18. A visual inspection data collection and analysis system, comprising: a. a plurality of visual inspection appliances (VIA) configured to acquire visual inspection data relating to inspected items; and b. a data collection and analytics server (DCAS) in communication with the plurality of VIAs over a communications network and configured to automatically detect and register each VIA newly connected to the network.

19. The system of claim 18 wherein the DCAS is configured to register each VIA newly connected to the network under a unique identifier and initiate an activity in each VIA based on the identifier of each VIA.

20. The system of claim 19 wherein the activity comprises one or a combination of: collecting visual inspection data, timing collection of the visual inspection data, storing received data, analysis of the received data and issuing reports.

21. The system of claim 19 wherein the activity is selected from the group consisting of one or a combination of: a. DCAS checks operational status of the VIA; b. DCAS checks software version running on the VIA; c. DCAS checks security status of the VIA; d. DCAS accesses a real-time view of an inspected item from the VIA; e. DCAS requests specific data from the VIA; f. DCAS changes inspection or other settings of the VIA; g. DCAS performs software upgrades to the VIA; h. DCAS initiates inspection to be performed by the VIA; i. DCAS changes a region of interest to be inspected by the VIA; j. DCAS initiates re-inspection of previously inspected items; and k. DCAS initiates re-inspection of previously inspected items with changed inspection parameters.

22. The system of claim 18 wherein the DCAS is configured to receive the visual inspection data from the plurality of VIAs and to analyze said received information to form a big data analysis.

23. The system of claim 22 wherein the inspected items are different types of items.

24. The system of claim 23 wherein the big data analysis comprises a combination of information related to the different types of items.

25. The system of claim 22 further comprising a display and wherein said DCAS outputs said analysis to said display.

26. The system of claim 22 wherein said analysis is selected from the group consisting of: a. root cause analysis of detected defects; b. predictive maintenance analysis; c. intensity of defects; d. significance of defects; e. analysis of product deviations from ideal that are not defects but indicate a trend towards decreasing quality; f. analysis of defect shape, area and type of defect; g. cost of defect; h. product recall, latent product fault vs. defect or product deviation history analysis; i. supplier analysis comparing product raw material suppliers vs defects; and j. relationship analysis between different production stages of a same item.

27. The system of claim 22 wherein said DCAS is adapted to issue reports based on received inspection data wherein said reports are selected from the group consisting of: a. % defects detected per item; b. defect report including images of item showing where defects were detected; c. % defects detected per manufacturing area; d. number of items inspected per period of time; e. personnel vs item defect report; f. % defects per shift; g. % defects per manufacturing type; h. % defects per defect type; i. defect report per period of time and production area; and j. a combination of the above.

28. The system of claim 22 wherein said DCAS is adapted to store said received visual inspection data and wherein the stored inspection data can be searched.

29. The system of claim 28 wherein said DCAS is adapted to run 3rd party applications adapted to produce analyses and reports based on said stored inspection data.

30. The system of claim 22 wherein said DCAS is adapted for issuing alerts based on said analysis.

31. The system of claim 18 wherein the visual inspection data from each one of the plurality of VIAs is selected from the group consisting of: a. image/s of an inspected item; b. record of decision by VIA whether an item has a defect; c. image of the defects; d. number of defects; e. records of deviations from good item samples which are not significant enough to be reported as defects but can imply issues in a production line; f. item unique ID; g. plant work/job order; h. batch ID; i. personnel in charge of a production line or station; j. production tool ID; k. part name; l. part serial number; m. production tool ID; and n. a combination of the above.

32. The system of claim 18 wherein said visual inspection data is communicated from a VIA to said DCAS according to timing selected from the group consisting of: a. after inspection of each item by each VIA; b. after inspection of a configurable number of items per VIA; c. after a configurable period of time per VIA; d. based on a date schedule; e. based on a time of day schedule; and f. a combination of the above.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0029] The invention will now be described in relation to certain examples and embodiments with reference to the following illustrative figures so that it may be more fully understood. In the drawings:

[0030] FIGS. 1A-1B are illustrative schematic drawings showing collection of data from automated visual inspection appliances on a production line according to at least some embodiments of the present invention; and

[0031] FIG. 2 is a flow diagram showing a process for collection of data from automated visual inspection appliances on a production line according to at least some embodiments of the present invention.

DETAILED DESCRIPTION

[0032] The present invention in at least some embodiments is for a system comprising multiple automated visual inspection appliances (VIA) for a production plant and a centralized data collection and analytics server (DCAS) that gathers and analyzes data from the VIAs. Reference is now made to FIGS. 1A-1B which are illustrative schematic drawings showing collection of data from automated visual inspection appliances on a production line according to at least some embodiments of the present invention. As shown in FIG. 1A an automated visual inspection system 100 comprises multiple visual inspection appliances (VIA) 110A, B,C, n in communication with a data collection and analytics server (DCAS) 150. Although four VIAs 110A, 110B, 110C and 110n are shown, it should be appreciated that any number of VIAs may be in communication with DCAS 150. VIA is preferably provided as an integrated appliance for use in a manufacturing environment or plant. VIAs 110A, B,C, n are optionally installed in one plant or optionally multiple VIAs are installed in multiple plants. Each VIA connects to DCAS 150 using wired or wireless communications protocols and methods as known in the art.

[0033] DCAS 150 is a computing device as defined above and may optionally comprise a server, distributed server, cloud computing environment, data cluster or any other suitable computing device. DCAS 150 preferably comprises analysis engine 152, database (DB) 154, DCAS user interface (UI) 156, and notification engine 158.

[0034] Analysis engine 152 receives data from VIAs 110A, B, C and n and analyses the received data to output insights, recommendations, summaries, trends, alerts, and root cause analysis of defects all related to the items inspected and the production environment as described below. Analysis engine 152 optionally uses big data analysis methods.

[0035] DB 154 is a database (e.g., as known in the art) and stores data transmitted by VIAs 110A, B, C and n and also results and interim results of analysis by engine 152. DB 154 also stores configuration data defined in DCAS 150 for system 100 including VIA profiles. A VIA profile includes information about each VIA in system 100 including but not limited to: unique identifier, name, physical mounting details, position in plant, plant geolocation, items inspected, reference images of items inspected, profiles of items inspected, inspection results and so forth. Optionally a manufacturing area 170 is defined for DCAS 150 where each manufacturing area 170 includes one of more VIAs. A manufacturing area 170 optionally comprises VIAs from one plant or optionally comprises VIAs from multiple plants. The manufacturing area 170 defined in FIG. 1A includes VIAs 110B and 110C but it should be appreciated that any number or any of VIAs could be included in a manufacturing area 170 and any number of manufacturing areas 170 may be defined for DCAS 150. Where more than one manufacturing area 170 is defined these manufacturing areas 170 may optionally overlap, i.e.: a single VIA may optionally be part of several manufacturing areas 170.

[0036] DCAS UI 156 enables display of the results of analysis engine 152 and also interaction with DCAS 150 by a human operator (not shown). DCAS UI 156 optionally comprises a monitor or screen and information provided to a user of DCAS 150 may be visual (e.g., text or other content displayed on the monitor). Alternatively or additionally, DCAS UI 156 comprises an audio player to emit a sound. DCAS UI 156 preferably enables accepting user input such as by a touch screen, keyboard and/or mouse. Optionally, DCAS UI 156 is provided on a multi-purpose device such as a smartphone, tablet or personal computer in communication with DCAS 150. Optionally DCAS UI 156 can be accessed remotely optionally from within the plant where it operates and outside of the plant where it operates.

[0037] Notification engine 158 is in communication with external communication networks 70 and provides push notification of alerts or other outputs from analysis engine 152. Non-limiting types of notification methods include email, SMS, WhatsApp or any mobile notification mechanism. Notification engine 158 can be configured via DCAS UI 156 to define recipients and notification methods for different types of alerts, reports or analyses.

[0038] In some embodiments, VIAs 110A, B, C, n and DCAS 150 communicate over the external network 70. DCAS 150 may automatically detect when a VIA is connected to the external network 70 and may then register the newly connected VIA and perform data collection and analysis of the VIA performance and of data obtained by the VIA, as described herein.

[0039] Thus, in one embodiment, a visual inspection data collection and analysis system, includes a plurality of VIAs configured to acquire visual inspection data relating to inspected items, and a central server, such as a DCAS configured to identify a newly connected VIA on a communications network, to register the newly connected VIA and enable data collection and analysis of each registered VIA.

[0040] The DCAS 150 may identify each VIA based on an ID, IP address or other unique identifiers connected to each VIA and each VIA may be registered under a unique identifier. Data collection and analysis of each registered VIA may be done according to the registered unique identifier. E.g., data from VIAs registered under an identifier related to inspection line A may be analyzed differently from data from VIAs registered under an identifier related to inspection line B.

[0041] DCAS 150 may detect when a VIA is connected to the external network 70 based on signals sent over the network (e.g., ethernet) by DCAS 150 and/or VIAs 110A, B, C and n. Signals may include, for example, packets transmitted by multicast addressing using, for example User Datagram Protocol (UDP). Based on the signals, which may be transmitted periodically by the DCAS and/or VIA, the DCAS can determine that a VIA is connected to the network and the DCAS may then compare the VIA identifier to already registered VIA identifiers to determine if the VIA is newly connected or not.

[0042] DCAS 150 may perform one or more different actions for each registered VIA, as described herein. For example, collecting visual inspection data and the timing of the collection of data may be done based on the registered VIA identifier. Storing the received data, analysis of the received data and issuing reports may be controlled based on the registered VIA identifier. The DCAs may initiate different activities in each VIA based on the registration of each VIA.

[0043] DCAS 150 is optionally in communication with an external monitoring system 60. Monitoring system 60 is a computing device as described above. Monitoring system 60 is typically a production plant management system such as for gathering and monitoring key performance indicators for manufacturing efficiency. Monitoring system 60 is optionally a production resource management platform.

[0044] DCAS 150 optionally runs 3.sup.rd party applications 159 where 3.sup.rd party application 159 are operative to produce analyses and reports based on the collected data, which may be stored in DCAS 150. Optionally the 3.sup.rd party applications 159 can operate VIAs according to the capabilities of DCAS 150.

[0045] As shown in FIG. 1B each VIA 110 comprises a controller 130, camera assembly 111, and mounting assembly 108. Camera assembly 111 comprises camera 102, and light source 106.

[0046] Camera 102 comprises a CCD or CMOS or other appropriate imaging chip. Camera 102 is a 2D camera or optionally a 3D camera. Optionally camera 102 comprises the camera integrated into a mobile device such as a smartphone or tablet where the device is attached to mounting assembly 108. Camera 102 optionally comprises a polarizing lens, tele-centric lens, narrow band, zoom lens, or other lens (not shown) placed over the lens of camera 102 or directly upon its imaging chip.

[0047] Light source 106 comprises LEDs or other known light source. The intensity (brightness) of light source 106 can be adjusted. Optionally, the color of light source 106 can be adjusted. Optionally, light source 106 comprises multiple controllable segments, each of which can be activated or provided with the same or different intensity and/or color. For example, but without intention to be limiting, light source 106 may comprise a circular array of LEDs surrounding camera 102 lens, where radial portions of circular light source 106 are controlled individually or alternatively the intensity and/or color of every LED or groupings of LEDs, can be controlled individually.

[0048] Light source 106 is shown as positioned above camera 102 for simplicity of the figures but this position should not be considered limiting. Optionally, light source 106 is mounted on the side of or below camera 102. Light source 106 is preferably attached to and surrounds or is otherwise fixed in relation to the lens of camera 102 so as to illuminate the field of view (FOV) 104 of camera 102 or portions thereof. Camera assembly 111 is attached to mounting assembly 108. Alternatively, camera 102 and light source 106 are separately attached to mounting assembly 108 allowing individual adjustment of the spatial position of either.

[0049] Mounting assembly 108 comprises mounts, segments and fasteners allowing adaptation and adjustment of mounting assembly 108 for optimal positioning of camera 102 and light source 106 for inspection of an item.

[0050] Camera assembly 111 is positioned using mounting assembly 108 such that items 20 to be inspected are within the field of view 104 of camera 102. Mounting assembly 108 is attached to a mounting surface 40. Surface 40 may remain in a fixed position relative to item 20 or alternatively may move so as to repeatedly bring camera assembly 111 into a position where items 20 to be inspected are within the field of view 104 of camera 102. A non-limiting example of a moving surface 40 is a robot arm. Where reference is made to FOV 104 herein it is to be understood that light source 106 is positioned to illuminate FOV 104. Surface 40 optionally comprises an aluminum profile including grooves for attachment of mounting brackets.

[0051] Items 20 to be inspected may be placed on an inspection line 30 which comprises means for supporting and moving items 20 such as but not limited to a conveyor belt, or a cradle or another holding apparatus, moving in direction 22, such that first item 20 is brought into FOV 104 followed by second item 20 which is brought into FOV 104, and so forth. Alternatively, items 20 are successively placed in FOV 104 and then removed such as by a robot or human operator.

[0052] Camera 102 and light source 106 are in communication with controller 130. Controller 130 is a computing device as defined herein. Controller 130 comprises one or more processors (not shown) such as but not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microprocessor, a controller, a chip, a microchip, an integrated circuit (IC), or any other suitable multi-purpose or specific processor or controller. Controller 130 activates light source 106 or any of its components or controllable segments as described above, which may or may not be activated depending on the item being imaged or the inspection lighting environment. Controller 130 preferably alters the intensity or color of light source 106 depending on the item being imaged or the inspection lighting environment. Controller 130 preferably alters the intensity or color of light source for regions of particular interest within the illuminated area.

[0053] Controller 130 further comprises a memory unit (not shown) which stores executable instructions that, when executed by the processor, facilitate performance of operations of the processor. The memory unit may also store at least part of the image data received from camera 102. Non-limiting examples of memory units include random access memory (RAM), dynamic RAM (DRAM), flash memory, volatile memory, non-volatile memory, cache memory, a buffer, a short term memory unit, a long term memory unit, or other suitable memory units or storage units.

[0054] Controller 130 further comprises a VIA user interface (UI) 132. VIA UI 132 may comprise a monitor or screen and notifications to a user may be visual (e.g., text or other content displayed on the monitor). Alternatively or additionally, VIA UI 132 comprises a light that may light up or change color. Alternatively or additionally, VIA UI 132 comprises an audio player to emit a sound. VIA UI 132 preferably enables accepting user input such as by a touch screen, keyboard and/or mouse. Optionally, VIA UI 132 is provided on a multi-purpose device such as a smartphone, tablet or personal computer.

[0055] Optionally DCAS 150 can check the operation status of one or more VIAs 110. Optionally DCAS 150 can check the software version running on one or more VIAs 110. Optionally DCAS 150 can check the security status of VIAs (e.g., that one or more VIAs 110 are updated with the most recent security updates). Optionally an operator can use DCAS 150 to access a real-time view of the inspection images from any VIA 110 for display on DCAS UI 156. Optionally an operator can use DCAS 150 to request specific data from any one or more of VIAs 110. Optionally DCAS 150 can change inspection or other settings of any one or more of VIA 110. Optionally DCAS 150 can perform software upgrades of any one or more of VIA 110. Optionally DCAS 150 can initiate inspection to be performed by any one or more of VIA 110. Optionally, DCAS 150 can change the region of interest to be inspected by any one or more of VIA 110. Optionally DCAS 150 can initiate re-inspection of previously inspected items further optionally with changed inspection parameters.

[0056] Reference is now made to FIG. 2 which is a flow diagram showing collection of data from automated visual inspection appliances on a production line according to at least some embodiments of the present invention. Use of automated visual inspection system 100 preferably proceeds according to process 200 as shown in FIG. 2. Before process 200 can begin, VIA 110 is set up to enable inspection of items 20. System 100 requires setup for each type of item or stage of item that is to be inspected. In the setup step, preferably at least two or more defect free samples of a manufactured item 20 of the same type are placed in succession within field of view 104 of camera 102. Each defect free sample of item 20 is imaged by camera 102. These images, which may be referred to as setup images, are optionally obtained by using different imaging parameters of camera 102 and lighting parameters of light source 106. The images comprise image data such as pixel values that represent the intensity of reflected light as well partial or full images or videos.

[0057] The setup images are analyzed by controller 130 using machine learning/artificial intelligence (AI) and computer vision algorithms to create a complete representation of item 20 used for defect detection, gating, sorting and/or other inspection tasks, on the production line. Following the setup step and based on the information collected from the sample, defect-free items, the inspection process can begin and controller 130 can preferably detect and inspect further items of the same type even if these further items were never previously presented, and determine whether these are defect-free. It should be noted that the inspection of items 20 for defect detection (determination of whether the item 20 is defect free or has a defect) or gating or sorting or counting can be performed by VIA 110 independently of DCAS 150.

[0058] In step 202, items 20 are inspected by each VIA 110 for defect detection, gating, or sorting purposes. The following data is collected by each VIA 110 per item 20 as a result of the inspection process. This data is herein referred to as “per-item collected data”, and one or more of per-item collected data is referred to as “collected data”: [0059] Image/s of the inspected item; [0060] Record of decision by VIA whether item has a defect; [0061] Image of the defects; [0062] Number of defects; [0063] Records of deviations from good item samples which are not significant enough to be reported as defects but can imply to issues in the production line; [0064] Item unique ID; [0065] Plant Work/Job order/Batch ID; [0066] Personnel in charge of the production line or station; [0067] Production tool ID (Die or molder, etc.). [0068] Part Name [0069] Part Serial Number [0070] Inspection Profile ID

[0071] In step 204 the collected data is transmitted by each VIA 110 to DCAS 150. As above the communication between VIA 110 and DCAS 150 may use standard communication infrastructure and protocols as known in the art. Collected data is stored in DB 154. Collected data is transmitted by VIA 110 to DCAS 150 according to one or more of the following: [0072] After inspection of each item by each VIA; [0073] After inspection of a configurable number of items per VIA; [0074] After a configurable period of time per VIA; [0075] Based on Date/Time schedule; [0076] Based on a combination of the above.

[0077] Collected data stored in DB 154 can preferably be searched and queried via DCAS UI 156, DB 154 of DCAS 150 functioning as an archive. A non limiting example of such a query is a search for images and other inspection data related to a specific item indexed by an item identifier such as but not limited to the item barcode or serial number.

[0078] In step 206 the collected data from VIAs 110 is analyzed by analysis engine 152 and/or used for generating reports. The analyses or use of collected data of step 206 optionally take place immediately following step 204. Alternatively, step 206 takes place some time after step 204. Reports and/or analyses are preferably generated using big data methods. Optionally the analysis is performed for a combination of different type items where different type items may be any of different products, different production stages, different plants, or different industries. One or more of the following reports are preferably generated including but not limited to: [0079] % defects detected per item; [0080] Defect report including images of item showing where defects were detected; [0081] % defects detected per manufacturing area; [0082] Number of items inspected per period of time; [0083] Personnel vs item defect report; [0084] % defects per shift; [0085] % defects per manufacturing type (e.g. casting lines vs molding lines); [0086] % defects per defect type; [0087] Defect report per period of time and production area;
One or more of the following analyses are preferably performed including but not limited to: [0088] Root cause analysis of detected defects; [0089] Predictive Maintenance analysis—based on detecting trends in defect or deviations that are not defects; [0090] Intensity of the defects—analysis of trends to increasing occurrences of defects per period of time; [0091] Significance of the defects [0092] analysis of trends of increasing effect of defects or deviations on the produced item; [0093] Analysis of product deviations from ideal that are not defects but indicate a trend towards decreasing quality; [0094] Analysis of defect shape, area and type of defect optionally in the form of a defect “map”; [0095] Cost of defect—i.e. the cost of discarded items or cost of repair of items determined to be defective; [0096] Product recall and/or latent product fault vs. defect and/or product deviation history analysis; [0097] Supplier analysis comparing product raw material suppliers vs defects; [0098] Relationship analysis between different production stages of the same item.

[0099] The analysis or reporting of step 206 preferably takes place based on one or more of: [0100] Periodic for set periods of time which are preferably configured independently per analysis; [0101] Operator initiated where an operator of DCAS 150 defines and initiates a specific analysis or report; [0102] Analysis and/or reporting per number of items inspected performed after a specific number of items have been inspected by a specific VIA 110 or specific manufacturing area 170.

[0103] In step 208 the results of the analyses and/or reports of step 206 are stored in DB 154 and also preferably displayed using DCAS UI 156. Optionally, results are exported to external systems such as but not limited to external monitor 60. Optionally results are displayed on a configurable dashboard presented on DCAS UI 156. Optionally results of step 206 generate alerts which are displayed on DCAS UI 156 or communicated to an operator of DCAS 150 such as via notification engine 158 sending, for example but not limited to, text or other messages to a mobile device. A non-limiting example of an alert is “% defects detected in a production area exceeding a defined threshold”.

[0104] It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.

[0105] Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.