AI-based Automatic Judgment Unit for Quality Classification of Semifinished Component Carriers of a Panel Based on Automatic Optical Inspection

Abstract

A method of manufacturing component carriers is disclosed. The method includes supplying a panel with a plurality of semifinished component carriers, to an automatic optical inspection unit for automatic optical inspection by comparison of a data set indicative of an actual image of a respective semifinished component carrier with a data set indicative of a reference image, forwarding the inspected panel to an automatic judgment unit for carrying out a quality classification of the semifinished component carriers, based on a result of the automatic optical inspection, by applying artificial intelligence, and taking an action based on the quality classification.

Claims

1. A method of manufacturing component carriers, wherein the method comprises: supplying a panel, comprising a plurality of semifinished component carriers, to an automatic optical inspection unit for automatic optical inspection by comparison of a data set indicative of an actual image of a respective semifinished component carrier with a data set indicative of a reference image; forwarding the inspected panel to an automatic judgment unit for carrying out a quality classification of the semifinished component carriers, based on a result of the automatic optical inspection, by applying artificial intelligence; and taking an action based on the quality classification.

2. The method according to claim 1, wherein the method comprises taking the action of continuing manufacturing the component carriers without intervening process steps.

3. The method according to claim 2, comprising at least one of the following features: wherein the method comprises taking the action of continuing manufacturing the component carriers without intervening process steps when the quality classification indicates no defect of a respective semifinished component carrier; wherein the method comprises taking the action of continuing manufacturing the component carriers without intervening process steps when the quality classification indicates an unrepairable defect of a respective semifinished component carrier.

4. The method according to claim 1, wherein the method comprises taking the action of repairing a respective semifinished component carrier when the quality classification indicates a repairable defect of a respective semifinished component carrier, in particular wherein the repairing comprises at least one of the group consisting of separating erroneously connected electrically conductive traces, connecting erroneously disconnected electrically conductive traces, adding solder material to or removing solder material from a defective solder structure, and repair of a defective core.

5. The method according to claim 1, wherein the method comprises at least one of the following features: taking the action of repairing a respective semifinished component carrier on basis of a data set indicative of a three-dimensional image of the semifinished component carrier or the panel; carrying out the quality classification by classifying a respective individual semifinished component carrier or an entire panel as pass; carrying out the quality classification by classifying a respective individual semifinished component carrier or an entire panel as fail; carrying out the quality classification by classifying a respective individual semifinished component carrier or an entire panel as repairable or to be repaired; carrying out the quality classification by indicating a type of defect among a number of predefined types of defect.

6. The method according to claim 1, wherein the method is carried out completely without involving a human operator.

7. The method according to claim 1, wherein the method comprises at least one of the following features: carrying out the automatic optical inspection based on a data set indicative of a two-dimensional image of the entire panel or a respective individual semifinished component carrier; carrying out the automatic judgment based on a data set indicative of a three-dimensional image of the entire panel or a respective individual semifinished component carrier.

8. The method according to claim 1, wherein the method comprises, when the automatic judgment unit is incapable of performing a quality classification, carrying out a further analysis.

9. The method according to claim 8, wherein the method comprises carrying out the further analysis by an additional automatic judgment unit applying artificial intelligence.

10. The method according to claim 9, wherein the method comprises exchanging data between the automatic judgment unit and the additional automatic judgment unit for learning, in particular in forward and/or backward direction.

11. The method according to claim 8, wherein the method comprises carrying out the further analysis based on a data set indicative of a three-dimensional image of a respective individual semifinished component carrier or the entire panel.

12. The method according to claim 1, wherein the method comprises applying the artificial intelligence in the automatic judgment unit by learning based on historical human-based judgments

13. The method according to claim 12, wherein learning based on historical human-based judgments is at least partially done under consideration of a comparison of judgments of the automatic judgment unit with human-based judgments.

14. The method according to claim 1, wherein the method comprises marking, in particular laser marking, a respective semifinished component carrier or an entire panel based on the quality classification.

15. The method according to claim 1, wherein the method comprises carrying out the automatic optical inspection during front-end processing of the semifinished component carriers on panel level.

16. The method according to claim 1, wherein the method comprises carrying out the automatic optical inspection and the quality classification after patterning a metal layer for forming electrically conductive traces of the semifinished component carriers or the entire panel for assessing a quality of the formed traces.

17. An apparatus for handling a panel during manufacturing component carriers, wherein the apparatus comprises: an automatic optical inspection unit configured for carrying out an automatic optical inspection of the panel, comprising a plurality of semifinished component carriers, by comparison of a data set indicative of an actual image of a respective semifinished component carrier with a data set indicative of a reference image; an automatic judgment unit configured for subsequently carrying out a quality classification of the semifinished component carriers, based on a result of the automatic optical Inspection, by applying artificial intelligence; and an action control unit configured for taking an action based on the quality classification.

18. The apparatus according to claim 17, wherein the apparatus is configured to carry out or control a method comprising: supplying a panel, comprising a plurality of semifinished component carriers, to an automatic optical inspection unit for automatic optical inspection by comparison of a data set indicative of an actual image of a respective semifinished component carrier with a data set indicative of a reference image; forwarding the inspected panel to an automatic judgment unit for carrying out a quality classification of the semifinished component carriers, based on a result of the automatic optical inspection, by applying artificial intelligence; and taking an action based on the quality classification.

19. A computer-readable medium, in which a computer program of manufacturing component carriers is stored, which computer program, when being executed by one or a plurality of processors, is adapted to carry out or control a method comprising: supplying a panel, comprising a plurality of semifinished component carriers, to an automatic optical inspection unit for automatic optical inspection by comparison of a data set indicative of an actual image of a respective semifinished component carrier with a data set indicative of a reference image; forwarding the inspected panel to an automatic judgment unit for carrying out a quality classification of the semifinished component carriers, based on a result of the automatic optical inspection, by applying artificial intelligence; and taking an action based on the quality classification.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0055] FIG. 1 illustrates an apparatus for handling panels during manufacturing component carriers according to an exemplary embodiment of the invention.

[0056] FIG. 2 illustrates a flowchart of a semiautomatic method of manufacturing component carriers according to an exemplary embodiment of the invention.

[0057] FIG. 3 illustrates a flowchart of a fully automatic method of manufacturing component carriers according to an exemplary embodiment of the invention.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

[0058] The illustrations in the drawings are schematically presented. In different drawings, similar or identical elements are provided with the same reference signs.

[0059] Before referring to the drawings, exemplary embodiments will be described in further detail, some basic considerations will be summarized based on which exemplary embodiments of the invention have been developed.

[0060] According to an exemplary embodiment of the invention, a smart automatic optical inspection and automatic judgement system for manufacturing component carriers may be provided, which may also involve a decision making concerning the necessity of repairing defective semifinished component carriers or panels. In an embodiment, an artificial intelligence working station may be applied to filter defects identified by an automatic optical inspection unit. This may improve quality and may remove manually occurring errors without manual handling by human operators. Such a system may also allow obtaining a better efficiency, in particular a shorter process flow and cycle time. Moreover, head count and machine quantity may be reduced so as to improve the efficiency of manufacturing component carriers.

[0061] In a preferred embodiment, a component carrier or panel may be inspected by an automatic optical inspecting unit to obtain defect data. This may be done by comparing a data set indicative of an actual image of such a semifinished component carrier or panel with a data set indicative of a target or reference image stored in a database. As a next procedure, the semifinished component carrier or panel may be forwarded to an automatic judgement unit which may carry out a judgement of the validity or invalidity of a decision made during automatic optical inspection. As a result of this judgement, an automated optical repair and/or automated optical shaping procedure may be carried out directly, in particular without any manual handling. At the same time, defect data may be transferred to an artificial working station forming part of the automatic judgement unit. Here, actually defective component carriers or panels may be filtered out. The filtered data may be forwarded or used for automated optical repair. During automated optical repair, the data may be verified by catching different images of the component carrier or panel and may then be passed to a second artificial intelligence working station. The second artificial intelligence working station may filter the false defect (for instance classified erroneously as defective, while in fact being intact) semifinished component carriers or panels and may decide how to proceed. Possibilities of proceeding may be in particular shaping, scrapping or the need to verify again. Exemplary embodiments of the invention may be implemented as a full artificial intelligence process or alternatively as a semi-artificial intelligence-based process for quality assessment of semifinished component carriers or panels.

[0062] Advantageously, any mechanical handling of the semifinished component carriers or panels by human operators may become dispensable. Implementing the above-described artificial intelligence working station may reduce the defect quantity for verification. As a result, a fast and reliable automatic optical inspection process flow may be obtained with a full or at least partial automation. This may increase quality and efficiency. Required machine quantity can be reduced. The manufacturing process can be sped up. By obtaining fewer errors (caused by manual classification in conventional systems), the yield may be improved.

[0063] Thus, in particular upstream of back-end processes, i.e. in the front-end before carrying out a solder mask process or a surface finish process, a quality assessment of a semifinished component carrier or panel may be carried out, for instance for each layer thereof. For example, after a copper foil patterning process of a respective layer, it can be checked whether the copper patterning process has been carried out correctly. In this context, the quality of a respective component carrier or preform thereof may be assessed and, if it has been found to be potentially defective, a judgement may be made whether this preliminary classification has been correct and, if appropriate, a repair mechanism may be triggered. For instance, when two traces are erroneously connected, they may be separated during repair.

[0064] A gist of an exemplary embodiment of the invention is an artificial intelligence-based judgement of defects in front-end patterning. In this context, PCB (printed circuit board) related bodies such as semifinished component carriers, array or panels may be inspected. A comparison may be made between a data set indicative of an actual image of such a PCB related body and a data set indicative of a reference image of a PCB related body meeting a specific specification. Then, artificial intelligence may be implemented in an automatic judgement unit for comparing a classification of the semifinished component carriers or the panels on the basis of the previously carried out optical inspection. Then, an action may be taken based on the classification which has been carried out.

[0065] According to an exemplary embodiment of the invention, a smart automatic optical inspection may be combined with a repair system. Advantageously, this may ensure that no manual handing process is necessary in embodiments of the invention. Furthermore, it may be advantageously possible to apply an artificial intelligence working station to reduce the defect quantity for verification. This may guarantee a short, fast and smart automatic optical inspection process flow. Advantageously, this may enable a full automation process, with improved quality and reduced effort in terms of handling. Furthermore, the hardware and manpower effort may thus be reduced significantly. Moreover, the process flow and process time may be reduced, and the efficiency may thus be increased. Less manual error may occur, so that also the excursion may be reduced. Less copper comprising chips without manual scrap may be obtained, which may improve the yield. In particular for producing modules, the described manufacturing method may be in particular suitable (for instance to achieve a thinner thickness, a fine line to space ratio, more cards per array, etc.). Descriptively speaking, a smart working station with a proper filter function may be provided. Furthermore, it may be possible to specifically target a reduction in defects and to identify defects when present automatically.

[0066] FIG. 1 illustrates an apparatus 120 for handling a panel 102 during manufacturing component carriers 100 according to an exemplary embodiment of the invention.

[0067] FIG. 1 shows apparatus 120 for carrying out optical inspection and quality classification of semifinished component carriers 100 of panel 102 during manufacturing of said component carriers 100. The apparatus 120 comprises an automatic optical inspection unit 104 configured for automatic optical inspection of the panel 102 by comparison of a data set indicative of an actual image of a respective semifinished component carrier 100 with a data set indicative of a reference image. Furthermore, the apparatus 100 comprises an automatic judgment unit 106 configured for subsequently carrying out a quality classification based on a result of the automatic optical inspection. For this purpose, the automatic judgment unit 106 may apply artificial intelligence. An action control unit 112 is provided for taking an action in accordance with a result of the quality classification. During operation of the apparatus 100, panel 102 with multiple semifinished component carriers 100 being still integrally connected on panel level may be supplied to the automatic optical inspection unit 104 for automatic optical inspection. This automatic optical inspection may be done based on a comparison of a data set indicative of an actual image of a respective semifinished component carrier 100 with a data set indicative of a reference image. Thereafter, it is possible to forward the inspected panel 102 to automatic judgment unit 106 for carrying out a quality classification based on a result of the automatic optical inspection by applying artificial intelligence. Thereafter, an action may be taken based on the quality classification, when this action may be defined by the action control unit 112.

[0068] In one embodiment, the taken action may be to simply continue the manufacturing process, i.e. to take no specific action deviating from the workflow, when the quality classification indicates no defect of a respective semifinished component carrier 100 or indicates an unrepairable defect of a respective semifinished component carrier 100. The action to be taken may be to repair a respective semifinished component carrier 100 when the latter has been classified as defective, but repairable. For instance, the repairing may comprise separating erroneously connected traces 108, connecting erroneously separate traces 108, repairing an erroneous solder structure (not shown) and/or an erroneous core 110, etc. In an embodiment, it may be possible to take the action of repairing a respective semifinished component carrier 100 on the basis of a data set indicative of a three-dimensional image of the semifinished component carrier 100 or the panel 102.

[0069] Now referring to the various options for the quality classification, it may be possible to carry out the quality classification to classify a respective individual semifinished component carrier 100 or an entire panel 102 as “pass”. Alternatively, it is possible to carry out the quality classification to classify a respective individual semifinished component carrier 100 or an entire panel 102 as “fail”. It may however also be possible to carry out the quality classification to classify a respective individual semifinished component carrier 100 or an entire panel 102 as “repairable” or “to be repaired”. In case of a defect, it may be advantageous to carry out the quality classification by indicating a type of defect among a number of predefined types of defect. Of course, the possible amount of such classification categories is unlimited and can be adjusted to a specific application. For instance, it can be reasonable to grade “repairable” into several subcategories.

[0070] In particular, it may be possible to carry out the described procedure completely without human operator. Alternatively, it is also possible to carry out the described procedure in a semiautomatic way which may involve for instance a human operator for making critical decisions. Optical inspection and at least the primary stage of quality classification may however always be made automatically.

[0071] Carrying out the automatic optical inspection may be performed based on a data set indicative of a two-dimensional image of the entire panel 102 or a respective individual semifinished component carrier 100. In particular, it may be possible to carry out the automatic judgment based on a three-dimensional image of the entire panel 102 or a respective individual semifinished component carrier 100. In case the automatic judgment unit 106 is incapable of performing quality classification (or cannot guarantee sufficient precision), it may also be possible to carry out a further analysis before making a quality classification. Carrying out the further analysis may be done by an additional automatic judgment unit 110 applying artificial intelligence, for instance on a higher level as compared to the artificial intelligence applied by automatic judgment unit 106. Exchanging data between the automatic judgment unit 106 and the additional automatic judgment unit 110 may be performed for learning in a forward and/or a backward direction (wherein learning in the backward direction is preferred). Applying the artificial intelligence may be accomplished by learning based on historical human-based judgments, in particular under consideration of a comparison of judgments proposed by the automatic judgment unit 106 and/or by the additional automatic judgment unit 110 with human-based judgments. It may also be possible to carry out the further analysis based on a data set indicative of a three-dimensional image of a respective individual semifinished component carrier 100 or the entire panel 102.

[0072] It is also possible that the described procedure comprises laser marking of a respective semifinished component carrier 100 or an entire panel 102 based on the quality classification. For instance, defective semifinished component carriers 100 may be marked with a laser engraved cross.

[0073] FIG. 1 shows a cross-sectional view of the panel 102 composed of various semifinished component carriers 100 being still integrally connected within a frame 122 of the panel 102. As can be taken from the illustration of the panel 102 on the left-hand side of FIG. 1, a copper foil on a core 110 has just been patterned to thereby form electrically conductive copper traces 108 on the various semifinished component carriers 100.

[0074] In the automatic optical inspection unit 104, a two-dimensional camera 124 captures a two-dimensional actual image of the surface of the panel 102 and forwards said image data to a first processor 126. The first processor 126 compares the actual image data captured by two-dimensional camera 124 with two-dimensional reference image data stored in a database (not shown). The data concerning the reference image indicate how a surface of the panel 102 should look like if the manufacturing process would have been carried out to comply with a specification. In other words, the first processor 126 determines as to whether the panel 102 meets the specification or fails to meet the specification based on said comparison in the framework of the automatic optical inspection. For example, if panel 102 fails to meet the specification based on said comparison in the framework, a conductive short or open area on panel 102 may be repaired, for instance by a dedicated laser head.

[0075] The result of said automatic optical inspection is forwarded to the automatic judgement unit 106, which can be configured as a further processor. The automatic judgement unit 106 reviews, rechecks or filters the determination made by the automatic optical inspection unit 124. For this purpose, artificial intelligence tools are used by automatic judgement unit 106. Furthermore, it is optionally possible that one or more further cameras 128, 130 capture a data set indicative of a three-dimensional image of the surface of the panel 102 to be inspected, or of only part thereof. Thus, more refined image data can be consulted by automatic judgement unit 106 for deciding about the decision or proposal made by automatic optical inspection unit 104.

[0076] In the shown example, semifinished component carrier 100a has met the specification in the analysis made by automatic optical inspection unit 104. If desired, automatic judgement unit 106 may further analyze the semifinished component carrier 100a. Since in the present example, also the automatic judgement unit 106 comes to the conclusion that the semifinished component carrier 100a meets the specification, it can be classified as “pass”, since it has passed successfully the automatic optical inspection. For semifinished component carrier 100b, the automatic optical inspection unit 104 has identified two separated traces 108 rather than one connected trace (as in the semifinished component carrier 100a), so that the automatic optical inspection unit 104 has classified the semifinished component carrier 100b as “fail”. Reviewing this proposal or decision of the automatic optical inspection unit 104, the automatic judgement unit 106 may determine that the classification of the semifinished component 100b as “fail” may be correct as such (since it does not meet the specification), but may be repaired. In particular, the automatic judgement unit 106 may assess that repairing the semifinished component carrier 100b may be possible by connecting the two erroneously separate traces 108. As can be taken from reference numeral 132 illustrating a repair unit, the semifinished component carrier 100b may be repaired to connect the two erroneously separated traces 108 so as to establish one connected trace 108 in the semifinished component carrier 100b, as shown in the illustration of the panel 102 in the repaired state on the right-hand side of FIG. 1. Furthermore, a semifinished component carrier 100c shows a significant indentation in core 110 and has therefore been indicated by automatic optical inspection unit 104 as defective. This decision of the automatic optical inspection unit 104 can also be reviewed by the automatic judgement unit 106. Analyzing the artefact in the semifinished component carrier 100c by automatic judgement unit 106 under consideration of a data set indicative of three-dimensional image data captured by cameras 128, 130 can result in the judgement that the semifinished component carrier 100c is defective and non-repairable. As a result, the automatic judgement unit 106 may also indicate to a laser marking unit 134 that the semifinished component carrier 100c should be marked with a physical marker 136 as defective and non-repairable. In the shown embodiment, the physical marker 136 is a cross which is laser engraved in the semifinished component carrier 100c. Concerning to a semifinished component carrier 100d, the automatic optical inspection unit 104 has also identified a deviation between a target size and an actual size of the trace 108 on the exterior surface. Descriptively speaking, trace 108 is too large in the semifinished component carrier 100d so that the automatic optical inspection unit 104 considers semifinished component carrier 100d as “fail”. This decision can be reviewed by automatic judgement unit 106, for instance on the basis of an additional three-dimensional image data captured by cameras 128, 130 of the respective surface of the semifinished component carrier 100d. When analyzing the image captured from the surface of the semifinished component carrier 100d, the automatic judgement unit 106 may determine that, although the trace 108 is in fact too large to match with reference image data, it does not contact erroneously any other trace and therefore does not negatively influence the performance of the semifinished component carrier 100d. In this case, the automatic judgement unit 106 may overrule the decision of the automatic optical inspection unit 104 and may classify the semifinished component carrier 100d nevertheless as “pass”.

[0077] As will be understood by a person skilled in the art, the blocks illustrated with reference numerals 132,134 do not need to be one after the other in the process flow. It may also be possible to omit one or both of them.

[0078] If desired, an additional automatic judgement unit 110, which may also implement artificial intelligence and which may comprise at least one processor, may be provided. For instance, it is also possible that the automatic judgement unit 106 comes to the conclusion that the semifinished component carrier 100d with the excessively large trace 108 is a borderline case and that it is doubtful whether the semifinished component carrier 100d should be classified as pass, fail or repair. In such an event, it is possible that a further three-dimensional camera composed of additional camera elements 138, 140 captures a detailed image of the semifinished component carrier 100d for a refined analysis of its classification. The additional automatic judgement unit 110, based on this additional analysis, may then classify the semifinished component carrier 100d selectively as pass, fail or repair.

[0079] In a data feedback loop 142, the automatic judgement unit 106 and the additional automatic judgement unit 110 may exchange data so as to further refine the learning process of their artificial intelligence modules.

[0080] The resulting classification of each semifinished component carrier 100a-100d may be stored in a database 144. For instance, action control unit 112 (which may also comprise one or more processors) may laminate and pattern further electrically insulating layer structures and electrically conductive layer structures on top of the panel 102 shown on the right-hand side of FIG. 1. The procedure of classifying said further layer structures (for instance traces 108) on a surface of such a further processed panel 102 may then undergo again the procedure described referring to FIG. 1.

[0081] FIG. 2 illustrates a flowchart of a semiautomatic method of manufacturing component carriers 100 according to an exemplary embodiment of the invention.

[0082] In FIG. 2, a physical product flow or processing path is denoted with reference numeral 150. As described referring to FIG. 1, the semifinished component carriers 100 of the panel 102 undergo an automatic optical inspection in automatic optical inspection unit 104. Here, a two-dimensional grey image may be captured, and this may be used for the automated defect inspection. The latter process is indicated schematically by reference numeral 152 in FIG. 2. Thereafter, a data analysis flow 154 of the result of the automatic optical inspection is forwarded to first artificial intelligence based working station, i.e. automatic judgement unit 106. As indicated with reference numeral 156, automatic judgement unit 106 carries defect image verification. For instance, a proposal made by the automatic optical inspection unit 104 may be verified (for instance faults, shape, not sure, or scrap).

[0083] A further data analysis flow 158 then logically connects automatic judgement unit 106 with automatic optical repair unit 132. Coming back to the processing of the panel 102 with the semifinished component carriers 100, the panel 102 may undergo, after the automatic optical inspection by automatic optical inspection unit 104, a manual verification process, see block 160. As can be taken from reference numeral 162, the data analysis flow from the automatic judgement unit 106 not only logically connects with automatic repair unit 132, but also with manual verification process unit 162. During the manual verification process, a video and an UV (ultraviolet) image of the panel 102 may be captured, and a verification and classification may be carried out, for instance scrap. In the automatic optical repair unit 132, shaping and repairing of one or more semifinished component carriers 100 of panel 102 may be carried out. Video and ultraviolet image data may be used for this purpose. A previous classification may be verified or a semifinished component carrier 100 may be classified as scrap.

[0084] As shown in FIG. 2 as well, a data communication is possible, along a bidirectional data analysis flow 164 between the automated optical repair unit 132 and additional automatic judgment unit 110. As shown by a block 166, data analysis corresponding to block 156 may be carried out.

[0085] As shown by block 168, the described procedure may be followed by an optional manual verification process. The data analysis made in a block 170 in the manual verification process may be repeated, see block 172.

[0086] Thus, the embodiment of FIG. 2 shows a semi-automated artificial intelligence-based monitoring and inspection process of an exemplary embodiment of the invention.

[0087] FIG. 3 illustrates a flowchart of a fully automatic method of manufacturing component carriers 100 according to an exemplary embodiment of the invention.

[0088] A difference between the embodiment of FIG. 2 and the embodiment of FIG. 3 is that, in the embodiment of FIG. 3, a fully AI (artificial intelligence) based optical inspection and repair system is provided. The automation process is continuous, as indicated by arrows 180, 182.

[0089] It should be noted that the term “comprising” does not exclude other elements or steps and the article “a” or “an” does not exclude a plurality. Also, elements described in association with different embodiments may be combined.

[0090] Implementation of the invention is not limited to the preferred embodiments shown in the figures and described above. Instead, a multiplicity of variants is possible which variants use the solutions shown and the principle according to the invention even in the case of fundamentally different embodiments.