Computer-implemented method of automatically generating inspection templates of a plurality of known good fasteners
11543364 · 2023-01-03
Assignee
Inventors
- Michael G. Nygaard (Grand Blanc, MI, US)
- Nathan Andrew-Paul Kujacznski (Swartz Creek, MI, US)
- Christopher Michael Alexander (Fenton, MI, US)
- Laura L. Poletti (Grand Blanc, MI, US)
- Gregory M. Nygaard (Clarkston, MI)
- James W. St. Onge (Bloomfield Hills, MI, US)
Cpc classification
G05B2219/32179
PHYSICS
Y02P90/02
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
G01N21/9515
PHYSICS
G01N21/952
PHYSICS
International classification
Abstract
A computer-implemented method of automatically generating inspection templates of a plurality of known good fasteners to identify fasteners at an inspection station is provided. The method includes providing a data entry mechanism to provide content needed to identify a plurality of unidentified mixed fasteners. The method also includes storing the content in a database, extracting the content from the database and creating the inspection templates from the extracted content. Each of the templates including a fastener profile and a set of features. Each of the features includes a range of acceptable values. Each of the templates has a fastener identification code associated therewith.
Claims
1. A computer-implemented method of (i) automatically generating inspection templates of a plurality of known good fasteners to identify a plurality of unidentified mixed fasteners at an inspection, station, and (ii) sorting the plurality of unidentified mixed fasteners at the inspection station, the method comprising: inputting data into a database via a user interface to provide content needed to identify the plurality of unidentified mixed fasteners; storing the content in the database; extracting the content from the database; creating the inspection templates from the extracted content, each of the templates including a fastener profile and a set of features wherein each of the features includes a range of acceptable values and wherein each of the templates has a fastener identification code associated therewith; assigning fastener identification codes to bins in response to generating each inspection template having an associated fastener identification code, wherein the bins are configured to receive subsets of the plurality of unidentified mixed fasteners; optically inspecting (i) the fastener profile and (ii) the set of features of each fastener of the plurality of unidentified mixed fasteners at the inspection station; assigning a first of the fastener identification codes to each fastener within a first of the subsets of the plurality of unidentified mixed fasteners that have fastener profiles and sets of features within the acceptable values according to a first of the templates during the optical inspection; assigning a second of the fastener identification codes to each fastener within a second of the subsets of the plurality of unidentified mixed fasteners that have fastener profiles and sets of features within the acceptable values according to a second of the templates during the optical inspection; directing each fastener of the first of the subsets of fasteners to a first of the bins, wherein the first of the bins is assigned the first of the fastener identification codes; and directing each fastener of the second of the subsets of fasteners to a second of the bins, wherein the second of the bins is assigned the second of the fastener identification codes.
2. The method as claimed in claim 1, further comprising displaying the templates on a user interface.
3. The method as claimed in claim 1, wherein each of the templates includes at least one defect detection region.
4. The method as claimed in claim 1, wherein the data entry mechanism comprises a spreadsheet.
5. The method as claimed in claim 1, wherein the content comprises print specifications organized by fastener family type.
6. The method as claimed in claim 5, wherein the print specifications include part member, part family, measurements and attributes.
7. The method as claimed in claim 1, wherein the database is an object-oriented database.
8. The method as claimed in claim 1, wherein the content includes data and wherein the method includes the step of populating fastener objects with the data.
9. The method as claimed in claim 8, further comprising generating an overlap report from the data and utilizing the overlap report to sort the fastener objects.
10. The method as claimed in claim 1 further comprising: assigning a reject code to each fastener within the plurality of unidentified mixed fasteners that have sets of features outside of the acceptable values of each template during the optical inspection; and directing each fastener assigned the reject code to a reject bin.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
DETAILED DESCRIPTION
(30) As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention.
(31) Referring now to
(32) Some of the many benefits of the use of the system 10 and its corresponding method include but are not limited to: eliminate or reduce waste; time savings; improve quality; and positive impact on the bottom line.
(33) In other words, the system 10 or machine is a technical system which can be used to recertify aerospace fasteners 36 as well as self-supporting parts. Bolts, nuts, and collars are introduced into the system for the purpose of part identification and certifying parts which fall into one of two classes. The first class is new and unused parts. These parts are capable of being re-introduced into inventory. The second class is parts that are not fit to be used and must be removed. The system 10 includes many devices and subsystems working together to create seamless, consistent, automated and accurate results.
(34) Referring now to
(35) Parts are then fed through a gentle-handling, vibratory carpet feeding subsystem thereby transporting parts into multiple lanes. The pre-screen subsystem is a multi-lane vibratory v-shaped transport system that moves parts into single files, gently transitioning them onto a pick conveyor 19 (i.e. “3” in
(36) The pick conveyor 19 is a multi-lane conveyor controlled by the master controller (
(37) The vision-guided robots 21 have the ability to pick up any part within a specified range of allowable parts using multiple end-of-arm tooling or grippers 17. The robots 21 pick up the bolts, nuts and collars and orient them at two inspection stations, each of which is referred to as a VisionLab. Each of the parts has a part axis and each of the inspection stations has a measurement axis 13 (i.e.
(38) The robots 21 are preferably six axis robots located adjacent the pick conveyor 19 and at the VisionLab inspection stations. Each robot 21 is vision-guided to identify, pick, orient, and present the parts 36 “head down” so that they are self-supporting on the VisionLab glass stage 14. The grippers 17 accommodate multiple part families.
(39) Benefits of Vision-based Robot Automation include but are not limited to the following:
(40) Smooth motion in high speed applications;
(41) Handles multiple parts 36;
(42) Slim designs to operate in narrow spaces;
(43) Integrated vision; and
(44) Dual end-of-arm tooling or grippers 17 designed to handle multiple part families.
(45) A master control station or system controller (
(46) In some embodiments, multiple cameras such as the cameras 32 can be situated at fixed locations on the frame structure 34 at the inspection stations, or may be mounted on the arms of the robot 21.
(47) The master controller at the master control station can include a processor and a memory on which is recorded instructions or code for communicating with the robot controllers 23, the vision systems, the robotic system sensor(s), etc. The master controller is configured to execute the instructions from its memory, via its processor. For example, the master controller can be a host machine or distributed system, e.g., a computer such as a digital computer or microcomputer, acting as a control module having a processor and, as the memory, tangible, non-transitory computer-readable memory such as read-only memory (ROM) or flash memory. The master controller can also have random access memory (RAM), electrically-erasable, programmable, read only memory (EEPROM), a high-speed clock, analog-to-digital (A/D) and/or digital-to-analog (D/A) circuitry, and any required input/output circuitry and associated devices, as well as any required signal conditioning and/or signal buffering circuitry. Therefore, the master controller can include all software, hardware, memory, algorithms, connections, sensors, etc., necessary to monitor and control the vision subsystem, the robotic subsystem, etc. As such, a control method can be embodied as software or firmware associated with the master controller. It is to be appreciated that the master controller can also include any device capable of analyzing data from various sensors, comparing data, making the necessary decisions required to control and monitor the vision subsystem, the robotic subsystem, sensors, etc.
(48) An end effector on the robot arm may include a series of grippers 17 supported to pick up the fasteners 36. The robotic arm is then actuated by its controller 23 to pick up the fasteners 36 with the particular gripper 17 from the conveyor 19, positioning the gripper 17 relative to the fasteners 36 using the determined location from the visual position data of the particular vision subsystem including its camera 32 and image processor (
(49) In general, each VisionLab inspection subsystem is a inspection subsystem equipped with a rotating glass stage 14. Vision Lab inspection includes 360 degrees of coverage for dimensional and visual defect purposes. Each VisionLab subsystem identifies and inspects the given self-supporting part/fastener 36 based on criteria specific to the part.
(50) Each VisionLab may have:
(51) Side, top and bottom view high resolution cameras, lenses (some of which are electronically-controlled liquid lenses having a variable focal length so that fasteners of various sizes and shapes can be identified via inspection) and lighting (bottom view through VisionLab's rotating sapphire glass support or stage 14);
(52) Color vision;
(53) Sequenced back and front lighting for 360° dimensional and visual defect inspection;
(54) Internal view of some fasteners 36; and
(55) Smart motor and software combined to provide measurements and visual inspection every 1 degree of rotary motion for a total of 720 inspection points.
(56) Referring now to
(57) Capabilities of the VisionLab station or subsystem (i.e. ARIS Inspection Station) include:
(58) Side-View Dimensional Characteristics
(59) Diameters
(60) Threads
(61) Tapers
(62) Lengths
(63) Concentricity
(64) Straightness
(65) Radii
(66) Length from specified diameter (head protrusion)
(67) Diameter from specified length position
(68) Parallelism/Perpendicularity
(69) TIR
(70) Theoretical Intersection
(71) Formula
(72) Surface Characteristics
(73) Seams
(74) Cracks
(75) Visual Defects
(76) Internal Thread Inspection (with the liquid lens technology)
(77) Color Camera (with the liquid lens technology)
(78) Plating Inspection, color verification
(79) Top and Bottom-View Inspection (top view with the liquid lens technology)
(80) Head Stamp Verification
(81) Visual Defects (bottom surface)
(82) Head Diameter
(83) Head Roundness
(84) Drive/Recess Inspection
(85) Referring now to
(86) Example self-standing parts, such as threaded parts with ball-shaped end portions and a threaded fastener or bolt, has threads, a length between its ends, a width, and a part axis which, preferably, is central to the part and parallel to its length. A variety of manufactured parts which may be inspected are shown in the drawing figures, including FIG. 1 of the '297 patent. In one example embodiment, the parts may have a length of 10″, a diameter of 2″, a length repeatability of 10 microns, a diameter repeatability of 2 microns, and an inspection speed of about 7 parts per minute. Also, typically the self-standing parts are capable of rotating without falling over (i.e. without the need for a part fixture).
(87) The threaded bolt 36 is supported on a transparent (i.e. scratch-proof glass, plastic, or sapphire) window 11 of the motorized rotary or rotation fixtureless stage 14. The part is able to stand on and rotate with the window 11 without falling over (i.e. the part is capable of staying up or upright) without the need for a part fixture (i.e., the part is self-supporting).
(88) As shown in
(89) The weight sensor 16 may be implemented as a strain gauge including a strain gauge load cell and resistive wire connected to a load plate coupled to the bottom surface of the window 11. The load plate and the window 11 typically displace a small distance corresponding to the part weight.
(90) The weight sensor 16 alternatively may be a piezo-based weight sensor, a QTC sensor, a capacitance-based weight sensor or an inductance-based weight sensor to measure microgram or milligram changes in weight caused by the weight of the part on the top surface of the window 11.
(91) Because the part is self-supporting on the transparent window 11 of the stage 14 without the need for fixtures or other devices, top and bottom cameras and corresponding strobed illuminating LED ring lights are provided to obtain top and bottom end views of the part, respectively. The rotation stage may be a precision rotation stage having relatively large (i.e. 100 mm) central aperture over which the transparent window is fixedly secured to rotate with an annular plate. An encoder 15 (i.e.,
(92) The bottom vision subsystem includes the bottom lighting and the bottom camera 210 both of which are located below the glass window as shown in FIGS. 2 and 9 of the '297 patent at a bottom imaging location. The camera 210 is preferably a single view camera with image analysis software that minimizes surface and lighting variations. The lighting illuminates the part.
(93) Detects which the bottom vision subsystem can detect with respect to the part:
(94) Min thru-hole;
(95) Cracks on flange;
(96) Functional OD Hex;
(97) Cracks on ball portion;
(98) Flange side ID cracks; and
(99) Top and bottom ID crack.
(100) The illuminator and the lens detector subsystem (i.e. camera) may be replaced by a high-speed 2D/3D laser scanner available form Keyence Corporation of Japan.
(101) The rotary part stage 14 typically includes an electric actuator or motor which may be stepper motor or a DC motor.
(102) The parts of FIG. 1 of the '297 patent can be rotatably driven by the motor via motor-driver or controller via system controller while being supported vertically (i.e., the parts are self-supporting) as shown in FIG. 9 of the '297 patent.
(103) The various components or functions of the motor driver or controller of FIG. 9 of the '297 patent may be implemented by the separate motor controller as illustrated, or may be integrated or incorporated into the system controller, or other controller, depending upon the particular application and implementation. The MCU (i.e. motor control unit) typically include the control logic to control the rotary stage 14. The control logic may be implemented in hardware, software, or a combination of hardware and software.
(104) One or more memory devices within the system controller and/or the motor controller may store a plurality of activation schemes for the rotatable window 11 and may represent any one or more of a number of known processing strategies such as event-driven, interrupt-driven, multi-tasking, multi-threading, and the like. As such, various steps or functions may be performed in sequence, in a modified sequence, in parallel, or in some cases omitted. Likewise, the order or operation or processing is not necessary required to achieve the objects, features, and advantages of the invention, but is provided for ease of illustration and description.
(105) Preferably, the control logic is implemented primarily in software executed by a microprocessor-based controller or the microcontroller (i.e. MCU). Of course, the control logic may be implemented in software, hardware, or a combination of software and hardware depending upon the particular application. When implemented in software, the control logic is preferably provided in a computer-readable storage medium having stored data representing instructions executed by a computer to control angular position and rotation of the window 11 of the system 10 through the rotary stage 14. The computer-readable storage medium or media may be any of a number of known physical devices which utilize electric, magnetic, and/or optical devices to temporarily or persistently store executable instructions and associated calibration information, operating variables, and the like.
(106) In one example embodiment, the stage 14 is electromechanically driven by a rotary actuator such as the DC motor 28 and associated transmission (not shown) in the form of a worm gear or the like. The DC motor may be, for example, a brushed or brushless DC servomotor, the operation of which is controlled by the motor controller via a motor drive or driver within the motor controller. The brushed or brushless motor may have its rotary speed and position controlled by pulse width modulation (PWM) control.
(107) The motor controller outputs motor drive commands to the DC motor based on outputs from the rotary encoder 15 and/or decoded commands from the system controller. The motor controller controls the DC motor through the motor drive of the motor controller so that the angular position of the stage 14 is changed. In other words, the system controller outputs servomotor drive commands to the motor controller which controls the DC motor and, through its transmission, the stage 14.
(108) As described in U.S. Pat. No. 8,550,444 (also owned by the assignee of the present application), the system 10 may include a part-centering and aligning subsystem. The subsystem or apparatus ensures that a part is centered in the system 10 and that the part is aligned with the measurement axis (Z-axis) 13 without the need to measure any distances or angles. In other words, the apparatus ensures that the part is properly placed or positioned on the window 11 in the system 10.
(109) As described in U.S. Pat. No. 8,550,444, the part-centering apparatus typically includes a carrier which defines a part receiving cavity. The apparatus also has a central axis substantially parallel to the measurement axis 13 or Z-axis and includes a plurality of members or levers having open and closed positions. The members having holding faces which are substantially equidistant from the central axis during movement between the open and closed positions. At least one of the members applies a force on an exterior side surface of a part, disposed between the holding faces during movement between the positions to reposition the part. The repositioned part is centered and aligned with respect to the measurement axis 13. The holding faces releasably hold the repositioned part in the holding position between the open and closed positions of the members.
(110) The part-centering apparatus may also typically include automatically operable lever arms which are coupled to their respective relatively moveable, spring-biased ring members of the carrier. Movement of one of the lever arms either towards or away from the other lever arm (depending on the biasing of the spring(s)) causes the members to move from their open position to their holding position against the part to center and align the part.
(111) The system 10 may also include a moveable stage subsystem coupled to the part-centering apparatus for sliding the apparatus relative to the repositioned part along the central or measurement axis in the open position of the members to allow the exterior side surface of the repositioned part to be measured. In turn, the slide/base unit moves the moveable stage subsystem up and down. A horizontal support member couples the subsystem to the apparatus to move the apparatus along the central or measurement axis 13.
(112) The system 10 may further include a mechanism which is coupled to one end of the support member for translating the support member and the apparatus a limited extent relative to the subsystem along the central axis.
(113) Referring again to FIGS. 3, 9 and 10 of the '297 patent, the system also includes a backside illumination assembly, generally included at 300. In general, back lighting provides measurement of profile characteristics. This provides maximum, minimum or average measurements, simultaneously or separately for features like: Radii Concentricity, Straightness, Lengths, Diameters and Threads for threaded parts.
(114) The illumination assembly 300 directs a beam of collimated radiation at substantially the entire backside surface of the self-supporting part 36 at predetermined angular increments of movement of the part about the measurement axis 13 of the system 10 during a rotational scan. The beam is occluded by the self-supporting part at each increment of movement to create a stream of unobstructed portions of the beam in rapid succession passing by and not blocked by the self-supporting part.
(115) Preferably, substantially the entire backside surface is completely enclosed by a beam profile of the beam. The beam profile is generally rectangular with a height greater than or equal to the length of the part and a width greater than or equal to the width of the part as shown in
(116) The assembly 300 may be moveable up or down via a motor driven or controlled by a driver/controller upon receiving a control signal from the system controller as shown in U.S. Pat. No. 9,370,799.
(117) The illumination assembly or radiant source 300 illuminates an object such as a threaded bolt to be imaged, and a telecentric optical lens 302 (i.e. FIGS. 2, 3, 9 and 10 of the '297 patent) receives the radiation passing by and not blocked by the part and guides it towards an image plane of the image acquisition device or detector, generally referred as 304. Consequently, the radiation source 300 preferably comprises a LED emitter including a plurality of LED emitter elements serving to emit radiation in either the visible or ultraviolet range. The LED emitter of the source 300 is preferably high power, capable of generating 100 optical mW or more for each emitting element. A lens (not shown) collimates the radiation.
(118) As shown in U.S. Pat. No. 9,370,799, the back light 300 and the detector 304 may be coupled together by a yoke to rotate together about the part 36 via a motor via a driver/controller upon receiving a command signal from the system controller.
(119) An optical or optoelectronic device for the acquisition of images (for example the camera or telecamera 304) has the image plane which can be, for example, an electronic sensor (CCD, CMOS). The self-supporting fastener, bolt or other manufactured part, is received on an retained at a position and orientation for optical inspection by the fixtureless transparent glass or plastic of the window 11 of the system 10. Preferably, the device 304 is a high speed, high resolution digital telecamera, having an electronic sensor with individual pixels of lateral dimensions equal to or less than one or more microns.
(120) As shown in U.S. Pat. No. 9,370,799, the assembly 304 can be driven up and/or down by a motor via a driver/controller upon receiving an appropriate control signal from the system controller. Typically, movement of the assembly 304 and the backlight 300 is coordinated by the system controller so that they move in unison.
(121) As described in U.S. Pat. No. 9,370,799, the lens 302 typically comprises a forward set of optical elements proximal to the manufactured part, a rear optical element proximal to the acquisition device and an aperture diaphragm interposed between the forward and rear sets of optical elements. The aperture diaphragm comprises a circular window transparent to the radiation, which is referred to as a diaphragm aperture. For example, the aperture diaphragm can comprise an opaque plate preferably of thickness of a few tenths of a millimeter, and the diaphragm aperture can be defined a simple hole in the plate.
(122) The diaphragm aperture or window is coaxial to the optical axis of the forward set of optical elements, and positioned on the focal plane of the forward set defined for the wavelength range of radiation emitted by the radiant source.
(123) The position of the focal plane of a set of optical elements mostly depends on the refraction index of the material from which the lenses are made, which, in turn, depends on the wavelength of the electromagnetic radiation passing through the lenses.
(124) The lenses 302 only accepts ray cones exhibiting a main (barycentric) axis that is parallel to the optical axis of the forward set. Thereby the lens 302 is a telecentric lens configured for the particular radiation. The rear set of optical elements serves to compensate and correct the residual chromatic dispersion generated by the forward set optical elements for the wavelength in question.
(125) The optical axis of the rear set coincides with the optical axis of the forward set and the focal plane of the rear set defined for the wavelength cited above, coincides with the plane on which the aperture diaphragm is located. Consequently, rays of radiation conveyed by the rear set towards the image lane form light cones, the main (barycentric) axis of which is parallel to the optical axis of the lens 302.
(126) The forward set preferably includes two positive lenses, which can exhibit a flat-convex, bi-convex, or meniscus shape. The positive lenses can both be made in common optical glass. For example, they can both be made in low chromatic dispersion crown glass.
(127) The rear set of optical elements preferably comprises four lenses. The lens which is proximal to the diaphragm can be a negative lens serving to partially or completely correct the chromatic aberrations generated by the forward set. The negative lens can be bi-concave, flat-concave, or meniscus shaped, and can be made of common optical glass. For example, it can be made of high chromatic dispersion flint glass.
(128) The three rear lenses are positive lenses that can all be made of optical glass, for example, in low chromatic dispersion crown glass.
(129) The lens 302 is therefore both telecentric on the object side and telecentric on the image side, and overall the lens 302 is a bi-telecentric lens configured for light such as visible light or ultraviolet light.
(130) Dimensional features that can be measured via the above-described side vision devices include:
(131) Diameters;
(132) Tapers
(133) Lengths
(134) Concentricity
(135) Straightness
(136) Parallelism
(137) Perpendicularity
(138) Threads
(139) Pitch
(140) Major Dia
(141) Pitch Dia
(142) Referring now to FIGS. 9 and 10 of the '297 patent, there is illustrated a triangulation-based sensor head, generally indicated at 400. Alternatively, the device 400 may comprise a color camera having a liquid lens. The offset color camera 400 preferably utilizes liquid lens technology and advanced image analysis software that minimizes surface and lighting variations. A strobe LED ring light illuminates the outer diameter surface for dedicated coating/color inspection. The liquid lens technology allows the camera to stay stationary and not move because the effective focal length of the camera lens can be electronically controlled. Color inspection is thus provided for proper fastener coating, color and quality.
(143) The sensor head 400 may comprise a high-speed, 2D/3D laser scanner available from Keyence Corporation of Japan. Such a sensor head from Keyence generates a laser beam that has been expanded into a line and is reflected from the side surface of the part as well as any radially extending surfaces of the part, such as the threaded bolt 36. The reflected line of light is formed on a sensor, and by detecting changes formed on a sensor, and by detecting changes in the position and shape of the reflection, it is possible to measure the position of various points along the surface of the part.
(144) Alternatively, during the scans of the side profile, a laser line may be painted on the part and a vision subsystem positioned on either side of the laser line receives reflected laser light and the resulting images provides a 3D image (including Z axis or depth). In this way, both visual defects and measurement features or characteristics that require a depth component are simultaneously extracted.
(145) Alternatively, (not shown herein but shown in U.S. Pat. No. 9,370,799 the entire disclosure of which is hereby incorporated by reference herein), the sensor head 400 may rotate and/or linearly move via a motor via a rotary driver/controller and/or a linear driver/controller, respectively, upon receiving command signals from the system controller. A transmission may convert the rotary motion of the motor output shaft to linear motion.
(146) As the manufactured part rotates, corresponding sets of 2D profile signals are generated by the sensor head 400. At least one processor processes the sets of 2D profile signals to obtain a 3D view of the complete side and any radially extending surfaces of the part.
(147) The system controller provides control signals based on the signals from the rotary sensor or encoder 15. Alternatively, sensor(s) and/or encoder(s) are not required if stepper motor(s) are provided. Alternatively, or additionally, the signals from the rotary encoder 15 are directly utilized by the sensor head 400 at the station to control the sensor head 400. The control signals are utilized to control the sensor head 400 which preferably have encoder inputs which allow precise control over the position of 2D profile signals samples.
(148) At least one signal processor may process the sets of 2D profile signals to identify a defective part as described in greater detail in U.S. Pat. No. 9,370,799, also owned by the assignee of the present application. The at least one processor may process the sets of 2D profile signals to obtain one or more measurements of the part.
(149) The operator may tell the system controller via a display 500 and user interface 502 (i.e.
(150) The 2D profile signals may be processed by the at least one processor under system control to obtain a 360 degree panorama composite view or image which is used by the processor to determine at least one of a dent, a split, a perforation, a crack, a scratch, a wrinkle, a buckle, or bulge, and a surface blemish located at the side surfaces of the part where the part is an ammunition case or a fastener 36.
(151) A top vision subsystem typically includes the frontside illumination device which may include the strobed ring LED illuminator 350. The illuminator 350 typically includes a curved array of LED light sources, groups of which are under control of the system controller to provide direct illumination of the front of the case or fastener 36 and are used to enhance defects in the front surface of the fastener or bolt 36. Alternatively, the frontside illumination device may be side-mounted so that the front light comes from the side of the part and not from above the part, i.e., basically like painting a thin line along the length of the part.
(152) The top vision subsystem also typically includes a single view camera with image analysis software that minimizes surface and lighting variations. A lens of the camera may also utilize liquid lens technology as described above. Lighting which illuminates the part includes software. The lens of the camera may be up to 2 inches in diameter. The top vision subsystem may detect such defects as castle chip out; functional ID Hex; and top and bottom ID cracks. Front lighting provides surface defect detection for tool chatter, cracks and other surface imperfections.
(153) The detected optical images are processed by the image processor to determine at least one of a dent, a split, a perforation, a crack, a scratch, a wrinkle, a buckle, a bulge, and a surface blemish located at the side surfaces of the fastener.
(154) Referring to
(155) Referring to
(156) Data/Image Processor for the Detection of Surface Defects on Small Manufactured Parts
(157) The vision system is especially designed for the inspection of relatively small manufactured parts (i.e. especially the parts of
(158) At least one embodiment of the present invention utilizes many of the teachings found in U.S. Pat. No. 7,633,635 ('635 patent), assigned to the assignee of the present application to automatically identify unidentified mixed fasteners.
(159) This embodiment (of the '635 patent) has a part setup procedure for a user to capture an image of a known dimensioned part and define a set of features with acceptable range of limit values for them. The part profile and features are referred to as the part template. During part ID mode, the profile of each inspected part is captured and its features are compared to their limit values. If any feature of a part is determined to be outside its range of limits, then it is not identified.
(160) Referring to FIG. 1 of the '635 patent, there is illustrated one embodiment of a system, generally indicated at 20, for automatically identifying non-identified fasteners. Such fasteners may include, as illustrated in
(161) The data and signal processing system described therein (i.e. the '635 patent) illustrates how the system processes sensor data and discovers the ID of the fastener presented to the system. Using calibration data, sensor data is transformed to a description of the outline of the fastener, specified in calibrated physical coordinates. Feature processing extracts values for each feature contained in the entire part template data set. Match metric processing identifies the best match to the sensor data among the fastener templates. ID generation evaluates the best match; if the match is good enough, the part is said to be identified, otherwise the part is not identified. After ID generation, a message is sent to the master computer containing the part ID or a “not matched” indication.
(162) When a new part is added to the system 20, a file called a “template” is automatically created as described in detail hereinbelow. The template file contains information about the fastener that is used to identify it. The template is set up so that any fastener of the given type will match the template, and any fastener not of the given type will not match.
(163) When a robot 21 of the present application places a self-supporting fastener to be identified on its fixtureless stage 14, the software acquires data containing the profile and eddy signature of the fastener. The software then checks all the templates in the list to find a match. If the fastener matches one of the templates, then the fastener is identified. If none of the templates match, the fastener is not identified.
(164) In general, when setting up a new fastener, the user chooses “features” of the part to be measured. The measurements of the features will distinguish the new fastener from the other fasteners in the system. The types of features include total length, internal length, diameter, thread, taper, and eddy current signature. For most features, the user chooses a region of the fastener where the measurement will be made, a nominal value of the measurement (the value the fastener should have if it's the right fastener, and plus and minus tolerances which determine if the measurement is close enough to match the fastener. For some features, such as total length and eddy current, the measurement region is the whole fastener. Also, for eddy current the user chooses a rectangular on the eddy screen of a display instead of a nominal value and tolerances. If the eddy signature hits the rectangle, then the fastener is a match.
(165) The user chooses which features are needed to distinguish the new fastener. For a wire nut, for example, the user would typically add a total length feature and a taper feature. A bolt may need total length, thread, and one or two diameters. If it is necessary to distinguish the type of material or coating to distinguish a bolt from one another bolt, the user would add the eddy feature.
(166) When all of the necessary features have been set up automatically in a template, the user saves the template. This is added to the list of templates to check when a fastener is in the part identification operation, as previously described.
(167) More particularly, in automatically creating a template a gold or master part with known good dimensions is not typically used. An image of the fastener is displayed on a screen, as generally shown in the screen shot of FIG. 6 of the '635 patent.
(168) After a good image of the part is automatically obtained, features automatically are added to the template as previously mentioned. For example, when adding an internal length, points are determined on the fastener when one wishes to measure the internal length (i.e., here the length of the head of the bolt). One can add multiple internal lengths for each fastener. Internal lengths can be use to measure features like: thread length, shoulder length, head height, under the head to the start of a part, and any length measurement needed inside of a fastener.
(169) Such predefined points are also useful for other template features like diameters and tapers. Such predefined points are useful when looking for rising and falling edges of the fastener as well as when looking for minimum and maximum diameters of the fastener.
(170) The diameter feature is used to measure diameters around the fastener. Multiple diameters can be added for each fastener. One can select minimum and maximum diameters for a selected area (or a small groove within a selected area) or one can average all the diameters in the area selected.
(171) With respect to taper features, tapers are used to measure tapered angles on a fastener. Multiple tapers can be added for each fastener.
(172) The external/overall length feature is automatically added to the list of features. With respect to the thread features, the tolerances on the following thread features can change: thread count, thread pitch, pitch diameter, functional size, lead deviation, minor diameter, and major diameter.
(173) With respect to the code, the designated code can be entered on a touch screen to identify the fastener corresponding to a particular template, which also shows various features of a bolt.
(174) With respect to eddy current, a frequency parameter is initially set up for a particular fastener. A relatively low frequency such as 1 KHz may be used to check for material and a relatively high frequency such as 50 KHz may be used to check for plating of a fastener. A template for eddy current is automatically generated. After obtaining a signature, one may have to adjust the parameters of the frequencies and the gains while testing a good fastener, until a good image is obtained on the screen of a display. A good image should have a defined area, like a loop, that will have some space inside it. After establishing the eddy current signature of a good fastener, the area of the signature one wants to inspect may be highlighted.
(175) Referring now to
(176) There are two linear slides or post inspection conveyors 44, one for each VisionLab output. These slides 44 receive the parts into basket-like, transparent devices 45. Each part is transported to one of a plurality (i.e. here 40) of bin locations 49. Once a transport device 45 arrives at the proper bin location 49, an electric part pusher cylinder 46 pushes the part out of the basket-like device 45 and into one of six locations 49 of a bin module assembly 48 (“7” in
(177) Good Part Transport to Holding Bin Location of Bin Assembly
(178) The high speed linear slides 44 transport accepted parts in holding cups or devices 45 that the parts fall into after the VisionLab inspection. One high-speed slide 44 is dedicated to each VisionLab. Each transport cup 45 takes each accepted part to the assigned bin 49 of one of the bin module assemblies 48 and the cylinder 40 pushes the part, allowing the part to transition into a particular bin location 49 within the designated assembly 48. The transport device 45 then returns via its slide 214 to retrieve the next accepted part from its VisionLab.
(179) Reject Chute
(180) Bad parts are sorted out to a reject bin (not shown). Rejected parts will drop into the first “bin” location which is designated for fail parts only “containment bin.”
(181) Each of the good bin locations 49 will know what part number and quantity reside in that bin location 49 via the master controller. Once the pre-determined quantity of parts has been met at a particular bin location 49, the floor 51 of that bin location 49 will be slid out by an electronic cylinder 51 having a gripper allowing the parts from that one bin location 49 to drop onto a bagging conveyor 53 (i.e. “8” in
(182) The bagging conveyor 53 then transports the parts to a bagger machine of a bagging system 52 (i.e. “9” in
(183) Good Part Holding Station/Bins
(184) Once the parts are removed from its bin location 49 and transported to the bagging machine, the bin location 49 becomes available for the next successfully passed part to occupy.
(185) Conveyor System Transfers Parts to Bagger Machine
(186) Full bins 49 are opened to the bottom bagging conveyor 53. Once a bin location 49 is emptied, it is assigned a new part number.
(187) Bagging System
(188) Good parts enter the bagging system 52 from the transfer or bagging conveyor 53.
(189) The bagging system 52 prints labels with part number and lot information, places the labels on the bags and finishes with sealing.
(190) Referring now to
(191) 1. Customer Provides Part Print Data in a pre-defined Spreadsheet Format of
(192) 2. Spreadsheet Data Uploaded in to a VisionLab Database of
(193) An import utility takes the spreadsheet data and converts it into a format that the database can use.
(194) 3. Database data is transferred into VisionLab Fastener Inspection Templates by Part Number
(195) A physical fastener from a family of fasteners can be utilized to “train” and set up the template. The software uses the information in the database to take the “trained” regions and automatically grow or shrunk them depending on the size of the fastener.
(196) Fastener templates created through the spreadsheet data transfer process described above include:
(197) 1. Part Number
(198) 2. All Selected Features
(199) 3. All Dimensions and Tolerances
(200) 4. Visual Defect Detection Regions of Interest and Level Settings
(201) As noted in
(202) As further described, the spreadsheet is created and provides all the data needed to identify the part including: Part number, Part Family, Measurements and Attributes. Software is used to read each line of the spreadsheet and populate the information into objects. The data is dynamically read as there can multiple options for each bolt, nut and collar that change the finish of the part or modify the measurements.
(203) After all the data is read from the spreadsheet, it is populated into the database using 4 normalized tables. The first table stores each family type with the date it was added into the system and the type of part that family provides (bolt, nut, collar, sleeve, etc.). The second table stores the full part number, and each value of the part number broken down. Lastly, the data for each part is stored between two tables. One table stores the measurement values as minimums, nominals, maxes, ranges and the other table stores attribute values such as true/false, colors and string values that match items such as head types.
(204) Lastly, using the data an overlap report is generated. The overlap report sorts all the part objects and compares the measurement values to find where all the measurements are an exact match.
(205) General Machine Overview
(206) As previously described, a vibrating feed table divides the parts by part height into 3 lanes. The parts are vibrated down each lane onto a slow-moving conveyor. Parts go through an optical sensor such as a camera that allows robots to identify the orientation and length of the part. The robots will then move into position to pick up the part. There are 3 grippers on each robot that allow them to pick up a range of part sizes. The robot will move with the correct gripper to grab the part and place it onto the center of one of the Vision Labs. Each robot has its own Vision Lab. The Vision Lab stage will then spin and images of the part are collected to identify it. After the Vision Lab has completed its cycle and has all its results, it will push the part off the table or stage and spin to dump the part down a chute. After the part falls it will be added into the queue station. The queue station can hold one part.
(207) A cup attached to a slide will collect the part from the queue and deliver it to the appropriate bin location. Each bin location will hold one type of part. When the cup gets to the correct bin it will open to allow all the parts to fall into the bin. There are 42 bins in total: 1 bin for failed parts, 1 bin for dumped parts, and 40 bins for good identified parts.
(208) When bins get full, or the number of bins runs out, the fullest bin is selected according to their max part count, another slide will go to the appropriate bin and pull a trap door to release the parts onto a lower conveyor. The parts will be pushed from the conveyor to a bagger which will be printed with a label. The label will contain the part information, a timestamp and how many parts are in the bag. The bagger will cycle and drop the bag onto a final conveyor which will take the recertified parts away.
(209) Referring now to
(210) At block 50, a top part is found that is closest to a number of measurements.
(211) At block 52, if the part fails the “foreign” part check (i.e. the part does not have a corresponding template), the part is dumped at block 54. If the part doesn't fail the check, block 56 is entered.
(212) At block 56, the question is asked if this is the only part found that matches the measurements. If yes, the part is identified at block 58. If not, block 60 is entered.
(213) At block 60, the list of matching measurement parts is narrowed down using identified attributes.
(214) At block 62, if there is only one part remaining, the part is identified at block 64. If not, the part is dumped at block 66.
(215) In summary, the system 10 includes in each VisionLab inspection station:
(216) Side Vision Profile Dimensional Inspection and Visual Defect Detection
(217) Bottom Vision with Front LED Lighting
(218) Top Vision with Front LED Lighting
(219) InternalView with Front LED Lighting
(220) Color Vision with Front LED Lighting
(221) MultiView Back Lighting: At the heart of VisionLab is the precise gauging side vision station that combines a high resolution CCD camera, telocentric lens, collimated back lighting with a smart motor and control logic. This combination provides 360 measurements per part (a measurement every degree of rotation) for unmatched repeatable profile measurements of part dimensional characteristics such as radii, concentricity, straightness, lengths, diameters, threads, tapers, etc . . . .
(222) Software: Part templates are created through an automatic database operation for instant dimensions and tolerances of all pertinent part features. With thousands of parts requiring inspection, this process saves valuable set-up time.
(223) Bottom Vision: Single view camera with advanced image analysis control logic that minimizes surface and lighting variations. A strobe LED ring light illuminates the bottom surface for dedicated inspection.
(224) Inspections: Identify Head Markings and Inspect Recess/Drive Features.
(225) MultiView Front Lighting detects surface imperfections such as—scratch gouges, flakes, burrs, smears, discoloration, chips, cracks, dents, damaged threads.
(226) Using a sequenced lighting technique, milliseconds after completion of the 360 dimensional inspection, the parts are illuminated with top and bottom front LED lighting to provide the side cameras a view of the full surface of the part. As with the dimensional inspection, the VisionLab processes 360 surface inspection images (one image per every degree).
(227) By processing 360 images, smaller surface defects are detected as the inspection window narrows to sections allowing more defined surface analysis.
(228) Software: Part templates are created through an automatic database operation for instant visual defect detection. Defect analysis settings are preprogrammed per Region of Interest (ROI), as shown in the threaded region in the image of
(229) Top Vision: Single view camera with liquid lens technology and advanced image analysis control logic that minimizes surface and lighting variations. A strobe LED ring light illuminates the top surface for dedicated inspection.
(230) By providing the liquid lens, neither the camera nor its lens assembly is required to move for varying part lengths (i.e., the focal length of the liquid lens is electronically controlled).
(231) Inspection: Top recess feature measurement and inspection.
(232) Top Vision: Single view camera with liquid lens technology and advanced image analysis control logic that minimizes surface and lighting variations. A strobe LED ring light illuminates the internal surface for Inner Diameter wall inspection including internal thread defect detection.
(233) By providing the liquid lens, neither the camera nor its lens assembly is required to move for varying part lengths (i.e., the focal length of the liquid lens is electronically controlled).
(234) Inspection: Inner diameter walls for visual defects and internal thread inspection.
(235) Identifying Parts Process via the Flowchart of
(236) Using machine vision, software images are taken that will take measurements down to the micron level for each region of the part. The measurements are then compared with the data extracted version (i.e. templates) to find the closest matching parts. When a handful of parts have been found, attributes are used to narrow down to the best possible choice. Attributes include: Coating, Color, Identification, Head Type, Drive Type, Hole in Shank, Hole in Head and grip markings. A foreign part check is done on the part to determine if all the measurements are within the tolerance levels. If any of the measurements are outside of the specified regions, it is determined that this part is not a part that should be sorted, and the part will get dumped.
(237) While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.