VEHICLE PHOTOGRAPHIC SYSTEM FOR IDENTIFICATION OF SURFACE IMPERFECTIONS

Abstract

A system and method are provided for determining surface defects of an object under study. A photographic tool is employed to produce real-time blending of a series of images of a static object where each of the individual images are illuminated under a different lighting condition with respect to the static object. The achieved effect is similar to what is experienced when looking at a painted surface from different perspectives in order to see if there are any imperfections. For example, in the case of automotive photography, paint problems, sheen differences, scratches, dents, and small dings may only be visible when the vehicle is lit from a specific or correct angle.

Claims

1. A system for determining vehicle surface defects, the system comprising: an enclosable drive-in structure having one or more doors, a wall and a ceiling; a plurality of cameras, with at least one of camera mounted in each of said one or more doors, said wall and said ceiling, said camera providing images or video to a computing system; a cluster of lighting elements associated with each of said plurality of cameras, said clusters of lighting elements positioned in said one or more doors, said wall and said ceiling; a trigger mechanism for each of said plurality of cameras to take exposures through an automated lighting cycle of each of the associated clusters of lights as a light beam generated by said cluster of lighting elements sweeps across a vehicle surface.

2. The system of claim 1 wherein said clusters of lighting elements are in a semicircle.

3. The system of claim 1 wherein said clusters of lighting elements are in a circle.

4. The system of claim 1 wherein said cluster of lighting elements are arranged as concentric rings of lighting for converting an image to a point cloud for a three dimensional (3D) rendering of the vehicle.

5. The system of claim 1 wherein at least one of said one or more doors, said walls and said ceiling are painted black.

6. The system of claim 1 wherein all of said one or more doors, said walls and said ceiling are painted black.

7. The system of claim 1 further comprising a machine learning capability to identify surface defects.

8. The system of claim 1 further comprising a central process unit (CPU), said CPU coordinates and controls said cameras, said lighting system; and wherein a communication interface connects said plurality of cameras via one or more of wired and wireless links, optical drives, universal serial bus (USB), flash memory slot, and combinations thereof for receiving and sending data.

9. The system of claim 1 wherein an image stack of photos taken by a camera with a series of different lighting angles is supplied to a computer workstation, where the image stack of photos are combined into a single blended image by a user.

10. The system of claim 9 wherein said computer workstation further comprises a joy stick controller, a mouse, a keyboard, and a display monitor.

11. The system of claim 9 wherein said computer workstation further comprises a graphical user interface (GUI) with which an operator can, zoom, pan, and change the lighting as needed to revel any surface issues that may need a closer look as seen in the blended image.

12. The system of claim 1 further comprising a switchable filter or light polarizer that is positioned in front of one or more of a lens of a camera or cluster of lighting elements.

13. The system of claim 12 wherein at least one of infra-red (IR) lighting, polarization, color polarization, and cross polarization is used to identify vehicle surface defects.

14. The system of claim 1 further comprising a radio frequency identification (RFID) reader or alphanumeric reader for identifying vehicle and client parameters upon entry into said structure.

15. The system of claim 14 wherein said RFID reader or said alphanumeric reader obtains the vehicle identification number (VIN) of the vehicle to be processed; wherein a set of information related to the vehicle is encoded in said VIN, said set of information comprising one or more of vehicle make, model, body style, and vehicle color.

16. The system of claim 1 wherein the images of the vehicle are uploaded in a database.

17. The system of claim 16 wherein the database further comprises vehicle and client information.

18. A process for photographing a vehicle with the system of claim 1 comprising: driving the vehicle through said door into said enclosable drive-in structure; selectively illuminating a first cluster of lights arrayed in said structure and associated with a first camera; collecting a first set of photographs of the vehicle with illumination from said first subset of lights; supplying the first set of photographs to a computer workstation; creating a blended photograph made up of the first set of photographs at the computer workstation; and analyzing the blended image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The subject matter that is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0013] FIGS. 1A and 1B are typical non-studio quality vehicle photographs rendered as line drawings;

[0014] FIG. 1C illustrates a studio shot with uncontrolled reflections in a vehicle surface photographs rendered as a line drawings;

[0015] FIGS. 2A and 2B illustrate an existing studio configuration for generating a contrast break line on a vehicle with additive lighting photographs rendered as line drawings;

[0016] FIGS. 3A and 3B illustrate an existing studio configuration for generating a contrast break line on a vehicle with subtractive lighting photographs rendered as line drawings;

[0017] FIG. 4 illustrates a photographic booth with a cameras mounted in the walls and ceiling that provides controlled surface reflections on surfaces of the vehicles being photographed as well as lighting configurations that assist in detecting surface defects, dents, and previous body work in accordance with embodiments of the invention;

[0018] FIG. 5 illustrates a partial cutaway view with the roof removed of the photographic booth of FIG. 4 with a vehicle positioned in the center of a photographic stage with cameras and lighting elements arranged in the walls and door of the photographic booth;

[0019] FIG. 6 illustrates the selective activation of lighting elements in the photographic booth shown in FIGS. 4 and 5 where the selective lighting of each of the lighting groups and multiple exposures from the related cameras allows for detection of surface defects, dents, and previous body work in accordance with embodiments of the invention;

[0020] FIG. 7 illustrates a partial cutaway view of a photographic booth with a lighting cluster associated with a fixed camera according to an embodiment of the invention;

[0021] FIG. 8A illustrates a beam of light generated from one of the lighting elements of the lighting cluster of FIG. 7, where the lighting element rakes across the surface of the vehicle to reveal the texture of the vehicles surface in accordance with embodiments of the invention;

[0022] FIG. 8B is a series of pictures showing a light beam raking across the driver side surfaces of a vehicle in accordance with embodiments of the invention;

[0023] FIG. 9 illustrates an image stack of photos taken by a camera with a series of different lighting angles that are combined into a computer generated single blended image in accordance with embodiments of the invention;

[0024] FIG. 10 illustrates a workstation and interface for identifying surface imperfections and defects based on the obtained images in accordance with embodiments of the invention;

[0025] FIG. 11 illustrates concentric rings of lighting for converting an image to a point cloud for a three dimensional (3D) rendering of a vehicle.

[0026] FIG. 12A is an initial photograph (opening blend) of a vehicle in accordance with embodiments of the invention;

[0027] FIG. 12B shows a series of blended photographs with lighting at different positions on the surface of the vehicle in accordance with embodiments of the invention;

[0028] FIG. 12C illustrates a portion of the surface of the vehicle shown in FIGS. 12A and 12B where the surface defects are not visible in a standard automotive photograph become extremely pronounced when the lighting is at the correct angle for that defect in accordance with embodiments of the invention;

[0029] FIG. 13A is an initial photograph (opening blend) of a vehicle in accordance with embodiments of the invention;

[0030] FIG. 13B shows a series of blended photographs with lighting at different positions on the surface of the vehicle in accordance with embodiments of the invention;

[0031] FIG. 13C is a close up view of one of the twelve photographs highlighting surface imperfections;

[0032] FIG. 14A is an initial photograph (opening blend) of an artifact with light rolled around the object in accordance with embodiments of the invention;

[0033] FIG. 14B is a series of photographs of the artifact of FIG. 14A with light rolled around the object in accordance with embodiments of the invention;

[0034] FIG. 15 is a photograph of a switchable filter or light polarizer that may be positioned in front of a lens of a camera; and

[0035] FIG. 16 is a system block diagram of the major components for photographing vehicles in accordance with embodiments of the invention.

DESCRIPTION OF THE INVENTION

[0036] The present invention has utility as a photographic tool that allows real-time blending of a series of images of a static object where each of the individual images are illuminated under a different lighting condition with respect to the static object. The achieved effect is similar to what is experienced when looking at a painted surface from different perspectives in order to see if there are any imperfections. For example, in the case of automotive photography, paint problems, sheen differences, scratches, dents, and small dings may only be visible when the vehicle is lit from a specific or correct angle.

[0037] In a specific inventive embodiment, a vehicle is subjected to an automated process of specialized photography with many lighting angles and styles to generate a series of images. A playback interface may then be used to “roll” the lighting around the subject vehicle to be inspected. In this way the image set revels more surface detail than any normal static photo could. Thus, embodiments of the inventive process and system may be used in detecting surface defects, dents, or previous body work performed on a subject vehicle.

[0038] In embodiments of the invention with respect to each camera position a lighting angle change is made for each of the series photos taken from the camera position that allows remote vehicle inspection for surface defects. Inventive embodiments allow an untrained operator to capture vehicle inspection images during the intake process for that vehicle. The detailed inspection is then done remotely by a highly skilled technician. Thereby one technician at a display monitor using an inventive embodiment of the photo blending interface may inspect many more vehicles per day across multiple facilities. Optimizing the work of that inspector while reducing physical fatigue. Embodiments of the photo blending may also be made available to the buying public as in the case of vehicle auctions allowing the bidders to better preview the vehicle. In such a case the auction house would simply capture the assets and post them online without the need for any skilled technician. This puts the responsibility of the vehicle's surface condition on the bidding consumer.

[0039] In inventive embodiments a predetermined blend of images may be used to create an opening view representing a normally lit object. When the playback interface is active the viewer may control the blend between different lighting angles giving the same experience as one would get in person by moving their head around to get a better angle of light across the surface to be inspected. In embodiments the intensity of the cross lighting may be controlled from a subtle highlighting of an area to complete crosslight, as well as highlight fading across to black.

[0040] Inventive embodiments may be scaled to photograph large subjects such as vehicles as well as smaller versions of the system may be used to examine the textures of smaller objects or the condition of such things where surface quality would affect the value of the object. illustrative examples include the condition of collectables (baseball cards, comic books, toys, coins, stamps) or artwork.

[0041] It is to be understood that in instances where a range of values are provided herein, that the range is intended to encompass not only the end point values of the range, but also intermediate values of the range as explicitly being included within the range and varying by the last significant figure of the range. By way of example, a recited range of from 1 to 4 is intended to include 1-2, 1-3, 2-4, 3-4, and 1-4.

[0042] Referring now to the figures, FIG. 4 illustrates a photographic booth 10 with camera apertures 12 mounted in the walls 14, ceiling 16, and door(s) 18. Cameras 12C are fix mounted at the camera apertures 12. While only one door 12 is shown it is appreciated that a second door as an opposing door may be present to allow the drive through of the vehicle. FIG. 5 illustrates a partial cutaway view with the ceiling 16 removed from the photographic booth 10 of FIG. 4 with a vehicle V positioned in the center of a photographic stage The interior of the walls 14, ceiling 16, and door(s) 18 are painted black or another color to control reflections, and provide controlled surface reflections on surfaces of the vehicles being photographed. The black interior surfaces stop the sheet metal of the subject vehicle V from reflecting any of the environment. The vehicle exposure will be made only with direct light from the lighting elements 20 avoiding any reflections from adding false shapes to the surface of the vehicle V. Each fixed camera takes exposures through the automated lighting cycle of the cluster of lights 20 positioned on the wall 14, ceiling 16, and door(s) 18 of a particular camera 12C that generates an image stack. A trigger mechanism 19 for each of the cameras take exposures through an automated lighting cycle of each of the associated clusters of lights as a light beam generated by the cluster of lighting elements sweeps across a vehicle surface. The clusters of lighting elements 20 are shown in a semicircle, though it is appreciated that alternative configurations or arraignments of the lighting elements are possible. The lighting configurations assist in detecting surface defects, dents, and previous body work in accordance with embodiments of the invention.

[0043] FIG. 6 illustrates the selective activation of lighting elements in the photographic booth 10 shown in FIGS. 4 and 5 where the selective lighting of each of the lighting groups and multiple exposures from the related cameras allows for detection of surface defects, dents, and previous body work on the vehicle V in accordance with embodiments of the invention.

[0044] FIG. 7 illustrates an embodiment 30 of a partial cutaway view of a photographic booth with a circular ring of lighting elements 20 associated with a fixed camera 12C according to an embodiment of the invention. Each lighting element 20 rakes the surface to reveal the texture of the vehicles surface.

[0045] FIG. 8A illustrates a beam of light 22 generated from one of the lighting elements of the lighting cluster of FIG. 7, where the beam of light 22 rakes across the surface of the vehicle V to reveal the texture of the surface of the vehicle. FIG. 8B is a series of pictures showing a light beam raking across the driver side surfaces of a vehicle.

[0046] FIG. 9 illustrates an image stack of photos 40 of a vehicle V taken by a camera with a series of different lighting angles that are combined into a computer generated single blended image 42 in accordance with embodiments of the invention.

[0047] FIG. 10 illustrates a computer workstation 50 with an input device illustratively including a mouse or joy stick controller 52, a data entry keyboard 54, and display interface 56 for identifying surface imperfections and defects based on the obtained images. The image stack 40 for each angle arrives at the computer workstation 50 and is imported in the viewer where the user selects images from the stack 40 to combine as a single blended image of that angle to see areas of interest better. Through the graphical user interface (GUI) of the workstation an operator can, zoom, pan, and change the lighting as needed to revel any surface issues that may need a closer look. It is appreciated that the display interface may be a touchscreen for controlling the GUI. The operator may add point annotations based on their findings and a grading scale may be applied to the image. Examples of annotations may include “this location needs further review”; “there is a mild defect here”; “there is a major defect here”; etc.

[0048] Through the use of the computer workstation 50 the user has the effect that is the same as looking down the side of the car and moving their head around to see a dent or scratch. In a similar manner that moving one's head causes the angle of light relative to the sheet metal to change—when light is aligned correctly dents and scratches become more visible. The same result can be had by keeping one's head in one place and moving a light source around the dented area.

[0049] As noted above a photography system produces a series of photographs of an automobile from a static camera. Each photograph is lit with directional lighting coming from a different direction. Within the “examination” software the user would see a single image. That image is a blend of several light sources. The user may then move the mouse or joystick 52 around which changes the percentage of which light sources get blended.

[0050] FIG. 11 illustrates an embodiment 60 of concentric rings of lighting 62 for converting an image to a point cloud for a three dimensional (3D) rendering of a vehicle. The usage of many cross lights would be used with one or more cameras to generate photogrammetric depth data. This data could be analyzed against known reference data for the same make and model. Any differences between the two models would indicate body damage on the subject car.

[0051] FIG. 12A is an initial photograph (opening blend) of a vehicle.

[0052] FIG. 12B shows a series of blended photographs at different positions on the surface of the vehicle of FIG. 12A.

[0053] FIG. 12C illustrates a portion of the surface of the vehicle shown in FIGS. 12A and 12B where the surface defects not visible in a standard automotive photograph become extremely pronounced when the lighting is at the correct angle. It is noted that modern digital single-lens reflex (DSLR) cameras create images that are 8000 pixels wide. This resolution allows large amounts of data to be examined and allows the ability to zoom in much further than ever before.

[0054] FIG. 13A is an initial photograph (opening blend) with equal amounts of the twelve photographs shown in FIG. 13B taken with lighting swept across the surface. A user at a workstation while moving their mouse or joystick will see light that will blend and shift around the twelve extremes as shown in FIG. 13B. FIG. 13C is a close up or zoomed in view showing vehicle surface imperfections including rust, scratches, and dents.

[0055] FIG. 14A is an initial photograph (opening blend) of an artifact with light rolled around the object.

[0056] FIG. 14B is a series of photographs of the artifact of FIG. 14A with light rolled around the object.

[0057] FIG. 15 is a photograph of an embodiment 70 of a switchable filter or light polarizer 72 that may be positioned in front of a lens of a camera and/or of the lighting elements of the lighting clusters. In a specific embodiment the wavelength of the light may be varied to expose surface imperfections. For example infrared lighting, color polarization, and cross polarization (on and off axis) filtration may also be used to identify surface imperfections.

[0058] In specific inventive embodiments, machine learning may be employed for finding the imperfections in an examined surface automatically. The software would have to know the make and model of vehicle and the vehicle would have to be clean. The machine would be taught what the reference vehicle's natural body lines look like and through examining the sheet metal in all lighting scenes determine where an imperfection is. A high resolution laser scan may also be used to provide a comparison model. Each image is examined automatically running through the full lighting and filtration series. Areas of interest are flagged, and the flagged sections of each camera angle are compared and graded based on criteria discovered during the machine training process. Highly likely areas of surface imperfections are confirmed by humans and then feed back into the system to help further the artificial intelligence (AI) training.

[0059] FIG. 16 is a system block diagram 100 of the major components for photographing vehicles for surface imperfections in accordance with embodiments of the invention. A central process unit (CPU) 122 coordinates and controls the overall operation of the photographing system 100 that may be operated in the photographic structure 10. The communication interface 124 is representative of various links and input connections and devices illustratively including but not limited to wired and wireless links, optical drives, universal serial bus (USB), flash memory slot, and combinations thereof, for receiving and sending data that may or may not be in real time. The bus 102 links the various components in the system. Memory 116 serves as storage for operating programs and firmware for the photographic system 100. A database with vehicle and client information is stored in Memory 116. Memory 116 is made up of read-only memory ROM and random access memory (RAM). Graphics chipset 120 drives a display 56 of FIG. 10. The display 56 may be liquid crystal display (LCD), light emitting diode (LED), or other known display technologies. Control interface 106 may include knobs, buttons, and other touch sensitive controls for selecting various menu items or for inputting alphanumeric information. Sensors 104 sense the presence of a vehicle and the vehicle position. RFID/barcode 108 is a reader that detects and interprets tags mounted to vehicles or driver identification (ID) cards. Cameras 110 and lighting 114 are controlled and sequenced by the CPU 122. Indicators 112 provide visual feedback to system users. In inventive embodiments, the central process unit (CPU) 122, or master and node computers, in the case of many cameras in use—these processing nodes speedup download and system throughput, and the CPU 122 coordinates and controls the plurality of cameras in the photographic structure 10. In specific inventive embodiments, a communication interface connects the plurality of cameras via one or more of wired and wireless links, optical drives, universal serial bus (USB), flash memory slot, and combinations thereof for receiving and sending data.

[0060] The use of sequenced and automated image capture allows for the rapid image processing of vehicles for auction sales, dealer records, as well as car condition assessments for rental agencies, fleet management companies, public safety agencies, municipal and government agencies, etc. A complete set of vehicle images are collected in between 5 and 90 seconds allowing for high throughput imaging associated with an auto auction or manufacturer.

[0061] Embodiments of the inventive enclosable photographic booth allow for a complete set of multi perspective high quality vehicle images to be obtained and recorded into a template or placed in a shared network folder in less than 90 seconds, and in some instances in approximately 5, 10, or 15 seconds. Thus, once a subject vehicle is photographed in the inventive enclosable oblong hemispherical domed photographic booth, the vehicle is available and ready for sale online based on the uploaded images and VIN uploaded information. The rapid image processing and recordation of vehicle condition allows for new business models such as creating virtual or cyber dealerships where a wholesale customer never takes possession of a subject vehicle, and the vehicle is dropped shipped to the end retail customer. Photos obtained with the inventive system may be sold along with the subject vehicle for use by a purchasing used car retailer, and as it typically takes four days for the buyer to take delivery of the wholesale vehicle, with instant access to the vehicle photos the purchaser can start advertising the vehicle four days prior to physically taking possession of the vehicle.

[0062] Embodiments of the inventive photographic booth may utilize radio frequency identification (RFID) tags to identify and record vehicles as the vehicles are processed through the system. The RFID may be related to the vehicle identification number (VIN) of a vehicle to be processed. It is further appreciated that additional identifiers may be used illustratively including barcodes that relate to the VIN. Information related to the vehicle, such as vehicle make, model, body style, and color encoded in the VIN may be used to automatically adjust lighting. Further information that may be related to the RFID or barcode may include a dealership setting that may also indicate which dealer the vehicle is being shot for, and incorporate that dealer's specific preferences such as lighting style, file size and format, number of photo shots and angles to be recorded. RFID or bar code information, read manually or automatically may also be used to project text onto the background of the shot or overlay text onto the file during processing, such as price, dealer name, vehicle specifications, mileage, etc.

[0063] As a person skilled in the art will recognize from the previous detailed description and from the figures and claims, modifications and changes can be made to the preferred embodiments of the invention without departing from the scope of this invention defined in the following claims.