Method for Improved Acquisition of Images for Photogrammetry
20220046189 · 2022-02-10
Inventors
Cpc classification
G06T17/10
PHYSICS
G01C11/02
PHYSICS
G01B21/20
PHYSICS
International classification
G01C11/02
PHYSICS
G06T17/10
PHYSICS
Abstract
A method for improved image acquisition for photogrammetry includes focusing a camera on one end of an object, capturing one or more images of the object, incrementally adjusting the focal length of the camera toward the opposite end of the object, and capturing images at each new focal length. Once the object has been photographed at varying focal lengths that run the entire length of the object, the multitude of images are then combined using focus stacking to create a singular image that is more in focus for the entire length of the object. A method for utilizing thermographic cameras to aid in the acquisition of images for photogrammetry includes applying thermal textures to the object and isolating an object from the background due to thermal differences.
Claims
1. A method of acquiring images for photogrammetry, comprising: positioning an object in a field of view of a thermographic camera; creating a pattern of temperature differentiation across the object; and capturing one or more thermographic images of the object with the thermographic camera; repositioning the object or the thermographic camera to a plurality of perspectives; capturing one or more thermographic images of the object in each of the plurality of perspectives while continuing to create the temperature differentiation to provide a comprehensive set of thermographic images which represent the object.
2. The method of claim 1, further comprising: creating a different pattern of temperature differentiation across the object in at least two different perspectives of the plurality of perspectives; and creating one or more control points by conserving one or more points across the different pattern of temperature differentiation and the different perspectives.
3. The method of claim 2, wherein the one or more control points present in each of the thermographic images captured from the at least two different perspectives serve as reference points for photogrammetric modeling of the object.
4. The method of claim 2, further comprising: creating a thermographic mesh from the comprehensive set of thermographic images; capturing one or more images of the object from a visible light camera in each of the plurality of perspectives; and projecting a texturing from the visible light images on to the thermographic mesh.
5. The method of claim 1, reapplying the temperature differentiation in the same patterned manner across the object as necessary to ensure the pattern persists over the range of the thermographic image perspectives.
6. The method of claim 5, wherein capturing one or more images from a visible light camera from the same perspectives as the thermographic camera and then projecting the texturing from the visible light images on to a thermographic mesh created from the thermographic images.
7. A method of claim 7, wherein the pattern of temperature differentiation is applied using a projected beam of light.
8. A method of claim 7, wherein the pattern of temperature differentiation is applied by a physical means.
9. A method of claim 7, wherein the pattern of temperature differentiation is applied by a non-physical means.
10. A method of subject isolation, comprising: positioning an object in a field of view of a thermographic camera; creating a discernable temperature difference between the object and a background surround the object by applying a temperature differentiation to at least one of the object or the background; capturing one or more thermographic images of the object and the background surrounding the object; and utilizing the discernable temperature difference between the object and the background to isolate the object from the background.
11. The method of claim 10, wherein the temperature differentiation is applied to the object or the background using a projected beam of light.
12. The method of claim 10, wherein the temperature differentiation is applied to the object or the background by a physical means.
13. The method of claim 10, wherein the temperature differentiation is applied to the object or the background by a non-physical means.
14. The method of claim 10, further comprising: using a visible light camera to capture one or more images of the object and the background from the same perspective as the thermographic camera; using the thermographic images to separate the object from the background; and overlaying the visible light camera images to produce an isolated object image.
15. The method of claim 10, further comprising: repositioning the object or the thermographic camera to a plurality of multiple perspectives; capturing one or more thermographic images of the object and the background in each of the multiple perspectives; and performing multiple-perspective isolation of the object.
16. A method of claim 10, further comprising: capturing one or more visible light camera images of the object and the background in each of the multiple perspectives; overlaying the visible light images onto the multiple-perspective isolation of the object; and creating an isolated multiple-perspective representation of the object in the visible light spectrum.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description and the accompanying drawings, in which:
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
DETAILED DESCRIPTION
[0023] In the following descriptions, like numbers refer to like elements.
[0024] The present disclosure will now be described in detail with reference to the Figures.
[0025] Referring now to
[0026] In
[0027]
[0028] High dynamic range photography is a process that can be utilized to create an image of an object that possess a larger range of luminosity for the entire object. The process involves the capturing of multiple images from the same location and at the same focus setting with varying exposure settings. This in turn ensures that at each location and focus setting each portion of an object is imaged at the substantially optimal exposure setting. The multitude of photos are then combined such that the resulting image has the larger range of luminosity values for each portion of the object. The photo can then be processed to create a singular image file wherein each portion of the object possesses its substantially optimal luminosity. This processing is accomplished through the use of a computer utilizing the requisite processing. The use of high dynamic range photography helps to ensure that the object is more clearly illuminated, which in turn aids in creating a more detailed and accurate photogrammetric model. The application of high dynamic range photography in acquiring images for photogrammetry is particularly useful when an object's coloring could potentially introduce exceptionally dark or light areas, which may distort the resulting model.
[0029] In step 207, the focal length of the camera is adjusted in a predetermined increment towards an end of the object. At this new focal length, in step 209, a new series of photos are taken. Again, these photos may be high dynamic range photos or not as needed. The process of incrementally changing the focal length (the increments need not be equal) and taking a series of photos is repeated in step 211, until photos have been captured along the entire length of the object. The result is a series of images that collectively have most if not all portions visible and in sufficient focus. In step 213, the camera or the object is repositioned so as to allow the capturing of a new perspective of the object. Whether it is the object or the camera that is repositioned depends on what is easiest. For a small object perhaps repositioning the object is easier, but if the object is a building then the camera is far easier to relocate. From this new perspective, the above process of taking photos at varying focal lengths is repeated in step 215. The process of repositioning the camera or object and taking photographs at incremental focal lengths is repeated for all the necessary angles in step 217 until a multitude of images have been captured at all necessary or desired perspectives, focal lengths, and exposure settings. In step 219, the batch of images of the object for a given perspective are then combined.
[0030] The resulting combination of images is processed, in step 221, through focus stacking and, if utilized, high dynamic range photography methods. Focus stacking or focal length merging is a process that can be utilized to create an image of an object that is more in focus across its entire depth as compared to traditional methods. Focus stacking is particularly relevant and useful when dealing with objects that are either very small or very large, when it is difficult to capture a single image of the object that is fully in focus. The process of focus stacking comprises the taking of two or more of images at varying focal lengths along the depth of the object. The images are then compared and processed to select the sharpest pixels of each image. This processing is generally done through the use of a computer running focus stacking algorithms. The result of the focus stacking process is a singular image where most if not all portions of the object are “in focus.” With an image of an object in better focus along each point of the object (as compared to a single image in which only a portion of the object will be in focus), a more accurate and detailed photogrammetric model can be created. In addition, this method aids in the acquisition and modelling of objects on the micro, macro, and close-range scale. Thus, the implementation of this method helps to overcome some previous limitations on photogrammetric modelling.
[0031] The result of this focus stacking process is a single image for a given perspective that is more in focus than traditional photo acquisition methods, which do not utilize focus stacking. In step 223, the focus stacking and high dynamic range processing is repeated for all the perspectives. The result is a series of images that fully resemble the object from all necessary perspectives and that each possess a wider range of luminosity and are more in focus as compared to traditional methods. From these images then, photogrammetric modelling can be performed to create a three-dimensional model, in step 225. By utilizing focus stacking and high dynamic range photography in this manner, then the depth of field and narrow luminosity range limitations of traditional photogrammetry image acquisition can be overcome. The result is a model that has far superior detail and accuracy. In addition, it allows for photogrammetry to be performed well on objects that previously were thought not ideal, such as on micro-scale objects.
[0032]
[0033] In addition, visible light images can also be captured from the same perspective as the thermographic camera. These visible light images can then be overlaid onto the three-dimensional model created by thermal texturing to create a three-dimensional model that accurately shows the geometry of the model in addition to the appropriate coloring and texturing of the model. However, it is not necessary to overlay these visible light images as the model created by the thermographic texturing would be sufficient to capture the full geometry of the object. Thus, depending on the purpose intended for the three-dimensional model, visible light images may or may not be taken in conjunction with the thermographic images.
[0034]
[0035] In step 405, a temperature differential is applied to the object in a patterned manner. The temperature differential could be applied in any pattern. Perhaps the most common would be in a somewhat checkboard pattern. The key is to ensure that only selective portions of the object are warmer or cooler than other portions and that there is not simply one uniform temperature. In addition, the user should avoid applying a temperature differential that would result in the edges of the object having the same temperature as the background of the object. The temperature differential may also be applied through a variety of means. For example, the temperature differential may be applied using some physical means or non-physical means. An exemplary but non-limiting list of physical means of applying the temperature differential may include a warmed or cooled sponge, some other hot object, an ice cube, a freezer pack, a heating blanket, a hand, or other such means known to one in the art. An exemplary but non-limiting list of non-physical means of applying the temperature differential may include a spot light, laser array, a blow drier, a flame, a flash light, light bulb, the user's breath, or other such means known to one in the art.
[0036] In step 407, one or more thermographic images are captured of the thermal pattern on the object using the thermographic camera. These images provide the basic surface point information needed by the photogrammetric modeling software to create a three-dimensional model when combined with multiple perspectives. In step 409, a series of visible light images are captured of the object using the visible light camera and from the same perspective as the thermographic camera. These images, as discussed above, are optional, but when combined with the three-dimensional model created from the thermographic images allow for a three-dimensional model to be created that more accurately resembles not only the geometry of the object but also the coloring and texturing of the object. These visible light images could encompass the focus-stacking and high dynamic range methods discussed above and shown in
[0037] Once all the images have been captured, the images are assembled together for each perspective in step 421. In step 423, the thermographic images are processed to create a thermal image mesh of the object. The mesh is used to provide the requisite data of the object's geometry for the photogrammetric three-dimensional modeling. In step 425, the visible light images that were captured may be overlaid onto the model created from the thermal mesh to give the model the appropriate coloring and texturing. This is not a necessary step but helps to produce a more accurate model. The overall process of using thermographic texturing allows for the photogrammetric modeling of highly reflective or transparent objects in an efficient and accurate manner; something not easily achievable under traditional photogrammetric image acquisition methods.
[0038]
[0039] The object 503 in this exemplary implementation is a smaller object and
[0040]
[0041]
[0042] The object 703 in this exemplary implementation is a larger object, such as the Statue of Liberty, as shown. However, the object 703 could be of any shape or size. Behind the object is a background 707, such as clouds in the sky. In this implementation the object 703 has a higher temperature than the background 707. The object has a higher temperature because it is being warmed by an external source 705, in this case the sun. Due to the fact that the object 703 is metallic and has been exposed to the external source 705, i.e. the sun, the object 703 is warmer than the background 707 behind it. As a result, the background 707 has a discernably different temperature than the object 703. Due to this temperature difference, the captured thermographic images can be used to isolate the object 703 from the background 707 as further exemplified in
[0043]
[0044]
[0045] As discussed, the temperature differential may be applied through a variety of means. For example, the temperature differential may be applied using some physical means or non-physical means. An exemplary but non-limiting list of physical means of applying the temperature differential may include a warmed or cooled sponge, some other hot object, an ice cube, a freezer pack, a heating blanket, a hand, or other such means known to one in the art. An exemplary but non-limiting list of non-physical means of applying the temperature differential may include a spot light, laser array, a blow dryer, a flame, a flash light, light bulb, the user's breath, or other such means known to one in the art. The means utilized is not important so long as the result is that a discernable temperature differential exists between the object and the background. It is helpful in attaining this goal if the temperature differential is applied in a uniform manner, however it is not necessary.
[0046] In step 907, the thermographic camera is used to capture thermographic images of the object and the background. These thermographic images show the temperature differential between the object and the background to allow for thermal subject isolation to be performed. In step 909, the thermographic camera or the object is repositioned as needed if multiple perspectives of the isolated object are desired. For example, this step would be needed if this method is applied to photogrammetric modeling photo acquisition, since multiple perspectives are needed to create a full three-dimensional model. In step 911, an image mask is created of the object from the thermographic images by using the temperature differential.
[0047] In step 913, the image mask is used to isolate the object from the background. The thermal image mask may then be overlaid on to visible light images of the object that were captured from the same perspectives as the thermographic images. Through this process a visible light image can be created of an isolated object as was shown in
[0048]
[0049] In step 1005, a temperature differential is applied to either the object or the background. It may be the case that a temperature differential does not need to be applied since it naturally exists, as was shown and discussed in
[0050] In step 1007, the thermographic camera is used to capture thermographic images of the object and the background. These thermographic images will show the temperature differential between the object and the background to allow for thermal subject isolation to be performed. In step 1009, a series of images are captured by the visible light camera at a first focal length, which generally would be located at one end of the object. The images in step 1009 may be high dynamic range images, meaning that a series of images are captured at various exposure settings at that first focal length. Although high dynamic range photographs are not necessary, they are useful when the coloring of the object would result in either exceedingly bright or dark portions that would skew the resulting photogrammetric model. In step 1011, the focus length of the visible light camera is adjusted incrementally towards an end of the object. At this second focal length, in step 1013, a new series of visible light photos are taken. Again, in this exemplary implementation the photographs may be high dynamic range photos, but this is not required. The process of incrementally changing the focus length and taking a series of photos is repeated in step 1015 until photos have been captured along the entire length of the object. The goal is to end with a series of images that when taken as a whole have the object more focused and, if desired, with a larger range of luminosity (compared to traditional methods) for a given perspective.
[0051] In step 1017, the cameras or the object is repositioned so as to allow the capturing of a new perspective of the object. In step 1019, the temperature differential is reapplied as necessary to ensure the thermal subject isolation method performs well. From this new perspective, in step 1021, the above process of capturing thermographic images of the object and background, as well as taking photos at varying focal lengths is repeated. In step 1023, the cameras or the object are repositioned and the process is repeated for all the necessary perspectives until a multitude of thermographic images and of visible light images have been captured at all necessary perspectives, focal lengths, and exposure settings to fully photograph the object from all angles.
[0052] In step 1025, the batch of images of the object for a given perspective are assembled together. In step 1027, the thermographic images are used to create an image mask of the object utilizing the temperature difference between the object and the background. The image mask can then be utilized to isolate the object from the background in all the images for the same perspective. The isolation step could occur here, or it could occur after the focus stacking and high dynamic photography methods have been implemented. In step 1029, focus stacking (as discussed above) is utilized to combine all the images of a given perspective. The result of the focus stacking is a singular image that is more in focus along its entire depth as compared to traditional photographic image acquisition without focus stacking. With the addition of the high dynamic range photography, the image also has a wider range of luminosity values. As such the resulting image from this perspective not only has a wider luminosity range and a more complete focus, as compared to traditional methods, but it is also isolated from the background. The thermal subject isolation and focus stacking process is then repeated for the remaining perspectives in step 1031. Once this step is complete a series of images should remain of an isolated object with a more complete focus and wider luminosity range. Thus, in step 1033, photogrammetry techniques are utilized to build a three-dimensional model from these images. Due to the processes utilized, this model will be highly detailed and more accurate regardless of the size, shape, transparency, reflectiveness, or color of the object or background. Thus, this method helps to overcome many of the current difficulties in photogrammetric image acquisition.
[0053] Any of the above methods can be utilized separately or in combination (as disclosed in
[0054] The above is a detailed description of exemplary implementations to illustrate the principles of the disclosure. The implementations are provided to illustrate aspects of the disclosure, but the disclosure is not limited to any embodiment. In addition, for the purpose of clarity, technical material that is known in the technical fields related to the disclosure has not been described in detail so that the disclosure is not unnecessarily obscured.