System, computing device, and method for document detection and deposit processing
11900755 ยท 2024-02-13
Assignee
Inventors
Cpc classification
G06Q20/042
PHYSICS
G07D7/2016
PHYSICS
International classification
Abstract
An image of a check is captured by an imaging device and processing of the digital image of the check for deposit at a remote server may be accomplished with a downloaded software application on a portable computing device associated with the imaging device. The downloaded application may include one or more trained machine learning models for processing the captured image. The portable computing device may utilize deterministic algorithms for certain image processing tasks and machine learning models for others. The selection between machine learning and deterministic processing may be made locally on the portable device or in response to instructions from an institution server to use a particular processing method.
Claims
1. A portable computing device, comprising: a processor; memory; and instructions stored on the computer-readable medium, the instructions configured to, when executed by the processor, cause the processor to: capture a digital check image depicting a check; input the data from the captured digital check image into a trained machine learning image cropping model to determine a plurality of edge coordinates corresponding to the check depicted in the digital check image; detect a check image included in the digital check image based on the plurality of edge coordinates; generate a digital image file including the digital check image and the edge coordinates; and transmit the digital image file to a remote server for deposit.
2. The computing device of claim 1, wherein the instructions are configured to, when executed by the processor, further cause the processor to: remove the background image from the digital check image by cropping the digital check image based on the edge coordinates determined by the trained machine learning model.
3. The computing device of claim 1, wherein the instructions are configured to, when executed by the processor, further cause the processor to generate the digital image file to include check data received via user input.
4. The computing device of claim 1, wherein the instructions are configured to, when executed by the processor, further cause the processor to: input the data from the captured digital check image into a trained machine learning image denoising model; remove handwriting from printed text on the check depicted in the digital check image based on an output of the trained machine learning image denoising model.
5. The computing device of claim 1, wherein the instructions are configured to, when executed by the processor, further cause the processor to: input the data from the captured digital check image into a trained machine learning image dewarping model; obtain a coordinate conversion map for dewarping the check depicted in the digital check image based on an output of the trained machine learning image dewarping model.
6. The computing device of claim 5, wherein the instructions are configured to, when executed by the processor, further cause the processor to: input the data from the captured digital check image into a trained machine learning duplicate model, wherein the trained machine learning duplicate check model comprises a perceptual hash trained model; when the trained machine learning duplicate model is stored on the portable computing device, prevent transmission of the data from the captured check to the remote server in response to a determination by the trained machine learning duplicate model that the check depicted in the digital check image based is a duplicate check.
7. The computing device of claim 4, wherein the instructions are configured to, when executed by the processor, further cause the processor to: input the data from the captured digital check image, after removing the handwriting from the printed text based on an output of the trained machine learning image denoising model, into a trained machine learning image dewarping model; obtain a coordinate conversion map for dewarping the check depicted in the digital check image based on an output of the trained machine learning image dewarping model.
8. A portable computing device, comprising: a processor; a memory; and wherein the processor is configured to: download an app for performing image processing on, and remotely depositing to an institution, a digital check image of a check, wherein the downloaded app comprises a trained machine learning model for executing a first image processing task: control, using the downloaded app, an image capture device associated with the portable computing device to capture the digital check image; input, using the downloaded app, the captured digital check image into the trained machine learning model to modify the digital check image in accordance with the first image processing task; process the modified digital check image, using the downloaded app, with a deterministic algorithm to execute a second image processing task that is different from the first image processing task; transmit the modified and processed digital check file to a remote server for deposit.
9. The portable computing device of claim 8, wherein the trained machine learning model for executing the first image processing task comprises an image cropping model configured to cause the processor of the portable computing device to remove a background image from the digital check image by cropping the digital check image based on edge coordinates determined by the trained machine learning model.
10. The portable computing device of claim 8, wherein the trained machine learning model for executing the first image processing task comprises an image demorphing model configured to cause the processor of the portable computing device to remove a skew from the digital check image.
11. The portable computing device of claim 10, wherein the image demorphing model causes the processor to generate a spatial transformation matrix to apply to the digital check image.
12. The portable computing device of claim 8, wherein the trained machine learning model for executing the first image processing task comprises a denoising model configured to cause the processor of the portable computing device to remove handwritten marks from overlapping printed text in the digital check image.
13. The portable computing device of claim 8, wherein the portable computing device comprises a smart phone and image capture device associated with the portable computing device comprises a digital camera.
14. The portable computing device of claim 8, wherein the downloaded app comprises a plurality of trained machine learning models and the processor is configured by the downloaded app to select between two or more trained machine learning models for use in a particular image processing task after determining a state of the captured digital check image.
15. A portable computing device, comprising: a processor; a memory; and wherein the processor is configured to: download an app for performing image processing on, and remotely depositing to an institution, a digital check image of a check, wherein the downloaded app comprises a first trained machine learning model; control, using the downloaded app, an image capture device associated with the portable computing device to capture the digital check image; input, using the downloaded app, the captured digital check image into the first trained machine learning model to identify which of a plurality of second trained machine learning models is a best match for handling an image processing task; and based on the determined best match for handling the image processing task, select and execute a second trained machine learning model to modify the captured digital check image.
16. The portable computing device of claim 15, wherein the first trained machine learning model is trained to cause the processor to: recognize one of a specific check type or check design type of the captured digital check image; and select one of the plurality of second trained machine learning models that is a best match for handling the image processing task based on the check type or check design type.
17. The portable computing device of claim 16, wherein the first trained machine learning model is further trained to cause the processor to: recognize at least one environmental factor associated with the captured digital check image, wherein the environmental factor comprises one of a lighting or shadow detection in the captured digital check image; and cause the processor to select the one of the plurality of second trained machine learning models based on the recognized environmental factor.
18. The portable computing device of claim 16, wherein the processor is configured to, when the second trained machine learning model determined as the best match for handling the image processing task is not stored in the memory on the portable computing device, download the second trained machine learning model from a remote computer.
19. The portable computing device of claim 15, wherein the first trained machine learning model is further configured to cause the processor to alter a device setting of the portable computing device for an capture operation based on a currently detected environmental condition.
20. The portable computing device of claim 15, wherein the first trained machine learning model is further configured to cause the processor to alter a default device setting of the portable computing device for a future image capture operation after a current image capture operation based on a currently detected environmental condition.
Description
DESCRIPTION OF THE FIGURES
(1) The present disclosure may be better understood with reference to the following drawings and description. Non-limiting and non-exhaustive descriptions are described with reference to the following drawings. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating principles. In the figures, like referenced numerals may refer to like parts throughout the different figures unless otherwise specified.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
DETAILED DESCRIPTION
(19) The methods, devices, and systems discussed below may be embodied in a number of different forms. Not all of the depicted components may be required, however, and some implementations may include additional, different, or fewer components from those expressly described in this disclosure. Variations in the arrangement and type of the components may be made without departing from the spirit or scope of the claims as set forth herein.
(20) Within the technological field of remote deposit, a payor may present a check, either in paper or electronic versions, to a payee for deposit. To deposit the check with a financial account at a financial institution associated with the payee, the payee may scan a check in a digital image using a scanner, digital camera, or other imaging device in communication with a computing device. A financial institution may then receive from the payee the digital image of the check. The financial institution may then use the digital image to credit funds to the payee. Such remote deposit techniques utilize the efficient and accurate detection of a check depicted within a digital image, where part of the check image detection includes detecting and extracting the check image from background portions included in the digital image that are not part of the check image.
(21) Traditional, deterministic, image processing algorithms can fail to clearly and reliably detect the border of a check. Such a failure can prevent successful processing of check contents and the failure of traditional algorithms can often be the result of insufficient contrast between the background portions and the check in a captured image. Another common challenge for successful image processing to find a check in an image with traditional algorithms is the presence of reflections or shadows in the image. As described herein, a machine learning model that may be more resilient over a range of color and lighting conditions than any single deterministic technique can be trained to identify a check from the background by, for example, identifying the bounding box coordinates (corners).
(22) Yet another challenge presented by processing images captured of checks is that of de-morphing or de-warping a check image. Distortions in an image can occur when there is a large angle between the camera and the check, or where portions of the check are folded and appear at a different angle than another portion of the check in the image. Again, traditional, deterministic algorithms exist that can address certain of these distortions, however a deep learning (also referred to herein as machine learning) model is disclosed herein that may be trained to learn and apply the necessary translation, scaling, rotation and shearing of an image in a more robust manner that stand-alone algorithms.
(23) Issues with identifying information within a check image may also be a challenge for systems that capture and process check images. For example, the printed text on a check often cannot be read correctly due to background noise. Traditional systems and deterministic algorithms can have difficulty consistently removing noise from the printed text. This noise may be in the form of the overlap of a signature or other handwritten notes on the printed text, reflections or shadows that are present in the image, creases or wrinkles in the check, and so on. Here again, a machine learning model may be generated and trained to identify and remove background noise so that a clean image is left for processing.
(24) Yet another check processing task that institutions perform is duplicate check detection. Members of a bank or other financial institution may attempt to deposit the same check multiple times, either fraudulently or by accident. Existing techniques and systems for handling duplicate check detection typically involve a database of check numbers with a limited lookback window, or simply involve manual check reviews. These existing techniques are often subject to missed detection opportunities, and are ultimately susceptible to fraud losses. As also described herein, a machine learning model may be trained to recognize duplicate checks in a greater range of conditions than may be available in current techniques. In one implementation, a perceptual hash of an image may be encoded from the image that allows for an efficient image similarity check on a large scale that can handle many lighting and shadow variations.
(25) The image processing steps for detecting and extracting the check image from the background portions may be implemented in whole, or in parts, on either a user device having captured the digital image, a remote server running an application for remote deposit, or both. Implementing these image processing steps are known to expend significant computing resources on a computing device, and so there are benefits to offloading, or sharing, some or all of the image processing between the user device and the remote server. However, with the improvements to the computing capabilities of mobile computing devices, it is within the scope of this disclosure for one or more, or all, of the image processing features to be implemented directly on the mobile computing device (e.g., user device). For example, the ability to select between existing deterministic algorithm and trained machine learning models may be implemented at the initial remote deposit application download stage, where the mobile computing device and/or remote server device communicate to exchange mobile computing device capabilities.
(26) In one implementation, only a limited set of total available deterministic or machine learning enables techniques that the mobile computing device can efficiently be expected to execute will be transferred to the mobile device with the remote deposit application. In other implementations, the complete set of deterministic and trained machine learning models for image processing may be included in the downloaded app and the mobile computing device may select which of the algorithms or machine learning models to apply for a given image capture and processing session. Alternatively, the machine learning model need not be persistent on the mobile device and may be temporarily downloaded, on an as-needed basis, via a web interface or a previously downloaded application programming interface (API). Additionally, the download of the machine learning model(s) to the phone or other mobile device need not be directly from the financial institution or third party remote check deposit handling service. Instead, the model(s) or deterministic algorithms may be downloaded to the phone or mobile device from a local network inside a user's home.
(27) In the disclosed embodiments, features are disclosed for dedicating image processing from the remote server to the user device, as well as for dedicating image processing from the user device to the remote server. The determination of where the image processing will be implemented may be determined, for example, according to detected attributes of the captured digital image depicting the check, computing capabilities of the mobile device including the image capture device, and/or environmental conditions (e.g., ambient light levels) detected when the digital image was captured. Given the processing advantage typically afforded a server-based system as compared to a remote device, such as a handheld tablet or smart phone, the bulk of the processing may be described below as primarily taking place on a server of an institution rather than on the remote device of a user. However, some or all of the functionality and circuitry may be included on the user device in different implementations. Also, with respect to the training of deep learning (machine learning) models, the processing power of a remote server is generally used, regardless of the location of the processing that later implements the trained model. In some alternative implementations, training for one or more of the models discussed herein may be performed at the user device. The training of a machine learning model can be updated with localized data from end users. The end user data may be transmitted back to a central server periodically or in real-time for periodic or real-time training updates to better optimize the model.
(28)
(29) A negotiable instrument typically includes a type of contract that obligates one party to pay a specified sum of money to another party. Negotiable instruments may include checks, money orders, cashier's checks, drafts, bills of exchange, promissory notes, and the like. A check instructs a financial institution to pay an amount of money from a specific account held in the payor's name with that financial institution to an account held in the payee's name. A money order is a trusted financial instrument that is a payment order for a pre-specified amount of money. It is a more trusted method of payment than a personal check because the funds for the amount shown on the money order must be prepaid. A cashier's check (also known as a bank check, official check, teller's check, bank draft or treasurer's check) is a check guaranteed by a bank and may be purchased from a bank. Cashier's checks are usually treated as cash since most banks clear them instantly.
(30) For example, the computing device 109 may be a personal computer (PC), a laptop computer, a handheld computing device, a personal digital assistant (PDA), a mobile phone, or a smartphone, for example. The computing device 109 includes an image capture device 115 for capturing an image of the check, where the image capture device 115 may be a digital camera, image scanner, or other device in which the image of the check 107 may be obtained. In some embodiments, the image capture device 115 may be camera with multiple different lenses, or multiple separate cameras each with its own lens. Each camera or camera lens of a multiple lens image capture device 115 may be controlled individually or concurrently to capture one or more images of a check at the same time or in sequence. Each camera lens may be the same or different to capture the image of a check from differing angles or offsets from each other camera lens. Alternatively or additionally, the lenses and/or the associated camera sensor or circuitry in a multiple camera embodiment of an image capture device 115 may be configured to capture a different portion of the electromagnetic spectrum, for example each lens or camera sensor may be calibrated to a respective one of visible frequencies, infrared frequencies or ultraviolet frequencies.
(31) The user 105 may be an individual or entity who has an account 165 held at the institution 200 and is accessible via the institution system 205. The account 165 may be any type of account for depositing funds, such as a savings account, a checking account, a brokerage account, and the like. Although only one account 165 is shown, it is contemplated that the user 105 may have any number of accounts held at the institution 200. The user 105 may deposit a check 107 or other negotiable instrument document into the account 165 either electronically or physically. The institution 200 may process and/or clear the check 107, as well as other types of negotiable instruments.
(32) The user 105 may remotely electronically deposit the check 107 at the institution 200 via the computing device 109 being operated by the user 105. For example, an application (sometimes referred to herein as an app) for remotely electronically depositing checks may be downloaded, from a third party provider such as the APPLE App Store or from the institution system 205, and installed on the computing device 109, where running the application on the computing device 109 enables the user 105 to remotely electronically deposit the check 107. It is noted that although examples and implementations described herein may refer to a check, the techniques and systems described herein are contemplated for, and may be used for, any negotiable instrument, such as a money order, a cashier's check, a check guaranteed by a bank, or the like. Similarly, the techniques and systems described herein are contemplated for and may be used with any form or document whose image may be captured with a scanner, camera, or other imaging device for subsequent storage and/or processing.
(33) The user 105 may access the institution 200 via the institution system 205 by opening a communication pathway via a communications network 140 using the computing device 109. The communications network 140 may be representative of an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless fidelity (Wi-Fi) network, a public switched telephone network (PSTN), a cellular network, a voice over Internet protocol (VoIP) network, or the like. The user 105 may also communicate with the institution 200 or the institution system 205 by phone, email, instant messaging, text messaging, web chat, facsimile, postal mail, and the like.
(34) There may be several ways in which the communication pathway may be established, including, but not limited to, an Internet connection via a website 218 of the institution system 205, where the website 218 includes content 219 for accepting remote deposits. The user 105 may access the website 218 and log into the website 218 using credentials, such as, but not limited to, a username and a password.
(35) The user 105 may operate the image capture device 115 installed on the computing device 109 to generate a digital check image of the check 107. The digital check image may be used to create a digital image file 135 that is sent to the institution system 205 and used by the institution 200, in conjunction with the institution system 205, to process a deposit of the check 107 whose image is depicted within the digital image file 135. In an implementation, the digital image file 135 may be augmented by secondary data which may be information relating to the deposit of the check 107, such as an account number and a deposit amount, for example.
(36) When capturing the digital check image, the check 107 may be placed on a background, such that the digital check image that is captured by the image capture device 115 comprises an image of the check (e.g., a check image) and a portion of the background (e.g., a background image). Any background may be used, although a dark background or a consistently colored background may provide more optimal results for distinguishing the check image from the background image within the overall initial image. In an implementation, white and yellow backgrounds may not be used. It is noted that although examples and implementations described herein may refer to a check image and check data, the term check image may refer to the check in a digital image (as opposed to the background image) and the term check data may refer to any data in a digital check image (as opposed to background data). Thus, check image and check data may refer to the non-background portion of the image and data from the non-background portion of the image in implementations involving any negotiable instrument, form, or document.
(37) In an implementation, the digital check image generated by the image capture device 115 comprises check data and background data. The check data pertains to the check image in the digital check image and the background data pertains to the background in the digital check image on which the image of the check 107 is disposed. An example of such a digital check image 201 is further described with respect to
(38) In an implementation, a minimum ratio of check image size (or area) to background image size (or area) may be provided in the digital check image generated by the image capture device 115. Examples include, and are not limited to, at least a minimum portion (e.g., five percent) of the digital check image may comprise a check image. Other examples of check image to background size include at least a minimum portion (e.g., 1 percent) of the digital check image may comprise a background image, and the check image may be surrounded by at least one pixel of background all the way around the check image, or other predetermined requirements.
(39) Referring back to the computing device 109 shown in
(40) When a deterministic algorithm is selected for processing the digital check image 201 depicting the check 107, the background processor circuitry 121 may perform various tasks with respect to the background image 250 included on the digital check image 201. The background image 250 may be a surface on which the check 107 is placed when the digital check image 201 is captured by the image capture device 115. In conjunction with the image capture device 115, the background processor circuitry 121 detects portions of the digital check image 201 corresponding to the background image 250, and distinguishes the background image 250 from the check image 210. The background processor circuitry 121 may further generate a modified digital check image that replaces the background image with a replacement background image. The replacement background image may be selected from one or a plurality of predetermined background images. For example, a predetermined background image may be comprised of a solid color. In an implementation, the background processor circuitry 121 may store the background image 250 and/or the modified digital check image in a storage memory associated with the computing device 109.
(41) Based on the background image 250 being separately detected from the check image 210 according to the techniques applied by the background processor circuitry 121, the image processing engine 120 may identify specific edge coordinates of the check image 210, such as edge coordinates 301, 302, 303, 304 illustrated in
(42) An example of an existing deterministic algorithm, as compared to the machine learning model developed from deep learning approaches discussed in greater detail below, is first discussed. In an implementation, the deterministic approach to finding edges of a check may utilize chroma key technology to detect edge coordinates 301, 302, 303, 304 of the check image 210, and/or provide a uniform or consistent replacement background for the check image 210 in the digital check image 201. Chroma key is a technique for mixing two images together, in which a color or color range from one image is removed (or made transparent), revealing another image behind it. This technique is also known as color keying, green screen, and bluescreen. The chroma key analyzer 122 may use any known chroma key technique for removing the original background image 250 and replacing it with (e.g., revealing) a replacement background image. Based on the background image 250 being separately detected from the check image 210 according to the chroma key technique(s) applied by the chroma key analyzer 122, the image processing engine 120 may identify specific edge coordinates for the check image 210, such as the edge coordinates 301, 302, 303, 304 illustrated in
(43) Alternatively or additionally to the deterministic chroma key approach, the background image 250 may be separately distinguished from the check image 210 within the digital check image 201 based on a difference in image characteristics between the background image 250 and the check image 210 analyzed by the image comparator 123. After generating a modified digital check image that replaces the background image 250 with a predetermined replacement background image, the image comparator 123 may compare the modified digital check image with the predetermined replacement background image. By making this known image comparison, the image comparator 123 determines that overlapping the overlapping image portions correspond to a background portion, and the remaining image portions correspond to the check image 210. The image comparator 123 may then determine this difference and subtract the background portion from the modified digital check image to result in obtaining the check image 210 alone without background portions. Based on the check image 210 being separately extracted from the digital check image 201 according to the image comparison technique(s) applied by the image comparator 123, the image processing engine 120 may identify specific edge coordinates for the check image 210, such as the edge coordinates 301, 302, 303, 304 illustrated in
(44) In addition, or alternatively, edge detection for the check image may be implemented by comparing the digital check image 201 to a predetermined rectangular shape that represents an outline of the check. This way, the check image 210 may be separately distinguished from the background image 250 within the digital check image 201 based on a matching of the predetermined rectangular shape with edge outlines of the check image 210 analyzed by the image comparator 123. After matching the predetermined rectangular shape to the edge outlines of the check image 210, the image comparator 123 may identify the edge coordinates of the check image 210. Based on the check image 210 being separately identified from the digital check image 201 according to the image comparison technique(s) applied by the image comparator 123, the image processing engine 120 may identify specific edge coordinates for the check image 210, such as the edge coordinates 301, 302, 303, 304 illustrated in
(45) According to some embodiments, image processing, such as the edge detection, may be applied to segmented pieces of the digital check image 201. The grayscale converter 124 may convert the digital check image 201 generated by the image capture device 115 to grayscale using known techniques. In photography and computing, a grayscale digital image is an image in which the value of each pixel is a single sample, that is, it carries only intensity information. Images of this sort are composed exclusively of shades of gray, varying from black at the weakest intensity to white at the strongest. Conversion of a color image to grayscale is well known and any known technique(s) may be used.
(46) The grayscale image generated by the grayscale converter 124 may be provided to the image segmenter 125, where the image segmenter 125 divides the image into a predetermined number of segments, such as 4 segments, 6 segments, 8 segments, or other predetermined number. Each segment may comprise a portion of the check image 210 (i.e., the check data) and a portion of the background image 250 (i.e., the background data) and an edge between the two portions. The segments may be equal in size or may differ in size. An example of a segmented image 400 divided into four segments is shown in
(47) The segment processor circuitry 126 may process each of the segments to identify and/or remove background data. In an implementation, histograms may be used to detect or identify background data and check data. The histogram of each segment can be used to determine an edge of the check image 210 from the background image 250 within the segment, so that the background image 250 may be removed or disregarded for subsequent processing.
(48) A histogram is a well-known graph and may be used to display where all of the brightness levels contained in an image are found, from the darkest to the brightest. These values may be provided across the bottom of the graph from left (darkest) to right (brightest). The vertical axis (the height of points on the graph) shows how much of the image is found at any particular brightness level. An example of a histogram 500 for a segment of an image comprising check data and background data is further described with respect to
(49) The segment recombiner 127 recombines the processed segments to generate a recombined image comprising a depiction of the check 107. Based on the recombined image, the image processing engine 120 may identify specific edge coordinates for the check depicted in the recombined image, such as the edge coordinates 301, 302, 303, 304 illustrated in
(50) In addition to, or instead of, the deterministic techniques for various image processing tasks, such as the edge detection and corner identification noted above, the image processing engine 120 may include a machine learning model repository 129 containing one or more trained machine learning models for each desired image processing task. Referring to
(51) In order to improve on standard algorithmic/deterministic techniques for identifying the check 210 from the background and finding the corners or edges of the check, an ML model, such as an image cropping model 700 (See
(52) Referring to
(53) In one implementation of training this image cropping model 700, a feature may be a two dimensional array of pixels (for example a 33 array of 9 total pixels) with a particular pattern. The pattern may be, for example, a specific number and location in the 33 array of dark pixels. That feature is then convolved with different 33 windows 704 of pixels of the test check and the result is stored in the first level feature map 706 for that feature. Essentially, the 33 window 704 of a feature is shifted by one pixel and convolved with the test check pixels at that location until the window has been scrolled through the entire test check image. This process of convolution is repeated for each feature (each different 33 pixel pattern in this example) until all of the predetermined features have had a respective feature map 706 generated of a plurality of features. Although the feature sets used in this step may include every possible permutation of numbers dark pixel and patterns for a 33 array, the computational power and time necessary to handle all permutations may be unnecessarily large and so only a fixed number of patterns, which may be predefined by the particular machine learning model, are typically convoluted with the test check into respective feature maps.
(54) After creating the initial feature maps 706, the process of training the image cropping model illustrated in
(55) Similarly, rather than use a discrete, deterministic algorithm to de-skew or de-warp check images, the model repository 129 of the image processing engine 120 may also include a trained demorphing transformation machine learning model 800. Referring to
(56) As illustrated in
(57) As illustrated in
(58) As shown in
(59) Yet another trained model that may be included in the model repository 129 of the image processing engine 120 is a duplicate check detection model 1000. Unlike the image cropping model 700, the demorphing model 800 and the denoising and binarization model 900, the duplicate check detection model 1000 is trained determine if a currently processed check is similar enough to be flagged as a duplicate of a previously deposited check. As illustrated in
(60) As shown in
(61) As noted earlier with respect to
(62) The image processor circuitry 128 of the computing device 109, alone or in combination with other of the components comprising the image processing engine 120, may operate to implement one or more image processing features such as to remove warping or de-warp a digital image, to crop a digital image, to de-skew a digital image (e.g., rotate the image to horizontal or vertical alignments), to identify the corners/edges within a digital image, or adjust brightness, tint, contrast, or other image attribute of a digital image, or other known digital image processing capability. In one implementation, the image processor circuitry 128 may include a single processing element, or multiple separate processing elements, such as embedded processors including specialized graphics processors known as graphics processing units (GPUs) available from NVIDIA Corporation of Santa Clara, Calif.
(63) The user 105 may thus generate a digital check image of the check 107 using the image capture device 115 on the computing device 109. For example, after endorsing the check 107, the user 105 may operate the image capture device 115 to capture a digital image of the front sides and/or back sides of the check 107 and store the digital image(s) as a digital image file on a storage memory of the computing device 109. The images of the front side and the back side of the check 107 may be processed using the techniques described herein. The images may be processed as separate files or as images in a single file. The images of the front side and the back side of the check 107 may be captured sequentially, e.g., pursuant to the user 105 flipping the check 107 over after an image of the front of the check 107 has been captured.
(64) The digital image file 135 comprising the digital image depicting the check 107 may be transmitted to the institution system 205. The digital image may be the original digital image captured by the image capture device 115, or a modified digital image having one or more image processing techniques applied. For example, the modified digital image may include the edge coordinates 301, 302, 303, 304 described herein. The computing device 109 transmits the digital image file 135, which may include secondary data, to the institution system 205 along with a request to deposit the check 107 into an account, such as the account 165. In an implementation, the user 105 may attach the digital image file 135 to an email, short message service (SMS) text, or other form of electronic message, and send the digital image file 135 to the institution system 205 using the computing device 109. However, other techniques for sending a digital image file 135 to the institution system 205 may be used, such as providing a digital image file 135 from storage to the website 218 associated with the institution system 205.
(65) The institution 200 in conjunction with the institution system 205 may process the deposit request according to the digital image and any secondary data included in the digital image file 135. Thus, the institution 200 in conjunction with the institution system 205 may process the digital image file 135 comprising the digital image depicting the check 107 for deposit.
(66) In an implementation, the institution system 205 may extract the check image 210 from the digital image file 135 and process the check 107 from the digital image for deposit, according to any one or more of the image processing techniques described herein. The institution system 205 may use only machine learning models, such as described in
(67) Upon receipt and processing of the digital image file 135 and approval of the check 107 associated therewith, the institution 200 may credit the funds of the check 107 to the account 165. It will be appreciated that the examples herein are for purposes of illustration and explanation only, and that an embodiment is not limited to such examples.
(68) In an implementation, the computing device 109 may be a mobile device that comprises a digital camera which can capture a digital image of the check 107 by taking a picture of the front and/or back of the check 107. The back of the check may provide endorsement verification, such as the signature of the person or party the check is made out to. The user 105 may send the digital image file 135, including the check image, to the institution system 205 using the mobile device. An exemplary computing device 109 is described with respect to
(69) The institution system 205 may include any combination of systems and subsystems such as electronic devices including, but not limited to, computers, servers, databases, or the like. The electronic devices may include any combination of hardware components such as processors, databases, storage drives, registers, cache, random access memory (RAM) chips, data buses, or the like and/or software components such as operating systems, database management applications, or the like. According to an embodiment, the electronic devices may include a network-based server that may process the financial information and may receive the digital image file 135 from the computing device 109.
(70) The electronic devices may receive the digital image file 135 and may perform an initial analysis on the quality of the image of the check 107 in the digital image file 135, the readability of the data contained therein, or the like. For example, the electronic devices may determine whether the amount payable and other information may be readable such that it may be obtained and processed by the institution system 205 to credit the account 165 associated with the user 105.
(71) The institution system 205 may include a user interface circuitry 220, an image processor 222, and a data source access engine 227. The user interface circuitry 220 may generate and format one or more pages of content 219 as a unified graphical presentation that may be provided to the computing device 109 or a representative computing device 187. In an implementation, the page(s) of content 219 may be provided to the computing device 109 and/or the representative computing device 187 via a secure website 218 associated with the institution system 205.
(72) In an implementation, the institution system 205 may use the image processor 222 to process the digital image file 135 comprising the image(s) 137 of the check 107 received from the user 105 for use by the institution 200 in the processing and/or clearance of the check 107. The image processor 222 may process multiple frames of the image if the image is comprised of multiple frames (e.g., the front side and the back side of a check).
(73)
(74) The GUIs 1205 and the I/O interface circuitry 1206 may include touch sensitive displays, voice or facial recognition inputs, buttons, switches, speakers and other user interface elements. Additional examples of the I/O interface circuitry 1206 includes microphones, video and still image cameras, headset and microphone input/output jacks, Universal Serial Bus (USB) connectors, memory card slots, and other types of inputs. The I/O interface circuitry 1206 may further include magnetic or optical media interfaces (e.g., a CDROM or DVD drive), serial and parallel bus interfaces, and keyboard and mouse interfaces.
(75) The communication interfaces 1208 may include wireless transmitters and receivers (transceivers) 1210 and any antennas 1212 used by the circuitry of the transceivers 1210. The transceivers 1210 and antennas 1212 may support Wi-Fi network communications, for instance, under any version of IEEE 802.11, e.g., 802.11n or 802.11ac, or other wireless protocols such as Bluetooth, Wi-Fi, WLAN, cellular (4G, LTE/A). The communication interfaces 1208 may also include serial interfaces, such as universal serial bus (USB), serial ATA, IEEE 1394, lighting port, I2C, slimBus, or other serial interfaces. The communication interfaces 1208 may also include wireline transceivers 1214 to support wired communication protocols. The wireline transceivers 1214 may provide physical layer interfaces for any of a wide range of communication protocols, such as any type of Ethernet, Gigabit Ethernet, optical networking protocols, data over cable service interface specification (DOCSIS), digital subscriber line (DSL), Synchronous Optical Network (SONET), or other protocol. The communication interfaces 1208 may communicate with remote computing devices via a network, such as the communications network 140.
(76) The system circuitry 1202 may be representative of any combination of hardware, software, firmware, application programming interface, or other circuitry for implementing the features of the remote deposit application described herein. For example, the system circuitry 1202 may be implemented with one or more systems on a chip (SoC), application specific integrated circuits (ASIC), microprocessors, discrete analog and digital circuits, and other circuitry. The system circuitry 1202 may implement any of the image processing features described herein. As an example, the system circuitry 1202 may include one or more processors 1216 and memory 1220.
(77) The memory 1220 stores, for example, control instructions 1223 for executing the features of the remote deposit application running on the computing device 109, and/or institution system 205, as well as an operating system 1221. In one implementation, the processors 1216 execute the control instructions 1223 and the operating system 1221 to carry out any of the features for the remote deposit application. For example, the control instructions 1223 for the remote deposit application includes instructions that, when executed by the processors 1216, implement the features corresponding to the image processing engine 120, which may include the deterministic image processing algorithms and or the ML models discussed previously. The memory 620 also includes control parameters 1222 that provide and specify configuration and operating options for the control instructions 1223 (such as the image processing technique selection instructions discussed herein), operating system 1221, and other functionality of the computer architecture 1200.
(78)
(79) At 1301, an account owner (e.g., payee to an amount written on a check document, referred to herein as a user) may receive a check from a third party (e.g., payor of the check), and may endorse the check by signing the back of the check in the designated field. If the user wishes to deposit the check into an account, such as a savings and/or checking account, the user may also write in an account number below the signature on the back of the check.
(80) At 1302, the user may operate the computing device 109 to open a communication pathway with the institution system 205 associated with an account of the user for depositing funds. The communication pathway may be implemented by logging into a website operating within the institution system 205, for example. In addition to, or alternatively, a remote deposit application may be downloaded onto the computing device 109. Executing the remote deposit application to run on the computing device 109 may initiate opening of the communication pathway. The communication pathway may be established after the user authenticates access to the institution system 205 using user specific authentication credentials such as, but not limited to, a username and a password, or biometric information such as fingerprints, voice recognition, or facial recognition information.
(81) At 1303, a digital check image of the check may be captured by the image capture device 115. The user may manually operate the image capture device 115 to capture the digital check image to include the check and a background portion. According to some embodiments, the remote deposit application running on the computing device may automatically capture the digital check image when a set of predetermined criteria is determined to be satisfied. For example, the remote deposit application may control the image capture device 115 to capture the digital check image after detecting the check is aligned within an alignment guide displayed on a display screen of the computing device 109, where the display screen displays a field of view of the image capture device 115. According to another example, the remote deposit application may control the image capture device 115 to capture the digital check image after detecting the check is in focus for the digital image capture by the image capture device 115. According to another example, the remote deposit application may control the image capture device 115 to capture the digital check image after detecting the check comprises at least a predetermined portion of the digital check image to be captured by the image capture device 115.
(82) Also at 1303, a deposit request is initiated. The deposit request may include selecting an account in which to deposit the check. In an implementation, the user may select a deposit check option provided on a graphical user interface (GUI) displayed on the computing device 109. The user may also input one or more of the following check information through the GUI: check amount, date, the account the check funds should be deposited in, comments, routing number, or other check data.
(83) At 1304, edge detection and corner identification is executed by the image processing engine 120 according to any one or more of the edge detection processes described herein. In one implementation, the cropping machine learning model 700 is executed by processing the captured digital image using the trained model. The edge detection process includes distinguishing the check image from the background image that comprises the overall digital check image captured by the image capture device 115. By executing the edge detection on the digital check image, edge coordinates for the check image are determined. For example, the edge detection may determine the edge coordinates 702 as shown in
(84) According to some other embodiments, deterministic algorithms may be used instead of applying the machine learning model. In those other embodiments, the edge detection implemented at 1304 may include one or more of the following processes: scaling down the digital check image for faster processing (scale factor depends on image size), converting the digital check image to grayscale, using close and/or open morphs on the digital check image to eliminate noise, binarizing the digital check image using a dynamic threshold algorithm (e.g., Otsu algorithm), running through multiple cropping logic algorithms, e.g., using OpenCV, until a check is found within the digital check image (e.g., 4 predetermined algorithms may be applied). To determine if a crop was successful, the crop result may be run through a crop validator that checks aspect ratio, area, skew, zoom, and/or other image analysis for validating a successful crop. The resulting digital check image may be run through focus detection to determine if the digital check image is blurry, and a refocusing process may be applied if blurriness is detected within a predetermined range. After multiple successful digital check images have been processed, the final digital check image is ready (e.g., considered a modified digital check image), along with four corner check coordinates and/or an image size of the modified digital check image.
(85) At 1305, a modified digital check image (e.g., modified digital check image 300) is generated that includes the edge coordinates (e.g., bounding box coordinates 702 from the cropping model as shown in
(86) At 1306, the modified digital check image may undergo one or more further image processing steps. The image processing steps may include any one or more of the image processing functions attributed to the image processor 128, as described herein. These one or more additional image processing steps may include only processing the modified image via machine learning models such as those described above, only processing the image using techniques based on deterministic algorithms, or a mixture of both.
(87) At 1307, the resulting processed image is stored within a digital image file. According to some embodiments, the digital image file may also include supplemental information, as described herein. At 1308, the digital image file is transmitted to the institution system 205. After receipt by the institution system 205, the institution system may further process the digital image file, and deposit the funds corresponding to the check depicted in the digital image file. The digital image file may include the digital check image in a known digital image format (e.g., a JPEG digital image file). According to some embodiments, the digital image file may further include edge coordinate information for edges detected within the digital check image by the image processing engine 120 at the computing device 109. The edge coordinate information may be in the following exemplary formats (exemplary coordinates are provided):
(88) {
(89) frontImageCoordinates: [ 186.0, 282.0, 1368.0, 294.0, 1374.0, 828.0 174.0, 798.0
(90) ],
(91) frontImageWidth: 1440,
(92) backImageWidth: 1500,
(93) frontImageHeight: 1080,
(94) backImageHeight: 1000
(95) }
(96) Further description of the processes that may be implemented on the digital image file by the institution system is described with reference to the flow diagram 800 shown in
(97) At 1402, edge detection is executed by the institution system 205 according to any one or more of the edge detection processes described herein. The edge detection process results in distinguishing the check image from the background image that comprises the overall digital check image captured by the image capture device 115. By executing the edge detection on the digital check image, edge coordinates for the check image are determined. For example, the edge detection may determine bounding box coordinates 702 from the trained machine learning cropping model as shown in
(98) According to some embodiments, the edge detection executed by the institution system 205 may be different from the edge detection executed by the image processing engine 120 of the computing device 109. In one implementation, the edge detection at the institution system may be a machine learning model such as discussed above, while the edge detection at the computing device may be a deterministic algorithm version. In other implementations, the computing device and the institution system may both use different trained machine learning models, or may each use different deterministic algorithm edge detection techniques.
(99) For example, with respect to the deterministic techniques of edge detection, the edge detection at 1402 at the institution system 205 may include one or more of the following: scaling the digital check image by a predetermined amount (e.g., scale down the digital check image by a half ()), converting the digital check image to a gray scale image, using close and/or open morphs on the digital check image to eliminate noise, binarizing the digital check image using a dynamic threshold algorithm (e.g., Otsu's method), and/or applying Hough transform algorithm to identify lines in the digital check image. The edge detection at 1402 may further include iterating through all the identified lines from the digital check image, and discarding away any lines that do not correspond to a check edge, where a check edge may be determined according to one or more of the following rules: vertical lines must be angled between 95 and 85 degrees or between 275 and 265 degrees, horizontal lines must be angled between 195 and 175 degrees or between 360 and 355 degrees, discard away lines that lie completely in the top fourth or bottom fourth of the image as these lines have a high likelihood of being image noise data, discard away lines that lie completely in the right fourth or left fourth of the image as these lines have a high likelihood of being image noise data, and/or find the average distance between every line and the center of the digital check image and throw away any outlier lines that are more than a predetermined distance from the center of the digital check image. The edge detection at 1402 may further include creating a bounding box around remaining lines after the line removal process, as the remaining lines have a high likelihood of actually corresponding from the check depicted in the digital check image. If creating the bounding box around the remaining lines does not produce a check21 valid image, then another attempt to create a bounding box around contours of the digital check image may be implemented. The contours may be detected according to a contour detection algorithm such as the Suzuki85 algorithm.
(100) At 1403, conditions detected during the edge detection implemented by the image processing engine 120 may be compared to conditions detected during edge detection implemented by the institution system 205. The conditions may include any environmental conditions (e.g., brightness levels) that may have affected the quality of the digital check image captured by the image capturing device 115. In addition or alternatively, as the edge detection implemented by the institution system 205 may be different from the edge detection implemented by the image processing engine 120 of the computing device 109, an accuracy of the edge detection implemented by the image processing engine 120 may be compared to an accuracy of the edge detection implemented by the institution system 205.
(101) At 1404, the institution system 205 selects one of the edge coordinates from the edge coordinates included in the digital image file received from the computing device 109, or the edge coordinates determined by the institution system 205. The institution system 205 may select the edge coordinates based on predetermined criteria such as, for example, related to the detected conditions, or related to the determined accuracy of the edge coordinates. When the selection is based on the detected conditions, the institution system 205 may select the edge coordinates determined by the image processing engine 120 on the computing device 109 when the conditions indicate the edge detection processes implemented by the image processing engine 120 would result in a greater accuracy than the edge detection processes implemented by the institution system 205. When the selection is based on the accuracy of the edge detection, the institution system 205 may select the edge coordinates determined to be more accurate.
(102) At 1405, the institution system 205 identifies the check image from the digital image file based on the selected edge coordinates. From the identified check image, the check data contained within the check image may be extracted.
(103) At 1406, the institution system 205 processes the check depicted in the check image to extract the check data contained therein. Processing of the digital image file may include retrieving financial information regarding the check. The financial information may comprise the MICR number, the routing number, a check amount, or other information found on the check. The processing step may include implementation of one of more of ML models such as the dewarping 800, or denoising 900, models discussed above, or may implement deterministic techniques using known algorithms, before extracting financial information from the image.
(104) At 1407, after retrieving the financial information from the check in an electronic data representation form, the institution system 205 determines whether the financial information is valid. For example, the institution system 205 may perform an initial analysis on the quality of the data representation, the readability of the data representation, or the like. For example, the electronic devices may determine whether the account number, amount payable, or the like may be readable such that they may be parsed and processed by the institution to credit an account associated with the user. If the financial information is determined to be valid, the electronic data representation may be processed by the institution system 205, thereby depositing the money in the user's account 165. If the financial information is determined to be invalid, then the user may be advised. For example, the institution system 205 may transmit an email, a web message, an instant message, or the like to the user indicating that the financial information associated with the electronic data representation may be invalid. The user may determine how to proceed by selecting an option on the web message, replying to the email, or the like.
(105) The remote deposit software application, or app, that a user computing device 109 downloads from a third party or institution may be customized by the third party or institution. In one embodiment, the ability for a computing device 109 that is user operated, such as a smartphone, tablet or other computing device, to handle certain machine language models for image processing or duplicate check detection may be determined when the app is requested, such that the computing device is only provided those features or types of image processing software that the computing device can properly execute. For example, the hardware and operating system information for the computing device 109 may be provided to the third party or institution system 205 and the computing device 109 capabilities assessed by the third party app provider or institution. Referring to
(106) Upon receipt of the information, the institution system 205 may determine if the computing device of the user is capable of handling the processing and memory requirements for each machine learning model that the institution system 205 or third party app store can include in the downloadable remote deposit app (at 1504). The institution system 205 may, for example, be able to select from one or more machine learning models and deterministic models and match the computing device capabilities with only machine learning models, in type or number, that the computing device can effectively handle (at 1506). Alternatively, or in addition, to the computing device capability check, the institution system 205 may selectively include machine learning models or deterministic models in the remote deposit app that will differ from any machine learning or deterministic algorithm techniques the institution system 205 will itself be using on check images sent via the app during a remote deposit process.
(107) After selecting the machine learning modes and/or deterministic technique instructions the computing device 109 can best utilize, the downloadable app may be configured to include those models or techniques and the institution system 205 may transmit them over the network connection to the computing device (at 1508). In yet other implementations, the downloadable app may be configured by the institution server 205 with a plurality of machine learning models and/or deterministic algorithm instructions at the time the app is first downloaded, and the institution server 205 may dynamically select which of the machine learning models or deterministic techniques to activate at the computing device 109 at the time a remote deposit query is initiated by the computing device.
(108) In another embodiment illustrated in
(109) In an alternative embodiment, the process of
(110) The first trained machine learning model may be executed on the computing device 109 to capture an initial image of the check and automatically determine the image capture environment (for example, lighting, shadows, skew and/or noise) for the check that is captured, and may also may determine the type of check and the check design in the image (at 1704). Based on the information it has been trained to look for regarding image capture environment and check type and design, the first machine learning model may determine if there is an optimal trained second machine learning model that may be used to handle image processing and extract the check information (at 1706). The determination of whether a second model is optimal may be a comparison of percentage success rates, for the determined environment and check type/design, for a known plurality of specialized second machine learning models (each trained more rigorously for particular different environment or check type/design) in comparison to a default second trained model or deterministic technique that already may be part of the earlier downloaded application. If there is no second trained model that matches the determined environment and/or check type/design such that it would be more optimal than a more generalized default second machine learning model already present on the computing device 109, then the default second machine learning model may be used (at 1712). If there is a better match to the environment or check type/design in a second machine learning model known to the first machine learning model, then that optimal second machine learning model may be downloaded to the computing device 109 and used to process the captured image (at 1708, 1710).
(111) In yet other embodiments, alone or in combination with any of the above, it is further contemplated that a trained machine learning model may also output computing device specific instructions to alter one or more settings of the computing device 109. Referring to the example of
(112) In embodiments where the image capture device consists of a camera with multiple lenses or multiple cameras as noted previously, the processor of the user's computing device 109 may execute downloaded instructions to assist with check edge detection/corner identification, via a trained machine learning model or a discrete algorithm, by adding in the factor of depth perception available from simultaneously captured images from the multiple cameras that are separately positioned as the image capture device.
(113) The use of multiple camera lenses/multiple cameras as the image capture device on a single computing device 109 can not only increase the detail provided on the checks themselves to improve the processing and accuracy of data pulled from the images, but can enhance security. In one implementation, the computing device 109 can use different information acquired by each lens of the same check to better identify/authenticate the user of the user device. As one example, the rotational angle and distance of each lens to the check during image capture of the check maybe used to determine a more precise angle of the user device to the check. This information may then, in turn be used in a deterministic algorithm or a machine learning model to compare to prior measurements of user device angle of rotation and distance from a check typically measured for that user. The multiple camera lens image capture device information may be used in combination with, or separately from, other user identifiable information entered into or gathered by the computing device. User entered data may include name, password, biometric, token or other direct verification information. This information may be combined with Global Positioning System (GPS) location information and sensed user information, such as how the computing device is being held via the multiple camera lenses, to modify a confidence level that the user is who he or she claims to be.
(114) Deposit suspension actions, or additional verification tasks, may be triggered at the computing device when one of more of the verification criteria do not match expectations of the system for a given user. If one of more the rotational angle or distance measurements do not result in the computing device being held in a manner expected for that particular user, then an additional authentication step may be triggered at the computing device 109 and any further deposit processing stopped. For example, the processing circuitry may execute an application running on the computing device 109 that suspends check processing for that image until an additional authentication step (e.g. user is presented with an additional authentication challenge question, or an additional biometric assessment or measurement is requested) is successfully completed. Alternatively, a machine learning user verification model may be implemented that has been trained to assess all of the available user-entered and device-sensed parameters and automatically allow a deposit transaction to proceed when enough of the parameters provide a threshold confidence level of user authenticity.
(115) The machine learning model may implement different levels of verification according to predetermined transaction risk levels. For example, the computing device may only utilize the rotational angle and/or distance measurements from each lens to recognize a particular user's usage pattern of the computing device when an amount of the check deposit is above a minimum amount. In this manner higher fraud risk transactions will trigger the user device usage pattern measurement and lower fraud risk (e.g. lower transaction amounts) will not trigger the elevated user verification mechanism. Computing device 109 processing resources and power usage may thus be reduced except for when higher risk transactions are involved.
(116) Other implementations of a multiple camera lens image capture device may include examining details of the check the user plans to deposit to verify user-specific features. In this embodiment, the user-specific aspects of the check to be deposited may can be relied on alone, or in combination with other user verification methods, such as the device orientation pattern noted above, to verify a user or strengthen the collection of user verification parameters used to verify the user of the computing device. Rather than utilizing the various different perspectives of the multiple lenses to identify a way a user is holding the user device, the multiple camera lenses can take a more critical look at the check to be deposited to identify bends, creases, folds or tears in the check that match a pattern of those parameters found in prior checks deposited by that user. As one example, a machine learning model may be trained from past deposits from the user to look for a location of creases in a check representing a fold pattern in checks that the particular user puts in checks he or she deposits (e.g. from a habit of folding a check in a certain way, placing it in a wallet, and then later unfolding it for a remote deposit operation). As another example of check condition parameters that may be used to identify the user via a trained machine learning model, the multiple lens image capture device may be used to acquire information on the depth of the indentations made by the user's signature endorsing the check, the color and content of the ink used in the signature, the positioning of the endorsement signature and the endorse, endorsement signature itself. One or more of these pieces of information may be applied to a trained machine learning that will either permit a deposit transaction to move forward, or will interrupt and pause the deposit process for additionally authentication instructions to be displayed to the user and acted on before further processing is permitted.
(117) In addition to the computing device usage and distinguishing check characteristics that may be tracked to determine or verify the user of the computing device, the background surrounding the check in the captured image may be used to verify authenticity of a user and raise the confidence level that a transaction is not fraudulent. Assuming a user typically deposits checks from a limited number of locations (e.g. kitchen counter or desk at home), then a same background or set of different backgrounds surrounding the check can be expected for deposits from the user. In one implementation, the image processing engine 120 of the computing device 109 may include in its machine learning model repository 129, or download from a remote server, a trained machine learning user verification model that identifies one or more of the device usage patterns, check feature patterns or background patterns of image capture transactions for a particular user. With respect to use of background patterns to verify user identity, a multi-camera lens image capture device may not only capture the overall pattern of the background, but may determine texture (e.g., wood grain, surface roughness or carpet fiber texture) and finer light quality details (content of light spectrum of lighting in area where image is being captured, amount or direction of types of lighting that may be analyzed in the trained machine learning model to verify a user or at least a trusted location that is being used for the user's deposit activities. This background determination, via a trained machine learning model established from that user's past deposits, may provide another layer of verification that the deposit transaction is from a legitimate source.
(118) The features of how a computing device is being held or operated, where the computing device is located, and of the user treatment of the check to be deposited, may be the focus of separate of combined trained machine learning models that execute on the computing device or remotely via data provided by the computing device. Each user may have a separate account stored locally on the user computing device, at a financial institution server, or in cloud storage that stores previously captured checks and images from that user that is used to build a history for that user and that may be used to train a machine learning/artificial intelligence-type model to profile the user for authentication and fraud prevention in future remote deposit transactions.
(119) In alternative embodiments, the computing device may also use the trained machine learning model(s) related to user verification to switch transaction modes from a check deposit transaction to a different transfer format. In one implementation, the computing device may search for the availability of another funds transfer mechanism to substitute for the more rigorous check deposit process. As one example, a person-to-person (P2P) transaction that bypasses typical check clearance and other check deposit procedures may be automatically investigated by the computing device. For example, if enough of the authentication/verification parameters have been satisfied in a predetermined number of transactions for the user, the computing device may automatically use the data gathered from the current check image to trigger a P2P transaction from the payor bank to the payee bank rather than proceed with a check deposit transaction. In this embodiment, the current image captured by the computing device may use the heightened authenticity scrutiny available from processing the check image captured (e.g. via the multiple camera lens image capture device) through the one or more machine learning models trained with user specific data on device usage, check features and/or background parameters. If the current image meets the authenticity and user verification threshold, and the user has met a threshold number of prior check deposit transactions also satisfying the authenticity and user verification tests, then the computing device may either automatically contact the payor and payee banks to initiate a P2P transaction, or it may interrupt the current check deposit operation to query the user, via the display of the computing device, regarding whether to proceed with a check deposit or to instead switch the transaction to a P2P transaction.
(120) It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination of both. Thus, the methods and apparatus of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium. When the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. One or more programs may implement or use the processes described in connection with the presently disclosed subject matter, e.g., through the use of an API, reusable controls, or the like. Such programs may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language and it may be combined with hardware implementations.
(121) Although exemplary embodiments may refer to using aspects of the presently disclosed subject matter in the context of one or more stand-alone computer systems, the subject matter is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, aspects of the presently disclosed subject matter may be implemented in or across a plurality of processing chips or devices, and storage may similarly be spread across a plurality of devices. Such devices might include personal computers, network servers, and handheld devices, for example.
(122) Various implementations have been specifically described. However, other implementations that include a fewer, or greater, number of features for each of the apparatuses, methods, or other embodiments described herein are also possible.