System and methods for performing automatic data aggregation
11526579 · 2022-12-13
Assignee
Inventors
Cpc classification
G06N7/01
PHYSICS
International classification
Abstract
Systems, apparatuses, and methods for automated data aggregation, automated webpage navigation, or automatically performing a task by entering data into multiple webpages. In some embodiments, this is achieved by use of techniques such as natural language processing (NLP) and machine learning to enable the automation of data aggregation and other tasks involving websites without the use of pre-programmed scripts.
Claims
1. A method of automating performance of a task for a user, comprising: receiving from a user an identification of a website and a task they want to have automated; navigating to a first webpage associated with the website; extracting one or more intent model features from the first webpage, the one or more intent model features including visual and textual features; accessing a trained intent model and providing the extracted intent model features from the first webpage as an input to the trained intent model, wherein the trained intent model operates to generate a prediction of one or more actions associated with the webpage and a corresponding confidence level for each of the one or more actions; extracting one or more target model features from the first webpage, the one or more target model features including both visual and textual features; accessing a trained target model and providing the extracted target model features from the first webpage and the generated prediction of one of the one or more actions associated with the webpage as an input to the trained target model, wherein for each of the one or more actions associated with the webpage, the trained target model operates to generate a prediction of one or more elements of the webpage that will cause the action to be performed and a corresponding confidence level for each of the one or more elements; executing one of the one or more actions associated with the webpage by interacting with a highest ranked element of the webpage for causing the one of the one or more actions, wherein executing the one of the one or more actions results in either navigating to a second webpage or performing the task; and if the task has not been performed, then repeating one or more of the preceding steps on the second webpage and on any subsequent webpages until the task is performed.
2. The method of claim 1, wherein the trained intent model comprises a multiple component model, and further, wherein the trained intent model utilizes one or more of a natural language processing technology and a machine learning technology to predict the action associated with the webpage.
3. The method of claim 1, wherein the method further comprises receiving a set of credentials from the user and using the received credentials to login to a user account accessible from the website.
4. The method of claim 1, wherein the extracted intent model visual features include an image of the webpage, and the textual features include one or more of text contained in a document object model or file describing elements of the webpage and text visible on the webpage.
5. The method of claim 1, wherein the extracted target model visual features include an image of an element on the webpage and the textual features include one or more of text contained in a document object model or file describing elements of the webpage and text visible on the webpage.
6. The method of claim 1, wherein the trained target model comprises a multiple component model, and further, wherein the trained target model utilizes one or more of a natural language processing technology and a machine learning technology to predict the element of the webpage that will cause the action associated with the webpage to be performed.
7. The method of claim 1, wherein the task is one of logging into an account, accessing account data, entering data, submitting a form, or providing payment for a product or service.
8. The method of claim 1, further comprising storing a record of which element on the webpage has been interacted with and which action has been executed for the webpage, and further, determining if executing the most recently executed action associated with the webpage has increased or decreased the likelihood of performing the task.
9. A non-transitory computer readable medium containing a set of computer-executable instructions which when executed by a processor or processors cause the processor or processors to automate performance of a task for a user by: receiving from a user an identification of a website and a task they want to have automated; navigating to a first webpage associated with the website; extracting one or more intent model features from the first webpage, the one or more intent model features including visual and textual features; accessing a trained intent model and providing the extracted intent model features from the first webpage as an input to the trained intent model, wherein the trained intent model operates to generate a prediction of one or more actions associated with the webpage and a corresponding confidence level for each of the one or more actions; extracting one or more target model features from the first webpage, the one or more target model features including both visual and textual features; accessing a trained target model and providing the extracted target model features from the first webpage and the generated prediction of one of the one or more actions associated with the webpage as an input to the trained target model, wherein for each of the one or more actions associated with the webpage, the trained target model operates to generate a prediction of one or more elements of the webpage that will cause the action to be performed and a corresponding confidence level for each of the one or more elements; executing one of the one or more actions associated with the webpage by interacting with a highest ranked element of the webpage for causing the one of the one or more actions, wherein executing the one of the one or more actions results in either navigating to a second webpage or performing the task; and if the task has not been performed, then repeating one or more of the preceding steps on the second webpage and on any subsequent webpages until the task is performed.
10. The non-transitory computer readable medium of claim 9, further comprising instructions which cause the processor or processors to receive a set of credentials from the user and use the received credentials to login to a user account accessible from the website.
11. The non-transitory computer readable medium of claim 9, wherein the task is one of logging into an account, accessing account data, entering data, submitting a form, or providing payment for a product or service.
12. The non-transitory computer readable medium of claim 9, further comprising instructions which cause the processor or processors to store a record of which element on the webpage has been interacted with and which action has been executed for the webpage, and further, determine if executing the most recently executed action associated with the webpage has increased or decreased the likelihood of performing the task.
13. A system for automating a task for a user, comprising: a set of computer-executable instructions stored in a memory; and a processor or processors configured to execute the set of instructions, wherein when executed, the instructions cause the processor or processors to perform a set of operations comprising receiving from a user an identification of a website and a task they want to have automated; navigating to a first webpage associated with the website; extracting one or more intent model features from the first webpage, the one or more intent model features including visual and textual features; accessing a trained intent model and providing the extracted intent model features from the first webpage as an input to the trained intent model, wherein the trained intent model operates to generate a prediction of one or more actions associated with the webpage and a corresponding confidence level for each of the one or more actions; extracting one or more target model features from the first webpage, the one or more target model features including both visual and textual features; accessing a trained target model and providing the extracted target model features from the first webpage and the generated prediction of one of the one or more actions associated with the webpage as an input to the trained target model, wherein for each of the one or more actions associated with the webpage, the trained target model operates to generate a prediction of one or more elements of the webpage that will cause the action to be performed and a corresponding confidence level for each of the one or more elements; executing one of the one or more actions associated with the webpage by interacting with a highest ranked element of the webpage for causing the one of the one or more actions, wherein executing the one of the one or more actions results in either navigating to a second webpage or performing the task; and if the task has not been performed, then repeating one or more of the preceding steps on the second webpage and on any subsequent webpages until the task is performed.
14. The system of claim 13, wherein the instructions cause the processor or processors to receive a set of credentials from the user and use the received credentials to login to a user account accessible from the website.
15. The system of claim 13, wherein the task is one of logging into an account, accessing account data, entering data, submitting a form, or providing payment for a product or service.
16. The system of claim 13, wherein the instructions cause the processor or processors to store a record of which element on the webpage has been interacted with and which action has been executed for the webpage, and further, determine if executing the most recently executed action associated with the webpage has increased or decreased the likelihood of performing the task.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Embodiments in accordance with the present disclosure will be described with reference to the drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
DETAILED DESCRIPTION
(13) The subject matter of embodiments of the present disclosure is described herein with specificity to meet statutory requirements, but this description is not intended to limit the scope of the claims. The claimed subject matter may be embodied in other ways, may include different elements or steps, and may be used in conjunction with other existing or later developed technologies. This description should not be interpreted as implying any required order or arrangement among or between various steps or elements except when the order of individual steps or arrangement of elements is explicitly noted as being required.
(14) Embodiments of the disclosure will be described more fully herein with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, exemplary embodiments by which the disclosure may be practiced. The disclosure may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy the statutory requirements and convey the scope of the disclosure to those skilled in the art.
(15) Among other things, the present disclosure may be embodied in whole or in part as a system, as one or more methods, or as one or more devices. Embodiments of the disclosure may take the form of a hardware implemented embodiment, a software implemented embodiment, or an embodiment combining software and hardware aspects. For example, in some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by one or more suitable processing elements (such as a processor, microprocessor, CPU, GPU, TPU, controller, etc.) that is part of a client device, server, network element, remote platform (such as a SaaS platform), an “in the cloud” service, or other form of computing or data processing system, device, or platform.
(16) The processing element or elements may be programmed with a set of executable instructions (e.g., software instructions), where the instructions may be stored on (or in) one or more suitable non-transitory data storage elements. In some embodiments, the set of instructions may be conveyed to a user through a transfer of instructions or an application that executes a set of instructions (such as over a network, e.g., the Internet). In some embodiments, a set of instructions or an application may be utilized by an end-user through access to a SaaS platform or a service provided through such a platform.
(17) In some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by a specialized form of hardware, such as a programmable gate array, application specific integrated circuit (ASIC), or the like. Note that an embodiment of the methods or processes described may be implemented in the form of an application, a sub-routine that is part of a larger application, a “plug-in”, an extension to the functionality or available services of a data processing system or platform, or other suitable form. The following detailed description is, therefore, not to be taken in a limiting sense.
(18) Conventional data aggregation services and task automation efforts suffer from several disadvantages or sub-optimal aspects. These include one or more of the following: existing data aggregation services rely on the availability of an API (Application Programming Interface) or pre-programmed scripts to access data from a source on behalf of their clients; note that the API enables a programmatic access to data and is hosted by the website itself; however, most websites offer no such API provided access; pre-programmed scripts (software) are software instructions that while running, programmatically navigate and retrieve data from the websites on behalf of the account owner. However, such scripts rely on pre-existing knowledge of the website's layout and flow (i.e., the movement from element to element within a webpage and between pages). As a result, if a website changes its web layout and/or flow, then the script will “break” and not function as intended. Further, it typically requires a significant amount of resources to maintain such scripts; this is an important disadvantage to current approaches; note that efforts to automate tasks that involve data entry and navigation between webpages often encounter the same sorts of difficulties; Due to the lack of API support for some websites, and the unreliability of pre-programmed scripts, there are frequent disruptions experienced by existing data aggregation services. At present, there is also a lack of data aggregation coverage for certain industries, such as utility, healthcare and insurance, due to the high cost of developing and maintaining pre-programmed scripts.
(19) Note that with a process based on pre-programmed scripts (such as conventional approaches to data aggregation), it is assumed that each step in the script will be executed on a pre-set (i.e., previously defined/known) webpage. Conventional processes also assume that the webpage's HTML (Hyperlink Markup Language) DOM (Document Object Model) will contain a given Html Element. The script dictates or defines taking an “action” based on a specific Element. Typically, the action will either retrieve data it needs, or cause navigation to a different webpage. Once the action is performed, the script will dictate or define the next step to take. As is apparent, if the website changes its flow or the identification of a data field or action, then the script will not operate correctly.
(20) In contrast, with the embodiments of the system and methods described herein, there is no presumed pre-existing knowledge of a page's HTML DOM, or of what action to take on the page. Instead, the embodiments described herein are directed to systems, apparatuses, and methods for more efficiently performing data aggregation across multiple sources without previously knowing the amount of information about a website's elements and flow required by conventional approaches. In a typical use case, the sources of the data are websites and embodiments operate or function to automate the aggregation of the data from multiple websites. Further, embodiments can automate the aggregation of the data from multiple websites independently of the website data schema or format, or of the website flow between webpages.
(21) As noted, different websites (i.e., the sources of the data to be accessed and aggregated) may have different and in some cases inconsistent layouts and flow, and this can create a significant problem or obstacle to navigating between webpages and aggregating the data contained on the website or entering data into fields in a webpage. For example: assume that website A has a Login page with a userID element <input type=“text” id=“UserID”>User ID</input>, and a password element <input type=“text” id=“Password”>Password</input>; for a manually written script to enter userID and password on this page, it must set userID to an element with a given id “UserID” and password to an element with a given id of “Password”; on Website B, the Login page may differ—it may be <input type=“text” name=“UserName”>LogIn Name</input> and the password entry field shows on the second page after user clicks ‘Next’. This requires a completely different script that set userID to an element with a given name of ‘UserName’ and to hold off setting text to password until after clicking the ‘Next’ button; thus, to retrieve data from both Website A and Website B requires the data aggregator/consumer to write and maintain two completely different scripts. In a more realistic situation, instead of two sources of data, there may be hundreds of such sources (or more), each with its own website design, element names, and flow.
(22) The embodiments described herein (and others based on those) enable the automation of the data aggregation process without using such scripts. This is at least in part because of the following features or aspects of one or more of the embodiments: embodiments use natural language processing (NLP) and/or machine learning (ML) techniques to automate data aggregation from websites without use of pre-programmed scripts; embodiments can retrieve data from a website “as is”. In some cases, embodiments simulate or model how a human brain would perceive and acquire data from the website, which is independent from the underlying website implementation. Embodiments make a decision on which action to take at runtime during the embodiment's execution. If a website changes its layout or flow, embodiments can adjust to the change automatically. Among other aspects, this enables embodiments of the data aggregation service described herein to achieve higher reliability while incurring lower (or no) maintenance cost; embodiments remove the dependency on a website API for use in data aggregation; and conventionally, it may take hundreds or thousands of software developer-hours of work to write a sufficient set of pre-programmed scripts and to maintain them. However, in contrast, embodiments are capable of operating using a minimal set of training data; this makes it easier to expand the data aggregation service to industries not presently accessible or sufficiently accessible, such as utility, healthcare and insurance.
(23)
(24) By way of further explanation, in the context of the described embodiments, features can be regarded as input variables or parameters, and the “model” can be regarded as a complicated multivariable function, where the coefficients of variables are determined through a training process. In the example of predicting the “intent” of a web page (which may be characterized as the purpose, function, capability, or goal of that page), the features of the current page (e.g., HTML DOM, page image, and page texts) are converted into numerical values for the variables. Some values are in the form of multi-dimensional numerical vectors. The variable values are then “inserted” into the multivariable function, which outputs a numerical vector that represents the probability of the current page being of each intent type. For example, if there are 10 intent types, the output will be a 1×10 vector with each element representing the probability of how likely the given page can be classified as corresponding to or representing the indicated intent type. Given the “prediction” result(s), the process will typically select the intent type with the greatest probability as the current page's intent.
(25) Note that the extracted features are provided/exported (stage or step 105) to a training repository 106 (such as a local or remote data storage medium) and used in a training process (as suggested by stage or step 108). Training step 108 (which may be termed part of a continuous training or updating process) is used to incorporate the extracted features and information (such as metadata, labels, or other characteristics) into the trained intent and target models (as represented by the paths between continuous training step 108 and the Intent and Target models at steps 110 and 114).
(26) The extracted features are sent to the trained Intent model (as illustrated in step or stage 110) to predict (i.e., to determine, decide, identify, select, etc.) in real time which type of intent (or purpose, goal, function, etc.) is represented by or associated with the current web page; for example, whether the current page provides for the ability to perform a specific function or access a type of data, such as “Login”, “Dashboard”, “Download”, etc. (note that this may include other types or categories, such as a specific intent, purpose, operation, etc.). Based on the intent type predicted by the model (as suggested by stage or step 112), and a record of the previous steps taken during the aggregation process, the data aggregation process then sends the extracted features to a trained target model (as suggested by stage or step 114), to “predict” which target type each HTML Element in the current page represents (or is associated with or corresponds to, as suggested by stage or step 116). Note that typically, each target can be interacted with to perform a certain “action”.
(27) In some embodiments, actions are predefined based on target type and are applicable to all websites. For example, an element with “UserNameTarget” target type is typically a textbox for a user to enter their username. An action defined for the “UserNameTarget” type element may be to set its innerText to the entered username. Similarly, an element with “LoginTarget” target type is typically a button or link for a user to click or select in order to log in. An action defined for the “LoginTarget” type element may be to invoke a “click” or “select” function on the element. In general, embodiments initiate or take actions against target elements, in a manner similar to how a human user interacts with the website or webpage to navigate and retrieve/access information.
(28) An action on the target (executed at step 118) may trigger a page navigation or HTML DOM change, which will lead to the next iteration of the process (as suggested by path/step 120 and its return of control to accessing the website, and hence possibly a new webpage at step 102), until the process achieves its goal and exits/terminates (as suggested by stage or step 122).
(29)
If the process is not proceeding correctly, then the current state is corrected by going back to the previous page (as suggested by the path from the “No” result of Step 206 to Step 201), and use the tracked target elements to self-correct the action; By this is meant that if the process is not proceeding correctly, then it means the previous action was taken on an incorrect element. In this situation, the process restores the current page to the previous page, and since an incorrect element was interacted with, the process selects a different element to act upon. As embodiments of the data aggregation process track the previous element acted upon, as well as a list of “candidate” elements to act upon, the process will pick/select the remaining candidate element with the highest predicted probability to retry the action on the current page; Step 207: Features extracted for (or from) every HTML DOM element in the current web page at step 202 are sent to a multi-component target model that “predicts” which target type or category the element belongs to. In some embodiments, target types are a set of predefined categories for actionable elements. Target types are defined and typically stored in a suitable data storage medium—note that the medium or data storage element may include configuration files, embedded resource files, a database etc. The target types are typically generic and applicable to most any website without prior knowledge of the website's specific implementation details; in this sense, a target type represents a generic or canonical capability or function of a web page (e.g., data entry, selection of an option, entry of login username and password, etc.); With regards to the description of Target types as being predefined (as opposed to discovered or identified) categories, note that Target types are limited, as most websites share a common pattern, and overall, at present, there are possibly no more than a few hundred target types across all websites; For example, having a username textbox is standard for any website using login credentials, and such a textbox, regardless of its label or id as defined in different webpages, is categorized as “UserNameTarget” in embodiments of the systems and methods described herein; Similarly, a textbox for a user to enter a password is categorized as “PasswordTarget” in embodiments of the systems and methods described herein; and As another example, websites that allow users to download a transaction history typically have a button or link for users to select/click to initiate the downloading process. Such a button is categorized as a target type “DownloadTarget”; If the methods described herein were desired to be used on a different or expanded set of Target types, then the additional or different Target types would need to be discovered, identified or otherwise added. One possible method of doing this discovery or identification is to utilize unsupervised learning to group elements into “clusters” of potential target types. Instead of manually defining and labeling target types, this enables a system to automatically identify clusters of similar elements and treat each cluster as a potential target type. This approach may be implemented by use of machine learning technologies such as “K-means”, “LDA”.
Returning to the embodiments of the system and methods being described herein, in some embodiments, the system and methods enumerate every HTML DOM element in a web page and provide the extracted features as an input to the target model, which then outputs a numerical vector representing a probability or likelihood of the element corresponding to each target type. In some embodiments, the target type with the greatest probability is chosen as the target type of the HTML element. For visual layout, embodiments may utilize a Neural Network such as a CNN (Convolutional Neural Network), a classifier such as SVM (Support Vector Machines), or a Decision Tree for classification or identification of an image; For DOM and text, embodiments may utilize a Neural Network such as a RNN (Recurrent Neural Network), a classifier such as Decision Tree, Support Vector Machine (SVM), Naive Bayes, Max Entropy for classification; For DOM and text modeling, embodiments may incorporate modeling results from Natural Language Processing (NLP) technologies such as Stemming and Lemmatization, POS tagging (part-of-speech tagging), Word Embedding, Conditional Random Fields (CRF), LDA (Latent Dirichlet allocation) or LSA (Latent Semantic Analysis); The predicted results from the visual model and NLP model may then be ensembled with an algorithm such as Bagging methods, Random Forest, AdaBoost, Gradient Tree Boosting, Stacking, or Voting Classifier to achieve better prediction performance. Step 208: rank the elements by predicted probability for each target type: The DOM element with the highest predicted probability for the given target type is typically chosen as the target element—for example, assume that with “DownloadTarget” as the target type, there are two DOM elements predicted to be of this type: a <button id=“downloadButton”>Download<button>, with a predicted probability of 0.80; and a second <a id=“help”>Need help with download?</a> with a predicted probability of 0.30. In this example, the download button has the greatest probability, and therefore is chosen as the DownloadTarget element. The help anchor element, with a lower probability, may be added to the candidate list for “DownloadTarget” type for the current page; Track the selected target element(s) throughout the process for evaluation (in the manner described with reference to Step 206). If this element has already been selected and acted upon in a previous iteration, then it indicates that no navigation event or DOM change occurs from the previous action, or that an incorrect navigation has taken place—therefore, skip this element and choose the one with next highest probability instead. If no more elements remain, generate an error or notification and exit the program; Step 209: access a knowledge-base or other data or information repository to determine which action to take with a given DOM element; the knowledge base is typically predefined and stored in a suitable storage medium—it may include configuration files, embedded resource files, database, etc. The knowledge base defines rules that are generic to HTML DOM Elements and are applicable to most websites without pre-existing knowledge of the website implementation: for example, with <input> element with type=“text”, the action to take is “Set InnerText”—for example if Step 208 determines that the target element is “<input type=“text” id=“UserName”>User Name</input> element”, then at Step 209 the action to take is to set the InnerText of the <input type=“text” id=“UserName”>User Name</input> element to be <input type=“text” id=“UserName”>given userName</input>, or; with <button> element, the action to take is invoke “Click” or “Select”—for example if Step 208 determines that the target element is “<button id=“downloadButton”>Download</button>”, then at Step 209 the action to take is to invoke “Click” on <button id=“downloadButton”>Download</button> element; Step 210: Evaluate if the end goal has been achieved; if so, exit the program. If not, then the Action taken at Step 209 may trigger a navigation event to a new page or DOM changes on the current page, both of which lead to repeating of the stage or process at Step 201 until the end-goal is achieved (as suggested by the path from Step 210 to Step 201).
(30) As mentioned, the embodiments of the system and methods described herein do not rely on pre-existing knowledge of a page's HTML DOM, or of what action to take on the page. When an embodiment navigates to a website page, it will not only extract the HTML DOM information, but also other information, including page image and page text, as “features”. Page image usually includes a screenshot of the page. Text data on a webpage includes both readable texts from elements' innerText as well as text attributes such as “id”, “name”, and “title”. The following aspects of one or more embodiments are also noted: if an embodiment decides that the current page is the correct page to be on, then the process evaluates which action to take on that page; embodiments have no prior knowledge of which HTML Element to use to initiate an action—instead, the process uses the extracted features with a trained model, which may be used to “predict” what Element is the target element (e.g., <button id=“downloadButton”>Download</button>); and embodiments typically have a pre-defined action associated with a given HTML Element target type—for example, with a “DownloadTarget”, the process performs a “Click” or “Select” action.
Note that in contrast to conventional approaches, with use of an embodiment of the system and methods described herein, if a webpage changes a button id from “download” to “download1”, it has no impact on the execution of the data aggregation process.
(31)
(32) Example Use Case 1:
(33) Bob has signed up at a city's utility service's website, UtilityA.com. He has created an account with username: Bob@bob.com, password “Bob1234”. He owns property, Property1. Bob wants to track his utility expenses in his personal financial tool, FinanceB, automatically without repetitively logging into his utility account. There are tens of thousands of utility companies in the US alone, most of which offer no API support. For a conventional data aggregation service, it would be expected to take a large number of developers (and multiple person-years of effort) to support all of the utility companies; In contrast, using one or more of the embodiments described herein, it only requires a small training sample and a short period of time to provide a data aggregation service for utility service data from a large number of such services (if not all) throughout the country. Note that if a utility company changes their website, then the embodiments can adjust to the change automatically without manual intervention; Bob opens FinanceB, which enables him to send his utility account's username and password to the system or service platform that implements an embodiment of the data aggregation service described herein. The system or service platform automatically retrieves Bob's current account balance from UtilityA.com and sends data back to FinanceB. Bob can now see his utility expenses in FinanceB. Afterwards, whenever Bob opens FinanceB, it automatically “calls” or accesses the system or service platform to update the utility account balance, and the system or service platform keeps the account balance updated behind the scene. Bob can now track his utility expenses and reconcile his bank payments with utility bills more easily.
(34)
(35) 1: As suggested by stage or step 201 in
(36) 2. As suggested by stages or steps 202, 203, 204 and 205 in
(37) 3. As suggested by stage or step 206 in
(38) 4. As suggested by stages or steps 207 and 208 in
(39) for <input type=“text” name=“UserName”/> its target type is “UserNameTarget”; for <input type=“text” name=“Password”/> its target type is “PasswordTarget”; for <input type=“submit” name=“LoginButton”/> its target type is “LoginTarget”;
5. As suggested by stage or step 209 in
6. As suggested by stage 210 in
7. As suggested by stages or steps 202, 203, 204 and 205 in
8. As suggested by stage or step 206 in
9. As suggested by stages or steps 207 and 208 in
10. As suggested by stage or step 209 in
11. As suggested by stage or step 210 in
(40)
(41) Example Use Case 2:
(42) Alice has accounts with multiple hospitals/labs. She wants to track all of her medical bills at one place automatically, so she can send them to her insurance company to file claims. Most of the hospitals/clinics offer no API and have their own unique website flow, making it overly expensive to write pre-programmed scripts to provide data aggregation services for such data; Alice logs into the system or service platform that implements an embodiment of the data aggregation process or method described herein at the appropriate website and sets her username and password for each hospital. The system or service platform automatically retrieves Alice's bills from all her hospitals and labs. Alice can now see all her bills at one place. Alice can easily download all the bills and send them to her insurance company for claims.
(43)
(44) 1: As suggested by stage or step 201 in
(45) 2. As suggested by stages or steps 202, 203, 204 and 205 in
(46) 3. As suggested by stages or steps 206 in
(47) 4. As suggested by stages or steps 207 and 208 in
(48) for <input type=“text” name=“UserName”/> its target type is “UserNameTarget”; for <input type=“text” name=“Password”/> its target type is “PasswordTarget”; for <input type=“submit” name=“LoginButton”/> its target type is “LoginTarget”;
5. As suggested by stage or step 209 in
6. As suggested by stage 210 in
7. As suggested by stages or steps 202, 203, 204 and 205 in
8. As suggested by stages or steps 206 in
9. As suggested by stages or steps 207 and 208 in
10. As suggested by stage or step 209 in
11. As suggested by stage or step 210 in
(49) The above examples represent uses of the system and methods disclosed herein to navigate between webpages for the purpose of acquiring certain data and in some cases to execute a specific action. As mentioned, another application of the disclosed system and methods is to automate software tasks that involve data entry, such as making a payment, submitting a form, extracting certain data from a set of webpages, logging into a website, etc. This form of automation can not only relieve a user of performing a tedious task but may also reduce errors in processing requests and ensure accuracy of entered data.
(50) As an example, tasks such as making payments, transferring funds, or filing taxes may be automated (at least in part) on behalf of users by designing a website and data processing flow to use an account holder's PIN or access code. This would be advantageous for users; however, there are obstacles to a business or data consumer being able to provide users with this capability. For one thing, there are a very large number of websites that a user might want to enter data into as part of performing a task; there are over 26,000 bill payment websites in the United States for different purposes such as utility, phone, internet, toll or other services, each with its own website design and data schema or format. The majority of these websites do not provide an API for a software application to integrate with directly. This places a significant burden on a company if they wish to offer a user with the ability to automatically (or even semi-automatically) perform a task that requires data entry. It also means that from a user perspective, it is likely that at least some of the websites they wish to use will lack this capability.
(51) Embodiments of the system and methods disclosed herein can be used to enable users to automate tasks that require data entry and/or navigation between webpages for one or multiple businesses. In some embodiments, this capability can be made available to a user through a SaaS platform where a user is able to select those websites for which it desires to automate a data entry process as part of performing a task. This provides a benefit to users as well as a way to assist a business to augment the capabilities of the data processing that occurs in support of its website. In this sense, a business may want to encourage its customers to utilize the services provided by the system and methods disclosed herein as part of improving the timeliness and accuracy of customer orders and payments.
(52)
(53) Example Use Case 3:
(54) Bob has a business that pays quarterly income tax to the Department of Revenue (e.g., at DOR.com). He has created an account with the username: Bob@bob.com, and a password “Bob1234”. At present, he uses an accounting service to manage his business and gives his DOR.com login credentials to the accounting service so it can automatically pay the business' income tax on his behalf. Note that there are hundreds of DOR type agencies in the US alone, most of which offer no API support. For a conventional software task automation service, it would be expected to take a large number of developers (and multiple person-years of effort) to support all of the DOR websites; In contrast, an embodiment of the system and methods described herein requires a small sample of training data and a relatively short period of training time to provide a software task automation service for business income tax payment that can be used on a large number of such services (if not all) throughout the country. Note that if a DOR agency changes their website, then the described embodiments can adjust to the change automatically without manual intervention by a user; Bob may arrange with an accounting service or business to send his DOR account's username and password to a system or service platform that implements an embodiment of the software task automation techniques described herein. The system or service platform automatically submits Bob's payment information and tax due calculated by the accounting business on the DOR website and sends back the submission result to accounting business. Bob can see that his quarterly business income tax is paid off by accessing the accounting business website and examining his account. Whenever Bob's business tax is due, the accounting business can automatically “call” the system or service platform to submit the new tax payment, and the system or service platform is able to maintain up to date tax payments for Bob's business.
(55)
(56) 1: As suggested by stage or step 201 in
(57) 2. As suggested by stages or steps 202, 203, 204 and 205 in
(58) 3. As suggested by stage or step 206 in
(59) 4. As suggested by stages or steps 207 and 208 in
(60) for <input type=“text” name=“UserName”/> its target type is “UserNameTarget”; for <input type=“text” name=“Password”/> its target type is “PasswordTarget”; for <input type=“submit” name=“LoginButton”/> its target type is “LoginTarget”;
5. As suggested by stage or step 209 in
6. As suggested by stage 210 in
7. As suggested by stages or steps 202, 203, 204 and 205 in
8. As suggested by stage or step 206 in
9. As suggested by stages or steps 207 and 208 in
10. As suggested by stage or step 209 in
11. As suggested by stage or step 210 in
(61)
(62) Example Use Case 4:
(63) Alice has accounts with multiple utility companies. She wants to pay off all of her utility bills on time automatically, so she won't incur late fees. Most of the utility companies offer no API and have their own unique website flow, making it too expensive to write pre-programmed scripts to provide software task automation services for such data; Instead, Alice logs into a system or service platform (such as a SaaS platform) that implements an embodiment of the software task automation process or method described herein and sets her username and password for each of the utility companies she wishes to have payments made to. The system or service platform automatically manages the payment of Alice's bills when a utility bill is due. By using an embodiment of the task automation techniques described, Alice can be assured that her utility bills are paid on time.
(64)
(65) 1: As suggested by stage or step 201 in
(66) 2. As suggested by stages or steps 202, 203, 204 and 205 in
(67) 3. As suggested by stages or steps 206 in
(68) 4. As suggested by stages or steps 207 and 208 in
(69) for <input type=“text” name=“UserName”/> its target type is “UserNameTarget”; for <input type=“text” name=“Password”/> its target type is “PasswordTarget”; for <input type=“submit” name=“LoginButton”/> its target type is “LoginTarget”;
5. As suggested by stage or step 209 in
6. As suggested by stage 210 in
7. As suggested by stages or steps 202, 203, 204 and 205 in
8. As suggested by stages or steps 206 in
9. As suggested by stages or steps 207 and 208 in
10. As suggested by stage or step 209 in
11. As suggested by stage or step 210 in
(70) The system and methods described herein enable a user to arrange for the automated execution of a task that requires data entry and in some cases navigation through a plurality of webpages. The system and methods may be provided as services by a SaaS platform (i.e., in the cloud) where a user may register to have a desired task performed and provide any required data. Such data will typically include identification of a website at which the task is to be performed, the user's credentials for the website, and an identification of the desired task. Based on this information, the system and methods is able to train a model to navigate through the webpages of the website and automatically perform the task by a combination of entering the user data and selecting one or more operations to be performed as part of accomplishing the task.
(71) As described, the training process involves determining an intent (or goal) associated with each webpage, a target or element of a webpage that causes the goal to be performed or executed, data that needs to be inserted into a field of a webpage, and whether a desired task has been completed after navigating away from a webpage. This combination of functions or operations can be accessed by a user through a SaaS platform account and may be configured to automatically perform a task that involves navigating through a set of webpages and performing data entry as part of accomplishing a task.
(72) In the example described with reference to
(73) With regards to the model that is used to navigate through the pages of a website and execute one or more data entry or other actions, in one embodiment, the model training process may be implemented as follows. For training, the system needs to collect a large sample size of webpages from various websites. The feature extraction process described herein is applied to those webpages to obtain the features for each webpage. By manually classifying (annotating or labeling) each webpage with the correct intent, the model can be developed using a supervised training approach. After training, the model may be used in an inference or classification process in which a new webpage may be classified with regards to its intent after extracting a set of features from the webpage. The model output is a corresponding predicted classification, in this case the intent of the webpage. A similar training process is used to train the target model, with the trained model then being used to classify or predict the target on a webpage (where the target is an element that will cause a desired action). Depending upon the desired task being automated, different training data may be acquired, different intent or target labels may be applied, and a trained model may then operate to classify or predict a different aspect of a webpage.
(74) For example, if the task is one to automate the payment of an invoice, then the system operator may add webpages into the training data sample that are specifically for making payments, and manually classify the new training data's intent as a “MakePayment” intent type. The training process is then rerun which adds the “MakePayment” intent into the possible classifications. This expands the possible intent predicted for a webpage to include “MakePayment” as a result. After any required retraining, the process would proceed as follows:
(75) Logging into a website and navigating through the pages is performed using the properly trained or updated model as described herein for identifying an intent and the possible targets on a webpage. As mentioned, the intent classification and target classification are performed using a pre-trained model that is based on webpage elements and data, and typically is performed using supervised learning. In most cases, the intents are generic ones such as “Login”, “Dashboard”, “AccountSummary”, etc.;
(76) To enable a user to make a payment, the model has been trained further by adding specific sample data to the existing set of training data. In one example, the service may collect bill payment pages from a large number of websites and add these to the training data. The described feature extraction process is used, as well as the fitting/prediction methods. The end result is a new “MakePayment” intent that is added to the intent classifications, and may include one or more new targets: such as a “MakePaymentTarget” added to “AccountSummary” Intent, and an “AmountDueTarget”, “PaymentInfoTarget”, “SubmitTarget” added as target classifications to the “MakePayment” Intent;
(77) In one embodiment, a service's daily job set may include logging into the utility account's website, navigating to the account summary page and predicting it to be for an “AccountSummary” intent. Because the job's end goal is to make a payment, it predicts the “Pay” button as the “MakePaymentTarget” on the webpage. Activating (clicking) the “Pay” button lands on the payment webpage, which is predicted as the “MakePayment” Intent page. The process then predicts “AmountDueTarget”, “PaymentInfoTarget”, “SubmitTarget” as the targets and takes actions to fill these targets and then clicks “submit”, which completes the job or task of making a payment.
(78)
(79) In one embodiment, ServicePlatform.web (element, component, or process 530) is based on a standard MVC architecture, and its controller utilizes the API web service (element, component, or process 532) to interact with the model (data) indirectly. The API web service is composed of web service modules (element, component, or process 543) and one or more that execute an embodiment of the process(es) or functionality disclosed herein, that is a Data Aggregation service module (element, component, or process 545). When receiving a request, either directly from a service user or from the ServicePlatform.web Controller, the web service module (543) reads data from the model, launches or instantiates the Data Aggregation service module (545) to retrieve data, and saves that data to the model.
(80) The API Service may be implemented in the form of a standard “Restful” web service, where RESTful web services are a way of providing interoperability between computer systems on the Internet. REST-compliant Web services allow requesting systems to access and manipulate textual representations of Web resources using a uniform and predefined set of stateless operations.
(81) With reference to
(82) Application layer 538 is typically composed of one or more application modules 539, with each application module composed of one or more sub-modules 540. As described herein, each sub-module may represent executable software instructions or code that when executed by a programmed processor, implements a specific function or process. Web service layer 542 may be composed of one or more web service modules 543, again with each module including one or more sub-modules, with each sub-module representing executable instructions that when executed by a programmed processor, implement a specific function or process. For example, web service modules 543 may include modules or sub-modules used to provide support services (as suggested by support service-modules 544) and to provide the functionality associated with the data aggregation services and processes described herein (as suggested by data aggregation service-modules 545). Thus, in some embodiments, data aggregation service-modules 545 may include software instructions that when executed implement one or more of the functions described with reference to
(83) A user can invoke and access the functionality of an embodiment through either the website or the API. In this regard,
(84) Note that the system, elements, functions, operations, methods, and processes described herein may be used for purposes other than data aggregation—for example, the machine learning and natural language processing described may be used to not only collect data, but also to complete data entry work automatically, such as filing taxes and completing claim forms. For example, an embodiment may navigate to an insurance company's website for electronically filing claims. Similar to the process illustrated in
(85) The architecture of
(86) For example, a service platform may be able to provide deeper data analysis by taking advantage of data access across domains. For instance, by having medical bills, and insurance bills in one place over a period of time, users may be able to identify ways to optimize his or her health insurance plan based on the past medical spending. Another example service that might be provided is to enable a user to have a transaction history from all of their bank and credit card accounts in one place; the user can then leverage a service platform's data analysis tools to identify where most spending takes place and the best way to save money.
(87) Other data analysis tools can be implemented on the service platform (or may be accessible by it) to automate data-based tasks, such as reminding a user of due dates, automatically paying bills, notifying a user of a low account balance, etc. The service platform offers users additional flexibility in terms of data sharing as well. For example, if a small business owner applies for a loan, instead of sending lengthy bank statements, tax filings and other documents, he or she can grant access to specific data stores or functionality of a service platform account to the loan office. This will enable a loan officer to access data regarding a bank account, credit card activity, utility bills, or even tax filings in one place.
(88) Further, a user may be able to set an access restriction or remove an access restriction for a specific data set, data store, or entity, either separately or collectively. Thus, in some embodiments, the system and methods described herein enable a user to group or link together data from multiple accounts/sources and then to grant or revoke access to that set of data based on the identify of an entity (e.g., insurance agent, loan officer, or investment counselor), the source of the data (e.g., only granting access to certain of the aggregated data), the age of the data, etc.
(89)
(90)
(91) Integrated business system 602, which may be hosted by a dedicated third party, may include an integrated business server 614 and a web interface server 616, coupled as shown in
(92) The ERP module 618 may include, but is not limited to, a finance and accounting module, an order processing module, a time and billing module, an inventory management and distribution module, an employee management and payroll module, a calendaring and collaboration module, a reporting and analysis module, and other ERP-related modules. The CRM module 620 may include, but is not limited to, a sales force automation (SFA) module, a marketing automation module, a contact list module (not shown), a call center support module, a web-based customer support module, a reporting and analysis module, and other CRM-related modules. The integrated business server 614 (or multi-tenant data processing platform) may also (or instead) provide other business functionalities. Web interface server 616 is configured and adapted to interface with the integrated business server 614 to provide one or more web-based user interfaces to end users of the enterprise network 604.
(93) The integrated business system shown in
(94)
(95) The distributed computing service/platform (which may also be referred to as a multi-tenant business data processing platform) 708 may include multiple processing tiers or layers, including a user interface tier 716, an application server tier 720, and a data storage tier 724. The user interface tier 716 may maintain multiple user interfaces 717, including graphical user interfaces and/or web-based interfaces. The user interfaces may include a default user interface for the service to provide access to applications and data for a user or “tenant” of the service (depicted as “Service UI” in the figure), as well as one or more user interfaces that have been specialized/customized in accordance with user specific requirements (e.g., represented by “Tenant A UI”, . . . , “Tenant Z UI” in the figure, and which may be accessed via one or more APIs). The default user interface may include components enabling a tenant to administer the tenant's participation in the functions and capabilities provided by the service platform, such as accessing data, causing the execution of specific data processing operations, etc. Each processing tier shown in the figure may be implemented with a set of computers and/or computer components including computer servers and processors, and may perform various functions, methods, processes, or operations as determined by the execution of a software application or set of instructions. The data storage tier 724 may include one or more data stores, which may include a Service Data store 725 and one or more Tenant Data stores 726.
(96) Each tenant data store 726 may contain tenant-specific data that is used as part of providing a range of tenant-specific business services or functions, including but not limited to data related to ERP, CRM, eCommerce, Human Resources management, payroll, etc. Data stores may be implemented with any suitable data storage technology, including structured query language (SQL) based relational database management systems (RDBMS).
(97) Distributed computing service/platform 708 may be multi-tenant, and service platform 708 may be operated by an entity (such as a service provider) in order to provide multiple tenants with one or more of a set of business related applications, data processing capabilities, data storage, or other functionality (such as the data aggregation services described herein). These applications and functionality may include ones that a business uses to manage various aspects of its operations. For example, the applications and functionality may include providing web-based access to business information systems, thereby allowing a user with a browser and an Internet or intranet connection to view, enter, process, or modify certain types of business information.
(98) As noted, such business information systems may include an Enterprise Resource Planning (ERP) system that integrates the capabilities of several historically separate business computing systems into a common system, with the intention of streamlining business processes and increasing efficiencies on a business-wide level. Such functions or business applications are typically implemented by one or more modules of software code/instructions that are maintained on and executed by one or more servers 722 that are part of the platform's Application Server Tier 720.
(99) Another business information system that may be provided as part of an integrated data processing and service platform is an integrated Customer Relationship Management (CRM) system, which is designed to assist in obtaining a better understanding of customers, enhance service to existing customers, and assist in acquiring new and profitable customers. Such functions or business applications are typically implemented by one or more modules of software code/instructions that are maintained on and executed by one or more servers 722 that are part of the platform's Application Server Tier 720.
(100) As noted with regards to
(101) Rather than build and maintain such an integrated business system themselves, a business may utilize systems provided by a third party. Such a third party may implement an integrated business system/platform as described herein in the context of a multi-tenant platform, wherein individual instantiations of a single comprehensive integrated business system are provided to a variety of tenants. One advantage to such multi-tenant platforms is the ability for each tenant to customize their instantiation of the integrated business system to that tenant's specific business needs or operational methods. Each tenant may be a business or entity that uses the multi-tenant platform to provide business data and functionality to multiple users.
(102)
(103) With reference to
(104) The application layer 810 may include one or more application modules 811, each having one or more sub-modules 812. Each application module 811 or sub-module 812 may correspond to a particular function, method, process, or operation that is implemented by the module or sub-module (e.g., a function or process related to providing ERP, CRM, eCommerce or other functionality to a user of the platform). Such function, method, process, or operation may also (or instead) include those used to implement one or more aspects of the inventive system and methods, such as for performing a data aggregation process by (note that these functions or processes are also examples of those that may be implemented by one or more of the data aggregation service-modules 545 of
(105) The application modules and/or sub-modules may include any suitable computer-executable code or set of instructions (e.g., as would be executed by a suitably programmed processor, microprocessor, or CPU), such as computer-executable code corresponding to a programming language. For example, programming language source code may be compiled into computer-executable code. Alternatively, or in addition, the programming language may be an interpreted programming language such as a scripting language. Each application server (e.g., as represented by element 722 of
(106) The data storage layer 820 may include one or more data objects 822 each having one or more data object components 821, such as attributes and/or behaviors. For example, the data objects may correspond to tables of a relational database, and the data object components may correspond to columns or fields of such tables. Alternatively, or in addition, the data objects may correspond to data records having fields and associated services. Alternatively, or in addition, the data objects may correspond to persistent instances of programmatic data objects, such as structures and classes. Each data store in the data storage layer may include each data object. Alternatively, different data stores may include different sets of data objects. Such sets may be disjoint or overlapping.
(107) Note that the example computing environments depicted in
(108)
(109) Each Data Aggregation or Task Automation service module or sub-module may correspond to a particular function, method, process, or operation that is implemented by the module or sub-module (e.g., a function or process related to the operations or functionality of the service platform). As described with reference to
As noted, in some embodiments, the history of actions taken on each element may be recorded. The elements not selected for each target type are also tracked, in case the action proves incorrect, and the element with the next highest probability may be acted upon when the navigation is restored back to the current page.
(110) The Data Aggregation or Task Automation service module(s) and/or or sub-module(s) may include any suitable computer-executable code or set of instructions (e.g., as would be executed by a suitably programmed processor, microprocessor, CPU, or GPU), such as computer-executable code corresponding to a programming language. For example, programming language source code may be compiled into computer-executable code. Alternatively, or in addition, the programming language may be an interpreted programming language such as a scripting language. The computer-executable code or set of instructions may be stored in (or on) any suitable non-transitory computer-readable medium. In general, with regards to the embodiments described herein, a non-transitory computer-readable medium may include almost any structure, technology or method apart from a transitory waveform or similar medium.
(111) As described, the system, apparatus, methods, processes, functions, and/or operations for implementing an embodiment may be wholly or partially implemented in the form of a set of instructions executed by one or more programmed computer processors such as a central processing unit (CPU) or microprocessor. Such processors may be incorporated in an apparatus, server, client or other computing or data processing device operated by, or in communication with, other components of the system. As an example,
(112) The subsystems shown in
(113) Machine learning (ML) is being used more and more to enable the analysis of data and assist in making decisions in multiple industries. In order to benefit from using machine learning, a machine learning algorithm is applied to a set of training data and labels to generate a “model” which represents what the application of the algorithm has “learned” from the training data. Each element (or instances, or example, in the form of one or more parameters, variables, characteristics or “features”) of the set of training data is associated with a label or annotation that defines how the element should be classified by the trained model. A machine learning model is a set of layers of connected neurons that operate to make a decision (such as a classification) regarding a sample of input data. When trained (i.e., the weights connecting neurons have converged and become stable or within an acceptable amount of variation), the model will operate on a new element of input data to generate the correct label or classification as an output.
(114) In some embodiments, the methods or models described herein (such as those referred to with regards to
(115) In general terms, a neural network may be viewed as a system of interconnected artificial “neurons” that exchange messages between each other. The connections have numeric weights that are “tuned” during a training process, so that a properly trained network will respond correctly when presented with an image or pattern to recognize (for example). In this characterization, the network consists of multiple layers of feature-detecting “neurons”; each layer has neurons that respond to different combinations of inputs from the previous layers. Training of a network is performed using a “labeled” dataset of inputs in a wide assortment of representative input patterns that are associated with their intended output response. Training uses general-purpose methods to iteratively determine the weights for intermediate and final feature neurons. In terms of a computational model, each neuron calculates the dot product of inputs and weights, adds the bias, and applies a non-linear trigger or activation function (for example, using a sigmoid response function).
(116) Any of the software components, processes or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, JavaScript, C++ or Perl using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands in (or on) a non-transitory computer-readable medium, such as a random-access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a CD-ROM. In this context, a non-transitory computer-readable medium is almost any medium suitable for the storage of data or an instruction set aside from a transitory waveform. Any such computer readable medium may reside on or within a single computational apparatus and may be present on or within different computational apparatuses within a system or network.
(117) According to one example implementation, the term processing element or processor, as used herein, may be a central processing unit (CPU), or conceptualized as a CPU (such as a virtual machine). In this example implementation, the CPU or a device in which the CPU is incorporated may be coupled, connected, and/or in communication with one or more peripheral devices, such as display. In another example implementation, the processing element or processor may be incorporated into a mobile computing device, such as a smartphone or tablet computer.
(118) The non-transitory computer-readable storage medium referred to herein may include a number of physical drive units, such as a redundant array of independent disks (RAID), a floppy disk drive, a flash memory, a USB flash drive, an external hard disk drive, thumb drive, pen drive, key drive, a High-Density Digital Versatile Disc (HD-DV D) optical disc drive, an internal hard disk drive, a Blu-Ray optical disc drive, or a Holographic Digital Data Storage (HDDS) optical disc drive, synchronous dynamic random access memory (SDRAM), or similar devices or other forms of memories based on similar technologies. Such computer-readable storage media allow the processing element or processor to access computer-executable process steps, application programs and the like, stored on removable and non-removable memory media, to off-load data from a device or to upload data to a device. As mentioned, with regards to the embodiments described herein, a non-transitory computer-readable medium may include almost any structure, technology or method apart from a transitory waveform or similar medium.
(119) Certain implementations of the disclosed technology are described herein with reference to block diagrams of systems, and/or to flowcharts or flow diagrams of functions, operations, processes, or methods. It will be understood that one or more blocks of the block diagrams, or one or more stages or steps of the flowcharts or flow diagrams, and combinations of blocks in the block diagrams and stages or steps of the flowcharts or flow diagrams, respectively, can be implemented by computer-executable program instructions. Note that in some embodiments, one or more of the blocks, or stages or steps may not necessarily need to be performed in the order presented or may not necessarily need to be performed at all.
(120) These computer-executable program instructions may be loaded onto a general-purpose computer, a special purpose computer, a processor, or other programmable data processing apparatus to produce a specific example of a machine, such that the instructions that are executed by the computer, processor, or other programmable data processing apparatus create means for implementing one or more of the functions, operations, processes, or methods described herein. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement one or more of the functions, operations, processes, or methods described herein.
(121) While certain implementations of the disclosed technology have been described in connection with what is presently considered to be the most practical and various implementations, it is to be understood that the disclosed technology is not to be limited to the disclosed implementations. Instead, the disclosed implementations are intended to cover various modifications and equivalent arrangements included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
(122) This written description uses examples to disclose certain implementations of the disclosed technology, and also to enable any person skilled in the art to practice certain implementations of the disclosed technology, including making and using any devices or systems and performing any incorporated methods. The patentable scope of certain implementations of the disclosed technology is defined in the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural and/or functional elements that do not differ from the literal language of the claims, or if they include structural and/or functional elements with insubstantial differences from the literal language of the claims.
(123) All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and/or were set forth in its entirety herein.
(124) The use of the terms “a” and “an” and “the” and similar referents in the specification and in the following claims are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “having,” “including,” “containing” and similar referents in the specification and in the following claims are to be construed as open-ended terms (e.g., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely indented to serve as a shorthand method of referring individually to each separate value inclusively falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate embodiments of the disclosed subject matter and does not pose a limitation to the scope of the embodiment(s) unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to each embodiment.
(125) As used herein in the specification, figures, and claims, the term “or” is used inclusively to refer items in the alternative and in combination Insert or-and statement.
(126) Different arrangements of the components depicted in the drawings or described above, as well as components and steps not shown or described are possible. Similarly, some features and sub-combinations are useful and may be employed without reference to other features and sub-combinations. Embodiments have been described for illustrative and not restrictive purposes, and alternative embodiments will become apparent to readers of this patent. Accordingly, the embodiments are not limited to the embodiments described above or depicted in the drawings, and various embodiments and modifications can be made without departing from the scope of the claims below.