Query engine for recursive searches in a self-describing data system
11645285 · 2023-05-09
Assignee
Inventors
- Kevin Gillespie (Andover, MA, US)
- Kevin Richard (Andover, MA, US)
- Robert McAveney (Boxford, MA, US)
- Sergey Murashko (Minsk, BY)
- Valentsin Shapavalau (Minsk, BY)
- Andrei Samsonau (Minsk, BY)
Cpc classification
G06F16/2448
PHYSICS
International classification
Abstract
A method for performing recursive searching of items of a data structure having a data mode includes creating an instance of a query definition, the instance of the query definition comprising a unique identifier, specifying one or more elements of the query definition, providing the query definition as an input to a query engine. The method further includes the operations of determining, by the query engine, query execution instructions based on the query definition, the query instructions specifying a recursive level-by-level search until a terminal node of the data structure is reached, obtaining results of a query executed based on the query execution instructions; and outputting query results.
Claims
1. A method for performing recursive searching of items of a data structure having a data model, the method comprising: creating an instance of a query definition, the instance of the query definition comprising a unique identifier and defining a depth parameter having a name and a value that controls a depth of a recursive level-by-level search of the data structure; specifying one or more elements of the query definition to be executed as instructions in a single query by a query engine, wherein the one or more elements comprise a query condition including a first dynamic parameter of an execution path comprising a string defining a search route from a parent query item to a child query item and the string references a key associated with the name of the depth parameter having the value that controls the depth of the recursive level-by-level search of the data structure, wherein the first dynamic parameter and the depth parameter are each a respective instance of an item in the data model, and wherein the depth parameter is configured in a configuration document that also configures the key and the string defining the search route from the parent query item to the child query item; providing the query definition as an input to the query engine; determining, by the query engine, query execution instructions based on the query definition, the query instructions specifying the recursive level-by-level search, the query engine further calculating the dynamic parameter and using the dynamic parameter with the depth parameter to control the recursive level-by-level search; obtaining results of a query executed based on the query execution instructions; and outputting query results.
2. The method of claim 1, wherein outputting the query results comprises at least one of outputting the query results as a flat output, displaying the results in a tree grid view or displaying the results as a graph visualization.
3. The method of claim 1, wherein an element of the query definition is specified by a security rule.
4. The method of claim 1, wherein the data model of the data structure is at least one of a hierarchical data model, a dynamic data model or a self-describing data model.
5. The method of claim 1, wherein specifying the elements of the query definition comprises at least one of specifying a query item, a query item selection property, a query item sort property, a query item available property, or a query reference.
6. The method of claim 1, further comprising storing the query definition in the data structure.
7. The method of claim 1, wherein the query execution instructions specify at least one of traversing the data structure upwards or traversing the data structure downwards.
8. A query engine, comprising: a processor; a memory containing instructions, which when executed by the processor, cause the query engine to: create an instance of a query definition, the instance of the query definition comprising a unique identifier and defining a depth parameter having a name and a value that controls a depth of a recursive level-by-level search of the data structure, obtain one or more elements of the query definition to be executed as instructions in a single query by the query engine, wherein the one or more elements comprise a query condition including a first dynamic parameter of an execution path comprising a string defining a search route from a parent query item to a child query item and the string references a key associated with the name of the depth parameter having the value that controls the depth of the recursive level-by-level search of the data structure, wherein the first dynamic parameter and the depth parameter are each a respective instance of an item in the data model, and wherein the depth parameter is configured in a configuration document that also configures the key and the string defining the search route from the parent query item to the child query item, provide the query definition as an input to the query engine, determine query execution instructions based on the query definition, the query execution instructions specifying the recursive level-by-level search, the query engine further calculating the dynamic parameter and using the dynamic parameter with the depth parameter to control the recursive level-by-level search; obtain results of a query executed based on the query execution instructions, and output the query results.
9. The query engine of claim 8, wherein the memory contains instructions, which when executed by the processor, cause the query engine to output the query results as at least one of outputting the query results as a flat output, displaying the results in a tree grid view or displaying the results as a graph visualization.
10. The query engine of claim 8, wherein an element of the query definition is specified by a security rule.
11. The query engine of claim 8, wherein the data model of the data structure is at least one of a hierarchical data model, a dynamic data model or a self-describing data model.
12. The query engine of claim 8, wherein the elements of the query definition comprise at least one of a query item, a query item selection property, a query item sort property, a query item available property, or a query reference.
13. The query engine of claim 8, wherein the memory contains instructions, which when executed by the processor, cause the query engine to store the query definition in the data structure.
14. The query engine of claim 8, wherein the query execution instructions specify at least one of traversing the data structure upwards or traversing the data structure downwards.
15. A non-transitory computer-readable medium containing program code, which when executed by a processor, cause a query engine to: create an instance of a query definition, the instance of the query definition comprising a unique identifier and defining a depth parameter having a name and a value that controls a depth of a recursive level-by-level search of the data structure, obtain one or more elements of the query definition to be executed as instructions in a single query by the query engine, wherein the one or more elements comprise a query condition including a first dynamic parameter of an execution path comprising a string defining a search route from a parent query item to a child query item and the string references a key associated with the name of the depth parameter having the value that controls the depth of the recursive level-by-level search of the data structure, wherein the first dynamic parameter and the depth parameter are each a respective instance of an item in the data model, and wherein the depth parameter is configured in a configuration document that also configures the key and the string defining the search route from the parent query item to the child query item, provide the query definition as an input to the query engine, determine query execution instructions based on the query definition, the query execution instructions specifying the recursive level-by-level search, the query engine further calculating the dynamic parameter and using the dynamic parameter with the depth parameter to control the recursive level-by-level search, obtain results of a query executed based on the query execution instructions, and output the query results.
16. The non-transitory computer-readable medium of claim 15, comprising program code, which when executed by the processor, cause the query engine to output the query results as at least one of outputting the query results as a flat output, displaying the results in a tree grid view or displaying the results as a graph visualization.
17. The non-transitory computer-readable medium of claim 15, wherein an element of the query definition is specified by a security rule.
18. The non-transitory computer-readable medium of claim 15, wherein the data model of the data structure is at least one of a hierarchical data model, a dynamic data model or a self-describing data model.
19. The non-transitory computer-readable medium of claim 15, the elements of the query definition comprise at least one of a query item, a query item selection property, a query item sort property, a query item available property, or a query reference.
20. The non-transitory computer-readable medium of claim 15, comprising program code, which when executed by the processor, causes the query engine to store the query definition in the data structure.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) For a more complete understanding of this disclosure and its advantages, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
DETAILED DESCRIPTION
(18)
(19)
(20) According to certain embodiments, the foundational element of a self-describing data system is an item, instances of which may be maintained in persistent storage in a relational database. According to certain embodiments, the configuration and properties of an item may be expressed in a markup language, such as extensible markup language (XML), or Aras Markup Language (AML), which, as described in greater detail herein, follows a repeating “/Item/Relationships/Item/Relationships” pattern to describe item configurations.
(21) Further, in the non-limiting example of
(22) According to various embodiments, the instance of the item defined by <item> tag 100 comprises three principal attributes, a type 105, an ID 110 and an action 115. It should be noted that the following three attributes are not the only attributes which can be applied to an item.
(23) In the non-limiting example shown in
(24) According to various embodiments, ID 110 comprises a unique identifier for the instance of an item created by <item> tag 100. In the non-limiting example of
(25) In some embodiments, action 115 comprises a method to be applied to the instance of an item defined by <item> tag 100. In the non-limiting example of
(26)
(27) Referring to the non-limiting example of
(28) According to certain embodiments, the configuration 200 of an item may be expressed as a markup language document (for example, an AML document). In some embodiments, item 200's configuration may be expressed through an “/Item/Relationships/Item/Relationships” pattern in an AML document. Further, the document expressing the configuration 200 of the item may contain data 220 (which are themselves, items), structure or relationships 210 (which are hierarchical items) and logic, which, as shown in the example of
(29) In the non-limiting example of
(30) As shown in
(31) In some embodiments, when the RelationshipType 212 is created, is_relationship 214 is also created. Is_relationship 214 comprises an item, and its id is the value of the relationship_id property of RelationshipType 212. As such, is_relationship 214 operates to provide an ItemType pairing to RelationshipType 212, and to define a RelationshipType rule and an ItemType for storing the source_relationship 216 and target_relationship 218 properties of the RelationshipType item 212.
(32) According to certain embodiments, source_relationship 216 is a property of RelationshipType 212 which comprises a link pointing to a child item. Similarly, target_relationship 218 is a property of RelationshipType 212, which comprises a link to a child item.
(33) As shown in the non-limiting example of
(34) According to certain embodiments, a property 222 defines data for an item. Examples of properties may include, for example, a cost for an item, which could be expressed in AML or XML in the form: “<cost>232.13</cost>” indicating that a particular item has a cost value of “232.13” units.
(35) According to certain embodiments, items of data for an item may be further specified with an attribute 224, which may be analogized as metadata for the item or property, and controlling logic and methods associated with the item. For example, an attribute may define a conditional, producing an AML or XML expression of the form “<cost condition=“between”>10.00 and 50.00</cost>” In this example, the property “cost” is further specified through the “between” attribute for which the values 10.00 and 50.00 are specified.
(36) According to certain embodiments, the configuration 200 for an item may further include history data for the item, showing some or all of the previous configurations of the item.
(37)
(38) The properties 310 of the item are set forth, and include an “item_number” value (which, according to certain embodiments, may function as a unique identifier of the instance of the item) and a “description” value, which, in this case is “Some Assy” (an abbreviation of “some assembly.”)
(39) Container tag 315 specifies that the item has relationships, including a first relationship 320 with item indicating an “add” method with an item of the type “Part BOM.” Item configuration 300 further specifies a “related_id” (e.g., child relationship between the “Part BOM” item and a child “part” item 325. Thus, by applying the “/Item/Relationships/Item/Relationships” pattern, a part-to-part BOM relationship may be described.
(40)
(41) According to certain embodiments, database server 405 is a server hosting data and implementing one or more database applications supporting query functionalities. Database server 405 is generally platform-agnostic and may host data in a number of known database formats, including a relational database format (for example, by running an instance of .SQL server) or as a columnar database format. In the non-limiting example of
(42) According to certain embodiments, database server 405 is configured to receive queries expressed as statements in a domain-specific language (for example, structured query language), and return results from the database hosted on database server 405.
(43) According to certain embodiments, backend 410 comprises a server or other computer configured to implement a query engine 415 configured to receive, from front end 420 query requests expressed in the syntax of a self-describing data system (for example, AML). As noted elsewhere, embodiments according to this disclosure are platform-agnostic and may be practiced across a wide range of hardware configurations and development environments. In some embodiments, query engine 415 may be implemented as an ASP.NET web service.
(44) In the non-limiting example of
(45)
(46) According to the non-limiting example of
(47) In some embodiments, a query definition is an item, and creating an instance of a query definition at operation 505 comprises beginning a markup language document (for example, an AML document) defining the configuration of the query definition. Further, a query definition may define the set of data (otherwise known as a domain) which a user is interested in seeing, and which can be collected across one or more different items types and/or relationships using user specified rules for filtering. Because a query definition defines the domain of a query, it may also be utilized to implement domain-based access controls to data items within the data structure.
(48) According to certain embodiments, the AML document defining the configuration of the query begins with an instance of an <item> tag, an example of which is provided below:
(49) <Item action=“qry_Execute QueryDefinition” type=“qry_QueryDefinition”>
(50) As shown above, according to some embodiments, an <item> tag creating an instance of a query definition specifies, at a minimum, a type of the instance of the query, which in this case, is a query definition (specified as “qry_QueryDefinition”), and a method, or action associated with the item, which in this case, is an instruction to execute a query, (specified as “qry_Execute Query Definition”). In some embodiments, the <item> tag creating the instance of the query definition item may further comprise a unique ID for the item, which in certain embodiments, may be advantageous if queries or query histories are stored in the data structure.
(51) As shown in the non-limiting example of
(52) According to certain embodiments, method 500 includes operation 515, wherein the query definition is provided to a query engine. According to some embodiments, operations 505 and/or 510 may variously be performed at a front end client (for example, front end 420 shown in
(53) In some embodiments, method 500 also includes operation 520, wherein the query engine determines query execution instructions based on the received query definition. In the non-limiting example of
(54) Additionally, in the non-limiting example of
(55) According to various embodiments, at operation 525, the query engine obtains the results of a query executed based on the query execution instructions. According to certain embodiments, the results obtained at operation 525 may comprise generally unformatted data, and the query engine may assemble a response containing the results of the query.
(56) In some embodiments, at operation 530, the query engine outputs the assembled query results. According to certain embodiments, operation 530 comprises returning the query response back to a user or application from which the request for a query was received (for example, front end 420 in
(57)
(58) As shown in the non-limiting example of
(59) According to certain embodiments, data model 600 is a self-describing data model which follows an “/Item/Relationship/Item/Relationship” description structure. Accordingly, in data model 600, a federated set of relationship properties 610 through 640 follow query definition 605. These relationships include query item 610. According to certain embodiments, query item 610 may appear as one or more <item> tags within a <relationship> container, such as shown in the example given in
(60) TABLE-US-00001 TABLE 1 Name Label Type Description classification Classification Aggregation (GroupBy, SUM, AVG) Union Intersection Special Join itemtype ItemType Item ItemType which is described by Query Item (Item or Relationship) Alias Alias String Alias of Query Item which will be used in joins and conditions. condition_ref_id Referenced String Referenced Query Condition Condition. ref_id Reference ID String Reference ID of Query Item
(61) As shown in the non-limiting example of
(62) According to certain embodiments, the relationships specified by data model 600 comprise query item selection properties 615, which define or identify which properties from query item 610 to include in the query response. An overview of the properties in one example of query item selection properties 615 is set forth in Table 2, below:
(63) TABLE-US-00002 TABLE 2 Name Label Type Description property_ref_id Property String Reference to qry_QueryItemAvailableProperty via ref_id value.
(64) In some embodiments, the relationships specified by data model comprise query item sort properties 620, which define which properties from the associated query item are to be used for sorting data returned by the query, and how the sort is to be performed. An overview of properties of query item sort properties 620 is set forth in Table 3, below:
(65) TABLE-US-00003 TABLE 3 Name Label Type Description property_ref_id Property String Reference to qry_QueryItemAvailableProperty via ref_id value. sort_order Sort Order Integer Order of sorting sort_order_direction Sort Order List Values: Ascending, Descending Direction
(66) According to various embodiments, the relationships specified by data model 600 further comprise query item available properties 630. In the non-limiting example of
(67) TABLE-US-00004 TABLE 4 Name Label Type Description source_id Item Reference to qry_QueryItem (qry_Queryitem) name Name String label Label ML String type Type List Data Type of the QueryItem property ref_id Reference String Reference ID (GUID) ID
(68) In the non-limiting example of
(69) TABLE-US-00005 TABLE 6 Name Label Type Description parent_ref_id Parent Item String Referenced parent Query Item. child_ref_id Child Item String Referenced child Query Item. condition_ref_id Referenced String Referenced Query Condition Condition.
(70) According to certain embodiments, the relationships specified within query definition data model 600 comprise query condition 640. Query condition 640 is an instance of an item which defines the filter conditions for the data request. According to certain embodiments, the scope of query condition 640 is the entity on which it is referenced, and a query condition can be optionally associated with a query item and query reference items. In the case where query condition 640 is referenced by a query item (for example, query item 610), then query condition filters the items defined by the query item. If, however, the query condition is referenced by a query reference (for example, query reference 635), it operates to filter the items defined by a query item referenced as the child query item for the query reference. An overview of properties of query condition 640 is set forth in Table 7 below:
(71) TABLE-US-00006 TABLE 7 Name Label Type Description condition_xml Condition Xml Text Xml representation of specified conditions. ref_id Reference ID String Reference ID of Query Condition.
(72)
(73) As shown in the non-limiting example of
(74) Referring to the non-limiting example of
(75) Configuration document 700 further includes query items 715a, 715b and 715c which, set forth properties to be part of the query response, and the properties to be used in joins and filtering. For example, query item 715a specifies an item, having the name “part” and the attribute “keyed_name,” with the value “4F1AC04A2B484F3ABA4E20DB63808A88” as a filter for items to be returned by the query.
(76) In the non-limiting example of
(77) Additionally, in this illustrative example, query document 700 further comprises an instance 725 of a query item sort property. In the non-limiting example of
(78) As shown in the non-limiting example of
(79)
(80) In the non-limiting example of
(81) Data model 800 may, according to various embodiments, include a variety of types of items 810 specifying relationships within the query definition. These items may comprise, for example, items 610-640 in
(82) Additionally, items 815 belonging to the query parameter item type may also be utilized to track or control aspects of the execution of a query. For example, according to certain embodiments, a user designed parameter “@ExecutionPath” is a dynamic parameter which may be calculated while processing a query definition to determine the progress of a query. Additionally, according to certain embodiments, items 815 belonging to the query parameter item type may also be used to define a query execution path, reflecting a route from a parent query item to a child query item in a query definition. Still further, items 815 belonging to the query parameter item type may be used to control the depth (i.e., how many levels are traversed) of recursion of a recursive query. According to some embodiments, a query engine (for example, query engine 415 in
(83)
(84)
(85)
(86) According to various embodiments, “@ExecutionPath” is a parameter calculated by a query execution engine (which according to certain embodiments, may be embodied as part of a query engine, such as, for example, query engine 415 in
(87) In some embodiments, the query parameter “@Levels” is a parameter specifying the number of levels to “drill down” in a recursive search. Thus, in the example of
(88) After an execution engine implements execution instructions based on the query definition, query engines according to certain embodiments of this disclosure obtain the results of the executed query and output the query results.
(89)
(90) As shown in the non-limiting example of
(91) According to certain embodiments, a query engine may output query results in a structured format, such as the structured format of the query definition (for example, as shown in
(92) As shown by
(93) According to certain embodiments or under certain conditions (for example, when performing very, very large queries, such as queries of a bill of materials for a helicopter, which when expressed as items in a self-describing data structure, may comprise a data structure with ˜30,000,000 item nodes) the performance of the query engine may be improved by outputting the query results in a “flat” or unstructured format. In contrast to certain structured output formats according to embodiments of this disclosure, wherein the query results are outputted in a manner that reflects and allows reconstruction of, the hierarchy and relationships within the query structure and query execution path, a “flat” output may adhere to a simplified structure, wherein only “key properties” are displayed. In this way, the file size of the query result may be made more manageable.
(94)
(95)
(96) The functionality and performance of query engines according to embodiments of this disclosure may be further enhanced by through the use of extended classification items. Extending the data model of a self-describing data system through the use of extended classifications may enhance the ability of the query engine to perform queries of polyhierarchical relationships, equivalence and associative relationships. Further, extended classifications according to embodiments of this disclosure may enhance the operation of a query engine, by enabling users to add additional properties to an item, without changing the underlying item type of the item. In this way, searches across the additional properties may be conducted quickly, in that the result set will not necessarily include null classes for the item instances not having the newly added (or extended) properties.
(97) According to certain embodiments, an extended classification encompasses a kind of item, defining a collection of properties, which are specific to an object classified by a term. Further, in some embodiments, an extended property comprises a property which exists on a global scope and which is not specific to any one item type. According to certain embodiments, extended properties may be defined via one or more extended classifications.
(98)
(99) As shown in the non-limiting example of
(100) According to various embodiments, data model 1500 describes a self-describing system whose items follow an “/Item/Relationship/Item/Relationship” structural pattern. Further, data model 1500 comprises xPropertyDefinition ItemType 1530, which defines a property which is defined on a global scope and is not specific to any one item type. As shown in
(101) TABLE-US-00007 TABLE 8 Property Name Label Data Type name Name string (32) label Label ml_string data_type Data Type list (Data Types) data_source Data Source Item (ItemType) stored_length Length integer prec Precision integer scale Scale integer is_required Required boolean is_indexed Indexed boolean column_alignment Alignment list (Text Alignment) column_width Width integer default_value Default Value ml_string pattern Pattern string (512) readonly Read Only boolean help_tooltip Tooltip ml_string track_history Track History boolean
(102) According to certain embodiments, data model 1500 further comprises ItemType_xPropertyDefinition Relationship Type 1520, which describes a link between a particular ItemType and an xPropertyDefinition ItemType 1530. According to various embodiments, any xProperty Definition can be assigned to multiple ItemTypes and any ItemType may have multiple assigned)(Property definitions.
(103) As shown in the non-limiting example of
(104) According to certain embodiments, data model 1500 comprises xPropertyContainerItem 1535, which describes an ItemType which has at least one allowed xPropertyDefinition. When an xPropertyDefinition is assigned to this ItemType, this ItemType will be added to a list of polymorphic sources of xPropertyContainerItem 1535.
(105) In some embodiments according to this disclosure, data model comprises a table of xPropertyValues 1515. As noted elsewhere in this disclosure, the implementation of extended classifications and extended properties enables properties to be dynamically added or removed from an instance of an ItemType without changing the type of the item. According to some embodiments, this may be accomplished by maintaining the values of the extended properties in a separate table from the items to which they relate.
(106) As discussed elsewhere in this disclosure, an extended classification is a type of item which defines a collection of properties, which may be specific to an object classified by a term.
(107) In the non-limiting example of
(108) TABLE-US-00008 TABLE 9 Property Name Label Data Type name Name string (32) item_number Number string (32) description Description text classification_hierarchy Classification text Hierarchy label Label ml_string select_only_leaf_class Restrict Selection boolean to only Leaf Classes select_only_single_class Restrict Selection boolean to a Single Class
(109) According to embodiments, data model 1600 may further comprise xClassificationTree_ItemType RelationshipType 1610, which defines a list of dimensions available for xClassificationTree ItemType 1605. xClassificationTree_ItemType RelationshipType 1610 may further be associated with one or more ItemTypes 1615.
(110) In various embodiments according to this disclosure, data model 1600 may further comprise xClass Relationship Type 1620. As noted elsewhere instances of XClass represent a concept named by a term, which in turn define a collection of properties, further specified by xClass_XProperty Definition Relationship Type 1625.
(111) In the non-limiting example of
(112) Additionally, data model 1600 may further comprise instances of xClass_xProperty_Flatten Relationship Type 1630, which, describes a link between a particular xClass and xPropertyDefinition, and which contains all of the xProperties of a given xClass, including both the xClass's own properties and its inherited properties. According to some embodiments, a list of inherited properties may be calculated based on a hierarchy reflected in xClassificationTree ItemType 1605. As shown in the non-limiting example of
(113) None of the description in this application should be read as implying that any particular element, step, or function is an essential element that must be included in the claim scope. The scope of patented subject matter is defined only by the claims. Moreover, none of the claims is intended to invoke 35 U.S.C. § 112(f) unless the exact words “means for” are followed by a participle.