COMPUTER-IMPLEMENTED CONVERSION OF TECHNICAL DRAWING DATA REPRESENTING A MAP AND OBJECT DETECTION BASED THEREUPON
20220350946 · 2022-11-03
Inventors
- Jacob Møller Hjerrild (Norresundby, DK)
- Kristian Holm Vester (Norresundby, DK)
- Rasmus Steenberg Andersen (Norresundby, DK)
Cpc classification
G06F30/12
PHYSICS
G06F18/213
PHYSICS
G06F30/13
PHYSICS
International classification
G06F30/27
PHYSICS
G06F30/12
PHYSICS
Abstract
A computer-implemented method of converting map data. The method includes: obtaining unstructured map data according to a first data representation, the unstructured map data representing or including a number of geometric entities where the first data representation is a technical drawing representation or a CAD data representation, and converting the unstructured map data according to the first data representation to structured map data according to a second data representation, where the second data representation is a graph data representation.
Claims
1. A computer-implemented method of converting unstructured map data, the method comprising: obtaining unstructured map data according to a first data representation, the unstructured map data representing or comprising a number of geometric entities where the first data representation is a technical drawing representation or a CAD data representation, and converting the unstructured map data according to the first data representation to structured map data according to a second data representation, where the second data representation is a graph data representation.
2. The computer-implemented method according to claim 1, wherein converting the unstructured map data according to the first data representation comprises: for at least a first geometric entity of the number of geometric entities, the first geometric entity comprising a number, or a plurality, of line segments, each line segment of the first geometric entity comprising two opposite end points, where an end point of a line segment may be shared or may be non-shared, respectively, with an end point of another line segment of the first geometric entity or of another of the number of geometric entities, generating one node in the graph data representation for each non-shared end point, generating a single node in the graph data representation for each shared end point, generating one edge in the graph data representation for each line segment so that a generated edge is connecting two generated nodes in the graph data representation that are generated for respective end points of a line segment that the edge is generated for.
3. The computer-implemented method according to claim 1, wherein the computer-implemented method further comprises for at least one, some, or all geometric entities of the number of geometric entities that comprises at least a circular segment and/or one or more other non-line segments, converting or replacing such geometric entities to or by an approximating line segment version before or when converting such geometric entities to the graph data representation.
4. The computer-implemented method according to claim 2, wherein the computer-implemented method further comprises assigning a weight for at least one edge, the weight for an edge corresponding to a length value of a line segment that the edge is or was generated for.
5. The computer-implemented method according to any one of claim 1, wherein the computer-implemented method further comprises determining a plurality of end points of line segments of the first data representation that are within a first predetermined vicinity or length of each other or of a single one of the plurality of end points, and replacing the determined plurality of end points within the first predetermined vicinity or length of each other or of a single one of the plurality of end points by a single end point retaining or having the line segments of the determined plurality of nodes, and/or determining an end point of the first data representation, among end points of two line segments, that is located within a second predetermined vicinity or length of at least one of the two line segments, and replacing the determined end point by a new single end point on one or both of the line segments at the location where the two line segments intersect or, if not intersecting, would intersect if at least one of the two line segments is extended until the two line segments intersect, and/or determining an end point of the first data representation, connected with only a single other end point and being located within a third predetermined vicinity or length of a further other end point, and connecting the determined end point with the further other end point by a new line segment, and/or determining two at least substantially parallel line segments of the first data representation that at least partly overlaps in their length direction and are distanced apart by less than a fourth predetermined vicinity or length in a direction substantially perpendicular to the length direction of the at least two substantially parallel line segments, and replacing the two at least substantially parallel line segments with a single line segment comprising the combined end points of the replaced two at least substantially parallel line segments.
6. The computer-implemented method according to claim 1, wherein the computer-implemented method further comprises determining a position or set of coordinates of where two line segments of the number of geometric entities of the first data representation intersect, and determining whether an end point is located within a fifth predetermined vicinity or length of the determined position or set of coordinates, and if so then replacing the line segment for each of the two intersecting line segments with two line segments and connecting respective line segments to the end point determined to be within the fifth predetermined vicinity or length of the determined position or set of coordinates.
7. The computer-implemented method according to claim 1, wherein the computer-implemented method further comprises assigning a value for each of a number of predetermined features for each node of the graph data representation, wherein each of the predetermined features characterises an aspect of the node in question and its context.
8. The computer-implemented method according to claim 7, wherein the values of the predetermined features are provided to an input part or input layer of a graph neural network, or a graph convolutional neural network, for subsequent data processing.
9. The computer-implemented method according to claim 7, wherein the predetermined features for a particular node comprises one or more selected from the group of: a minimum length of edge(s) connected to the particular node, a maximum length of edge(s) connected to the particular node, indication of which angle group(s), if any, of a plurality of different angle groups, the particular node is determined to belong to, for each angle group, a number of occurrences that the particular node is determined to belong to a respective angle group, a number of one or more neighbouring nodes determined to be orthogonal to each other as seen from the particular node, a circle probability representing a probability of the particular node being determined to be part of a circle, an indication of whether the particular node is determined to be part of a circle or not, an indication of whether the particular node is determined to be part of a half circle, an indication of whether the particular node is determined to be part of a quarter circle, an indication of whether the particular node is determined to be part of an angle group representing a corner, a shortest Euclidean distance or length between the particular node and a node determined to belong to an angle group representing a quarter circle, a shortest topological graph distance or length between the particular node and a node determined to belong to the angle group representing a quarter circle, a shortest topological graph distance or length between the particular node and a node determined to belong to an angle group representing a half-circle, a shortest Euclidean distance or length between the particular node and a node determined to belong to the angle group representing a half-circle, and an identifier or indication of which type of geometric entity the particular node arises from.
10. The computer-implemented method according to claim 1, wherein the nodes of the graph data representation are non-ordered, the graph data representation is undirected, and/or the graph data representation is a non-connected graph representation.
11. The computer-implemented method according claim 1, wherein the first data representation is a layered or a non-layered two-dimensional or three-dimensional drawing data representation and/or the first data representation comprises data representing a drawing of a building or at least a part thereof.
12. The computer-implemented method according claim 1, wherein the unstructured map data is or comprises unstructured indoor map data and/or is or comprises unstructured indoor floor plan data.
13. The computer-implemented method according claim 1, wherein each geometric entity of the number of geometric entities is not connected to another geometric entity of the number of geometric entities.
14. A computer-implemented method of detecting or predicting a presence of at least one object in unstructured map data according to a first data representation, the unstructured map data representing or comprising a number of geometric entities, wherein the first data representation is a technical drawing representation or a CAD data representation and one or more of the number of geometric entities represents or defines an object to be detected or to have its presence predicted, converting the unstructured map data according to the first data representation to structured map data according to a second data representation, or according to the method according to claim 1, where the second data representation is a graph data representation, detecting or identifying one or more objects in the structured map data according to the second data representation in response to providing the structured map data according to the second data representation to a computer program or routine implementing a trained graph artificial intelligence or machine learning method or component, or a trained graph neural network (GNN), to generate or output the detected or identified one or more objects.
15. The computer-implemented method according to claim 14, wherein the trained graph artificial intelligence or machine learning method or component is or implements a graph neural network (GNN).
16. The computer-implemented method according to claim 14, wherein the trained graph neural network (GNN) is a graph convolutional (neural) network (GCN) node classification system.
17. The computer-implemented method according to claim 14, wherein the one or more objects being detected or identified in the structured map data according to the second data representation is one or more of accessibility objects and/or access points/connections/objects, and/or obstacles.
18. The computer-implemented method according to claim 14, wherein the trained graph neural network (GNN) is a graph attention network (GAT).
19. An electronic data processing system, comprising: one or more processing units connected to an electronic memory, wherein the one or more processing units are programmed and configured to execute the computer-implemented method according to claim 1 and/or to execute the computer-implemented method according to claim 14.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0088] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099]
[0100]
[0101]
[0102]
[0103]
DETAILED DESCRIPTION OF THE INVENTION
[0104] Various aspects and embodiments of a computer-implemented method of converting unstructured map data, a (related) computer-implemented method of detecting (or predicting a presence of) at least one object in unstructured map data, and an electronic data processing device or system configured to execute one or both of the computer-implemented methods as disclosed herein will now be described with reference to the figures.
[0105] The shown figures are schematic representations for which reason the configuration of the different structures as well as their relative dimensions are intended to serve illustrative purposes only.
[0106] Some of the different components are only disclosed in relation to a single embodiment of the invention, but are meant to be included in the other embodiments without further explanation.
[0107]
[0108] Illustrated are four different ways of drawing (and thereby representing) the exact same object—namely a door and doorway—taken from different technical drawings, being as an example a CAD drawing. This illustrates in a simple way, the diversity—and thereby the inconsistency—typically found in technical drawings even for simple things, which as mentioned presents a challenge in automated or semi-automated generation of map data from such technical drawing data and often necessitates a need for a GDS or other professional having to review and e.g. mark which objects are what and even carry out corrections before a final map is obtained or a map is obtained that can be used for reliable automatic or semi-automatic object detection used in connection with map data generation.
[0109]
[0110] Illustrated in
[0111]
[0112] Technical drawings/CAD drawings can range from being very elaborate, for example naming/showing construction materials, walls with a width and insulation material, ventilation ducts, technical installations, etc., to be simpler in nature, for example just showing walls and other features as single lines.
[0113] In technical/CAD drawings, constructional features might not always be explicitly drawn, but can be implicit in nature, for example when a room is not digitized as a polygon but rather can be deduced by looking at a set of lines (i.e. representing walls) intersecting each other without having a shared point (i.e. two lines intersecting without sharing a vertex/point). Furthermore, technical/CAD drawings may contain drawing aids (e.g. helper lines with equal spacing) and/or non-constructional features such as legends, bill of materials, etc., which has nothing to do with the building layout as such.
[0114] Typically, the technical drawing data or the CAD data within the present context comprises a (often very large) number of geometric entities (e.g. ranging from a few thousands to more than a million geometric entities depending on purpose/application) where each geometric entity is represented or defined according to a vectorised (and lossless) geometric entity format.
[0115] Each geometric entity will be of one of a number of predetermined basic or fundamental types such as “point”, “polyline”, “polygon”, “circle”, “arc”, “spline”, “ellipse”, “annotation”, “multi-patch”, etc. and have an own associated set of relevant parameters.
[0116] Merely rasterising such technical drawings into digital 2D images and using—even well trained—ANNs (artificial neural networks), DNNs (convolutional or otherwise), or other suitable machine learning techniques have not produced suitably good results in a reliable way. All rasterising typically involve some loss of detail or information, but rasterising such technical drawings into a 2D image involves loss of many valuable details and much information, such as (precise) geometric coordinates, resolution, attributes (CAD layer info), etc. that can be significant for proper map data generation and further uses.
[0117] This can be mitigated to some extent—but not fully or sufficiently—by rasterising into 2D images of (very) high resolution potentially leading to gigabyte-sized images. However, that increases the computational complexity and computational burden to a very large degree for subsequent data processing (such as object detection, map data creation and/or augmentation, etc.), increases storage requirements, and so on.
[0118]
[0119] Illustrated is an example of a portion of the drawing data of
[0120] Generally, geographic Information Systems (GISs) are typically used to analyse geographic data's spatial relations. It is thus significant that relations between geographic objects such as roads, streams, and areas, are mapped in detail. For example, if two roads have an intersection they must share a vertex/point where they intersect.
[0121] GIS data typically also follow a specific scheme that ranges from very complex topological data models, for example roads are boundaries for areas that again contain buildings, to more simple data models for each object, for example one model for roads, another for areas, a third for buildings, etc.
[0122] Objects in GIS data can have one or more attributes describing the properties of the object, for example the width of a road, an area's class, or a specific type of building. GIS can combine spatial queries with attributes thus enabling users to perform complex and detailed analysis of spatial data and their properties and how they relate to each other and other objects.
[0123]
[0124] Illustrated is an image of a final graphical map that may be presented to an end user e.g. or preferably in map applications designed for web and/or mobile devices and/or used in any other suitable manner e.g. for further subsequent data processing. A colour scheme is applied to the map objects to aid users in distinguishing the map objects from each other. The graphical map here illustrates how a map may look like when applying aspects of the present invention to suitable input data.
[0125]
[0126] Illustrated is unstructured map data 101 according to a first data representation being technical drawing data, such as CAD data. In at least some embodiments, the first data representation is a layered or a non-layered two-dimensional or three-dimensional technical drawing data (e.g. CAD data) representation.
[0127] The unstructured map data 101 comprises one or more datasets or files of map data, each comprising a number (in principle one or more) of geometric entities as disclosed herein. Typically, a dataset or file of unstructured map data 101 in the present context will comprise a relatively large number of geometric entities of various types (see e.g.
[0128] As mentioned, technical drawing data and CAD data typically defines data according to a vectorised and lossless geometric entity format with relevant parameters and associated values depending on the specific basic or fundamental type of a particular geometric entity and how it is to be rendered/represented. The types may e.g. comprise one or more of “point”, “polyline”, “polygon”, “circle”, “arc”, “spline”, “ellipse”, “annotation”, “multi-patch”, etc., which are known as such by itself. Each geometric entity (depending on type) may also comprise or be associated with a spatial (relative or absolute) position within a suitable coordinate system or similar reference framework.
[0129] As an example, a polyline geometric entity (being a set of two or more vertices that form a connected set of lines) may e.g. comprise or represent a “Polyline” identifier or label and the data {“points”: [[100,100], [100,200], [200,300], [300,300]]} (representing a polyline having four vertices as an array of points). As further examples, a polygon geometric entity (being a polyline with three or more vertices that begin and end in a same point) may e.g. comprise or represent a “Polygon” identifier or label and the data {“points”: [[100,100], [200,100], [150,150], [100,100]]} (representing a triangular polygon as an array of points), and a circle geometric entity (often represented by a centre of the circle and the circle's radius) may e.g. comprise or represent a “Circle” identifier or label and the data {“point”: [100,100], “radius”: 50} (representing a circle as an object with a centre in 100,100 and a radius of 50).
[0130] A converter or conversion element 102 receives at least one file or dataset (or a part thereof) according to the first data representation and converts it as disclosed herein. Accordingly, unstructured map data in the first data representation is converted into structured map data according to a second data representation being a graph data representation (see e.g. 804, 805, 807 in
[0131] One or more, typically several, of the geometric entities according to the first data representation may also comprise or represent one or more non-line segments, i.e. segments that are not a line segment or made up solely of line segments. Such could e.g. be geometric entities of the types “circle”, “arc”, “spline”, “ellipse”, etc.
[0132] In at least some embodiments, such non-line segment entities are (prior or during conversion to the graph data representation) converted or replaced by an approximating line segment version.
[0133] For circle, arc, ellipse, etc. types of geometric entities, an approximating line segment version could e.g. be derived by any suitable piecewise linear approximation or similar and could e.g. be based on the official PostGIS ST_CurveToLine function or functionality (see e.g. postgis.net/docs/ST_CurveToLine.html). The sampling or approximation could e.g. be based on a variable specifying a sampling or step angle defined in relation to the centre of the circular shape to a first endpoint (becoming the first node). The angle is incremented in fixed steps and a point of the circular shape being at the incremented angle becomes the node for that step. Accordingly the circular shape is converted or replaced. Such angle could e.g. be 3°, 5°, 10°, 15° or any other suitable angle depending on specific use. 5° to 10° or about 5° to about 10° has been seen to offer a good balance between subsequent reliable detection/prediction and computational effort and storage requirements (the smaller the angle, the more line segments will be used). In some embodiments, if a circular shape is not divisible by a specified sampling or step angle, the sampling or step angle may be replaced (approximated by) one that does (e.g. replacing a sampling or step angle of 11° with one that is 10° for a quarter circle) to have approximating line segments of equal length.
[0134] One exemplary embodiment of a flowchart of such a computer-implemented conversion method is illustrated in
[0135] The conversion may also comprise one or more simplifying or optimising functions/methods (done prior, during, and/or after the actual conversion) to increase the usability of the graph data representation and/or to decrease the computational complexity of it for subsequent data processing. Some such simplifying or optimising functions/methods are described in connection with
[0136] The graph data representation enables efficient subsequent data processing such as reliable object detection or at least reliable object prediction and/or map data creation and/or augmentation as disclosed herein.
[0137] In at least some embodiments, the conversion 102 further comprises embedding relevant data of the unstructured map data according to the first data representation (e.g. CAD data) in the graph data representation. The relevant data may comprise values for relevant parameters (such as geometry parameters (e.g. stored according to formats such as GeoJSON, Well Known Text (WKT), etc.), CAD Layer information, version, date, etc.) for the converted geometric entities and in particular the relevant coordinates of the converted geometric entities within an overall/reference coordinate system. In this way, the graph data representation can carry any geometric and any topological relation between the geometric entities that is (decided to be) embedded in the graph data representation at any given CAD coordinate (such that the exact same coordinates are represented in the graph data representation) or put in other words—the graph data representation can be fully compliant with the original technical drawing/CAD data such that data and information can be transferred back and forth between technical drawing/CAD domain and the graph domain without any loss of relevant detail or orientation.
[0138] In some further embodiments, additional attributes, data, etc. such as CAD layer info, descriptive attributes, etc. can also be carried over.
[0139] In some further embodiments, during or after conversion a weight is assigned (or generated and assigned) for each edge of the graph data representation (or at least one or some of them), where the weight is or corresponds to the length of a line segment that the edge is or was generated for. The weight may e.g. be a normalized length instead of an actual length. The weight may e.g. be used to derive one or more features (such as a minimum or a maximum length of edge(s) connected to a particular node or other; see e.g. further in the following), setting (or at least influencing) the topological distances in the graph data representation according to the lengths of the line segments that the edges are based on (by setting their respective weights to be equal to the respective lengths), whereby e.g. a topological shortest path directly is influenced or based on respective lengths, and so on.
[0140] Additionally (at least in some embodiments), relevant values for a number of predetermined features, e.g. or preferably in the form of a feature vector (f) or similar, is generated or derived for each node of the graph data representation, where each of the predetermined features characterises an aspect of the node in question and its context. The relevant values for the features may e.g. be derived during the conversion. The features/feature vector may e.g. be supplied to an input layer or input part of a graph neural network (GNN) as disclosed herein. The particular selection of features will have a significant impact on the efficiency, reliability, and/or usability of a GNN subsequently processing the graph data representation for a particular purpose such as object detection or object prediction.
[0141] The number of features or the length of the feature vector is typically the same for all nodes of a particular graph data representation, but it may and typically will, for a particular node, contain non-zero values for only a sub-set thereof.
[0142] In some embodiments, the predetermined features (e.g. or preferably in the form of a feature vector) for a particular node comprises one or more selected from the group of (or one or more selected from the group consisting of) and as disclosed herein: [0143] a minimum length of edge(s) connected to the particular node, [0144] a maximum length of edge(s) connected to the particular node, [0145] indication of which angle group(s) (if any) of a plurality of different angle groups, the particular node is determined to belong to, [0146] for each angle group, a number of occurrences that the particular node is determined to belong to a respective angle group, [0147] a number of one or more neighbouring nodes determined to be orthogonal to each other as seen from the particular node, [0148] a circle probability representing a probability of the particular node belonging to a circle geometric entity, [0149] an indication of whether the particular node is determined to part of a circle geometric entity or not, [0150] an indication of whether the particular node is determined to be part of a half circle, [0151] an indication of whether the particular node is determined to be part of a quarter circle, [0152] an indication of whether the particular node is determined to be part of an angle group representing a corner, [0153] a shortest Euclidean distance or length to a node determined to belong to an angle group representing a quarter circle, [0154] a shortest topological graph distance or length to a node determined to belong to the angle group representing a quarter circle, [0155] a shortest topological graph distance or length to a node determined to belong to an angle group representing a half-circle, [0156] shortest Euclidean distance or length to a node determined to belong to the angle group representing a half-circle, [0157] shortest topological graph distance or length to a node determined to belong to the angle group representing a half-circle, and [0158] an identifier or indication of which type of geometric entity (e.g. arc, circle, ellipse, etc.) the particular node arises from.
[0159] The feature vector, f serves the purpose of describing/representing the qualities or characteristics of node, n, in a d-dimensional space of real numbers, R, i.e.
f:n.fwdarw.R.sup.d,
where each entry in f typically is a real number respectively representing each of the relevant/selected/used features (e.g. as mentioned above and/or disclosed herein). The features may e.g. comprise one or more of “minimum length”, “maximum length”, “circle probability”, “is circle”, “is half circle”, “is quarter circle”, “is corner”, “angle group 90”, “angle group 180”, or any other suitable angle group, etc. as disclosed herein.
[0160] After the unstructured map data has been converted into a graph data representation it is, at least in some embodiments and as illustrated, received and used as input by a suitable map object detection engine (also referred to herein as prediction engine) 103. The generated graph data representation (being provided to the map object detection engine 103) is herein also referred to as a geometric graph (data representation). In at least some embodiments, the map object detection engine 103 is or comprises a suitably trained GNN and the generated graph data representation (being the output of the conversion 102) is provided to the input layer of the (appropriately trained) GNN, which in turn outputs (as a result) the nodes and associated edges of the graph data representation being determined or predicted (by the GNN) to be one of a number of possible different types/categories of objects, such as a ‘door’, a ‘wall’, a ‘stair’, and/or any other type useful in creating or processing map data within the present context. The number of different detectible types (and their nature) depends on how the GNN is modelled and how it is trained and in particular on which features/which feature vector is used together with the conversion 102 (and used in the training of the GNN).
[0161] The generated graph data representation can in principle be provided to any suitably trained GNN—suitably trained to produce a meaningful output. In some embodiments, the GNN is a Graph Convolutional (Neural) Network (GCN) node classification system where an output graph (i.e. the output produced by the GCN) has the same dimension as the input graph of the GCN. The output graph comprises a label or similar identifier (of a plurality of such labels or identifiers) for each node designating a respective node as belonging to a predicted or determined object (as predicted or determined by the GCN) as described or identified by the respective label. As examples, if an input node is predicted or determined by the GCN to belong to a door object then the output node corresponding to the input node comprises a label or identifier designating a ‘door’, if an input node is predicted or determined by the GCN to belong to a wall object then the output node corresponding to the input node comprises a label or identifier designating a ‘wall’, etc.
[0162] In some embodiments, the GNN is trained as a supervised node classification system, by using any Graph Convolutional Network. In some further embodiments, a model is trained based on an optimally selected feature set, which is a subset of the full d-dimensional set, specifically selected for detecting the specified objects such as doors, walls, rooms, elevators, stairs, etc. The automatic feature selection can be done by principal component analysis, filter or wrapper methods or even by trying all combinations.
[0163] In some embodiments, the GNN/GCN receives the generated graph data representation at the input layer or input part or similar that performs aggregation and maps the input features, using a fully connected layer, to an output equal to the number of network nodes n in hidden layers of the GNN/GCN. The hidden layers perform aggregation followed by a mapping to size n. Additionally, an output layer performs aggregation and maps the result, e.g. or preferably using a fully-connected layer, to c nodes equal to the number of object classes. For a binary classifier for a single object (e.g. door vs non-door) c is equal to 2. Having initialized a network, a forward pass can be performed using the graph data representation and its feature matrix and a convolution layer of each network may comprise aggregation as generally known.
[0164] In some embodiments, the map object detection (or prediction) engine 103 is configured to function as a detection service meaning that it will predict one type of object using a specific model specifically trained for this given purpose of detection. One advantage of doing so, is that each type of object, e.g. door, wall, etc., can have a corresponding optimal detection or prediction model associated specifically for it, this particular model will have its own neural network depth, width, network hyper-parameters, etc. This is e.g. illustrated further in
[0165] In some further embodiments, the GNN is a so-called graph attention network (GAT) (see e.g. arxiv.org/abs/1710.10903), which is a specific subset of a convolutional (in a graph neural network context) graph neural network. It has surprisingly been determined that GAT GNNs are particular useful and well-performing in relation to reliable object detection on graphs representing or being converted from technical drawing data/CAD data as disclosed herein, since attention weights between neurons of the GNN are trained. There exist various types or subclasses of GAT GNNs.
[0166] As mentioned, in some embodiments, the GCN is a so-called Graph Attention Network (GAT). In alternative embodiments, the GCN is a graph data representation according to the GraphSAGE framework (see e.g. snap.stanford.edu/graphsage/), a Topologically Adaptive Graph Neural Network (TAGCN) (see e.g. arxiv.org/abs/1710.10370), an Attention Based GNN (AGNN) (see e.g. github.com/dawnranger/pytorch-AGNN), a GIN (see e.g. workshops.infed.ac.uk/accml/papers/2020/AccML_2020_paper_6.pdf), an APPNP (see e.g. arxiv.org/pdf/1810.05997.pdf), or any other suitable GCN within this class or corresponding ones.
[0167] As an alternative, to providing the geometric graph to a suitable GNN, the geometric graph may be applied to so-called semi-supervised auto-encoders for abnormality detection (i.e. finding abnormal objects in the CAD data) or unsupervised methods by leaving out any (class) labels or identifiers (e.g. ‘door’, ‘wall’, etc.) or selecting certain types of data based on a particular need or use.
[0168] Accordingly, the map object detection engine 103 generates data 104 of which nodes and edges of the input graph data representation that is determined (according to the engine 103) to be of a certain predetermined type or category. In some embodiments (and as illustrated), the map object detection engine 103 generates respective data 104 for each specific object type/category, and store such separately in a suitable data storage structure 105 such as one or more databases. As an example and as illustrated, respective data representing identified/determined doors 104, walls 104, stairs 104, desks 104, etc. is derived individually. This data can then be used for various purposes e.g. for reliable and efficient map creation, map augmentation, indoor routing, outdoor routing, etc.
[0169]
[0170] Illustrated is a map object detection or prediction engine 103 that may correspond to the engine of
[0171] Additionally shown is a suitable data storage structure 109 (such as one or more databases) storing different models—e.g. or preferably one specifically for each object type or category, e.g. door, wall, stair, desks, etc. (see e.g. 104 in
[0172]
[0173] Illustrated in
[0174] At step 801, the method 800 starts and potentially is initialized, etc.
[0175] At optional step 802, one or more simplifying or optimising functions/methods is executed for unstructured map data according to a first data representation being technical drawing data, such as CAD data (see e.g. 101 in
[0176] In some embodiments, one such simplifying or optimising function/method (which may be instead of or in addition to the simplifying or optimising function(s)/method(s) mentioned herein) comprises determining a plurality of end points of the first data representation that are within a first predetermined vicinity or length/distance of each other or of a single one of them and then replacing the determined end points (then being within the first predetermined vicinity or length of each other) by a single end point where the single end point retains or is assigned with all the line segments of the determined end points (to being within the first predetermined vicinity or length/distance). Stringently speaking, the determined end points need not actually be replaced by an end point but instead one of the existing end points (e.g. the most suitable end point according to one or more predetermined criteria) of the determined end points is selected to be kept (while removing the others). In some further embodiments, the value of the first predetermined vicinity or length can be controlled according to use and/or preference. The value may be set universally or can even be set dynamically (e.g. one region of the first data representation is set to one value while another region of the first data representation is set to another value, end points arising from a particular type of geometric entity is set to one value while end points arising from another particular type of geometric entity is set to another value, etc.). As examples, the value of the first predetermined vicinity or length may e.g. be selected as one value selected from/corresponding to 1, 2, 3, 4, 5, or more millimetres, e.g. about 1 to about 10 millimetres and even below 1 millimeter depending on specific implementation/use and e.g. as obtainable from the coordinates and associated scale of the underlying technical drawing/CAD data of the first data representation. Such function/method can and typically will greatly simplify first data representation and in turn the resulting graph data representation (as the number of end points and thereby nodes are reduced) and thereby reduce any subsequent computational processing thereof and/or reduce storage requirements for the graph data representation often without sacrificing too much accuracy and/or reliability, if any practically speaking. An example of a result of such function/method is e.g. illustrated in
[0177] In some embodiments, one other such simplifying or optimising function/method (which may be instead of or in addition to the simplifying or optimising function(s)/method(s) mentioned herein) comprises determining an end point among end points of two line segments (in vicinity of each other), where the end point is located within a second predetermined vicinity or length of at least one of the two line segments, and replacing the determined end point by a new single end point on one or both of the line segments at the location where the two line segments intersect (i.e. at the coordinate of intersection) or, if not intersecting, would intersect if at least one of the two line segments is extended until the two line segments intersect. In effect, line segments within a relatively small proximity (as determined by the second predetermined vicinity or length) is ‘snapped’ together. As examples, the value of the second predetermined vicinity or length may e.g. be selected as one value selected from/corresponding to between more than zero to about 1 centimetre in the low end of the range (e.g. 1-2 millimetres) depending on specific implementation/use. The value of the second predetermined vicinity or length may be set universally or can even be set dynamically. An example of a result of such function/method is e.g. illustrated in
[0178] In some embodiments, one other such additional simplifying or optimising function/method (which may be instead of or in addition to the simplifying or optimising function(s)/method(s) mentioned herein) comprises determining an end point connected with only a single other end point (i.e. it has only one direct neighbour connected by a single line segment) and where the end point located within a third predetermined vicinity or length of a further other end point (i.e. the further other end point is not connected to the determined end point or its neighbour), and connecting the determined end point with the further other end point by a new line segment. This connects disconnected line segments with other parts and in turn creates a more connected graph data representation. As examples, the value of the third predetermined vicinity or length may e.g. be selected as one value selected from/corresponding to between more than zero and up to 4 centimetres depending on specific implementation/use. The value of the third predetermined vicinity or length may be set universally or can even be set dynamically. An example of a result of such function/method is e.g. illustrated in
[0179] In some embodiments, one other such further simplifying or optimising function/method (which may be instead of or in addition to the simplifying or optimising function(s)/method(s) mentioned herein) comprises determining two at least substantially parallel line segments of the first data representation where the two at least substantially parallel line segments at least partly overlaps in their length direction and furthermore are distanced apart only by less than a fourth predetermined vicinity or length in a direction substantially perpendicular to the length direction of the two at least substantially parallel line segments, and replacing the two at least substantially parallel line segments with a single line segment comprising the combined end points of the replaced two at least substantially parallel line segments. In effect, two parallel or substantially parallel at least partly overlapping line segments within a certain (relatively small) distance of each other will be merged as one while retaining all end points of the relevant line segments (and thereby retaining all nodes in the resulting graph data representation). As examples, the value of the fourth predetermined vicinity or length may e.g. be selected as one value selected from/corresponding to between more than zero to about 0.5 or 1 centimetres depending on specific implementation/use. The value of the fourth predetermined vicinity or length may be set universally or can even be set dynamically. The value of the fourth predetermined vicinity or length may be set universally or can even be set dynamically. Two exemplary results of such function/method are e.g. illustrated in
[0180] In some embodiments, yet a further such simplifying or optimising function/method (which may be instead of or in addition to the simplifying or optimising function(s)/method(s) mentioned herein) comprises determining one or more, e.g. all, intersecting line segments (of the underlying technical drawing/CAD data and for each intersection involving two line segments determining or obtaining the set of coordinates of where two line segments intersect. After the set(s) of coordinates have been determined or obtained then the simplifying or optimising function/method checks (for each set of coordinates of an intersection) whether an end point is located within a fifth predetermined vicinity or length of a particular set of coordinates and if so, replacing the line segment for each of the involved two intersecting line segments with two line segments (then totalling four line segments) and connecting the respective line segments to the end point determined to be within the fifth predetermined vicinity or length of the particular set of coordinates. The line segments—at their respective opposite end—retains their connections to respective end points. In a sense, an existing end point (within a suitable distance or length) is used as a connection end point for an intersection. In case of two or more end points being within the fifth predetermined vicinity or length, one can be selected according to one or more additional criteria (e.g. selecting the closest). In this way, the complexity of the first data representation and in turn the graph data representation is reduced typically reducing any subsequent computational processing thereof often without sacrificing too much reliability if any, practically speaking. In effect, the graph data representation becomes more connected and connected in a less complex manner. In some further embodiments, the value of the fifth predetermined vicinity or length can be controlled according to use and/or preference. The value may be set universally or dynamically, e.g. as described above or in another suitable way. As examples, the value of the fifth predetermined vicinity or length can e.g. correspond to up to about 0.5 to 1 centimetres or up to about 2 centimetres or alternatively more in the ‘real world’ e.g. as obtainable from the coordinates and associated scale of the underlying technical drawing/CAD data. An example of a result of such function/method is e.g. illustrated in
[0181] Step 802 is as mentioned optional and may be fully omitted or may comprise one or more other simplifying or optimising functions/methods instead or in addition to one or more of the functions/methods mentioned above. One or more of the simplifying or optimising functions/methods (or parts thereof) may also be executed ‘later’ in the flow chart at suitable steps or locations (as new steps). One function/method of step 802 (or a part thereof) may also be executed later while another (or a part thereof) is executed at step 802.
[0182] The above mentioned simplifying or optimising functions/methods have been described as working for/on the first data representation/CAD data and at least in some embodiments they are respectively working only on data of a single layer (at least at a time).
[0183] Furthermore, the above mentioned simplifying or optimising functions/methods (or at least one or some of them) will typically be executed for whole data set/layer to find all suitable candidates rather than only a single instance.
[0184] Alternatively, one or more of the above mentioned simplifying or optimising functions/methods (or a part or parts thereof) may be implemented to process data in the graph data instead of the first data representation. The above description is still applicable changing end point to node and line segment to edge. It is however, more computationally efficient—at least for some applications—to process the data in the first data representation.
[0185] At step 803, data representing a (or at least one) geometric entity as disclosed herein is obtained or provided from unstructured map data according to a first data representation being technical drawing data, such as CAD data (see e.g. 101 in
[0186] At step 804, the type of the obtained geometric entity is determined among a number of possible types. The type is typically one of predetermined basic or fundamental types according to or as supported by the technical drawings/CAD data, e.g. one of a number of predetermined basic or fundamental types such as “point”, “polyline”, “polygon”, “circle”, “arc”, “spline”, “ellipse”, “annotation”, “multi-patch”, etc.
[0187] At step 805, respective end points of each line segment of the geometric entity is determined. The way to determine respective end points typically depend on the type of geometric entity, e.g. is it a polyline or a circle, etc. For example a point, line/polyline, polygon, and similar, only comprises none (for a point) or exclusively one or more line segments while a circle, arc, etc. does not disclose any line segments.
[0188] Step 805 may additionally comprise the functionality of converting or replacing non-line segments of the geometric entity in question to or by a suitable approximating line segment version thereof, e.g. as described in connection with
[0189] At step 806, a node is assigned or generated in a graph data representation as disclosed herein, e.g. as in connection with
[0190] At step 807, values for a number of features, e.g. or preferably in the form of a feature vector, is assigned or derived for each node. The number of features is typically the same for all nodes (i.e. the feature vector for such embodiments has the same length and ordering of values/parameters for all nodes). The features/the feature vector may e.g. be or preferably is as described in connection with
[0191] These features are to be used for subsequent processing by a suitable and suitably trained GNN as disclosed herein, e.g. in order to identify or predict certain object of the graph and thereby of the underlying technical drawing/CAD data (comprising the geometric entities as obtained at step 803).
[0192] In some further embodiments, step 807 may also assign or associate other data or meta-data for other purposes, e.g. as disclosed herein and e.g. in connection with
[0193] It is noted, that the functionality of step 807 could be executed as part of step 806 or in another suitable manner.
[0194] At step 808, it is tested whether additional geometric entities should be processed by method 800 (i.e. should generate additional node(s) and edge(s) in the graph data representation). In case of no, the method proceeds to terminating this execution of method 800 at step 809. This may e.g. be the case if all (relevant) geometric entities of the underlying technical drawing/CAD data has been processed. In case of yes, the method 800 loops back to step 803 repeating the relevant steps for a new/next geometric entity.
[0195] The generated graph data representation may be referred to as a main/geometrical graph data representation.
[0196] It is noted, that the resulting graph data representation typically or at least can comprise sub-graphs without connections/edges between them, e.g. two non-connected buildings in the unstructured map data according to the first data representation will result in a separate sub-graph representation for each building without connection, two separate objects (e.g. two tables) inside a room will result in one sub-graph representation for each object without connection between them, etc.
[0197]
[0198]
[0199] According to the illustrated simplifying or optimising function/method (also described in connection with
[0200] Note, for simplicity/clarity, only some end points/nodes are labelled with a reference number.
[0201]
[0202] According to the illustrated simplifying or optimising function/method (also described in connection with
[0203] Note, for simplicity/clarity, only some end points/nodes and some line segments/edges are labelled with a reference number.
[0204]
[0205] According to the illustrated simplifying or optimising function/method (also described in connection with
[0206] For simplicity/clarity only some end points/nodes and some line segments/edges are labelled with a reference number.
[0207]
[0208] According to the illustrated simplifying or optimising function/method (also described in connection with
[0209] For simplicity/clarity only some end points/nodes and some line segments/edges are labelled with a reference number.
[0210]
[0211] According to the illustrated simplifying or optimising function/method (also described in connection with
[0212] For simplicity/clarity only some end points/nodes and some line segments/edges are labelled with a reference number.
[0213]
[0214]
[0215]
[0216]
[0217] Shown is an electronic data processing apparatus 500 comprising one or more processing units 502 connected via one or more communications and/or data buses 501 to an electronic memory and/or electronic storage 503, and optionally one or more signal transmitter and receiver communications elements 504 (e.g. one or more of cellular, Bluetooth, WiFi, etc.) for communicating via a computer network, the Internet, and/or the like. In at least some embodiments, the electronic data processing apparatus 500 is configured to communicate with a cloud computing system 509 that may (or may not) be a distributed system. The one or more processing units 502 may e.g. include one or more CPUs, TPUs (tensor processing units), FPUs (floating point units), GPUs (graphics processing units), and/or the like.
[0218] The electronic data processing apparatus 500 may also comprise an optional display 508 and/or one or more optional (graphical and/or physical) user interface or user experience (UX) elements 507.
[0219] The electronic data processing apparatus 500 can e.g. be one or more programmed computational devices, e.g. like a PC, laptop, computer, client, server, smart-phone, tablet, etc. and is specially programmed to carry out or execute one or more of the computer-implemented methods and embodiments thereof as described throughout the specification and variations thereof.
[0220] The electronic data processing apparatus 500 (or a corresponding apparatus) may also be used to train one or more artificial intelligence, machine learning, or similar methods or components to be trained as disclosed herein, and in particular to suitably train a GNN as disclosed herein.
[0221] Some preferred embodiments have been shown in the foregoing, but it should be stressed that the invention is not limited to these but may be embodied in other ways within the subject matter defined in the following claims.
[0222] It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, elements, steps or components but does not preclude the presence or addition of one or more other features, elements, steps, components or groups thereof.
[0223] In the claims enumerating several features, some or all of these features may be embodied by one and the same element, component or item. The mere fact that certain measures are recited in mutually different dependent claims or described in different embodiments does not indicate that a combination of these measures cannot be used to advantage.
[0224] In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim. The word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
[0225] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to an advantage.
[0226] It will be apparent to a person skilled in the art that the various embodiments of the invention as disclosed and/or elements thereof can be combined without departing from the scope of the invention as defined in the claims.