METHOD FOR GENERATING TRIPLES FROM LOG ENTRIES

Abstract

A computer-implemented method, computer program product, and a technical system for generating triples including providing a plurality of log entries from respective log files, wherein each log entry of the plurality of log entries includes at least one text message, generating at least one template based on the plurality of log entries using unsupervised clustering, wherein the at least one template includes at least one variable part and at least one fixed part, assigning each log entry of the plurality of log entries to one respective template based on the generated at least one template using a similarity measure, extracting the corresponding at least one variable and at least one fixed part of each text message of the plurality of text messages as key/value pairs using the respective assigned at least one template based on the plurality of log entries, and providing the text messages, keys and values as triples.

Claims

1. A computer-implemented method for generating triples from log entries, the method comprising: providing a plurality of log entries from respective log files, wherein each log entry of the plurality of log entries comprises at least one text message; generating at least one template based on the plurality of log entries using unsupervised clustering, wherein the at least one template comprises at least one variable part and at least one fixed part; assigning each log entry of the plurality of log entries to one respective template based on the generated at least one template using a similarity measure; extracting the corresponding at least one variable and at least one fixed part of each text message of the plurality of text messages as key/value pairs using the respective assigned at least one template based on the plurality of log entries; and providing text messages, keys and values as triples.

2. The method according to claim 1, wherein the triples are an input data set for log mining or any other further analysis.

3. The method according to claim 2, further comprises loading the input data set into a knowledge graph.

4. A computer program product, comprising a computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor or a computer system to implement a method according to claim 1 when the computer program product is running on a computer.

5. A technical system for generating triples from log entries, the technical system comprising: a receiving unit for providing a plurality of log entries from respective log files, wherein each log entry of the plurality of log entries comprises at least one text message; a clustering unit for: generating at least one template based on the plurality of log entries using unsupervised clustering, wherein the at least one template comprises at least one variable part and at least one fixed part, and assigning each log entry of the plurality of log entries to one respective template based on the generated at least one template using a similarity measure; a triple extracting unit for extracting the corresponding at least one variable and at least one fixed part of each text message of the plurality of text messages as key/value pairs using the respective assigned at least one template based on the plurality of log entries; and a transmitting unit for providing the text messages, keys and values as triples.

Description

BRIEF DESCRIPTION

[0045] Some of the embodiments will be described in detail, with reference to the following figures, wherein like designations denote like members, wherein:

[0046] FIG. 1 illustrates a flowchart of the method according to an embodiment of the invention; and

[0047] FIG. 2 illustrates a technical system according to an embodiment of the invention.

DETAILED DESCRIPTION

[0048] FIG. 1 illustrates a flowchart of the method according to embodiments of the invention. The method for generating triples, comprises the following steps S1 to S5: [0049] a. In a first step a plurality of log entries 12 is provided. The log entries are derived from respective log files S1. Each log entry comprises one or more text messages. The first step can be performed by a receiving unit 10. [0050] b. In a second step at least one template is generated based on the plurality of log entries using unsupervised clustering S2. Each template comprises at least one variable part and at least one fixed part. The second step can be performed by a clustering unit. The clustering unit can be equally referred to extracting unit 20. [0051] c. In a third step each log entry of the plurality of log entries is assigned to one respective template based on the generated at least one template using a similarity measure S3. The third step can be performed by the same clustering unit or another clustering unit 20. [0052] d. In a further step the corresponding at least one variable and at least one fixed part of each text message of the plurality of text messages are extracted as key/value pairs using the respective assigned at least one template based on the plurality of log entries 12. This step can be performed by a triple extracting unit 30. [0053] e. In a last step the text messages, keys and values are provided as triples 42. This last step can be performed by a transmitting or sending unit 40. The aforementioned units of the technical system can be also referred to as modules.

[0054] FIG. 2 illustrates an exemplary technical system 1 with the aforementioned units 10 to 40. The resulting triples are loaded into a knowledge graph according to this embodiment.

[0055] Clustering According to Steps S2 and S3

[0056] The step S2, the templates are generated based on different log messages with usage of unsupervised clustering. Each template consists of fixed and variable parts. Exemplary templates are listed in the following: [0057] Log message: “Software McAfee tries to reach IP address 139.136.55.1” [0058] Template: “Software <*> tries to reach IP address <*>” [0059] Log message: “Software McAfee Solidifier detected deletion of C:\temp\test.vbs” [0060] Template: “Software <*> detected deletion of <*>”

[0061] In step S3, similar text messages are classified as the same template, e.g. similar in the manner that the fixed parts are the same, variable parts are not the same, but have the same structure and/or length.

[0062] Exemplary templates and a list of log entries with their corresponding templates are listed in the following: [0063] Template: “Software <*> tries to reach IP address <*>” [0064] Log messages: [0065] “Software McAfee tries to reach IP address 139.136.55.1” [0066] “Software Symantec Endpoint Protection tries to reach IP address 138.136.55.10” [0067] Template: “Software <*> detected deletion of <*>” [0068] Log messages: [0069] “Software McAfee Solidifier detected deletion of C:\temp\test.vbs” [0070] “Software ACME Detector detected deletion of C:\temp\test2.vbs”

[0071] Triple Extracting According to Step S4

[0072] In step S4, the varying and the fixed parts of the different text messages are extracted according to the structure of the templates. For every varying part the fixed part before is extracted and listed as key/value pairs.

[0073] Exemplary key/value pairs after said extraction are listed in the following: [0074] Key: “Software”, Value: “McAfee” [0075] Key: “tries to reach IP address”, Value: “139.136.55.1” [0076] Key: “Software”, Value: “Symantec Endpoint protection” [0077] Key: “tries to reach IP address”, Value: “138.136.55.10” [0078] Key: “Software”, Value: “McAfee Solidifier” [0079] Key: “detected deletion of”, Value: “C:\temp\test.vbs” [0080] Key: “Software”, Value: “ACME Detector” [0081] Key: “detected deletion of”, Value: “C:\temp\test2.vbs”

[0082] Further, the key/value pairs of fixed and varying text message parts are connected to entities of the text message itself for the graph creation as exemplary use case.

[0083] Exemplary connections are listed in the following: [0084] Create a connection “Software” from this message to the entity “McAfee” [0085] Create a connection “triesToReachIPaddress” from this message to the entity “139.136.55.1”, which could relate to a device [0086] Create a connection “Software” from this message to the entity “McAfee Solidifier” [0087] Create a connection “detectedDeletionOf” from this message to the entity “C:\temp\test.vbs”

[0088] Post-processing steps can be added after the generation of the triples before they are loaded into the knowledge graph, such as entity reconciliation e.g. nearly similar path names should be treated as the same entity.

[0089] Although the present invention has been disclosed in the form of preferred embodiments and variations thereon, it will be understood that numerous additional modifications and variations could be made thereto without departing from the scope of the invention.

[0090] For the sake of clarity, it is to be understood that the use of “a” or “an” throughout this application does not exclude a plurality, and “comprising” does not exclude other steps or elements.