An electronic device acquires, from program code, two or more program code loops having specified data dependencies. The electronic device places each of the program code loops into a corresponding blocking loop, each blocking loop including at least one blocking loop induction variable that is incremented by a corresponding block size and used to specify a number of iterations for at least one internal loop induction variable of the respective program code loop. The electronic device fuses the blocking loops into a fused loop by placing all of the blocking loops in the fused loop and replacing the blocking loop induction variables of the blocking loops with a fused loop induction variable that is incremented by the corresponding block size and used to specify the number of iterations for respective internal loop induction variables in the blocking loops.

In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can provide a service to recommend actions and transformations, on an input data, based on patterns identified from the functional decomposition of a data flow for a software application, including determining possible transformations of the data flow in subsequent applications. Data flows can be decomposed into a model describing transformations of data, predicates, and business rules applied to the data, and attributes used in the data flows.

In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system provides a programmatic interface, referred to herein in some embodiments as a foreign function interface, by which a user or third-party can define a service, functional and business types, semantic actions, and patterns or predefined complex data flows based on functional and business types, in a declarative manner, to extend the functionality of the system.

The invention comprises (i) a compilation method for automatically converting a single-threaded software program into an application-specific supercomputer, and (ii) the supercomputer system structure generated as a result of applying this method. The compilation method comprises: (a) Converting an arbitrary code fragment from the application into customized hardware whose execution is functionally equivalent to the software execution of the code fragment; and (b) Generating interfaces on the hardware and software parts of the application, which (i) Perform a software-to-hardware program state transfer at the entries of the code fragment; (ii) Perform a hardware-to-software program state transfer at the exits of the code fragment; and (iii) Maintain memory coherence between the software and hardware memories. If the resulting hardware design is large, it is divided into partitions such that each partition can fit into a single chip. Then, a single union chip is created which can realize any of the partitions.

A program conversion device includes converting a program including second loop processing that repeats first processing and determination processing a plurality of times to a program including third loop processing and fourth loop processing; converting the converted program to a first-number-of-times repeating processing that includes second-number-of-times repeating processing of the first loop processing and the second-number-of-times repeating processing of the determination processing; converting the first processing and the determination processing to processing that accesses to memory areas that are different for fourth loop processing and that are continuous in a processing order in the fourth loop processing; and exchanging a processing order of the fourth loop processing and the first loop processing, wherein the third loop processing is the-first-number-of-times repeating processing related to the second loop processing, and the fourth loop processing is the-second-number-of-times repeating processing related to the second loop processing in the third loop processing.

A method may include receiving, from a client, a request to customize an operator node corresponding to a data processing operation. The request may include a first key. The operator node may be selected for inclusion in a graph representative of a data processing pipeline. The operator node may be associated with a first file that includes at least one configuration parameter associated with the operator node. The at least one configuration parameter may be associated with a second key. In response to the first key being determined to match the second key, the operator node may be customized by modifying the at least one configuration parameter. Furthermore, a second file associated with a customized operator node may be generated to store the customizations made to the operator node including the modification of the at least one configuration parameter. Related systems and articles of manufacture are also provided.

Systems, apparatuses and methods may provide for technology that detects one or more local variables in source code, wherein the local variable(s) lack dependencies across iterations of a loop in the source code, automatically generate pipeline execution code for the local variable(s), and incorporate the pipeline execution code into an output of a compiler. In one example, the pipeline execution code includes an initialization of a pool of buffer storage for the local variable(s).

In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can include a software development component and graphical user interface, referred to herein in some embodiments as a pipeline editor, or Lambda Studio IDE, that provides a visual environment for use with the system, including providing real-time recommendations for performing semantic actions on data accessed from an input HUB, based on an understanding of the meaning or semantics associated with the data.

In accordance with various embodiments, described herein is a system (Data Artificial Intelligence system, Data AI system), for use with a data integration or other computing environment, that leverages machine learning (ML, DataFlow Machine Learning, DFML), for use in managing a flow of data (dataflow, DF), and building complex dataflow software applications (dataflow applications, pipelines). In accordance with an embodiment, the system can perform an ontology analysis of a schema definition, to determine the types of data, and datasets or entities, associated with that schema; and generate, or update, a model from a reference schema that includes an ontology defined based on relationships between datasets or entities, and their attributes. A reference HUB including one or more schemas can be used to analyze data flows, and further classify or make recommendations such as, for example, transformations enrichments, filtering, or cross-entity data fusion of an input data.

A multi-threaded programming language and compiler generates synchronous digital circuits that maintain thread execution order by generating pipelines with code paths that have the same number of stages. The compiler balances related code paths within a pipeline by adding additional stages to a code path that has fewer stages. Programming constructs that, by design, allow thread execution to be re-ordered, may be placed in a reorder block construct that releases threads in the order they entered the programming construct. First-in-first-out (FIFO) queues pass local variables between pipelines. Local variables are popped from FIFOs in the order they were pushed, preserving thread execution order across pipelines.