Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes repeatedly selecting, by a control system for the environment, control settings for the environment based on internal parameters of the control system, wherein: at least some of the control settings for the environment are selected based on a causal model, and the internal parameters include a first set of internal parameters that define a number of previously received performance metric values that are used to generate the causal model for a particular controllable element; obtaining, for each selected control setting, a performance metric value; determining that generating the causal model for the at particular controllable element would result in higher system performance; and adjusting, based on the determining, the first set of internal parameters.

A method of optimizing a plurality of control signals used in operating a vehicle is described. The operation has a plurality of associated measurable parameters. The method includes: for each control signal, selecting a plurality of potential optimum values from a predetermined set; operating the vehicle in at least a first sequence of operation iterations, where for each pair of sequential first and second operation iterations in the first sequence of operation iterations, the potential optimum value of one control signal in the first operation iteration is replaced in the second operation iteration with a next potential optimum value of the control signal, while the potential optimum values of the remaining control signals are maintained; for each operation iteration, measuring each parameter in the plurality of measurable parameters; and generating confidence intervals for the control signals to determine causal relationships between the control signals and the measurable parameters.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes identifying a procedural instance; selecting control settings for the procedural instance, comprising, for a particular one of the controllable elements: assigning the procedural instance to a cluster for the particular controllable element in accordance with current values of a set of clustering parameters for the particular controllable element; and selecting a setting for the particular controllable element for the procedural instances based on a causal model that is specific to the cluster; obtaining environment responses to the selected control settings that define a value of the performance metric for the procedural instance; and updating, for the particular controllable element, the causal model for the cluster for the controllable element to which the procedural instance was assigned based on the value of the performance metric.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes repeatedly selecting control settings for the environment based on (i) a causal model that identifies causal relationships between possible settings for controllable elements in the environment and environment responses that reflect a performance of the control system in controlling the environment and (ii) current values of a set of internal parameters; and during the repeatedly selecting: monitoring environment responses to the selected control settings; determining, based on the environment responses, an indication that one or more properties of the environment have changed; and in response, modifying the current values of one or more of the internal parameters.

A method of performing a process using a plurality of control signals and resulting in a plurality of measurable outcomes is described. The method includes optimizing the plurality of control signals by at least: receiving a plurality of process constraints; receiving, for each measurable outcome, an optimum range; receiving, for each control signal, a plurality of potential optimum values; iteratively performing the process, where for each process iteration, the value of each control signal is selected from among the plurality of potential optimum values received for the control signal; for each process iteration, measuring each outcome in the plurality of measurable outcomes; and generating confidence intervals for the control signals to determine a causal relationship between the control signals and the measurable outcomes. The method includes performing the process using at least the control signals determined by the causal relationship to causally affect at least one of the measurable outcomes.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for optimizing a manufacturing process. In one aspect, the method comprises repeatedly performing the following: i) selecting a configuration of control settings for a manufacturing process, based on a causal model that measures causal relationships between control settings and a measure of a success of the manufacturing process; ii) determining the measure of the success of the manufacturing process using the configuration of control settings; and iii) adjusting, based on the measure of the success of the manufacturing process using the configuration of control settings, the causal model.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes identifying a procedural instance; determining a temporal extent for the procedural instance based on temporal extent parameters for the one or more entities in the procedural instance; selecting control settings for the procedural instance; monitoring environment responses to the control settings that are received for the one or more entities; determining which of the environment responses to attribute to the procedural instance in a causal model; and adjusting, based at least in part on the environment responses that are attributed to the procedural instance, the temporal extent parameters for the one or more entities.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes repeatedly selecting, by a control system for the environment, control settings for the environment based on internal parameters of the control system, wherein: at least some of the control settings for the environment are selected based on a causal model, and the internal parameters include a first set of internal parameters that define a number of previously received performance metric values that are used to generate the causal model for a particular controllable element; obtaining, for each selected control setting, a performance metric value; determining that generating the causal model for the at particular controllable element would result in higher system performance; and adjusting, based on the determining, the first set of internal parameters.

A method of performing a process using a plurality of control signals and resulting in a plurality of measurable outcomes is described. The method includes optimizing the plurality of control signals by at least: receiving a plurality of process constraints; receiving, for each measurable outcome, an optimum range; receiving, for each control signal, a plurality of potential optimum values; iteratively performing the process, where for each process iteration, the value of each control signal is selected from among the plurality of potential optimum values received for the control signal; for each process iteration, measuring each outcome in the plurality of measurable outcomes; and generating confidence intervals for the control signals to determine a causal relationship between the control signals and the measurable outcomes. The method includes performing the process using at least the control signals determined by the causal relationship to causally affect at least one of the measurable outcomes.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes identifying a procedural instance; selecting control settings for the procedural instance, comprising, for a particular one of the controllable elements: assigning the procedural instance to a cluster for the particular controllable element in accordance with current values of a set of clustering parameters for the particular controllable element; and selecting a setting for the particular controllable element for the procedural instances based on a causal model that is specific to the cluster; obtaining environment responses to the selected control settings that define a value of the performance metric for the procedural instance; and updating, for the particular controllable element, the causal model for the cluster for the controllable element to which the procedural instance was assigned based on the value of the performance metric.