Systems and methods for providing secure evolution as a service

Abstract

Described herein is a process which facilitates segmented security between domain-specific data sets being evaluated as part of a candidate evaluation service and third-party evolution services, wherein the data sets are not transmitted to the evolution service which is evolving candidates for evaluation. This enables customers with secure data sets to use candidate evolution services securely by obtaining a population of potentially optimal candidate models to evaluate and then optimizing on those data sets in their own secure fashion.

Claims

1. A process for evolving candidate individuals for optimization against a secure third-party data set comprising: receiving at a first server of a receiving party a first secure request for evolution of a first population of candidate individuals in accordance with a set of domain factors established by a requesting party, wherein the set of domain factors includes one or more of: domain constraints, known domain parameters and formatting rules for a specific representation of each of the candidate individuals and the first secure request includes a first checkpoint key; creating by the receiving party a first population of candidate individuals and assigning a unique candidate identifier to each of the candidate individuals in the first population; transmitting a first secure response, including the first population of candidate individuals with assigned candidate identifiers, to a second server of the requesting party, wherein the first server and the second server are separate by a firewall; receiving at the first server, a second secure request for evolution of a second population of candidate individuals, where the second secure request includes the first checkpoint key and results of evaluation by the second server of one or more of the candidate individuals from the first population against the secure third-party data set; creating by the receiving party the second population of candidate individuals and assigning a unique candidate identifier to each of the candidate individuals in the second population; and transmitting a second secure response, including the second population of candidate individuals with assigned candidate identifiers, to the second server of the requesting party, wherein the second secure response includes a second checkpoint key.

2. The process according to claim 1, further comprising: translating by a translator of the receiving party each of the candidate individuals from a first format to a second format in accordance with the formatting rules.

3. The process according to claim 2, wherein the first format is a coded genome format.

4. The process according to claim 2, wherein the second format is in JSON code.

5. The process according to claim 1, further comprising: translating by the translator of the receiving party each of the second candidate individuals from a first format to a second format in accordance with the formatting rules.

6. The process according to claim 5, wherein the first format is a coded genome format.

7. The process according to claim 5, wherein the second format is in JSON code.

8. The process according to claim 1, wherein the candidate individuals are neural networks.

9. The process according to claim 1, wherein the first secure request and the first secure response are encrypted.

10. A process for evolving candidate individuals for optimization against a secure data set comprising: transmitting a first secure request from a first server for evolution of a first population of candidate individuals in accordance with a set of domain factors to a second server, wherein the first secure response further includes a first checkpoint key; receiving a first secure response, including the first population of candidate individuals with assigned candidate identifiers, at the first server, wherein the first server and the second server are separate by a firewall; evaluating one or more of the candidates individuals against the secure data set to determine measurements indicative of a fitness of each of the candidate individuals for a predetermined use; transmitting by the first server, a second secure request for evolution of a second population of candidate individuals, where the second secure request includes the first checkpoint key and results of the evaluation by the first server of the one or more candidate individuals from the first population against the secure data set; and receiving a second secure response, including the second population of candidate individuals with assigned candidate identifiers, at the first server, wherein the second secure response includes a second checkpoint key.

11. The process according to claim 10, further comprising: translating by a translator of the first server each of the first candidate individuals from a first format to a second format in accordance with the formatting rules.

12. The process according to claim 11, wherein the first format is a coded genome format.

13. The process according to claim 11, wherein the second format is in JSON code.

14. The process according to claim 10, further comprising: translating by the translator of the first server each of the second candidate individuals from a first format to a second format in accordance with the formatting rules.

15. The process according to claim 14, wherein the first format is a coded genome format.

16. The process according to claim 14, wherein the second format is in JSON code.

17. The process according to claim 10, wherein the first secure request and the second secure request are encrypted.

Description

BRIEF DESCRIPTION OF THE FIGURES

(1) The invention will be described with respect to specific embodiments thereof, and reference will be made to the drawings, in which:

(2) FIG. 1 is a high level diagram of an EaaS embodiment described herein.

(3) FIG. 2 illustrates an exemplary modules configuration which can be used to implement evolution services in accordance with one or more embodiments herein;

(4) FIG. 3 illustrates an exemplary first candidate format in accordance with one or more embodiments herein;

(5) FIG. 4 illustrates an exemplary second candidate format in accordance with one or more embodiments herein; and

(6) FIG. 5 illustrates and exemplary messaging flow in accordance with one or more embodiments herein.

DETAILED DESCRIPTION

(7) The embodiments disclosed allow for segmented security between domain-specific data sets being evaluated as part of a candidate evaluation service, wherein the data sets are not transmitted to the evolution service which is evolving candidates for evaluation. This enables customers with secure data sets to use candidate evolution services securely by obtaining a population of potentially optimal candidate models to evaluate and then optimizing on those data sets in their own secure fashion.

(8) The embodiments herein also allows data and code details of the Evolution Service implementation to remain secure from entities using the service, thus protecting data and intellectual property of the service provider.

(9) Evolution as a Service (EaaS)

(10) EaaS includes two primary components or subsystems/processes: an Evolution Service and a Candidate Evaluation System. FIG. 1 illustrates an exemplary schematic of EaaS 10 components/processes including representative inputs and data flows. The Evolution Service 15 communicates with the Candidate Evaluation System 20 (also called Experiment Host) across a firewall 17. Inputs to the Candidate Evaluation System 20 may include experiment framework code (e.g., Python) 25, experiment configuration (e.g., JSON/YAML) 30 and domain-specific evaluation code 35 (e.g., evaluate_candidate( )). As indicated in FIG. 1, the Candidate Evaluation System 20 may be hosted by a third-party, e.g., the EaaS service provider, or the customer. In an embodiment, the subsystems 15 and 20 run on physically distinct hosts, each within their own secure environment separated by the firewall 17. The firewall 17 is symbolic since it is created by software aspects of the embodiments described herein. Additionally, conventional hardware firewalls may be present as well as will be appreciated by one skilled in the art.

(11) The EaaS sub systems 15 and 20 communicate with each other via encrypted connection using standard network traffic. There can be one or more intermediary devices such as a content delivery network (CON) positioned between the Evolution Service 15 and the Candidate Evaluation System 20. In one embodiment, the CON is positioned on the same side of the firewall 17 as the Candidate Evaluation System 20. In another embodiment, the Candidate Evaluation System 20 is positioned on the same side as the Evolution Service 15. In yet another embodiment, parts of the CON are positioned on both sides of the firewall 17.

(12) Further to FIG. 1, customers may use candidate model management framework services such as those supported by Studio.ml, in order to manage, track, reproduce, and share the candidate experiments. Studio.ml uses publicly available cloud computing resources, e.g, Amazon EC2, Google Cloud Computer, Microsoft Azure to implement Studio.ml evaluation worker instances 40. Input 36 to the Studio.ml worker instances 40 includes, e.g., an evaluation worker request with single candidate, evaluation configuration, evaluation code and Python dependencies. Output 42 from the Studio.ml worker instances 40 includes single candidate with metrics.

(13) Evolution Service

(14) The Evolution Service 15 is responsible for:

(15) a) Accepting configuration information regarding the constraints of Evolution from the Candidate Evaluation System 20.

(16) b) Creating new populations of candidates of possible optimizations from no prior candidates.

(17) c) Creating new populations of candidates from previous populations of priors, based on fitness data for each prior candidate.

(18) d) Securely reading/writing checkpoints of hidden representation representing evolution state, so that such state can be resumed at any point in the future, only by the Evolution Service 15. Such state can be associated with an insecure key which is shared with the Candidate Evaluation System 20.

(19) e) Providing translations/interpretations of any new candidates generated by the Evolution Service 15 in a representation such that the Candidate Evaluation System 20 knows what to do with the candidates once it gets them.

(20) f) Assignment of unique identifiers for each of the candidates (“candidate ID”).

(21) Candidate Evaluation System

(22) The Candidate Evaluation System 20 is responsible for:

(23) a) Initiating requests from the Evolution Service 15 (with or without results from prior candidates, configuration updates, insecure checkpoint keys, etc.)

(24) b) Evaluating candidates against the secure data set (by a mechanism of its own choosing) such that enough measurements about the candidates can be taken to inform the creation of the next population.

(25) In the example below the Candidate Evaluation System 20 and the Evolution Service 15 are intended to be running on two distinct hosts, each within their own secure environments. Communication is limited to standard network traffic between the two, over a (potentially encrypted) socket connection. Preferably, the two hosts are physically distinct, but in an alternative embodiment they may be different virtual machines sharing a common physical computer platform.

(26) First, the customer uses the Candidate Evaluation System 20 to initiate contact with Evolution Service 15 by communicating configuration information regarding constraints of the search space for genetic material, variations and/or known parameters on algorithm, and selection of the representation by which the Candidate Evaluation System wishes to receive candidates, etc. The Evolution Service 15 accepts configuration and creates a new candidate population either originally or based on prior candidates (if any) and new algorithm configuration (if any). Each member candidate of the candidate population is assigned a specific candidate identifier (“candidate ID”), unique (at least) amongst other candidates in the present experiment. Internal representation of the population is put through a selected translator which translates each instance of candidate genetic material for each candidate in the population into a candidate representation known to the Candidate Evaluation System 20, each associated with its original candidate ID. A checkpoint key, unique to the present experiment and population, along with the translated candidate representations and their associated candidate IDs are communicated back to the Candidate Evaluation System 20.

(27) The Candidate Evaluation System 20 receives checkpoint key and corresponding population and evaluates each candidate of the population against its secure data set, in whatever secure environment is required (if any). For each candidate evaluation, the Candidate Evaluation System 20 records measurements of performance against the secure data set. The secure data set may be static or dynamic, such as where candidates are tested online against actual users. When all evaluation is complete (as determined by the domain-specific aspects of the Candidate Evaluation System 20), evaluation results each associated with their candidate ID's are potentially reported back to the Evolution Service 15 with the previous checkpoint key. The Evolution Service 15 repeats the process starting with creating a new candidate population as describe above, unless some experiment-specific termination criteria is reached.

(28) FIG. 2 illustrates representative modules that can be used to implement the Evolution Service 15 and Candidate Evaluation System 20. In FIG. 2, dotted lines indicate optional modules. The modules in FIG. 2 can be implemented in hardware or software, and need not be divided up in precisely the same blocks as shown in FIG. 2. Some of the modules can also be implemented on different processors or computers, or spread among a number of different processors or computers. In addition, it will be appreciated that some of the modules can be combined, operated in parallel or in a different sequence than that shown in FIG. 2 without affecting the functions achieved. Also as used herein, the term “module” can include “sub-modules”, which themselves can be considered herein to constitute modules. In particular, candidate testing module 238, competition module 251, and procreation module 253 are also considered herein to be sub-modules of a candidate processing module 270. Note that candidate testing module 238 is part of Candidate Evaluation System 20 running on a separate host from the Evolution Service 15 as described above. The blocks in FIG. 2, designated as modules, can also be thought of as flowchart steps in a method. A module also need not necessarily have all its code disposed contiguously in memory; some parts of the code can be separated from other parts of the code with code from other modules or other functions disposed in between.

(29) Referring to FIG. 2, at the Evolution Service 15 after accepting configuration information regarding the constraints of Evolution from the Candidate Evaluation System 20, the candidate individual population 232 is initialized by a population initialization module 202, which writes a preliminary pool of candidate individuals in the population 102. The preliminary pool can be created randomly, or in some embodiments, a priori knowledge can be used to seed the first generation. In another embodiment, candidate individuals from prior runs can be borrowed to seed a new run. At the start, all candidate individuals are initialized with performance measures 106 that are indicated as undefined. The population initialization module 202 also defines a candidate search space. As used herein, the term “candidate search space” refers to a space having M dimensions, where each dimension represents an axis along which different candidate individuals can have different values. The size of the space in each dimension is equal to the range of values available for that dimension. In one implementation, the operation of each of the modules shown in FIG. 2 is described in the U.S. Nonprovisional application Ser. No. 15/399,450 published as U.S. Patent Publication No. 20170192638 incorporated by reference herein.

(30) Let us consider application of the EaaS to an e-commerce example. In e-commerce, designing user experiences, i.e., webpages and interactions, which convert as many users as possible from casual browsers to paying customers is an important goal. While there are some well-known design principles, including simplicity and consistency, there are also often unexpected interactions between elements of the webpage that determine how well it converts. The same element may work well in one context but not in others. It is often difficult to predict the result, and even more difficult to decide how to improve a given webpage. A website host running a Candidate Evaluation System 20 may employ a website modification Evolution Service 15 as described herein to provide a presentation of its webpages that maximizes conversion.

(31) In a first embodiment, each candidate individual in the population generated by the Evolution Service 15 is in a “coded” genome form as shown in FIG. 3. For example, if the purpose is to evolve better web funnels, the coded genome form may be like FIG. 3 and the useful form can be as shown in FIG. 4. As described herein, the Evolution Service 15 and the Candidate Evaluation System 20 are separated from each other by the firewall 17. The Evolution Service 15 transmits the candidate individual 310 in an encrypted form as shown in FIG. 3 through the firewall 17 to the Candidate Evaluation System 20. The Candidate Evaluation System 20 then converts the candidate from the coded genome form (FIG. 3) to the useful form prior to experimentation and evaluation with secure data.

(32) By way of particular example, continuing with the example of webpage evaluation, consider a webpage has four elements: logo, main headline, sub headline, and action button. Each element has corresponding dimensions. For example, logo has two dimensions: logo text and logo formatting. Dimensions have corresponding rendering values for example, logo text has two rendering values: control value and value 1. As shown in FIG. 3, an individual encodes the rendering values in a binary sequence called a genome, which is further compartmentalized into subsequences called genes.

(33) FIG. 3 illustrates that each individual can represent a dimension of an element of a webpage and encode rendering values corresponding to the dimension. For each gene, only one of the bits can be active or hot (e.g., represented by 1) so as to identify a particular rendering value from a set of available rendering values. The leftmost bit can represent the control value of an individual, which, in one example, for main headline might indicate plain text. The second, third, and forth bits might, for example, indicate bold text, italics text, and bold underlined text, respectively. It will be appreciated that many other encodings are possible. For example, a two-bit numeric value might be used, with 0 indicating plain text, 1 indicating bold text, 2 indicating italics text, and 3 indicating bold underlined text.

(34) In a second embodiment, the candidate individual is sent through the firewall in a “useful” form as shown in FIG. 4. Prior to transmission, the Evolution Service 15 converts the candidate individual from coded genome form (e.g., FIG. 3) to a form useful for its purposes as identified in the initial request to the Evolution Service 15 from the Candidate Evaluation System 20. As described above in the first embodiment, the individual is transmitted through the firewall 134 in an encrypted form. This is one example of the individual of FIG. 3 in a usable JavaScript Object Notation (JSON) format 300. In JSON code 300, the additional scripts 216 are identified as “scriptUrls”, which further contains links to five scripts. These scripts, when executed by the client/customer, generate the updated interface version of the portion of the webpage, as discussed above.

(35) FIG. 5 is a message sequence diagram illustrating the communication between the Evolution Service 15 and Candidate Evaluation System 20. As indicated above, the two systems 15 and 20 are separated by a firewall 17 (not shown in FIG. 5) and all communication between the two systems is encrypted. The Candidate Evaluation System 20 establishes communications with the Evolution Service 15 in a message 1 (M1) by sending configuration information including, e.g., constraints, known parameters and usable representation of evolved candidates, to the Evolution Service 15. The Evolution Service 15 creates new population of individuals and new algorithm configurations. The Evolution Service 15 assigns unique candidate identifiers (candidate IDs) to each candidate individual in the created population. The candidate IDs are unique at least amongst individuals in the same experiment (message M2). In one embodiment, the Evolution Service 15 uses a translator to translated each instance of candidate genetic material in a first format into a second format representation known to the Candidate Evaluation System 20. Alternatively, as discussed above, this translation may be completed back at the Candidate Evaluation System 20.

(36) The translated representations of the candidates in the second format, along with a checkpoint key, are sent to the Candidate Evaluation System 20 via a message M3. The Candidate Evaluation System 20, evaluates each individual against its data set in a secure environment (message M4). The above process is repeated via a message M5 until an experiment specific criteria is reached (message M6).

(37) The technology disclosed can use any specific representation of evolved material and keep the secure data and implementation properties as described above. In one such implementation, the technology disclosed is used to generate candidate Neural Networks via evolution.

(38) In one example implementation, a similar service can be used to evolve anything from website GUI's, to motions of robots, shapes and properties of objects intended to be made physical at some later date.

(39) Aspects of the invention can also apply to other population-based algorithms and population-based machine learning algorithms beyond evolution as well.

(40) Beyond the description of the technology disclosed above, also incorporated are the following patent applications which are considered part of this disclosure. The examples presented in the following incorporated applications and research publications exemplify situations in which aspects of the invention can be used. These following documents are incorporated by reference herein in their entireties: U.S. Nonprovisional application Ser. No. 15/399,450 filed on Jan. 5, 2017, titled “Machine Learning Based Webinterface Production and Deployment System;” U.S. Nonprovisional application Ser. No. 15/399,523 filed on Jan. 5, 2017, titled “Webinterface Production and Deployment Using Artificial Neural Networks;” Golovin, et. al., (2017), “Google Vizier: A Service for Black-Box Optimization,” Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1487-1495; Liang, et. al., (2018), “Evolutionary Architecture Search for Deep Multitask Networks,” arXiv: 1803.03745; Meyerson, et. al., (2018), “Pseudo-task Augmentation: From Deep Multitask Learning to lntrastask Sharing and Back,” arXiv: 1803.04062; Rawal, et. al., (2018), “From Nodes to Networks: Evolving Recurrent Neural Networks,” arXiv: 1803.04439; Zhang, et al., (2011), “Evolutionary Computation Meets Machine Learning: A Survey,” IEEE Computational Intelligence Magazine, Vol. 6, No. 4, DOI 10.1109/MCI.2011.942584.