Intelligent control with hierarchical stacked neural networks
11514305 · 2022-11-29
Inventors
Cpc classification
Y10S128/925
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
G01C21/3602
PHYSICS
F03D7/046
MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
B60W30/00
PERFORMING OPERATIONS; TRANSPORTING
International classification
Abstract
A neural network method, comprising: modeling an environment; implementing a policy based on the modeled environment, to perform an action by an agent within the environment, having at least one estimated dynamic parameter; receiving an observation and a temporally-associated cost or reward based on operation of the agent in the environment controlled according to the policy; and updating the policy, dependent on the received observation and the temporally-associated cost or reward, to improve the policy to optimize an expected future cumulative cost or reward. The policy may represent a set of parameters defining an artificial neural network having a plurality of hierarchical layers and having at least one layer which receives inputs representing aspects of the received observation indirectly from other neurons, and produce outputs to other neurons which indirectly implement the policy, the plurality of hierarchical layers being trained according to respectfully distinct training criteria.
Claims
1. A neural network method, comprising: estimating at least one dynamically changing state parameter of an environment of an agent having a set of environment states modified by operation of the agent; modelling the environment states of the agent as a finite state Markov decision process, with at least one automated processor, comprising the estimated dynamically changing parameter; implementing a policy for operation of the agent within the environment based on the set of environment states with the at least one automated processor, and producing an output of the at least one automated processor to control performance of an action by the agent, the policy being defined as a conditional distribution over a set of possible actions given an observation of a state by the agent operating within the environment, the policy representing a set of parameters stored in a memory and defining an artificial neural network having a plurality of hierarchical layers and having at least one layer which receives inputs derived from the received observation, and produces outputs which influence the action, the plurality of hierarchical layers being trained according to respectively distinct training criteria; performing a classification analysis, with the at least one automated processor, to produce a noise vector with respect to the policy, the noise vector selectively representing information within the received observation which fails to specifically excite or suppress the outputs of the artificial neural network; receiving an observation of the state and a temporally-associated cost or reward resulting from operation of the agent in the environment controlled according to the policy; and updating the policy, with the at least one automated processor, dependent on the received observation and the temporally-associated cost or reward, to improve the policy to optimize an expected future cumulative cost or reward.
2. The neural network method according to claim 1, further comprising producing, by at least one layer of the artificial neural network, the noise vector with respect to the policy, representing at least information within the received observation that includes information not reflected in the updated policy and excludes information that is reflected in the updated policy.
3. The neural network method according to claim 1, further comprising processing, with the at least one automated processor, the noise vector within at least one artificial neural network layer hierarchically higher than an artificial neural network layer in which the noise vector is generated.
4. The neural network method according to claim 1, wherein the noise vector comprises information representing an observation to which the policy is insensitive, further comprising analyzing the noise vector according to a distinct analysis from the update of the policy.
5. The neural network method according to claim 1, wherein the classification analysis comprises statistical clustering.
6. The neural network method according to claim 1, wherein the noise vector is based on K-means clustering.
7. The neural network method according to claim 1, wherein the noise vector is based on fuzzy c-means clustering.
8. The neural network method according to claim 1, wherein the action comprises automatically piloting a vehicle.
9. The neural network method according to claim 1, wherein the action comprises a semantic task.
10. The neural network method according to claim 1, wherein at least two hierarchical artificial neural network layers are independently trained with different training data and according to different training criteria.
11. The neural network method according to claim 1, wherein at least one hierarchical artificial neural network layer required for the policy is not updated dependent on the received observation and the temporally-associated cost or reward.
12. A neural network method, comprising: modeling an environment of an agent as a finite state Markov decision process having a set of environment states S dependent on actions A of the agent, with at least one automated processor; implementing a policy π.sub.t for operation of the agent within the environment based on the set of environment states S and an estimate of at least one dynamic parameter of the environment, to perform at time t an action A.sub.t of a set of possible actions A(s.sub.t) by the agent, the policy π.sub.t being defined as a conditional distribution over the set of possible actions A(s.sub.t) given an observation of state s.sub.t∈S, by the agent operating within the environment, with probability distributions: instantaneous cost distribution P(r.sub.t|s.sub.t), observation distribution P(A(s.sub.t)|s.sub.t), and transition distribution P(s.sub.t+1|s.sub.t,A(s.sub.t)) receiving the observation of the state s.sub.t+1 and a temporally-associated cost or reward r.sub.t+1 resulting from operation of the agent in the environment controlled according to the policy π.sub.t; updating the policy π.sub.t with the at least one automated processor to produce updated policy π.sub.t+1, dependent on the received observation s.sub.t+1 and the temporally-associated cost or reward r.sub.t+1, to improve the policy π.sub.t to optimize an expected future cumulative cost or reward
13. The method according to claim 12, wherein at least one layer of the artificial neural network further produces the noise vector with respect to the policy π.sub.t, representing at least information within the received observation s.sub.t that includes information not reflected in the updated policy π.sub.t+1 and excludes information that is reflected in the updated policy π.sub.t+1.
14. The method according to claim 12, further comprising processing the noise vector within at least one artificial neural network layer hierarchically higher than an artificial neural network layer in which the noise vector is generated.
15. The method according to claim 12, further comprising analyzing the noise vector according to a distinct analysis from the updating of the policy π.sub.t to policy π.sub.t+1.
16. The method according to claim 12, further comprising performing a clustering analysis to produce the noise vector.
17. The method according to claim 12, further comprising independently training at least two hierarchical artificial neural network layers with different non-arbitrary organizations of actions.
18. The method according to claim 12, further comprising maintaining at least one hierarchical artificial neural network layer required for the policy π.sub.t, without updating, dependent on the received observation s.sub.t and the temporally-associated cost or reward r.sub.t.
19. A non-transitory computer readable medium storing therein instructions for causing a programmable automated processor to perform a method comprising: instructions for modeling an environment altered by actions of an agent as a finite state Markov decision process having a set of environment states, comprising estimating at least one dynamic parameter of the set of environment states; instructions for implementing a policy for operation of the agent within the environment based on the set of environment states, to perform an action by the agent within the environment, the policy being defined as a conditional distribution over actions given an observation of state, by the agent operating within the environment, the policy representing a set of parameters stored in a memory defining an artificial neural network having a plurality of hierarchical layers and having at least one layer of neurons which receives inputs representing the received observation of the state indirectly from other neurons, and produces outputs to neurons of another hierarchical layer which implement the action, the plurality of hierarchical layers being trained according to respectfully distinct training criteria; instructions for receiving an observation of the state and a temporally-associated cost or reward resulting from operation of the agent in the environment controlled according to the policy; instructions for updating the policy, dependent on the received observation and the temporally-associated cost or reward, to improve the policy to optimize an expected future cumulative cost or reward; and instructions for performing a classification analysis to produce a noise vector representing information within a respective set of inputs of at least one of the hierarchical layers which fails to specifically excite or suppress a respective output of the at least one of the hierarchical layers of the artificial neural network.
20. The computer readable medium according to claim 19, further comprising instructions for causing a programmable automated processor to process the noise vector within a hierarchical layer superior than a respective hierarchical layer in which the noise vector is generated.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
(15) Before constructing a hierarchical stacked neural network of the present invention, the builder must determine the order of hierarchical complexity of the tasks that the hierarchical stacked neural network will perform. The highest order of hierarchical complexity found from this determination dictates the number of neural networks that the hierarchical stacked neural network requires. In making this determination, the order of hierarchical complexity is measured by the number of recursions that the coordinating actions must perform on a set of primary elements. Actions at a higher order of hierarchical complexity:
(16) 1) are defined in terms of the lower-order actions;
(17) 2) organize and transform the lower-order actions;
(18) 3) produce nonarbitrary organizations of these lower-order actions that solve new, more hierarchically complex tasks.
(19) The following equation determines the order of hierarchical complexity of the task, i.e., the number of concatenation operations it contains. In a concatenation, a coordinating action is performed on two or more of the immediately preceding necessary lower-order actions. The products of this coordinating action then become the lower-order elements of the next higher-order coordinating action. The next higher-order coordinating action is performed again on the new, more hierarchically complex elements generated from the previous order. From this coordinating action one derives the products of the new, higher-order performance of the action, which become prerequisite elements of an action of a still higher order, and that higher-order action is performed again.
(20) The order, O, of hierarchical complexity of task T is denoted O(T), and defined as follows:
(21) (a) For a simple task E.sub.1 at the lowest order in the hierarchical complexity, O(E.sub.1)=1.
(22) (b) Otherwise, O(E)=O(E′)+1, where O(E′)=max(C(E.sub.1, C(E.sub.2, . . . C(E.sub.n))), for all E.sub.i=1 . . . n in E.
(23) In other words, the order of the next higher-order task is one order of hierarchical complexity more than the next lower-order task out of which it is built. If task E is built out of tasks of different orders of hierarchical complexity, then E′ has the maximum order of all of the tasks within it. Consider the example of distributivity, 3*(9+2)=(3*9)+(3*2)=27+6=33, where the numbers come from counting objects. The maximum order of the subtasks would be based on the “adding” and “multiplying” actions (order 7), not the “counting” actions (order 6) (See Table 1 below).
(24) The hierarchical complexity of any task in a task sequence may be determined through such task analysis. The hierarchical complexity of a task therefore refers to the number of concatenation operations it contains. An order-three task has three concatenation operations. A task of order three operates on the output of a task of order two, and a task of order two operates on the output of a task of order one (i.e., a simple task).
(25) Table 2 provides examples of 14 stages/orders of hierarchical complexity identified for human beings through research on human development. It uses in part mathematical operations to illustrate how tasks at a higher stage/order build on tasks at lower stages. Higher-order tasks are created from the preceding lower stage/order tasks by concatenation of lower-order tasks. Other examples, with fewer or more stages/orders of hierarchical complexity, are also available. In the example of spam filtering, there are, for example, five neural networks involved. The lowest level neural network N1 identifies text as text. The next highest neural network N2 separates text into letters. The next level N3 organizes text into strings of letters—words and/or sentences based on spacing and punctuation. Note that levels N1 through N3 are not necessary if the words are already parsed at the input. Level N4 distinguishes patterns of words, while level N5 derives concepts from the patterns of words.
(26) Referring to
(27) Within a neural network in the hierarchical stack, tasks may be combined in ways that do not produce a change in stage/order. For example, iterations may repeat lower stage/order tasks, e.g., repeatedly adding a sequence of numbers. A mixture may include combining simple addition and multiplication. A chain may include arbitrarily linking lower-stage/order tasks that are not conceptually related. In the case of a change in stage/order, a transformation of lower-stage/order operations may use a new construct to combine lower-order tasks that incorporate tasks at a lower stage/order but create a new way of structuring the set of relationships. One example is using matrix algebra so that lower-order operations, e.g., adding and multiplying, become part of the matrix algebra tasks within a multi-dimensional domain.
(28) Referring to
(29) Referring to
(30) The stacked hierarchical neural network of the present invention can be constructed only after the number of neural networks needed in the stack has been determined. The stage/order of the actions and tasks performed in a neural network in the hierarchical stack must equal the level of the neural network. Thus neural network N.sub.m performs tasks and actions at stage/order in.
(31) The actions and tasks in each successive neural network are a combination, reordering and transforming the tasks of the immediately preceding neural network in the hierarchical stack. At each level the neural network performs actions at a higher stage/order of human cognitive development. Thus a neural network at the lowest level may only process binary information, such as whether or not an event has occurred, where a neural network operating at a higher level in the hierarchical stack may perform actions in multiple dimensions. As in human cognitive development, the neural network at the higher level makes more complex discriminations and can perform more sophisticated tasks. For example, a neural network at level 6 can perform tasks such as counting objects, where a neural network at level 7 can perform tasks that involve simple mathematical operators. Thus level 7 has moved beyond counting to a system that creates counts through a new, higher-order set of tasks (See Table 1).
(32) Each neural network in the hierarchical stack can have one or more hidden layers of neurons and various interconnections. The number of hidden layers within a network and the interconnections between layers depend on the nature of the tasks that the neural network at that particular stage/order is performing. The number of tasks and types of actions that the neural network performs and the stimuli that it processes all affect the number of hidden layers, the number of neurons in a hidden layer, and the interconnections between layers. The stimuli processed by the neural network may be external, internal to the network, or previously conditioned stimuli. When the types of stimuli that the neural network processes increase, the number of hidden layers and/or the number of neurons within a layer increase.
(33) The algorithms a particular neural network in a hierarchical stack uses to assign connection weights between neurons also vary, depending on the nature of the problem that the neural network is solving and the input and internal stimuli that the neural network processes. Specific algorithms are not associated with particular stages/orders of neural networks in the hierarchical stack. For example, a type of algorithm that assigns connection weights in a neural network at stage/order m that names geometric objects may differ from an algorithm that assigns connection weights in a neural network at stage/order m that names people.
(34) Training is not specific to the stage/order of a neural network in a hierarchical stack. Rather it can vary based on the particular tasks being performed by a network in the hierarchical stack. Individual networks in the hierarchical stack may use unsupervised training, supervised training, or a combination of both to set initial weights. In the case of unsupervised training the neural network continues to learn, adapt, and alter its actions throughout the course of its operation. It can respond to new patterns not presented during the initial training and assignment of weights. This capacity allows a network to learn from new external stimuli in a manner similar to how learning takes place in the real world.
(35) A neural network in the hierarchical stack can train other neural networks that operate at the same order/stage of hierarchical complexity. In this training, the information from the neural network is transferred to an architecturally distinct unit that is analogous in structure to the original neural network. This type of training constitutes a transfer of learning from one neural network to another; the new neural network does not have to be independently trained, thereby saving time and resources.
(36) More formally stated, an intelligent control system of the present invention comprises up to O hierarchical stacked neural networks, N.sub.m, . . . , N.sub.m+(O−1), where m equals the lowest stage/order tasks performed by the hierarchical stacked neural network and O represent the highest stage/order tasks performed. The actions and tasks performed in N.sub.m correspond to the stage/order of cognitive development of the model of cognitive development employed in solving the problem. Thus N.sub.1 performs tasks at the level of hierarchical complexity that corresponds to stage/order 1, and N.sub.5 processes information and responds at the level of complexity that corresponds to stage/order 5. The number of neural networks in the stack depends on the order of hierarchical complexity of the tasks performed by the hierarchical stacked neural network. The number of networks always equals the highest order of hierarchical complexity of any task performed by the hierarchical stacked neural network. But, unlike the human brain, the number of networks in a hierarchical stack is not limited by the highest order of hierarchical complexity that a human can achieve. The number of networks in a hierarchical stack is unlimited and thus may exceed the capabilities of the human brain.
(37) Referring again to
(38) The stage/order at which a stacked neural network begins and ends and the number of neural networks in a hierarchical stack depend on the nature of the problem to be solved. Moreover, each neural network in a hierarchical stack may use different architectures, algorithms, and training methods. The type of architecture, algorithm, and training method selected for a particular neural network in the hierarchical stack depends on the order/stage of the neural network, the type of intelligent control system that is being implemented, the tasks to be performed and the stimuli to be processed. In addition to producing data as output, the neural networks described herein can be designed to provide error vectors depicting the cognitive noise produced at each neural level. Cognitive noise occurs when the information is of a form which is recognized at a lower level (and is thus presented at the input of the higher level), but it fails to represent a recognized pattern at a higher level. For example, if a text includes valid English characters, but there are many misspelled words or non-words, or alternatively if the words are correctly spelled but not combined into valid phrases and sentences. In another example, in an algebraic system, cognitive noise occurs when lower level calculations (e.g. addition) are done correctly, but higher level calculations (e.g. multiplication or distribution) are systematically done incorrectly. An example of the use of this cognitive noise is to maintain the representations of these presumptively erroneous calculations which might indeed, at a higher level, represent useful information, using a different calculus or symbology, or through a transformation of the data or symbols to a different representation.
(39) In the human brain, cognitive noise might be suppressed, as the brain is tuned to search selectively for certain signals. This allows us to concentrate on a conversation with a colleague in a noisy room (the background noise is cognitive noise) or read sloppily handwritten text (the imperfections in the formations of the letters are cognitive noise). See, generally, Selective attention and the suppression of cognitive noise. Neill, W. Trammell; Westberry, Richard L. Journal of Experimental Psychology: Learning, Memory, and Cognition. Vol. 13(2), April 1987, 327-334), describing this effect. However, in some situations the cognitive noise that is filtered out might comprise valuable information in itself. For example, the fact that a meeting is taking place at a noisy bar instead of in a quiet office might provide some information as to the intent of the parties. Similarly, the fact that a document is handwritten rather than typed might give one information as to the intent of the author. Similarly, in spam filtering, the fact that a message is not written in proper English increases the likelihood that it is spam.
(40) It is not necessary, for the present invention, for the neural network, especially at each level of analysis, to fully understand the meaning of each word or sentence that it identifies. (The output could only be a spam flag or a non-spam flag.) Rather, clustering techniques may be used to separate data into coherent signals and cognitive noise at each neural level.
(41) In one embodiment of the invention, k-means clustering is used to separate signal, cognitive noise and pure noise. The steps of k-means clustering, as applied to word identification are:
(42) 1. Select a number of clusters—one cluster for each word.
(43) 2. Determine the cluster centers, which represent the correct spelling of each word.
(44) 3. Assign each word in the input text to the nearest cluster center. The noise is the distance of each word in the input text from the nearest cluster center. Ideally, the distance would be some measure of how correct the spelling is. In other words, if the target word “hello” is spelled H-E-L-L-O, it would correspond to a distance of zero, but the spelling H-E-L-O would be closer than G-O-O-D-D-B-Y, which would correspond to “goodbye.”
(45) 4. Calculate the total distance between the target words and the closest match. Messages with a larger distance are more likely to be spam. In one embodiment, this is reported in an error vector.
(46) A similar algorithm can be used in other neural networks, for example, for character identification.
(47) K-means clustering is more difficult to apply to sentence structure identification because there is not a fixed number of clusters with which the sentence can be compared. There are a finite number of words in the dictionary, but it is nearly impossible to provide a list of all of the possible, grammatically correct, sentences with all of these words, and such clustering would present its own problems and might not simplify the problem at hand. Fuzzy c-means clustering can be used in this case. In fuzzy clustering, each point has a degree of belonging to a cluster rather than fully belonging to a cluster. This might correspond to the role a word plays in a sentence. For example, the word “Xerox,” might be a proper noun (Xerox Corporation), a common noun (photocopy machine) or a verb (to photocopy). The sentence structure parser would consider all three possibilities and probabilistically determine which one is most likely based on the other words in the phrase or sentence and their roles therein. For every point x in a fuzzy c-means clustering algorithm, we have a coefficient giving the degree of belonging in the kth cluster u.sub.k(x). Usually, the sum of those coefficients for any given x is defined to be 1.
(48)
(49) Note that, in one embodiment, one of the clusters would correspond to cognitive noise—groups of words that do not make sense together or do not represent a proper English (or foreign language) phrase or sentence. Note that it is also possible that a spoken representation might be a slurred rendition of “the rocks”. The clustering algorithm might also consider that possibility.
(50) With fuzzy c-means, the centroid of a cluster is the mean of all points, weighted by their degree of belonging to the cluster:
(51)
The degree of belonging is related to the inverse of the distance to the cluster center:
(52)
(53) then the coefficients are normalized and fuzzyfied with a real parameter m>1 so that their sum is 1. So
(54)
After this procedure, either the coefficient corresponding to the noise cluster or some measure of the distance of the points from the respective center points is returned to signify the degree of noise in this level.
(55) Spam Filter
(56) An example of an intelligent system based on the stacked neural network of the present invention is a spam filter. This example follows the model of cognitive development depicted in Table 1. (See R. Case (1987), supra; M. L. Commons et al (1998), supra; M. L. Commons & P. M. Miller (1998), supra.)
(57) Commons, M. L., & Richards, F. A. (2002). Organizing components into combinations: How stage transition works. Journal of Adult Development. 9(3), 159-177. (Expressly incorporated herein by reference).
(58) In one embodiment, the system and method involve a design based on skipped or bypassed cognitive levels. In other words, “noise,” which fails to specifically excite or suppress a trained network of neurons is forwarded to a higher level of the hierarchical stacked neural network, or other processor, for further processing.
(59) In one embodiment, “cognitive noise” provided as output in one layer of the neural network is ignored at the next layer but processed for meaning at a higher layer later on in the processing. For example, an email message might state, “This is the new car that I just purchased,” and include a signature file as shown in
(60) The signature file might be treated as cognitive noise in the layer that is looking for letters and converting the letters into words and words into sentences. However, when analyzed for meaning in conjunction with the words, the signature file could add “object representation” information.
(61) Another example involves processing an opera in a foreign language (in which the human or processor was not trained to work). While the phonemes are recognized as such, the phonemes are not combined to make understandable words and sentences. Nevertheless, the cognitive noise produced by the phonemes can be combined with the actions of the actors and the tone of the music in order to ascertain a meaning, and thereby permit the network to learn aspects of the new language.
(62) In yet another embodiment, information about a speaker can be gleaned from cognitive noise. The neural network would be trained to convert phonemes into meaningful words and sentences. However, if the speaker's pronunciation and grammar are different from what the processor expects, the processor can conclude that the speaker is a foreigner or from a distant part of the country. This information could be useful to understanding and responding to the cultural background of the speaker.
(63) Application
(64) In one embodiment of the present invention, a machine determines at which levels of development certain habits or ways of thinking are developed. This calculation is based on the number of sublevels of the thought involved. This machine could then be used to modify the maladaptive ways of thinking of affected individuals who are seeking treatment. For example, a written questionnaire or in-person interview or interactive machine analysis could be implemented to ascertain whether a subject has a certain phobia (snakes, spiders, people of different backgrounds, heights, flying in airplanes, etc.). A determination is then made, based on the acquired data, at which stage, or at which cognitive level of thinking the phobia is located. The knowledge of the cognitive stage responsible for the phobia could be useful for addressing and perhaps eliminating it. For example, if a phobia is in a high stage, logical arguments as to why the phobia is irrational might be most effective. However, if the phobia is at a low stage, and perhaps the holder of the phobia acknowledges it as irrational, more innate, instinctive programming is necessary to remove the phobia from the holder. A similar analytic framework may be used as part of an educational system, in which the presentation of facts and theories is adaptive to the determined cognitive level of analysis. Indeed, such a system may permit subanalysis of cognitive processing in context-sensitive manner; therefore, one need not assume that an individual operates at all times at the same cognitive level, and therefore a level may be determined for various activities and thought processes. One way of determining the cognitive level is to compare user responses with a model, selecting the level based on a best correspondence with the available models. Thus, the user response is compared with an exemplar response.
(65) In yet another embodiment of the invention, the Model of Hierarchical Complexity can be used to implement a child-like robot. An exemplary robot would have eleven to fifteen levels of neural networks and start with some pre-programmed “instinctive” responses. The robot would further comprise cameras (for eyes), microphones (for ears), and speakers (for a mouth). Further, various sensors, such as pressure, texture, and temperature sensors on its hands. Such a robot would be able to learn to interact with humans, for example, by reading, writing, and speaking, by further training its neural networks beyond a base state. The neural networks could also be trained to complete different tasks of varying complexity, from basic verbal logical and mathematical issues, to assisting a person with basic motor tasks, such as brushing teeth. At higher levels, the robot is trained to, and capable of, solving calculus problems and piloting an automobile. The networks may be individually trained, and functional capabilities provided ab initio.
(66) In some cases, a robot can be trained to complete a task more efficiently than a human because a robot is not limited by the imperfect perception, reaction, and ability to integrate information of humans. For example, human drivers have limited peripheral and reverse vision and limited reaction times. Also, some humans have poor spatial perception, making driving tasks such as reverse and parallel parking difficult. A robotic driver would not have such limitations. For example, the robotic driver might have a “head” located above the roof of the vehicle allowing it to look in all directions, while the body of the robot is inside the car. In another embodiment, the robot has an “eye” on the rear bumper of the vehicle to optimize reverse and parallel parking. The connections between the various parts of the robot could be wired or wireless. For example, an Ethernet, WiFi, or Bluetooth connection could be used.
(67) In yet another embodiment, a self-driving car trained through these neural networks is provided. As part of an advanced cruise control system, the human “driver”/main passenger of the car preferably has back-up controls in case of a failure of the automatic driver.
(68) It should also be noted that, while human reasoning may be limited to 14 stages according to the Model of Hierarchical complexity, the thinking of a human-made robot is not so limited. There is nothing preventing one from building a robot with more than 14 hierarchical neural network levels, thereby creating a robot that can think at higher stages than humans. Preferably, the cognitive levels emulate those of a human; however, in some cases, the cognitive levels implemented in an automated system can deviate from the human cognitive levels in purpose, capability and scope. Likewise, in some cases, automated systems are implemented with sets of neural networks at generally parallel levels of the hierarchy, which permits fusion and/or divergence of outputs at successive levels. However, it is also possible to have multiple hierarchies, with disparate inter-level definitions.
(69)
(70) In yet another embodiment, the neural networks are implemented on a general or special purpose computer, for example an Intel Core i7 based computer running Windows 7, having an nVidia Fermi (Tesla) coprocessor. See, Daniel L. Ly, Volodymyr Paprotski, Danny Yen, “Neural Networks on GPUs: Restricted Boltzmann Machines”, www.eecg.toronto.edu/˜moshovos/CUDA08/arx/NeuralNet_report.pdf; Jayram Moorkanikara Nageswaran, Nikil Dutt, Jeffrey L Krichmarl, Alex Nicolau, Alex Veidenbaum, “Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors”, www.ics.uci.edu/˜jmoorkan/pub/gpusnn-ijcnn.pdf; Mario Martínez-Zarzuela, Francisco Javier Díaz Pernas, José Fernando Díez Higuera and Míriam Antón Rodríguez, “Fuzzy ART Neural Network Parallel Computing on the GPU” Lecture Notes in Computer Science, Computational and Ambient Intelligence, Springer Berlin/Heidelberg, ISSN 0302-9743 Volume 4507 (2007), (expressly incorporated herein by reference).
(71) The computer could have multiple layers of separately trained neural networks according to the Model of Hierarchical Complexity. This computer system is preferably adapted to produce creative works of art, music, literature or cinema. This permits novel computer art, musical recordings, literary works, and films generated entirely by machine with or without the creative involvement of human authors.
(72) In general, an element of creativity is permutation of compatible concepts, and therefore one embodiment provides a genetic type algorithm (which may or may not be predicated on random permutations), to generate new higher level concepts and concept streams. See, e.g., U.S. Pat. No. 5,140,530, incorporated herein by reference.
(73) In one embodiment, the invention is adapted to essays and synthesis documents or reports. This embodiment of the invention could be accessible over an Internet website. The company implementing the system might charge user's a fee for its use, subscription, or it could be provided for free to members of a certain group, e.g. people who have an account at a certain bank. Various other commercial models may be implemented, for example product placement, advertisement (preferably contextually appropriate), or the like.
(74) The neural network is trained to comprehend instructions in natural language, and likewise interact with knowledge bases also having natural language records, as well as formatted or structured databases. This system might be used, for example, to generate term papers (or at least drafts thereof), or to review and/or improve such papers as part of an education or automated mentoring system. Thus, college students using the computer program could scan paper copies of their assignments and provide them as input to a website or on-line system. In addition to being able to understand natural language, the system is able to connect to generic, scholarly and literary databases to “learn” information that is relevant to the question. For example, the system may be used to analyze the use of harsh and soft sounds in Hamlet's “To be or not to be” speech. The hierarchical stacked neural network ascertains the meaning of ‘harsh’ and ‘soft’ sounds, as used in this context, as well as the text of Hamlet's speech and the background and plot of the play. In an educational embodiment, the neural network further seeks a syllabus or semantic description of the course material. For example, the paper regarding the use of sound in Hamlet's “To be or not to be” speech might take a different form if it were written for (1) an introductory English course, (2) an advanced course on Shakespeare, and (3) an advanced poetry course. In yet another embodiment, cultural data about either the student or the professor teaching the course could be taken into account. For example, if the professor is a devout Christian who enjoys seeing Biblical references in the students' writing, the system could provide such references. Alternatively, the student might note that she is interested in impressionist art, and the resulting document could comprise references to Monet and Renoir's paintings. A user model may be implemented to emulate the style and error probability of a person, and thus produce a document corresponding the work of the emulated person. Such a system may also be used to assist a user in improving the style, based on a cognitive model of the existing user process, and that of an improved paradigm.
(75) Another embodiment is illustrated in
(76) Using this type of system, it is possible to extract a particular skill of the user at a specific cognitive level, while permitting automation at other levels. Thus, if the goal is to judge the student's ability to generate a thesis, the mechanics of writing the paper may be automated based on the provided thesis. If the skill sought to be analyzed is the creation of a paper from an outline, then the process up to creation of the outline may be automated, and the remainder the manual effort of the student. On the other hand, in a technical writing environment, the goal of an analyst may be to amass and organize a body of information, which is then to be formulated into a technical report. Thus, a technician can be offloaded of the responsibility of data analysis and presentation, while generating a high level, well written report.
(77) A natural language translation neural network is described in detail by Koncar and Guthrie, “A Natural Language Translation Neural Network,” Imperial College of Science, Technology, and Medicine, London, U K, 1997 (Hereinafter “Koncar”). The method of Koncar can be slightly modified to yield a meaning-to-English (or meaning to any other spoken or written language) translator that could be used in an automatic essay writer as disclosed herein. Thus, it is clear that not all elements of the invention need be implemented as hierarchical stacked neural networks modeled according to human cognitive levels, and rather use available information sources and/or processing facilities as may be available.
(78) Combining hierarchical stacked neural networks and representation of meaning with a translation system and method, such as that of Koncar or that implemented by Google Translate at translate.google.com, or Bing Translator by Microsoft Corporation, www.microsofttranslator.com, would allow a translation from English (or any other language) to meaning, back to English (or to any other language). The document may be modified, at the meaning or conceptual level, thus permitting the user to impart a specific contribution, or change a premise.
(79) Therefore, in the example illustrated in
(80) In another embodiment, illustrated in
(81) Step 610 involves receiving a question about which a report is to be written. There are many ways in which the processor can receive the question. In one embodiment, implemented as a website on the Internet, the user could provide a text or image file containing the question, type the question, or speak the question into a computer. The question could be any topic about which the user wants information. In an exemplary embodiment, the question entered by the user is, “Are golden parachutes effective at reducing perverse incentives?”
(82) The next step 620 is researching the question. In one embodiment, the server further comprises a knowledge database which it can search. The knowledge database can be local to or remote from the server. In one embodiment, the Internet is the database to be searched. The Internet can be searched through many commercial search engines, for example, www.google.com or www.bing.com by Microsoft Corporation. Natural language search engines, such as Google and Bing, could receive as input the text of the question and provide as output a plethora of web pages comprising relevant information. For example, a Google search for “Are golden parachutes effective at reducing perverse incentives?” conducted on Jun. 1, 2010, returned about 5,450 results. The results include a Wikipedia article on golden parachutes, and several scholarly articles about golden parachutes and related topics. These would be provided as input to the hierarchical stacked neural network. Text would be provided to the N.sub.4 layer (either directly or through lower neural network layers), which converts text to words. However, images and text in files that have not been placed under optical character recognition (OCR) technology would initially be analyzed with the pattern detector in, for example, N.sub.1. The information in the articles would be propagated upward in the hierarchical stacked neural network to derive a meaning.
(83) According to a preferred embodiment, a stacked neural network is implemented based on the highest level of cognition required by the problem to be solved or addressed. In this case, the network would have about 12-14 levels, unless intended to present information to non-adults. The meaning is this represented at the highest level of the network. In a pure neural network, the output of the highest level is then used to excite A hierarchical stacked motor neural network which can then produce outputs usable in its context. In this case, while the paper is the desired output, the intervening steps each require various “actions”, and thus the information will pass through both ascending and descending hierarchies during use. In a hybrid network architecture, the “meaning” from the top-most level of the neural network is received by a logical processor, which can then organize the various tasks for processing by the cognitive neural networks. The logical processor also can solve issues of temporary storage, interfacing, and other issues which might be complex in a purely neural network implementation. However, it is understood that neural networks can be implemented with registers and the like to facilitate an implementation which does not require human-programmed logic.
(84) In the third step 630, the processor generates a thesis or a brief answer to the question. In some circumstances, all of the literature discovered would point in one direction. For example, if the question is, “Is the Earth round or flat?” the processor, after conducting the research step, would come to the conclusion that the Earth is round and discuss this thesis. However, one embodiment is able to provide a report arguing the contrary thesis (that the Earth is flat). Returning to the extended example regarding golden parachutes, if the scholarly literature is split roughly 50/50 on the topic, the processor may then select one of the two common answers to the question to argue. In an embodiment, the selection could be random, based on a random or pseudorandom number generator. In an exemplary embodiment, to answer the question, “Are golden parachutes effective at reducing perverse incentives?” the thesis is either: (1) “Golden parachutes are effective at reducing perverse incentives,” or “Golden parachutes are not effective at reducing perverse incentives.”
(85) In the fourth step 640, the processor generates an outline for the essay report. In one embodiment, the outline would summarize common themes among the resources that had been gathered in the information search step. For example, a paper trying to prove that golden parachutes are effective at reducing perverse incentives might begin by providing a high level definition of golden parachutes and the basic arguments as to why they do and do not provide perverse incentives to employees holding them. The next step in the outline would address all of the arguments that golden parachutes are effective and explain the incorrectness of the arguments that golden parachutes are not effective.
(86) In the fifth step 650, the portions of the outline are discussed in detail. In one embodiment, this step is completed by providing a summary of each of the discovered articles that has a meaning which is similar to the part of the outline which is being discussed. Techniques for summarizing documents are known in the art. See, e.g., U.S. Pat. Nos. 6,205,456, 5,978,820, 6,289,304, and 5,867,164, each of which is expressly incorporated herein by reference. It is also noted that LexisNexis.com implements a summarization technique for legal cases, law journal articles, and legal encyclopedia articles for subscribers to the LexisNexis legal research service.
(87) Finally, in step 660, the processor provides the report. There are many ways in which the report could be provided. For example, it could be sent to the user by email or provided on a webpage. Alternatively, a paper version of the report could be sent to a fax machine or a printer that is connected to the Internet or other communication network.
(88) Neural networks can be used to control systems and processes, as well as to gather data. One example of a neural network designed to control a system is provided by Schafer, US App. 2009/0271344, incorporated herein by reference. Schafer's method includes two steps, the learning of the dynamic with historical data based on a recurrent neural network and a subsequent learning of an optimal regulation by coupling the recurrent neural network to a further neural network. The recurrent neural network has a hidden layer comprising a first and a second hidden state at a respective time point. The first hidden state is coupled to the second hidden state using a matrix to be learned. This allows a bottleneck structure to be created, in that the dimension of the first hidden state is smaller than the dimension of the second hidden state or vice versa. The autonomous dynamic is taken into account during the learning of the network, thereby improving the approximation capacity of the network.
(89) This creative neural network embodiment of the invention could be created through genetic algorithms. Genetic algorithms are computation techniques based on evolutionary biology, implementing features such as inheritance, mutation, selection, and crossover, which are designed to solve computing optimization problems. See, generally, Wikipedia: Genetic Algorithm, available at en.wikipedia.org/wiki/Genetic algorithm.
(90) Sung-Bae Cho discusses a creative system based on genetic algorithms in Cho, “Towards Creative Evolutionary Systems with Interactive Genetic Algorithm,” Applied Intelligence 16, 129-138, 2002 (Hereinafter “Cho”).
(91) A hierarchical stacked neural network with 14 or more layers implementing an Interactive Genetic Algorithm (IGA) can be trained to write college level papers, and assigned grades by, for example, college writing professors that grade the neural network's output and give it feedback, which is then used to adapt the network.
(92) Several publications have been made with the application of the IGA to the problems that require creative solutions, and they can be classified into three categories: artistic, engineering and edutainment applications. See, Handbook of Evolutionary Computation, Section C2.9 by Banzhaf. We can apply GA for design by initializing the population of individuals encoded from design characteristics, setting and evolving the fitness as ‘how good the design is.’ However, there may be no standard of ‘goodness of design,’ and it is therefore difficult to organize the fitness function. IGA provides a possible solution in this case. IGA can reflect personal preference, because it perceives the fitness directly from human instead of computing some function.
(93) Fashion Design
(94) Though the meaning of design has changed by time, the works that designers do has not changed much when they design clothes. They start with a sketch and flesh it out into an illustration. With clippings from magazines, journals and photographs of similar or rival designs from fashion shows, they make these concepts into some sample pieces. These samples are tested for quality, feel and aesthetics very rigorously. Recently, the computer has begun to aid these works.
(95) Captured body models can be transformed to virtual mannequins.
(96) Some design-aid systems have been developed using Evolutionary Computations (EC). Nakanishi developed a fashion design aid system using genetic programming. He encoded a number of different lines from a dress into a “chromosome”. The system evolves each dress design according to the user's selection. But most of its productions were somewhat impractical, because encoded individuals did not contain realistic knowledge on the domain of fashion.
(97)
(98) Previous design aid systems using evolutionary computation may produce impractical or unacceptable designs because they do not consider domain-specific knowledge. To solve this problem, one may encode the detail model based on the knowledge of fashion design.
(99) For example, Cho suggests reclassifying general detail factors into three parts: neck and body, arm and sleeve, skirt and waistline. Next, Cho encodes them with three additional bits for each, which represent the color of each part. A design is made from combining them, and with IGA some combination that produces the design preferred by users can be discovered in the training stage, resulting in more realistic and reasonable designs.
(100) There are various basic strategies which might be used to generate designs. In some cases, the design is created de novo based on a “concept” from a high level of the network, for example based on a random pattern or internal excitation of the system. In other cases, the system is iterative, or design-cue based, and receives an external starting point. For example, a fashion design may be initiated based on a new fabric design, and therefore the fabric design is received as an input. The two strategies may be used together. In addition, the design generation capability may be used to assist or guide a human, or to interact with a human, and therefore need not generate an entire design unaided.
(101) In accordance with one embodiment of the invention, representations of the fashion designs could be analyzed by a hierarchical stacked neural network. The lowest level extracts the visual features, such as color and changes in light intensity, of the design. The higher levels of the neural network would extract the features of the designs and make conclusions relating to them. For example, certain shapes correspond to shoes. The shoes have certain features, such as raised heals, laces, etc. Other shapes correspond to pants, shirts, dresses, etc. The genetic algorithm taught by Cho would come up with features that a computer user can like or dislike, and the neural network would attempt to provide some high-level representation of the user's tastes, and then to create one or more outfits that would be appreciated by the user.
(102) In another embodiment, the processor receives as input from the user a representation of the user's tastes. For example, the user could speak into a microphone, “Please provide a man's white button down shirt and grey pants.” The processor would then generate a white button down shirt with grey pants to provide to display to the user, based on a cognitive analysis of the input.
(103) In yet another embodiment, the processor is taught to generate fashion designs which are associated with the features of a particular designer (e.g., Tommy Hilfiger) by being provided as input many designs and being told, by a user or automatic service, which ones are associated with Tommy Hilfiger. Using, for example, a hierarchical stacked neural network, the processor would then “learn” the features of Tommy Hilfiger clothing and provide new clothing incorporating these features as output using a clothing generator module based on the neural network representation. Many Internet databases, such as Google Images, images.google.com, and Bing Images by Microsoft Corporation, www.bing.com/images that can provide diagrams of general clothing as well as of clothing by certain designers, such as Tommy Hilfiger. In another embodiment, another Internet image source or a database local to the processor could be used instead of the Internet. In one embodiment, the step of incorporating features from former designs into new designs is completed in the neural network. The neural network selects a feature, such as a pocket, a button, or a decoration, from a former Tommy Hilfiger design and then copies and pastes the feature into a novel design, with or without modification of the emulated feature(s). For example, the pockets in one pair of jeans could be combined with the buttons in a second pair of jeans to generate a novel third pair of jeans.
(104) Image Retrieval
(105) A neural network-based control system, such as that provided by Schafer, US App. 2009/0271344, may be combined with the hierarchical stacked neural networks described herein, to create a neural network based control system that is useful to pilot a car. The automatic driver of the car (1) selects a travel path, (2) controls the car in accordance with the travel path, and (3) modifies the driving in accordance with perceived threats, road signs, and other visual and audio input.
(106) It should be noted that many driving mistakes or accidents are caused due to the driver of a vehicle that was involved either lacking information, such as failing to see an oncoming vehicle or a traffic control device, or poorly implementing driving rules, such as failing to obey a speed limit sign or running a yellow or red light. The automatic driver is designed, in part, to solve these problems by automatically processing more information than is typically available to a human driver and by automatically implementing all of the driving rules and modifying the driving behavior in response to a perceived threat of an accident.
(107) To select a travel path, the automatic driver would take as input from a user a destination address. The automatic driver would then ascertain the current location through a global positioning system (GPS) mechanism. An exemplary GPS machine is described by Hsu in U.S. Pat. No. 6,392,591, incorporated herein by reference. GPS machines well known in the art and are manufactured and sold by TomTom Corporation, Garmin Corporation, Magellan Corporation, and others.
(108) The travel path could be selected by a mapping algorithm calculating a shortest or approximately shortest path between the starting point and the destination. Techniques for doing so are well known in the art and are implemented on the Internet by Google Maps, Yahoo Maps, and Bing Maps, among others. In one embodiment, the automatic driver could access Google Maps, Yahoo Maps, or a similar service over the Internet or over a cellular network to obtain driving directions.
(109) The car could be steered in accordance with the selected travel path by driving along the path. GPS devices that tell the driver exactly when and in which direction to turn are known in the art and are provided by TomTom Corporation, Garmin Corporation, Magellan Corporation, and others. Therefore, these can be implemented by the automatic driver.
(110) Cameras (or other imaging or sensing devices) could be placed all around the car (or other vehicle) to provide a 360-degree view of the surroundings. This way, more data can be processed than by a human driver, who can only be looking in one direction at a time. To arrange for a 360-degree view from multiple parts of the car, a preferred embodiment of the invention uses several cameras or a single camera with a wide field of view or rapid scan rate. However, only one camera that represents the human driver's view might be necessary for a minimal embodiment.
(111) The images from the camera(s) are provided to a hierarchical stacked neural network which is trained to search the images for information relevant to driving, such as road signs, traffic control devices, other vehicles, and pedestrians. In one embodiment, the neural network is further adapted to develop motion vectors for the other vehicles and pedestrians on the road so that it could predict future movements. For example, if the vehicle is travelling at 55 mph on a highway and is thinking of moving to the left lane, the motion vectors could be used to ascertain that the vehicle is presently being passed and it is necessary to wait until the passing vehicle gets by.
(112)
(113) Neural network 2110 is trained using input arrays containing excitation patterns. A backward-propagation algorithm 2130 adjusts weights in neural network 2110 based on neural network 2110's successes and failures in detecting physical objects.
(114) Neural network 2112 is an adaptive-resonance (ART) neural network (G. A. Carpenter & S. Grossberg, “Neural dynamics of category learning and recognition: Structural invariants, reinforcement, and evoked potentials”, in M. L. Commons, R. J. Hermstein, S. M. Kosslyn, & D. B. Mumford, (Eds.), 9 Computational and clinical approaches to pattern recognition and concept formation. Quantitative analyses of behavior (Hillsdale, N.J., Lawrence Erlbaum Associates, 1990)). Neural network 2112 performs processing actions at stage/order 2, the Circular Sensory-Motor stage/order, in the model described in Table 1. At this stage/order an intelligent system distinguishes objects and tracks them. Using Circular Sensory-Motor stage/order tasks, neural network 2112 maps input excitation patterns from neural network 2110 to clusters. Cluster weights are adjusted each time a new excitation pattern enters neural network 2112 from neural network 2110. A second hidden layer of neurons tracks excitation patterns through the scene and links their movement centroids to the appropriate clusters. Repeated placement of input excitation patterns on the same cluster, when only the centroid coordinates of the patterns differ, triggers an output signal to neural network 2114 when a predetermined threshold rate of continuous placement is obtained. The output signal contains the excitation area's centroid, dimensions, store coordinates, and history of centroid positions at successive time intervals. The training of neural network 2112 is unsupervised. Cluster weights are adjusted each time a new excitation pattern, representing a new physical object is input into neural network 2112 from neural network 2110.
(115) Neural network 2114 is a feed-forward heteroassociative neural network that performs processing actions at stage/order 3, the Sensory-Motor stage/order, of the model described in Table 1. At this stage an intelligent system can recognize objects and place them in classes. Using Sensory-Motor tasks, neural network 2114 analyzes patterns output by neural network 2112 and determines whether a pattern is “a road sign”, “a traffic control device”, “another vehicle”, “a pedestrian”, etc. (Other data that is relevant to driving will be apparent to persons skilled in the art.) Patterns identified by neural network 2112 need not be identical to stored patterns to activate an output signal. Stored patterns that identify “a pedestrian” are based on major human features such as a head, a face, arms, and hands. The patterns associated with “another vehicle”, “a road sign”, and “a traffic control device” use a system of weights that weight some pattern components more heavily than others.
(116) Neural network 2114 is trained by inputting patterns of “a road sign,” a “traffic control device,” “another vehicle,” “a pedestrian,” etc. A backward-propagation algorithm 2132 adjusts neural network 2114's weights based on the network's successes and failures in identifying “a road sign,” a “traffic control device,” “another vehicle,” “a pedestrian,” etc. When neural network 2114 associates a pattern with “a road sign,” a “traffic control device,” “another vehicle,” “a pedestrian,” etc. the network outputs to neural network 2116 the pattern's classification as “a road sign,” a “traffic control device,” “another vehicle,” “a pedestrian,” etc., as well as the pattern's centroid, dimensions, store coordinates, and history of centroid positions at successive times.
(117) Neural network 2116 is a feed-forward neural network that performs processing actions at stage/order 4, the Nominal stage/order, of the model described in Table 1. At the Nominal stage/order, an intelligent system can identify simple relationships between concepts and label them. Neural network 2116 has one hidden layer. The neurons in this layer receive excitatory and inhibitory input based on the centroids, dimensions, coordinates, and history of centroid positions at successive one-second time points of objects and persons that was received from neural network 2114. The neurons also receive input from other neurons within the hidden layer. The determination that “another vehicle” has a “motion vector that may lead to a collision” is signaled by excitatory input from neurons, within the hidden layer, activated by patterns for “motion vector that may lead to a collision” and that share contiguous and overlapping store coordinates with “another vehicle.” When “another vehicle” has become associated with “motion vector that may lead to a collision,” an output signal is triggered. Neural network 2116 then outputs to neural network 2118 an array pattern for the “motion vector that may lead to a collision” and the history of store coordinates of the “another vehicle” array at successive times. The array pattern uniquely identifies the “another vehicle” and the “motion vector that may lead to a collision” as being associated with the “another vehicle.” This information can now be fed to a rule-based system that can calculate an appropriate response to avoid the collision. Persons skilled in the art will note that this rule-based system will need to take the motion vectors of the other vehicles and pedestrians, as well as the present road signs and traffic control devices, into account.
(118) Neural network 2116 network is trained using patterns of interaction between pedestrians and vehicles. A backward-propagation algorithm 2134 adjusts weights each time neural network 2116 successfully matches “another vehicle” or “a pedestrian” and “motion vector that may lead to collision.”
(119) Neural network 2118 is an adaptive-resonance (ART) neural network that performs processing actions at stage/order 5, the Sentential stage/order, of the model described in Table 1. At this stage/order, classes and patterns are ordered. Using Sentential stage/order tasks, neural network 2118 tracks “a vehicle” having “a motion vector that may lead to a collision” and issues an emergency alert to the vehicle control logic so that it might respond appropriately. Neural network 2118 has two hidden inner layers. The vehicle/motion vector array pattern input from neural network 2116 is mapped to a specific cluster in the first hidden layer and the weights associated with the cluster are set for that vehicle/motion vector array pattern. Each time this array pattern enters from neural network 2116, it maps to this cluster. The cluster is reset when it no longer matches incoming patterns. Each time that the cluster is activated, it sends excitatory signals containing the coordinates and time points associated with the person/object array to the second hidden layer. The areas of excitation in this layer track the other vehicle's movement in relation to the vehicle being operated. An alert 2122 to the vehicle control logic is generated when the pattern of neuron excitation indicates that the “another vehicle” having a “motion vector that may lead to a collision” is sufficiently close to the vehicle being operated or is approaching the vehicle being operated at a high speed.
(120) Neural network 2118 is trained using patterns of movement that a vehicle would follow while it is driving. In one embodiment, a driving simulation video game or computer model is used to train the neural network. In another embodiment, in the training stage, the neural network is placed in a car with an experienced driving instructor, who provides feedback to the neural network in order to train it. In this embodiment, there is little chance that the neural network will cause an accident because the driving instructor has access to an emergency brake and can take control of the car away from the neural network, or override any of the neural network's decisions, if necessary. A backward-propagation algorithm 2136 adjusts weights in each of the neural networks 2110, 2112, 2114, 2116, and 2118 based on the hierarchical stacked neural network's success in detecting and responding to information relevant to driving.
(121) These neural networks generate the information that is necessary to safely operate the vehicle. The next step is processing this information in order to actually drive the car (or other vehicle). While a neural network may be used to operate the car, a rule based system is preferred. The rules will have a hierarchy. In other words, some rules can be broken more easily than other rules. For example, while, as a general rule, the vehicle should not run red lights, running a red light might be acceptable if it is necessary to allow an emergency vehicle to pass and there is no oncoming traffic that might cause a collision.
(122)
(123) A more detailed hierarchy than the one presented in
(124) In one embodiment, the driving rules are stored locally to the automatic car pilot. In another embodiment, the driving rules are stored on a remote server and are accessible to the vehicle over an Internet or cellular connection. Preferably, the set of rules that is presently being implemented is stored locally to the server. For example, if the vehicle is in New York State, the New York Vehicle and Traffic Law is stored on the server. This embodiment is beneficial if there are occasional breaks in the Internet connection, as the automatic driver still has access to the traffic rules and can operate the vehicle.
(125) In one embodiment, the rules may be adjustable by the driver or owner of the automatic vehicle. For example, the driver may adjust the “no speeding” rule to allow for the vehicle to travel up to 5 mph above the speed limit. In another embodiment, the driver will “co-pilot” the automatic vehicle and be able to override any of the actions of the automatic vehicle.
(126) In another embodiment, the auto-pilot may further comprise a mechanism for accessing the Internet or another database to discover mechanisms for controlling and operating the car. This Internet or database access is useful in multiple different situations. For example, in an emergency situation, since the autopilot has control over steering, braking, throttle, transmission and suspension, a simple stopping of the vehicle is not the only available option, and therefore that the collision avoidance maneuver may include any available movement, such as a slide, or other “trick” maneuver more commonly associated with skilled stunt drivers.
(127)
(128) In step 1310, the human user enters a request to drive to a destination address, such as 300 Broadway Ave., Albany, N.Y. The auto-pilot notes that it is currently located at a starting location, such as 10 Bank St., White Plains, N.Y. The auto-pilot can determine its present location through many different methods. For example, global positioning system (GPS) satellite or cellular tower based triangulation techniques could be used. Such methods are implemented by TomTom Corporation in GPS devices and by Apple Corporation in iPhone Apps such as Google Maps and others.
(129) In step 1320, the auto-pilot notes that the car is running out of gasoline. There are many ways of making this determination. For example, the auto-pilot could note that the car has less than a quarter of a tank of gas, less than a certain volume of gas, or not enough gas to reach the destination, based on the average miles per gallon (MPG) rating of the vehicle or the manufacturer's expected city and highway MPG ratings. It is noted that many modern vehicles, such some models of the Chevrolet HHR 2010, track their own instantaneous MPG rating, lifetime average MPG rating, and average MPG rating during a given trip (where the start and end of the trip is provided by the driver).
(130) In step 1330, the auto-pilot connects to the Internet through AT&T's 3G cellular network. Of course, other methods of connecting to the Internet, such as Verizon's cellular network, public WiFi networks, or satellite-based Internet connections can also be used.
(131) In step 1340, the auto-pilot creates a cognitive model of the prices at some of the gas stations along the route and within a certain radius of the route to determine an optimal gas station to use to refuel the vehicle. This cognitive model can take many different features into a account. For example, the type of fuel used 1341—regular, premium, or diesel—is an important variable. The price of the fuel at various gas stations 1342 is also taken into account. Internet databases of gas prices are available or can be developed. An exemplary internet gas price database is provided at autos.msn.com/everyday/GasStationsBeta.aspx (last visited Jun. 9, 2011), which allows one to search for gas prices by city and state or zip code. Other considerations include: the ability of the vehicle to reach the gas station on available fuel 1343, a human user's interest in saving money on gas 1344, a human user's anxiety to reach the destination quickly 1345, extra tolls that need to be paid to reach the gas station 1346 (if the gas station requires a detour off a toll road), and the human user's preference for different gas stations 1347 (e.g. some human users prefer ExxonMobil gas to Gulf Gas). In one embodiment, the auto-pilot may recommend that the human user not fill the gas tank all the way. For example, if the vehicle is nearly out of gas and a price sensitive human user who is not anxious to reach his destination is involved, the auto-pilot may recommend that the human user purchase a few gallons of gas at an expensive gas station that is nearby, and then drive to a cheaper gas station a few miles off the route to fill the gas tank.
(132) In step 1350, the auto-pilot selects an optimal gas station to use on the basis of the cognitive model developed in step 1340.
(133) In step 1360, the auto-pilot directs the car to the optimal gas station, and then to the destination.
(134) It is noted that this system and method may be subsidized to the consumer by advertising. For example, a gas station company, such as Shell, could provide compensation in exchange for notifying users of Shell gas stations along the route and offering to navigate the vehicle to the Shell gas station. In another embodiment, Shell could be the default gas station to which the vehicle auto-pilot drivers whenever the vehicle needs gas. The human user is able to modify or override this setting.
(135) In other embodiments of the invention, the system and method for the gas station optimizer could be modified to select an optimal speed to travel, optimal locations for rest stops during a road trip, a system and method for avoiding potholes, etc. In one embodiment, each of these involves creating a cognitive model of the unique needs of a user.
(136) Neural Network Router Controller
(137) According to one embodiment, the inventive hierarchical stacked neural network can be used to control a WiFi or cellular router. These routers typically use orthogonal frequency-division multiplexing (OFDM) technology for high data rate wireless transmissions. OFDM is well-known in the art and is described, for example, in U.S. Pat. Nos. 5,694,389, and 5,732,113, each of which is expressly incorporated herein by reference.
(138) OFDM (orthogonal frequency division multiplexing) brings many benefits to wireless networking, and will be a part of the next-generation (4G) of mobile wireless systems. The technology however imposes a challenge due to the increased complexity of channel equalization. Wireless channels are multipath fading channels, causing deformation in the signal. To remove the effect (imposed by channel) from received signal, the receiver needs to have knowledge of CIR (Channel impulse response) that is usually provided by a separate channel estimator. One of the many goals of this invention the use of the inventive hierarchical stacked neural networks disclosed herein as a tool for MIMO-OFDM channel estimation and compensation.
(139) OFDM-based routers often need to make decisions based on a large amount of input data. For example, in channel coding or forward error correction technology, some of the other bits in a transmission may need to be considered. Under the Shannon Noisy Channel Theorem, if there exists a noisy channel with capacity C and data transmitted at rate R, where R<C, there exist codes that allow the probability of error at the receiver to be arbitrarily small. These error correction codes have traditionally been implemented through statistical methods. However, a problem with the statistical methods is that they often fail to take into account all of the nuances of the large data sets which they are provided, and assume a system which is consistent with the statistical methods employed.
(140) The hierarchical stacked neural networks are useful for error correction. The lowest level neural network would detect the signals being provided and immediately filter the pure noise. Similar to the manner in which the written character filter distinguishes between letters that look like they could represent two or more different letters at the next highest layer, the error correction neural network provides data that is capable of having multiple interpretations to the next highest level to derive the best interpretation.
(141) Benediktsson, “Neural Network Approaches Versus Statistical Methods in Classification of Multisource Remote Sensing Data,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 28, No. 4, July 1990, compares statistical methods and neural network methods in error correction. An advantage of neural networks, over statistical error detectors and correctors, is that neural networks are distribution-free and no prior knowledge is needed about the statistical distributions of classes in the data sources in order to apply neural network classification methods. Neural networks can also adjust the weight given to each data source in the classification. In one embodiment, input from various data sources is processed independently, and each input is characterized by an appropriate model in the neural network. In another embodiment, a layer of the neural network is responsible for selecting a model to use to classify the data.
(142) Hassoun discusses neural network technology data compression and error correction technologies, which are useful in wireless networking. Mohamed Hassoun, Fundamentals of Artificial Neural Networks (MIT Press, 1995) (Hereinafter Hassoun). Data compression techniques exploit the redundancy that naturally exists in most data for efficient storage and/or transmission purposes. Here, a data set is encoded with a much smaller number of bits than the total number of bits required to describe it exactly (i.e., lossy compression). After retrieval or at the receiver end of a transmission link, the encoded or “compressed” data may then be decoded into an approximation of the original data. The compression of images can be posed as an optimization problem where, ideally, the encoding and decoding is done in a way that optimizes the quality of the decoded data.
(143) In a neural network image processor having an input forced to be reproduced through a narrow hidden layer (bottleneck), back propagation attempts to extract significant features from the input vectors. Here, the hidden layer, which is also known as the representation layer, is expected to evolve an internal low-dimensional distributed representation of the training data. Empirical analysis of the trained compression network shows that the hidden unit activities span the principal component subspace of the image vector(s), with some noise on the first principal component due to the nonlinear nature of the hidden unit activation's.
(144) The addition of one or more encoding hidden layers with nonlinear units between the inputs and the representation layer, and one or more decoding layers between the representation layer and the output layer provides a network which is capable of learning nonlinear representations. Such networks can perform the nonlinear analog to principal component analysis, and extract “principal manifolds.” These principal manifolds can, in some cases, serve as low-dimensional representations of the data which are more useful than principal components. A three hidden layer auto-associative net can, theoretically, compute any continuous mapping from the inputs to the second hidden layer (representation layer), and another mapping from the second hidden layer to the output layer. Thus, a three hidden layer auto-associative net (with a linear or nonlinear representation layer) may, in principle, be considered as a universal nonlinear PCA net.
(145) Another way of interpreting the above auto-associative feed forward network is from the point of view of feature extraction. Here, the outputs from the representation layer are taken as low-dimensional feature vectors associated with complete images or data sets (or any other high-dimensional raw data vectors) presented at the input layer. Whereas, the decoder (reconstruction) subnet is only needed during the training phase and is eliminated during retrieval. The output from the representation layer can now be used as an information-rich, low-dimensional feature vector which is easy to process/classify. Reducing dimensionality of data with minimal information loss is also important from the point of view of computational efficiency. Here, the high-dimensional input data can be transformed into “good” representations in a lower dimensional space for further processing. Since many algorithms are exponential in the dimensionality of the input, a reduction by even a single dimension may provide significant computational savings.
(146) Similar techniques to those presented by Hassoun can be used in error correction neural networks applied to WiFi and cellular receivers running OFDM. The neural network can be implemented directly on an embedded processor within the receiver. Alternatively, these neural networks can be implemented in machine instructions stored on a computer readable medium external to the receiver that communicates with it. For example, if the receiver has a universal serial bus (USB) port, the instruction can be stored on a USB memory drive that is connected to the receiver to improve its operation. The receiver may be part of a router, and the USB port may be part of the router. In yet another embodiment, a router communicates with the computer readable medium containing these instructions over a wireless means, such as WiFi or Bluetooth.
(147) Hardware Overview
(148)
(149) Computer system 400 further includes a read only memory (ROM) 408 or other static storage device coupled to bus 402 for storing static information and instructions for processor 404. A storage device 410, such as a magnetic disk or optical disk, is provided and coupled to bus 402 for storing information and instructions.
(150) Computer system 400 may be coupled via bus 402 to a display 412, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 414, including alphanumeric and other keys, is coupled to bus 402 for communicating information and command selections to processor 404. Another type of user input device is cursor control 416, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 404 and for controlling cursor movement on display 412. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
(151) The computer system 400 may be used to implement the techniques described herein. According to one embodiment, those techniques are performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another machine-readable medium, such as storage device 410. Execution of the sequences of instructions contained in main memory 406 causes processor 404 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
(152) The term “machine-readable medium” as used herein refers to any medium that participates in providing data that causes a machine to operation in a specific fashion. In an embodiment implemented using computer system 400, various machine-readable media are involved, for example, in providing instructions to processor 404 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 410. Volatile media includes dynamic memory, such as main memory 406. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 402. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
(153) Tangible media are provided to enable the instructions carried by the media to be detected by a physical mechanism that reads the instructions into a machine.
(154) Common forms of machine-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, paper tape, any other physical medium with readable physical patterns, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, quantum physics effect processing elements, or any other medium from which a computer can read.
(155) Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 404 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 402. Bus 402 carries the data to main memory 406, from which processor 404 retrieves and executes the instructions. The instructions received by main memory 406 may optionally be stored on storage device 410 either before or after execution by processor 404.
(156) Computer system 400 also includes a communication interface 418 coupled to bus 402. Communication interface 418 provides a two-way data communication coupling to a network link 420 that is connected to a local network 422. For example, communication interface 418 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 418 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented, such as WiFi (a/b/g/n), DSRC, GSM, CDMA, 2G, 3G, 4G, LTE, etc.
(157) In any such implementation, communication interface 418 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
(158) Network link 420 typically provides data communication through one or more networks to other data devices. For example, network link 420 may provide a connection through local network 422 to a host computer 424 or to data equipment operated by an Internet Service Provider (ISP) 426. ISP 426 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 428. Local network 422 and Internet 428 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 420 and through communication interface 418, which carry the digital data to and from computer system 400, are exemplary forms of carrier waves transporting the information.
(159) Computer system 400 can send messages and receive data, including program code, through the network(s), network link 420 and communication interface 418. In the Internet example, a server 430 might transmit a requested code for an application program through Internet 428, ISP 426, local network 422 and communication interface 418.
(160) The received code may be executed by processor 404 as it is received, and/or stored in storage device 410, or other non-volatile storage for later execution. In this manner, computer system 400 may obtain application code in the form of a carrier wave.
(161) Having described preferred embodiments of the invention with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims.
(162) TABLE-US-00001 TABLE 1 A sequence of behaviors placed into different orders of hierarchical complexity Name of Order of Order/ Hierarchical Stage Complexity Example 0 Calculatory Simple Machine Arithmetic on 0's and 1's 1 Sensory & Seeing circles, squares, etc. or touching them. Motor 2 Circular Reaching and grasping a circle or square. Sensory- * * * * * motor □ □ □ □ □ # # # # # # / ″ } Q 3 Sensory- A class of filled in squares may be made motor 4 Nominal That class may be named, “Squares” 5 Sentential The numbers, 1, 2, 3, 4, 5 may be said in order 6 Pre- The objects in row 5 may be counted. The last count called 5, operational five, cinco, etc. 7 Primary There are behaviors that act on such classes that we call simple arithmetic operations: 1 + 3 = 4; 5 + 15 = 20; 5(4) = 20; 5(3) = 15; 5(1) = 5 8 Concrete There are behaviors that order the simple arithmetic behaviors when multiplying a sum by a number. Such distributive behaviors require the simple arithmetic behavior as a prerequisite, not just a precursor: 5(1 + 3) = 5(1) + 5(3) = 5 + 15 = 20 9 Abstract All the forms of five in the five rows in the example are equivalent in value, x = 5. Forming class based on abstract feature 10 Formal The general left hand distributive relation is x * (y + z) = (x * y) + (x * z) 11 Systematic The right hand distribution law is not true for numbers but is true for proportions and sets. x + (y * z) = (x * y) + (x * z) x ∪ (y ∩ z) = (x ∩ y) ∪ (x ∩ z) 12 Meta- The system of propositional logic and elementary set theory are systematic isomorphic x & (y or z) = (x & y) or (x & z) Logic .Math. x ∩ (y ∪ z) = (x ∩ y) ∪ (x ∩ z) Sets T(False) .Math. φ Empty set T(True) .Math. Ω Universal set 13 Paradigmatic Distributive Systems are part of the Mathematical Paradigm. Mathematics integrates algebra, set theory, elementary probability theory, analysis, and based upon such an integration generates measure theory, and the mathematics used in physics. 14 Cross- Mathematics and Physics are integrated in the Mathematical paradig- Physics of Quantum Mechanics (Einstein, 1950; Planck, 1922; maticmatic Bohr, 1934) including the Standard Theory of Particles (Gell- Mann, 1964, Weinberg, 1983, Salam, 1972) Special and General Relativity Theory (Einstein, 1950)
(163) TABLE-US-00002 TABLE 2 General Description of Sequence Discrim- MHC Kohlberg inations Verbal relationships Vocalizations and Words Examples Receptive Speech Examples 0 −1 Calcu- None Exact-no generalization None None latory 1 0 sensory & Babbling (Universal) Rate, Generalized None None motor actions 2 0 circular Phonemes Open-Ended Classes Gestures, “aa” sensory- Follows one act directions motor actions 3 0/1 sensory- Morphemes Concepts Phoneme action relationships Holding cup out and saying motor “wa” Follows simple commands 4 — Nominal Single words: ejaculatives & Relations among concepts Relating concepts to others Says “water” exclamations, verbs, nouns, through a word Follows simple chain of number names, letter names commands. 5 1 Sentential Pronouns: my, mine, I; yours, Imitates and acquires Incomplete sentences; then States a rule “Don't touch” you; we, ours; they, them sequences. complete sentences, crude Follows instructions: “Turn off Follows short dichotomies, ordered VCR when tape is over” sequential acts numbers & letters 6 1/2 pre- Connectives: as, when, then, Simple deductions but Multiple sentence stories with Says “The water is filling the operational why, before contradiction is not logically order sequenced bathtub” excluded. Follows list of events. Counting materials sequential acts laid out in a line 7 2 Primary Times, places, acts, actors Simple logical deduction Stories with logically order The water is filling the and empirical rules sequenced events connected bathtub so I can take a bath involving time sequence. solidly to reality with some Can follow directions without Simple arithmetic specified times, places, acts, being supervised for an actors. Either perspective of extended period of time. self or other. 8 2/3 Concrete Interactions social events, Full arithmetic Stories with some specified If I turn on the faucet, the what happened among times, places, acts, actors, and water will run out and fill the others the relationship of the above. bathtub. My parents like it Things, incidents, events, when I fill the tub myself. actors, actions, places in the They are more willing to buy context of the interaction me clothes when I am between self and other. always clean. 9 3 Abstract Variable time, place, act, Discrimates variables Variable time, places, acts, The water can run out quickly actor, state, type; Quantifies such as Stereotypes; actors, state, type or slowly and the temperature (all, none, some) play an Logical Quantification; Variables such as Stereotypes. may get colder after awhile. important role in the (all, none, some) Logical Quantification Words: All teachers do that! preadolescent's formulation all, none, some, never, of categorical assertions sometimes, always, no one, (e.g., “All teachers do that!”). somebody, everyone. Propositions with variables 10 3/4 Formal Words: linear, logical, one Argue using empirical or Variables connected by Turning of a faucet changes dimensional, if . . . then, logical evidence. Logic is “if . . . then” and other the size of the opening of the thus, therefore, because. linear, 1 dimensional relations; passage way and thereby Empirical, or logical controlling the amount of flow. evidence and argument. Relations among Variables Single variables on input side. Empirical, or logical evidence and argument. 11 4 Systematic Systems words appear in Constructs multi variate Systems of relations. Can follow directions without adult's utterances: legal systems and matrices. Relations among relationship being supervised for an system, society, our coordinating more than among variables extended period of time. company, the economy, one variable. Events and the country. ideas situated in a larger context. 12 5 meta- Metalogical, meta-analytic Integrate systems to Supersystems of systems The systems that describe sytematic words appear. Properties construct multisystems. non-turblent water flow and of systems can be considered: Compare systems and electric flow (amperes) are homomorphic, isomorphic, perspectives in a isomorphic. Flow equals incomplete, inconsistent systematic way (across pressure (volts) × resistence system, consistent system, multiple domains). (ohms) incomplete system, complete Reflects on systems. system, incommensurable system, orders of complexity of the system, commensurable systems, etc. 13 6 para- Newly created paradigmatic Discriminate how to fit There are many metasystems digmatic words are used whether metasystems together to that belong to non-relativistic metasystems may be form new paradigms and non statistical mechanics integrated. There are a few adults who can discriminate how to fit metasystems together to form new paradigms 14 7 cross- Newly created word about Discriminate how to form By crossing mathematics and para- integration of paradigms. new fields by crossing physical experimentation, digmatic Historically there a few paradigms. classical mechanics arises. people who have discriminated how to form new fields by crossing paradigms.