AD GENERATION WITH NEURAL NETWORKS

Abstract

Ads are generated based on product info and consumer profiles. A discriminator evaluates probabilities of ads being effective at causing consumer engagement. A decoder extracts product info from generated ads. Based on the probabilities of ads being effective and similarity of extracted and source product info, generated ads are labeled as examples. The examples are used in training an improved ad generator. Ads may be visual and/or audio containing speech. Ads may even contain humor, as recognized by mismatches between source and decoded product info.

Claims

1. A system comprising: an ad generator that generates ads based on input info about products; a discriminator that predicts probabilities that generated ads could be effective; a decoder that extracts, from the generated ads, info about products in the generated ads; and a database of examples of ads that have a probability of being effective above a threshold and have an acceptable match between the input info and the extracted info; wherein the examples could be used to train an improved ad generator.

2. The system of claim 1 wherein the probabilities that generated ads could be effective is with respect to probabilities that consumers would engage if presented with the generated ads.

3. The system of claim 1 further comprising a database of example ads that do not have a probability of being effective above the threshold.

4. The system of claim 1 further comprising a database of example ads that do not have an acceptable match between the input info and the extracted info.

5. The system of claim 1 wherein the ads are visual and extracting info about products comprises object recognition.

6. The system of claim 1 wherein the ads are audible and extracting info about products comprises automatic speech recognition.

7. The system of claim 1 wherein a match between the input info and the extracted info is acceptable even if there is a humorous mismatch between parameters of the product info.

8. The system of claim 1 wherein the ad generator customizes the generated ads based on individual consumer profiles.

9. A method comprising: generating ads based on input info about products; predicting probabilities that the generated ads could be effective; extracting info about products from the generated ads; labeling the generated ads as examples based on the probabilities that the generated ads could be effective and similarity between the input info and extracted info about products; and providing the examples as training data for an improved ad generator.

10. The method of claim 9 wherein the probabilities that generated ads could be effective is with respect to probabilities that consumers would engage if presented with the generated ads.

11. The method of claim 9 wherein labels are non-Boolean, the method further comprising training an improved ad generator using a regression algorithm.

12. The method of claim 9 wherein the ads are visual and extracting info about products comprises object recognition.

13. The method of claim 9 wherein the ads are audible and extracting info about products comprises automatic speech recognition.

14. The method of claim 9 wherein the similarity between the input info and the extracted info is acceptable even if there is a humorous mismatch between parameters of the product info.

15. The method of claim 9 further comprising training an improved ad generator, wherein training includes applying consumer profiles and wherein predicting probabilities that the generated ads could be effective is based on the consumer profiles.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG. 1 shows an overview of a system and the steps of its operation according to an embodiment.

[0021] FIG. 2 shows a mobile handset displaying a generated visual ad according to an embodiment.

[0022] FIG. 3 shows an engagement by a consumer with an audio ad according to an embodiment.

[0023] FIG. 4 illustrates an ad data structure according to an embodiment.

[0024] FIG. 5 illustrates a data structure of metadata corresponding to an ad according to an embodiment.

[0025] FIG. 6 illustrates system components and method steps for training an effective ad discriminator according to an embodiment.

[0026] FIG. 7 illustrates system components and method steps for training an ad generator according to an embodiment.

[0027] FIG. 8 illustrates a network-based system for delivering ads to a consumer according to an embodiment.

[0028] FIG. 9A shows a rotating disk non-transitory computer-readable medium according to an embodiment.

[0029] FIG. 9B shows a solid-state memory chip package non-transitory computer-readable medium according to an embodiment.

[0030] FIG. 10A shows the solder ball side of a packaged system-on-chip according to an embodiment.

[0031] FIG. 10B shows a block diagram of the functions within a system-on-chip according to an embodiment.

[0032] FIG. 11A shows a rack-based server system according to an embodiment.

[0033] FIG. 11B shows a block diagram of the functions within a server system according to an embodiment.

DETAILED DESCRIPTION

[0034] The following describes various embodiments of the present invention that illustrate various interesting features. Generally, embodiments can use the described features in any combination.

[0035] In relation to advertising, this disclosure uses the term product to describe both products and services that might be advertised.

System Overview

[0036] FIG. 1 shows an overview of system operation according to an embodiment. It comprises an ad generator 11 that takes as input a product info and generates as output ads. Some embodiments implement an ad generator using a neural network. However, any algorithm that converts product info to ads in a manner according to example-based training is appropriate.

[0037] Some embodiments of an ad generator 11 may also take consumer profile info, which affects the output to be specific to a consumer profile. Whereas ads based solely on product info are appropriate for billboards or other widely observed ads, an ad generator that uses consumer profile info creates ads specific to individual consumers, which, on personal devices such as mobile handsets, can be more effective than ads based on product info alone.

[0038] Some embodiments further comprise a step of presenting ads to one or more consumers, measuring the effectiveness of the ads, and labeling each ad as to its effectiveness 12. The embodiment of FIG. 1 identifies ads as either effective or ineffective and stores effective ads in an effective ads database 64 and ineffective ads in an ineffective ads database 65. Some embodiments store the ads in a single database but tag each ad as being effective or ineffective. This is useful such as for training a binary classifier model.

[0039] Some embodiments store ads in a database and associate a score with each ad, the score representing a non-Boolean value of its effectiveness. This is useful such as for training a regression model. Though not shown in FIG. 1, its implementation is readily apparent.

[0040] The embodiment shown in FIG. 1 performs a training step on the ads labeled as effective and ineffective 13. The training produces an effective ad discriminator model 14. The effective ad discriminator 14 takes an ad as input and produces an output indicating the estimated effectiveness of the ad. Some embodiments implement an effective ad discriminator using a neural network. However, any algorithm that takes in an ad and outputs a prediction of its effectiveness according to example-based training is appropriate. Some embodiments output a Boolean value for the effectiveness prediction. Some embodiments output a non-Boolean estimation of effectiveness.

[0041] In the embodiment shown in FIG. 1, a labeling step 15 uses the predicted effectiveness to send generated ads to a database of positive example ads 74 for ads predicted to be effective or to a database of negative example ads 75 for ads predicted to be ineffective. Some embodiments store the example ads in a single database but tag each ad as being effective or ineffective. This is useful such as for training a binary classifier model.

[0042] Some embodiments store example ads in a database and associate a score with each ad, the score representing a non-Boolean value of its effectiveness. This is useful such as for training a regression model. Though not shown in FIG. 1, its implementation is readily apparent.

[0043] The embodiment shown in FIG. 1 performs a training step 16 to train the ad generator 11. In some embodiments, training happens in batches, rerun at periods from 1 hour to several weeks. Some embodiments use a continuous retraining process. The training periods and algorithms may be different for the ad generator 11 and effective ad discriminator 14. While the presenting of ads, in order to collect effectiveness labeled data, is limited to a rate that consumers are able to observe the ads, ad generation for the purpose of creating positive and negative examples can occur much more frequently and can be useful to improve the effectiveness of the ad generator 11 without waiting for consumer impressions.

[0044] Some embodiments include consumer profile info with the ad labeling in both the presented ads databases and example ads databases.

[0045] The ad generator 11, effective ad discriminator 14, and the training process for each creates a feedback loop that, because of the step of measuring and labeling presented ads continuously improves its effectiveness.

Product Info Decoder

[0046] A system with discriminator feedback training a generator and generated ads training a discriminator will tend to fall into a local extremum of generating the essentially the same ads regardless of the product info or consumer profile input. Because this is a local extremum, it is likely that it not the optimum ad for any given product or consumer profile. Furthermore, because people find stimulus attractive if it is surprising (to a moderate degree), generating essentially the same ads would render them as boring and therefore less effective.

[0047] One way to avoid boringness is by adding randomization at some point in the feedback system. Randomization without constraints will result in ads with interesting variations. However, the variations would be irrelevant to the products that they advertise. This would make the ads less effective than they could be.

[0048] Some embodiments include a product info decoder 17 that takes as input an ad produced by the ad generator 11 and the product info that was the input to generate the ad. Some embodiments of product info decoders 17 also take a consumer profile as input. The product info decoder extracts product info from ads and compares it to the source info that generated the ads. If the extracted info for an ad matches the source info for the ad, then the product info decoder labels the ad as acceptable. Some embodiments allow for imperfect matching and labeling with a degree of acceptability. In embodiments with a product info decoder 17, its acceptability output contributes to the example ad labelling function 15.

[0049] Some embodiments could have a product info decoder along the path of presentation, measuring, and labeling of ads 11. However, since the effective ad discriminator path is independent of presenting ads to consumers, it can run more frequently for faster training. Also, it is preferable not to present ads to consumers if the ads have a high probability of performing poorly or failing in a product info decoder.

[0050] Some embodiments train or contribute to the training of the product info decoder by using human data labelers. These can be specialists or non-specialist proxy consumers, such as in focus groups or other consumer research studies. Accordingly, labelers observe ads, observe product descriptions, and indicate whether or to what degree it is clear that the ad is for the described product. Similarly, for a product info decoder that uses consumer profile input, it would be useful to segregate human labeler by their consumer profiles.

Humor

[0051] Some products or advertisers or ad campaigns succeed by using humor. It is possible to train a product info decoder by human labelers with humor as a desirable output feature. It is also possible to train on generated ads targeting an intentional mismatch between source product info and decoded product info. For example, if product info is related to shampoo for humans and the decoded product info is shampoo for non-human animals then the product info decoder labels the ad as successful.

[0052] One way to implement humor detection is to treat product info as an embedding and measure cosine distance between source product info and decoded product info. By using a threshold of maximum acceptable cosine distance for labeling an ad as a positive example, it is possible to allow humor that is recognizable as humor. Creating a threshold of minimum acceptable cosine distance for labeling an ad as a positive example ensures a certain degree of humor.

[0053] Another approach to creating acceptable humor is to mark only certain parameters of product info as ones appropriate for variation. A similar compatible approach is to create acceptable ranges or specific sets of allowable values of product info attributes.

Ad Structure

[0054] The embodiment of FIG. 1 and variations described above are applicable to generating ads with various application-appropriate types of elements. Some types of elements are audible, musical, spoken, graphic, textual, animated, and physically moving. These different types of elements are applicable to various human-machine interfaces that support presenting ads. Combinations of types of elements are appropriate for some interfaces. For example, a billboard near a road could have graphic and textual elements but not audible or musical elements. A radio ad could have musical and spoken elements but not graphic, textual, or animated elements.

[0055] FIG. 2 shows a mobile handset embodiment 21 with a screen portion 22. The screen is useful to display apps such as an app for reading messages 23. The screen portion 22 further displays an ad 24. FIG. 2 shows an example of an ad for a toothpaste of a specific brand called Star Paste. The ad comprises a graphic element 25 that is a product photo. The ad also comprises a first textual element 26 that is the product name rendered with a particular font. For consistency of trademark usage, a product name and font might be better rendered as a graphic element. The ad further comprises a second textual element 27 that has poetic words to make the product seem appealing to consumers.

[0056] FIG. 3 shows a robot assistant embodiment 31. It interacts with a human master 32. The robot assistant is anthropomorphic. Accordingly, it provides ads with spoken elements. Whereas text has fonts, speech elements have text-to-speech (TTS) synthesis voice parameters.

[0057] FIG. 4 shows an ad data structure within a computer file 41. The structure shown includes element types. It is possible for ads to have arrays of multiple instances of element types.

[0058] One type of element is displayed text. Such an element may have parameters such as font type, size, and style.

[0059] Another possible type of ad element are keywords. These can be useful for various purposes, including supporting conventional ad ecosystems that use keyword-based bidding or placement choosing.

[0060] Another type of element is vector graphics. This can include objects with sub parameters such as size, color, orientation, transparency, shading, patterns, etc. One example of a standard format for vector graphics is the Scalable Vector Graphics standard (SVG).

[0061] Another type of element is raster graphics. Ads generally require small amounts of data. They are, essentially, input vectors for presentation rendering and for a discriminator. Raster graphics, such as bitmaps or compressed bitmaps in formats such as Joint Picture Experts Group (JPEG) format usually comprise relatively large amounts of data. Some embodiments train discriminators on graphics. These operate like object recognition discriminators such as convolutional neural networks (CNNs). However, some embodiments keep ads small and strictly parameter based. Such an approach reduces model sizes, which improves training speed, storage space, bandwidth, and other key criteria. To support such an embodiment and still support raster graphic elements, such embodiments use pointers to raster art files. The renderer for presenting ads to consumers uses the pointer to lookup and read the graphic element.

[0062] Another type of element is text for the presentation step to render as synthesized speech audio. Like text has fonts, speech has TTS parameters, which are parameters of spoken text elements.

[0063] Some ads, such as ones for robot assistants or speech-based virtual assistants comprise conversation script elements. These provide for conditional responses to the consumer's behavior. For example, a toothpaste ad can initiate a conversation with a question such as, “have you heard about the new Star brand toothpaste?” The ad can then direct the presentation interface to recognize words from a target consumer. If the words mention breath, the ad instructs a spoken response about fresh breath and if the words mention teeth, the ad instructs a spoken response about healthy teeth.

[0064] One type of ad element is vector music. As displayed text has fonts and speech has TTS parameters, vector music has instrument parameters. One standard format for vector music is the Musical Instrument Digital Interface (MIDI) standard.

[0065] One type of ad elements is audio clips. For the same reasons that raster graphics might be embedded or might be linked by a pointer, so, too, can audio clip elements. Audio clips are useful for musical segments such as product jingles or sound effects.

[0066] One type of ad element is vector animation. This is appropriate for human-machine interfaces with a visual component. One industry standard format for vector animations is Adobe Flash. Another standard is Hyper-Text Markup Language (HTML) 5.

[0067] One type of ad element is video clips. For the same reasons that raster graphics and audio clips might be embedded or might be linked by a pointer, so, too, can video clip elements. Video clips are useful for pixel-based moving visual segments.

[0068] One type of ad element is a set of one or more action command. This is appropriate for human-machine interfaces that move. For example, assistant robots that move objects around or anthropomorphic robots that nod their head or shrug their shoulders.

[0069] Some embodiments include product profiles within ads. This can support ad inventory management and sales and can provide criteria for ad selection algorithms. There are various ways to represent product profiles, depending on the application. One conceivable way is to use World Intellectual Property Organization (WIPO) trademark classification codes. One way is to use embedding vectors with dimensions corresponding to attributes of advertised products. System designers can define embedding space dimensions. It is also possible to define an embedding space using machine learning from correlations between product profiles and ad elements in ads found to be effective or ineffective.

[0070] Furthermore, include a product profile within an ad is one way to provide the source product to the product info decoder for comparison to decoded product info decoded from generated ad elements.

Vectorizing

[0071] CNNs or equivalent models can be effective on elements with large amounts of data such as raster graphics, audio clips, or video clips. However, their training is much slower than training on relatively small purely vector-based (parametric) ad representations. As a result, vectorizing digitized elements can improve results. Some examples of vectorization are optical character recognition (OCR) to recognize text from raster graphics; object recognition to recognize semantically meaningful objects within raster graphics, video object recognition in video clips, music acoustic profiling from audio clips, and automatic speech recognition (ASR) to recognize words spoken in audio clips.

[0072] For systems that are purely parametric, vectorizing of content can be useful for training from successful historic ad campaigns such as classic ones from Coca-Cola, McDonalds, Nike, and Marlboro. Vectorizing is also useful for training on ads generated by incompatible systems.

Effective Ad Discriminator

[0073] An effective ad discriminator is a model that estimates probabilities of ads being effective. It can use, for example, a classification or a regression algorithm. Various ways are possible to measure effectiveness of an ad. A simple way is to measure engagement such as a click-through rate for visual ads or query (“tell me more”) rates for natural language conversational ads. A more accurate way to measure effectiveness is conversion rates. That considers whether the engagement caused a consumer to make a purchase. Ads can also be effective for changing feelings about products even if not causing a short-term measurable conversion. Another way of measuring effectiveness is to measure non-purchase behavior of consumers that indicates their feelings. Numerous measurable parameters can indicate ad effectiveness.

[0074] Not all consumers are the same. For any given ad, an effectiveness prediction is most accurately conditioned by consumer profile. For example, ads are usually most effective when they show people that have a similar age and race as the consumer. For another example, ads are most effective when they use words in a language that a consumer understands. Therefore, training an effective ad discriminator gives best accuracy when trained not only on the vectors of ads but vectors relating to the consumers on whom training data relies. Some example vectors are age, gender, race, home location, and income.

[0075] Furthermore, the effectiveness of ads depends on context. For example, ads containing food items are more effective at times of day when consumers are hungry. Ads with agricultural themes are more effective in rural locations. Ads with children are more effective when placed amid content having family-related words than amid content having business-related words.

[0076] FIG. 5 shows an ad metadata structure within a computer file 51. The metadata corresponds to a single ad after presenting it a multiplicity of times. The metadata comprises arrays of metadata for positive examples representing presentations of the ad that were successful and a similar array of negative examples representing presentations of the ad that were unsuccessful. Each array comprises consumer profile vectors including values known for age, gender, and home location. Each array also comprises context info vectors including time and location of the ad presentation. The context info vectors also comprise an unordered set of recognized words spoken either to the consumer or by the consumer within a window of time before the ad presentation. Some embodiments use multiple windows of time.

[0077] Some embodiments also store a bid amount as metadata for each ad. This is an indicator of the value of the corresponding ad to the advertiser, which publishers can use to optimize training for maximizing ad publishing revenue.

[0078] FIG. 6 shows an effective ad discriminator training process according to an embodiment. The method trains the discriminator on real ads and resulting consumer behavior. The ad presentation system presents ads from a database 61. From the presentation, the system compiles a database of metadata about successful and unsuccessful presentations of the ad 62. The system labels the ads 63 as either effective or ineffective and stores effective ones in a database 64 and ineffective ones in another database 65. The database comprises not just ad vectors, but a vector for each presentation of an ad. Each stored vector comprises both the vectors of a presented ad content and the metadata resulting from the presentation.

[0079] The system uses a model training process 13. Many methods of training models such as deep neural networks are well known. The training products an effective ad discriminator model 14.

[0080] Some systems present ad campaigns. Those are groups of ads that, cumulatively, serve to persuade consumers more effectively than repeated presentations of a single ad. For ad campaigns, a recurrent neural network would be appropriate, where the effectiveness model is dependent on previously presented or previously engaged ads.

[0081] One way to initially bring-up a system is to train on historical ads that are not ones that the system generated itself.

Ad Generator

[0082] Training an ad generator simply based on training from displayed ads would have at least two problems. One problem is that it would be slower than necessary since there is a limit to the rate at which a consumer can absorb ads. Another problem is that the system would present large numbers of ineffective ads while learning to present effective ones, and some ineffective ads, especially in large numbers, would be detrimental to their advertisers' brands. It is necessary to train an ad generator from large corpora of positive and negative example ads without presenting them to consumers.

[0083] FIG. 7 shows a system process for doing so. An initially trained ad generator 11 takes in a test vector. The text vector comprises a product description and ad metadata such as a user profile vector and context vector. Ad generator 11 generates large numbers of ad vectors. For each on, an effective ad discriminator 14 and product info decoder 17 indicate the likely effectiveness of the ad and whether its product is discernable. A labeling step 15 uses those results to sort ads and their metadata vectors into databases of positive examples 74 and negative examples 75. In a training step 16, the system proceeds to retrain or train a new ad generator 11 to produce more effective ads.

[0084] With a well-trained effective ad discriminator 14, random test vectors of product descriptions, consumer profiles, and presentation contexts can yield a well-trained ad generator that can produce effective ads for a wide range of products across a wide range of consumers in a wide range of contexts. Test vectors can be created randomly or in a directed-random way to train for especially common or desirable products, consumers, or contexts. A well-trained system can instantaneously generate, for any product, an optimal ad for a given consumer in a given context.

Personalization

[0085] In various embodiments, consumer profiles comprise info such as gender, age, ethnicity, current location, and location history.

[0086] A consumer profile can also comprise a word usage embedding vector that represents an average, mean, or other statistical representation of the cumulation of recognized words expressed by the consumer in an embedding space.

[0087] A consumer profile can comprise an array of such vectors organized in ways such as by time or by location or both.

The Cloud

[0088] FIG. 8 shows a consumer 32 communicating with a human interface device 81 coupled, as a client, through a network 82 to an ad server 83 according to an embodiment. The ad server 83 communicates ads over network 82 to the client 81, which presents the ad to the consumer 32.

[0089] Some examples of a consumer interface device 81 are mobile phones, home music players, automobiles, billboards, personal computers, augmented reality glasses, robots, and avatars. Some examples of a network 82 are the Internet, a local area network (LAN), a Bluetooth connection, and a 5G mobile network. Some examples of an ad server 83 are serverfs in cloud data centers, subsystems built into vehicles, and subsystems of portable devices.

CRMs

[0090] FIG. 9A shows an example non-transitory computer readable medium 91 that is a rotating magnetic disk. Data centers commonly use magnetic disks to store data and code comprising instructions for server processors. Non-transitory computer readable medium 91 stores code comprising instructions that, if executed by one or more computers, would cause the computer to perform steps of methods described herein. Rotating optical disks and other mechanically moving storage media are possible.

[0091] FIG. 9B shows an example non-transitory computer readable medium 92 that is a Flash random access memory (RAM) chip. Data centers commonly use Flash memory to store data and code for server processors. Mobile devices commonly use Flash memory to store data and code for processors within system-on-chip devices. Non-transitory computer readable medium 92 stores code comprising instructions that, if executed by one or more computers, would cause the computer to perform steps of methods described herein. Other non-moving storage media packaged with leads or solder balls are possible.

[0092] Any type of computer-readable medium is appropriate for storing code comprising instructions according to various embodiments.

The SoC

[0093] FIG. 10A shows the bottom side of a packaged system-on-chip device 100 with a ball grid array for surface-mount soldering to a printed circuit board. Various package shapes and sizes are possible for various chip implementations. System-on-chip (SoC) devices control many embedded systems and IoT device embodiments as described herein.

[0094] FIG. 10B shows a block diagram of the system-on-chip 100. It comprises a multicore cluster of computer processor (CPU) cores 101 and a multicore cluster of graphics processor (GPU) cores 102. The processors connect through a network-on-chip 103 to an off-chip dynamic random-access memory (DRAM) interface 104 for volatile program and data storage and a Flash interface 105 for non-volatile storage of computer program code in a Flash RAM non-transitory computer readable medium. system-on-chip device 100 also has a display interface 106 for displaying a graphical user interface (GUI) and an I/O interface module 107 for connecting to various I/O interface devices, as needed for different peripheral devices. The I/O interface enables sensors such as touch screen sensors, geolocation receivers, microphones, speakers, Bluetooth peripherals, and USB devices, such as keyboards and mice, among others. system-on-chip device 100 also comprises a network interface 108 to allow the processors to access the Internet through wired or wireless connections such as Wi-Fi, 3G, 4G long-term evolution (LTE), 5G, and other wireless interface standard radios as well as ethernet connection hardware. By executing instructions stored in RAM devices through interface 104 or Flash devices through Flash interface 105, the CPU cores 101 and GPU cores 102 perform steps of methods as described herein.

The Server

[0095] FIG. 11A shows a rack-mounted server blade multi-processor server system 110 according to some embodiments. It comprises a multiplicity of network-connected computer processors that run software in parallel.

[0096] FIG. 11B shows a block diagram of the server system 110. It comprises a multicore cluster of computer processor (CPU) cores 111 and a multicore cluster of graphics processor (GPU) cores 112. The processors connect through a board-level interconnect 113 to random-access memory (RAM) devices 114 for program code and data storage. Server system 110 also comprises a network interface 118 to allow the processors to access the Internet. By executing instructions stored in RAM devices through interface 114, the CPU cores 111 and GPU cores 112 perform steps of methods as described herein.

General Boilerplate

[0097] Practitioners skilled in the art will recognize many modifications and variations. The modifications and variations include any relevant combination of the disclosed features.

[0098] Various embodiments are methods that use the behavior of either or a combination of humans and machines. Method embodiments are complete wherever in the world most constituent steps occur. Some embodiments are one or more non-transitory computer readable media arranged to store such instructions for methods described herein. Whatever machine holds non-transitory computer readable media comprising any of the necessary code holds a complete embodiment. Some embodiments are physical devices such as semiconductor chips; hardware description language representations of the logical or functional behavior of such devices; and one or more non-transitory computer readable media arranged to store such hardware description language representations.

[0099] Descriptions herein reciting principles, features, and embodiments encompass both structural and functional equivalents thereof. Elements described herein as coupled have an effectual relationship realizable by a direct connection or indirectly with one or more other intervening elements.

[0100] Examples shown and described use certain spoken languages. Various embodiments operate, similarly, for other languages or combinations of languages. Examples shown and described use certain domains of knowledge. Various embodiments operate similarly for other domains or combinations of domains.

[0101] Some embodiments are screenless, such as an earpiece, which has no display screen. Some embodiments are stationary, such as a vending machine. Some embodiments are mobile, such as an automobile. Some embodiments are portable, such as a mobile phone. Some embodiments comprise manual interfaces such as keyboard or touch screens. Some embodiments comprise neural interfaces that use human thoughts as a form of natural language expression.