Systems and methods provide a customer engagement platform experimentation framework. Experiments are created to determine which variation of a particular promotion (i.e., an experiment) is most effective. Initially, users are split into buckets that define whether a particular user is in a control group or a treatment group for a particular experiment. Users are assigned into buckets differently for unrelated experiments and similarly for related experiments. This ensures the users are in either the control group or the treatment group for all related experiments. Key performance indicators (KPIs) of users that experienced the experiment are compared with KPIs of users that did not experience the experiment. An assessment of the experiment can then be provided based on the compared KPIs.

An online system identifies a candidate content item eligible for presentation to a viewing user of the online system, in which the candidate content item includes media (e.g., an image, a video, etc.). The online system identifies one or more media attributes for the media, such as color saturation, tone, brightness, sharpness, contrast, etc. The online system also predicts a value of a performance metric for the candidate content item that indicates a likelihood of user engagement with the candidate content item by the viewing user. For each modification that may be made to a media attribute, the online system predicts a change to the value of the performance metric. Based on the predicted change, the online system determines an optimal set of media attributes associated with a maximum predicted value of the performance metric. The online system modifies the media based on the optimal set of media attributes.

A method and apparatus for spawning and management of online surveys is provided. An administration and redirector subsystem (ARS) is used manage ongoing surveys for a plurality of sites. Surveys can be added to or removed from individual sites using the ARS. The ARS also tracks statistics for individual surveys. Redirector tags invite users to participate in surveys and track user survey participation through cookies. The system can be used to conduct advertisement effectiveness studies. Tracker tags and corresponding cookies are used to track users who have viewed content containing the tag. Each study is issued a unique content tag, but uses the same redirector tag.

A scene of a portion of a simulated environment within a field of view of a virtual camera from a virtual camera point of view (camera POV) is displayed on a display. The scene displayed is changed during a user's interaction with the simulated environment. A revised virtual camera path is generated that places more of an advertising target within the field of view of the virtual camera. The advertising target is characterized by a field of attraction of the camera POV.

A system and method for selecting portions of content, and media content sets to be delivered over a packet based network is provided. The selected portions of content (such as a web page) may be keywords such as a word or a group of words. The media content sets may include one or more of video content (that may include an audio portion), audio content, graphics, images, or other multimedia or single media content. The media content set may include advertising, informational, business, educational, governmental, institutional, customized (e.g., personalized) content and/or other content. According to an example embodiment, the present invention provides a heuristic model employing a piece-wise linear optimization function to select keywords and the media content sets for delivery by a media delivery provider.

A method of presenting information to a party through a messaging application is described. Responsive to receipt of a communication from a party (e.g., the first user), a reply is sent. The communication and the reply is presented in an interface to the sender. The messaging system determines matching content that is relevant to one or both of the communication and the reply and determines a quality of the match. Determining the quality of the match may include determining a score for an advertisement based on the advertisement's responsiveness to content identified in the reply message that was sent. Based on a determination that the quality is above a threshold, the matching content is presented along with the communication and the reply.

Methods and apparatus for conducting test promotions in a highly scalable and cost-effective manner using controlled experiment methodology are disclosed. Test promotions of interest are presented to visitors of a website in a controlled experiment manner wherein each page presents one test promotion of interest among other promotions. The other promotions presented in the same page are kept constant in the pages. The visitors' responses with respect to the test promotions of interest are then recorded and analyzed to determine the performance of each presented test promotion of interest.

In many embodiments, a system comprising one or more processor and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform: determining a set of questions associated with a campaign by extracting text from one or more advertisements; generating vector embeddings of one or more online activities of a user; receiving a search query from a graphical user interface of a computing device of the user; determining a second question from the set of questions to present to the user based on respective relevance probabilities; determining a confidence score associated with the second question; and when the confidence score associated with the second question exceeds a second predetermined threshold, presenting the second question to the user via the graphical user interface. Other embodiments of related methods and systems are also provided.

Methods and apparatus for determining a unique audience exposed to media while reducing memory resources of a computing device are disclosed herein. An example apparatus includes means for logging a plurality of impressions based on impression requests from a plurality of client devices, the plurality of impressions corresponding to media accessed at the client devices; means for obtaining counts, the obtaining means to: obtain a count of demographic impressions logged by a database proprietor; and obtain a count of registered users of the database proprietor exposed to the media; and means for determining a unique audience size by: multiplying a count of the plurality of impressions by a square of the count of the registered users to generate a product; dividing the product by the count of the demographic impressions to generate a quotient; and determining the unique audience size based on a square root of the quotient.

Techniques are provided for k-best option identification of options subject to a supplied tolerance. One technique includes: sampling the options for a first period on a plurality of computers; computing an average and a sample count for each option based on the sampling; splitting the options into a highest group and a lowest group based on the computed averages; selecting a weakest one of the highest group (option A) and a strongest one of the lowest group (option B); and deciding whether or not to terminate based on the supplied tolerance and the selecting of options A and B. In some cases, the technique further includes outputting the highest group and terminating in response to a termination decision; otherwise continue with sampling options A and B for a next period; and updating the computed average and the sample count for options A and B based on corresponding next period sampling.