The disclosed systems evaluate user responses to various potential meal-kit menu designs and isolate variables having an impact on customer conversion, enabling a meal-kit company to select more effective meal-kit menus and to avoid waste associated with inaccurately predicting demand for meal-kits.

An online system measures performance of content presented to a plurality of identifiable and non-identifiable individuals based on matching user identifying information included in data describing presentation of the content and data describing performance of an action associated with the content. To reduce measurement inaccuracy resulting from incomplete matching of user identifying information associated with non-identifiable individuals, the online system generates models to extrapolate data describing an amount of unique individuals presented with the content, an amount of unique individuals who performed an action associated with the content, and an amount of unique individuals who performed the action associated with the content attributable to presentation of the content by a content publisher. The models are applied to data collected by the online system describing presentation of the content and performance of actions associated with the content. Metrics describing performance of the content are generated based on the models.

Methods and systems for processing viewability metrics are provided. A method includes receiving, by a processor of an online advertising platform, online advertisement viewability metrics collected by a system of an online content publisher. The online advertisement viewability metrics indicate whether a user of a user device viewed an online advertisement provided to the user device. The method further includes dividing, by the processor of the online advertising platform, the received online advertisement viewability metrics into a first online advertisement viewability metric set and a second online advertisement viewability metric set. The method further includes providing, by the processor of the online advertising platform, the first online advertisement viewability metric set to a system of a third-party analytics provider.

Systems and methods are provided for discovering advertisements on publisher web pages and for determining whether the advertisements are obscured by other content on the web pages. An advertisement tracking and discovery system may use web crawler applications to explore multiple publisher websites. The web crawler applications may gather advertisement data that includes information associated with detected advertisements, placement pathways by which the advertisements were placed on the publisher web pages, and obscuration information for the detected advertisements. Obscured advertisements that have been detected may be ignored during advertisement tracking and discovery operations or obscuration information may be stored and provided with other advertisement information for the obscured advertisements. The advertisement data may be aggregated and formatted to be provided to a user. The provided data may include advertisement screenshots and web page screenshots containing the advertisements.

An ad server logs web traffic regarding web page traffic by users. The web traffic is analyzed to determine web activity indicating that a user is being paid to fake legitimate web traffic. These users are referred to as abusive users. The web traffic is also analyzed to determine web activity indicating that a web page is artificially boosting its own traffic in order to earn undeserved advertising revenue. These web pages are referred to as honeypot web pages. Scores are determined for both users and web pages, where the scores indicate the likelihood that the associated user or web page is an abusive user or a honeypot web page, respectively. The ad server uses these scores to determine an amount to bid on future impressions to those users and web pages.

An influence system for predicting advertisement impact for audience selection. An advertising probe campaign is operated by sending an advertisement to each entity in a treatment group of entities. A control group of entities which excludes the treatment group entities is selected and no campaign advertising content is sent to the treatment group entities. An influence model is created by comparing features of the treatment group converters to features of the control group converters. An individual frequency cap is selected for each entity that is a candidate for the advertising campaign based on a result of applying the influence model to the features of the candidate entity. The entity may be selected to receive an advertisement based on the individual frequency cap. Some embodiments are integrated with a real time bidding (RTB) exchange and a bid response may be configured based on the results of applying the influence model.

An information processing device applies a machine learning model to content for a specific thing for sale to generate changed content in which a predetermined portion other than the specific thing for sale in the content is changed, provides a user with the changed content having been generated, and applies a machine learning model to content for another thing for sale and changes a predetermined portion other than the other thing for sale in accordance with an action of the user with respect to the changed content in the content for the other thing for sale to generate changed content for the other thing for sale.

An example apparatus includes means for identifying a pattern in first cookies received via the Internet and stored in a storage device, the first cookies corresponding to devices accessing media via the Internet, means for determining an error between a first demographic estimate based on a potential pattern in the first cookies and a second demographic estimate based on user identifiers corresponding to the devices and received via the Internet, the means for determining the error to determine the potential pattern to be the pattern when the error is less than a threshold, means for accessing impression information received from a database proprietor via the Internet, the impression information including second cookies, and means for identifying a subset of the second cookies that are associated with a same person based on the pattern, the means for identifying the subset to associate impressions corresponding to the identified subset with the same person.

A system including a server of an information publishing subsystem to be accessed into an access subsystem. The access subsystem, according to the accessed information publishing system, provides an effective information publishing system list to a data object placement subsystem. The data object placement subsystem selects a target information publishing subsystem from the effective information publishing subsystem list for placement of data object information. A client terminal of the information publishing subsystem receives the data object list corresponding to the information publishing subsystem from the access subsystem, generates the virtual item according to the data object information, uses the virtual item to send information, and provides corresponding data object information when the data object corresponding to the virtual item is operated. The present disclosure enriches the placement channels of the data objects, and improves the click through rate and conversion rate of the data object.

At least one aspect is directed to determining an estimate of an intersection of user identifiers in a first set of user identifiers and a second set of user identifiers. The first and second sets of user identifiers can be populated with user identifiers that have interacted with the same content item or content item campaign. Estimates of intersections of the first and the second sets can be determined based on a binomial vector approach, a vector of counts approach, or a hybrid approach. The binomial vector approach generates vectors based on k hashes of each user identifier in the first set and summing the vectors to generate a first vector. The intersection can be determined based on a dot product of the first vector and a second vector similarly generated from the second set of user identifiers.