An online system obtains risk scores determined by a machine learning model for a content item provided by a user of an online system for display to users of the online system, where the risk scores indicate the likelihood of content items violating a content policy. The online system uses the risk scores to determine sampling weights used to select content items for inclusion in a sampled subset of content items. The sampling weights are determined from risk score counts indicating the relative frequency of the obtained risk scores and impression counts indicating the number of times content items have been presented to the users of the online system. The online system presents the selected content items for evaluation by a human reviewer using a quality review interface. Using the results of the quality review, the online system determines quality performance metrics of the machine learning model.

A software licensing verification and installation system includes a client computing system in signal communication with a service provider computing system. The client computing system is configured to generate a request for at least one software product. The service provider computing system is configured to determine hardware information and software information of the client computing system, to receive a client license provided by the client computing system corresponding to the at least one software product, and to verify the client license based on the hardware information and software information.

A verifying method for software authorization under an offline environment is disclosed. The method includes: executing an application software in an electronic device; obtaining a serial number to input to the application software; decrypting the serial number to at least retrieve a requested using term, and a hard-drive identification time information of a hard-drive of the electronic device recorded at the time the serial number being applied; obtaining a power-on time of the hard-drive of the electronic device; computing a using time of the application software in the electronic device according to the hard-drive identification time information and the power-on time; determining whether an authorization to the application software is expired according to the using time and the requested using term; and disabling the application software if the authorization to the application software is expired.

A system and method for media content management include creating, via a digital vault, a container file comprising media content submitted by a user and content metadata; verifying, via the digital vault, a completeness of the content metadata associated with the media content in the container file; classifying, via the digital vault, the container file based on the completeness of the media content; and capturing, via the digital vault, event metadata when a second user gains access to the container file, the event metadata comprising identification of the second user, an activation timestamp, a duration of access, portions of the container file accessed, and changes to the container file.

A method for preventing digital content misuse can include detecting, by a client-side computing device, that the client-side computing device is paired to a viewing device such that, after being paired, the client-side computing device can cause digital content received from a remote server to be presented on a display of the viewing device; after detecting that the client-side computing device is paired to the viewing device, detecting, by the client-side computing device, that the client-side computing device has been unpaired from the viewing device; and in response to detecting that the client-side computing device has been unpaired from the viewing device, executing a remedial action.

Disclosed in some examples are methods, systems, devices, and machine-readable mediums which utilize digital tracking tags attached to data to monitor and/or control the data as it moves between applications and/or computing devices. The digital tracking tag may be embedded in the data (e.g., as a digital watermark) or associated with the data e.g., as metadata. In some examples, the digital tracking tag may include an address of a tracking database with which to record one or more events related to the data. For example, recipients, senders, or other participants in a data transfer event may register the data transfer event with the tracking database.

A system and method for performing licensing monitoring and compliance within a service provider platform are provided. The system comprises a memory and a processor configured to execute instructions stored within the memory. The system further comprises a central instance that executes on the processor and comprises a license repository containing licensing data for application components. The system further comprises a customer instance that includes a third-party application component installed within the customer instance from an application store. The system further includes a licensing module. The third-party application component is switchable between a monitor mode in which the licensing module reports usage of the third-party application component to the central instance and an enforcement mode in which the usage is controlled on the customer instance based on the license repository.

The invention is directed towards enabling data volume and data type based licensing of software in a distributed system of a plurality of remote and/or local nodes. The invention enables measuring and optionally restricting the use of software based on one or more provided licenses that restrict the amount and type of data that may be processed by the software. New and older licenses may be added together for a single, bulk entitlement for a given volume of data processing for one or all types of data. Different users in the same enterprise may combine license entitlements too. Also, a new license can be acquired repeatedly, without requiring the issuance of combined licenses by the issuing authority and/or the revocation of prior licenses.

A method for preventing digital content misuse can include detecting, by a client-side computing device, that the client-side computing device is paired to a viewing device such that, after being paired, the client-side computing device can cause digital content received from a remote server to be presented on a display of the viewing device; after detecting that the client-side computing device is paired to the viewing device, detecting, by the client-side computing device, that the client-side computing device has been unpaired from the viewing device; and in response to detecting that the client-side computing device has been unpaired from the viewing device, executing a remedial action.

According to an example aspect of the present invention, there is provided a method, comprising: receiving a cryptographic proof of use indicative of decoding of a first media content element by a user device (400), receiving a cryptographic first expected proof of use value (410), and controlling access to an encrypted second media content element associated with the first media content element on the basis of verification of the received cryptographic proof of use and the first expected proof of use value (420).