Embodiments herein describe providing feedback to a shopper at a POS system using a computer vision system. Many items at a store may lack barcodes or other identifying marks such as produce. The shopper may have to perform an action to identify the item to the POS system. The computer vision system can double check the identify provided by the shopper to reduce mistakes and deter nefarious actors. If the computer vision system cannot independently confirm that the item being purchased matched the identity provided by the shopper, the POS system can display a graphical user interface (GUI) that includes an image of the item captured by the computer vision system along with identification data of the item identified by the shopper. This gives the shopper a chance to correct any mistakes.

Computer vision grouping recognition is provided by receiving training images that include unpackaged items; identifying, by a computer vision model, a candidate identities for unpackaged items in a given training image; receiving, from a human user, a selected identity for the unpackaged item as feedback for the candidate identity; constructing a confusion matrix tallying matches and mismatches between candidate identities and the selected identities as analyzed across the training images for each unpackaged item; identifying at least one product category that includes at least a first unpackaged item and a second unpackaged item that the confusion matrix indicates as being misidentified for each other by the computer vision model; and reconfiguring the computer vision model to identify the product category instead of the first unpackaged item or the second unpackaged item when analyzing a given image including one or more of the first unpackaged item or the second unpackaged item.

This disclosure describes an automated process for training an ML model used by a computer vision system in a point of sale (POS) system to recognize a new item. Instead of relying on a manual process performed by a data scientist, the automated process can use images of a new (i.e., unknown) item captured at one or more POS systems to then retrain the ML model to recognize the new item. That is, the images of the item are used to retrain the ML model and to test the accuracy of the updated ML model. If the updated ML model can confidently identify the new item, the updated ML model is then used by the computer vision system to identify items at the POS system.

A non-transitory computer-readable recording medium having stored therein an information processing program that causes a computer to execute a process including specifying, a first image area which includes a person who visits a store and a second image area which includes a container containing commodity products, specifying, positional relationship between each of a plurality of persons belonging to a first group and a specific machine placed in the store, identifying, from among the plurality of persons belonging to the first group, a first person who use the specific machine based on the positional relationship, specifying, a second person who carries the container in which the commodity products are contained from among the plurality of persons belonging to the first group, and performing authentication processing of the commodity products contained in the container carried by the second person has been completed.

A non-transitory computer-readable recording medium stores therein an information processing program that causes a computer to execute a process including, specifying, from a plurality of images taken by one or more camera devices, a person who visits a store and a first object that contains one or more first commodity products and is used by the person, specifying, from the plurality of images, a location of the person in the store, specifying, based on the location of the person and a location of each of a plurality of terminals in the store, a first terminal in which the person uses from among the plurality of terminals, associating the one or more first commodity products contained in the first object with the first terminal, and performing authentication processing of the one or more first commodity products contained in the first object by using the certificate information.

A mechanized store uses a mobile device to authenticate the user. Items removed from one or more displays of the mechanized store by the user are tracked and a list of items removed by the user is updated. The list of items removed is linked with an account of the user.

An information processing system includes a first imaging unit and a controller configured to perform processing relating to an image that is captured by the first imaging unit. The controller performs the processing based on an image captured at a first stage at which an object is put into a container and an image captured at a second stage at which the object is taken out of the container.

A system for generating a shopping checkout list of items selected by a shopper in a store including: in-store security cameras; a cart scanner and a cart analyzer configured to generate a shopping checkout list based on the data received from the cart scanner; and a shopping list builder (SLB) configured to record images from the security cameras of the shopper's activity in the store to form recorded images, wherein, when the SLB requires verification of an item in the generated shopping checkout list, the SLB is further configured to analyze the recorded images to determine selection of an item by a shopper to thereby verify the item on the shopping checkout list.

Provided is a method for performing accurate object recognition in a stable manner in consideration of changes in a shooting environment. In such a method, a camera captures an image of a shooting location where an object is to be placed and an object included in an image of the shooting location is recognized utilizing a machine learning model for object recognition. The method further involves: determining necessity of an update operation on the machine learning model for object recognition at a predetermined time; when the update operation is necessary, causing the camera to capture an image of the shooting location where no object is placed to thereby re-acquire a background image for training; and causing the machine learning model to be trained using a composite image of a backgroundless object image and the re-acquired background image for training as training data.

Apparatuses, systems, methods, and computer program products are presented for a propensity module based optimization. An apparatus comprises a processor and a memory that stores code executable by the processor to receive an electronic submission for a pass/fail interface, identify information from the electronic submission to suggest to a user for entering into an input field for the pass/fail interface prior to submitting the electronic submission to the pass/fail interface to reduce a likelihood that the electronic submission will be rejected at the pass/fail interface, determine the likelihood that the electronic submission will be accepted by the pass/fail interface, and submit the electronic submission to the pass/fail interface in response to the likelihood satisfying a threshold.