The present disclosure relates to an object selection system that accurately detects and optionally automatically selects user-requested objects (e.g., query objects) in digital images. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of a query object. In particular, the object selection system can identify both known object classes as well as objects corresponding to unknown object classes.

The present disclosure relates to an object selection system that accurately detects and optionally automatically selects user-requested objects (e.g., query objects) in digital images. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of a query object. In particular, the object selection system can identify both known object classes as well as objects corresponding to unknown object classes.

A computer-readable recording medium has stored a program that causes a computer to execute a process including: generating a trained model that includes performing machine learning of a 1st_model based on a 1st_output value that is obtained when a 1st_image is input to the 1st_model in response to input of training data containing pair of the 1st_image and a 2nd_image and containing a 1st_label indicating which of the 1st and 2nd_image has captured greater movement of muscles of facial expression of a photographic subject, a 2nd_output value obtained when the 2nd_image is input to a 2nd_model that has common parameters with the 1st_model, and the 1st_label; and generating a 3rd_model that includes performing machine learning based on a 3rd_output value obtained when a 3rd_image is input to the trained model, and a 2nd_label indicating of movement of muscles of facial expression of a photographic subject captured in the 3rd_image.

A computer-readable recording medium has stored a program that causes a computer to execute a process including: generating a trained model that includes performing machine learning of a 1st_model based on a 1st_output value that is obtained when a 1st_image is input to the 1st_model in response to input of training data containing pair of the 1st_image and a 2nd_image and containing a 1st_label indicating which of the 1st and 2nd_image has captured greater movement of muscles of facial expression of a photographic subject, a 2nd_output value obtained when the 2nd_image is input to a 2nd_model that has common parameters with the 1st_model, and the 1st_label; and generating a 3rd_model that includes performing machine learning based on a 3rd_output value obtained when a 3rd_image is input to the trained model, and a 2nd_label indicating of movement of muscles of facial expression of a photographic subject captured in the 3rd_image.

A system and method for training a text report identification machine learning model and an image identification machine learning model, including: initially training a text report machine learning model, using a labeled set of text reports including text pre-processing the text report and extracting features from the pre-processed text report, wherein the extracted features are input into the text report machine learning model; initially training an image machine learning model, using a labeled set of images; applying the initially trained text report machine learning model to a first set of unlabeled text reports with associated images to label the associated images; selecting a first portion of labeled associated images; re-training the image machine learning model using the selected first portion of labeled associated images; applying the initially trained image machine learning model to a first set of unlabeled images with associated text reports to label the associated text reports; selecting a first portion of labeled associated text reports; and re-training the text report machine learning model using the selected first portion of labeled associated text reports.

A system and method for training a text report identification machine learning model and an image identification machine learning model, including: initially training a text report machine learning model, using a labeled set of text reports including text pre-processing the text report and extracting features from the pre-processed text report, wherein the extracted features are input into the text report machine learning model; initially training an image machine learning model, using a labeled set of images; applying the initially trained text report machine learning model to a first set of unlabeled text reports with associated images to label the associated images; selecting a first portion of labeled associated images; re-training the image machine learning model using the selected first portion of labeled associated images; applying the initially trained image machine learning model to a first set of unlabeled images with associated text reports to label the associated text reports; selecting a first portion of labeled associated text reports; and re-training the text report machine learning model using the selected first portion of labeled associated text reports.

Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.

A positioning system and a method based on neural network models are provided. The positioning method includes collecting WI-FI® fingerprint data; configuring a computing device to receive the WI-FI® fingerprint data, and the computing device includes a processor and a database storing positioning map data and a group of neural network models including a global positioning model, a coarse positioning model and a fine positioning model; configuring the processor to input the WI-FI® fingerprint data and perform the following steps: estimating a global coordinate through the global positioning model; obtaining the corresponding coarse positioning model from a corresponding primary sub-region to estimate an estimated coarse coordinate of a current position; estimating a plurality of estimated fine coordinates of the current position from the corresponding fine positioning model; and performing a merging process on the estimated fine coordinates to generate a final coordinate.

A positioning system and a method based on neural network models are provided. The positioning method includes collecting WI-FI® fingerprint data; configuring a computing device to receive the WI-FI® fingerprint data, and the computing device includes a processor and a database storing positioning map data and a group of neural network models including a global positioning model, a coarse positioning model and a fine positioning model; configuring the processor to input the WI-FI® fingerprint data and perform the following steps: estimating a global coordinate through the global positioning model; obtaining the corresponding coarse positioning model from a corresponding primary sub-region to estimate an estimated coarse coordinate of a current position; estimating a plurality of estimated fine coordinates of the current position from the corresponding fine positioning model; and performing a merging process on the estimated fine coordinates to generate a final coordinate.

Disclosed is a computer-implemented method of transmitting identification information of a medical instrument. The method encompasses comparing a digital image of an instrument tray and an instrument to a digital image of just the instrument tray to determine the identity of the instrument. A characteristic geometry such as its envelope is assigned to the instrument, and a characteristic quantity of the envelope such as its aspect ratio may be used to identify the instrument. Based on determining, from the image of the instrument and the instrument tray, the relative position between those two entities, the method determines whether the instrument has been taken from the instrument tray, and accordingly instructs a medical computing system about this determination. The medical computing system may then determine whether the correct instrument has been taken from the instrument has been taken from the instrument tray, for example by comparison with medical procedure planning data.