A method includes receiving, from an image capture device in communication with the data processing hardware, image data for an area of interest of a user. The method also includes receiving a query from the user referring to one or more objects detected within the image data and requesting a digital assistant to discern insights associated with the one or more objects referred to by the query. The method also includes processing the query and the image data to: identify, based on context data extracted from the image data, the one or more objects referred to by the query; and determine the insights associated with the identified one or more objects for the digital assistant to discern. The method also includes generating, for output from a user device associated with the user, content indicating the discerned insights associated with the identified one or more objects.

A contextual query system is configured to perform operations that include: causing display of a graphical user interface at a client device, the graphical user interface including a display of image data that comprises a set of image features; generating a query based on the set of image features of the image data; accessing media content based on the query at a repository, the repository comprising a collection of media content; and causing display of a presentation of the media content within the graphical user interface at the client device.

A contextual query system is configured to perform operations that include: causing display of a graphical user interface at a client device, the graphical user interface including a display of image data that comprises a set of image features; generating a query based on the set of image features of the image data; accessing media content based on the query at a repository, the repository comprising a collection of media content; and causing display of a presentation of the media content within the graphical user interface at the client device.

Methods for retrieving images from a database are provided. A query image is obtained. A plurality of patches are extracted from the query image. A set of weightings is obtained according to a plurality of bases of a sparsity-based dictionary. The set of weightings includes a plurality of non-zero weightings. The patches are encoded with the set of weightings to obtain an encoding matrix. The database is searched based on the encoding matrix to retrieve the images corresponding to the query image. The database and the dictionary are dynamically updated to adapt to the query image encountered.

Methods for retrieving images from a database are provided. A query image is obtained. A plurality of patches are extracted from the query image. A set of weightings is obtained according to a plurality of bases of a sparsity-based dictionary. The set of weightings includes a plurality of non-zero weightings. The patches are encoded with the set of weightings to obtain an encoding matrix. The database is searched based on the encoding matrix to retrieve the images corresponding to the query image. The database and the dictionary are dynamically updated to adapt to the query image encountered.

Systems and methods of image searching include receiving content, receiving a request to select an image from content, selecting a plurality of items in the image, retrieving information about the selected item, and providing display data based on the retrieved information.

Systems and methods of image searching include receiving content, receiving a request to select an image from content, selecting a plurality of items in the image, retrieving information about the selected item, and providing display data based on the retrieved information.

Described is a system and method for enabling dynamic selection of a search input. For example, rather than having a static search input box, the search input may be dynamically positioned such that it encompasses a portion of displayed information. An image segment that includes a representation of the encompassed portion of the displayed information is generated and processed to determine an object represented in the portion of the displayed information. Additional images with visually similar representations of objects are then determined and presented to the user.

Described is a system and method for enabling dynamic selection of a search input. For example, rather than having a static search input box, the search input may be dynamically positioned such that it encompasses a portion of displayed information. An image segment that includes a representation of the encompassed portion of the displayed information is generated and processed to determine an object represented in the portion of the displayed information. Additional images with visually similar representations of objects are then determined and presented to the user.

Computer-implemented methods and systems are provided for curating medical images for a medical image database. An example method for curating involves operating a processor to obtain a query medical image, assign the query medical image to an initial set of related images, and generate a similarity indicator for each image of the initial set. The similarity indicator is representative of a measure of similarity between image data of that image and the initial set. The processor is operable to define an intermediate set from the initial set based on the similarity indicators; and generate a relevancy indicator for each image of the intermediate set. The relevancy indicator is representative of a measure of the relevancy between relevance of the image and the intermediate set. The processor is operable to define a curated set of images from the intermediate set based on the relevancy indicators.