A configured machine performs image matching and retrieval of natural images that may depict logos. The machine generates and uses color-localized spatial masks, which may be computationally less expensive than spatial verification techniques. Key points are detected within images that form a reference database of images. Local masks are defined by the machine around each key point based on the scale and orientation of the key point. To utilize color information presented in logo images, ordered color histograms may be extracted by the machine from locally masked regions of each image. A cascaded index may then be constructed for both visual descriptors and color histograms. For faster matching, the cascaded index maps the visual descriptors and color histograms to a list of relevant or similar images. This list may then be ranked to generate relevant matches for an input query image.

Detection of near-duplicate images is important for detecting the reuse of copyrighted material. Some applications require the clustering of near-duplicates instead of the comparison to an original. Representing images as bags of visual words is the first step for our clustering approach. An inverted index points from visual words to all the images containing that visual word. In the next step, matches are geometrically verified in pairs of images that share a large fraction of their visual words. Geometric verification may use affine, perspective, or other transformations. The verification step provides a similarity measure based on the fraction of the matching image points and on their distributions in the compared images. The resulting distance matrix is very sparse because most images in the collection are not compared to each other. This distance matrix is used as input for modified agglomerative hierarchical clustering approach that can handle a sparse distance matrix.

Various aspects of the subject technology relate to systems, methods, and machine-readable media for composition aware image querying. A system may receive user input identifying a search query for content from a client device, where the user input indicates one or more queries assigned to one or more regions of a search input page. The system may generate a query vector for each query using a computer-operated neural language model. The system may compare the query vector to an indexed vector for each region of an image. The system may determine a listing of composition-specific images from a collection of images based on the comparison. The system may determine a ranking for each image in the listing of composition-specific images, and provide search results responsive to the search query to the client device. The search results may include a prioritized listing of the composition-specific images based on the determined ranking.

The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.

A framework is provided for associating dense images with topics. The framework is trained utilizing images, each having multiple regions, multiple visual characteristics and multiple keyword tags associated therewith. For each region of each image, visual features are computed from the visual characteristics utilizing a convolutional neural network, and an image feature vector is generated from the visual features. The keyword tags are utilized to generate a weighted word vector for each image by calculating a weighted average of word vector representations representing keyword tags associated with the image. The image feature vector and the weighted word vector are aligned in a common embedding space and a heat map is computed for the image. Once trained, the framework can be utilized to automatically tag images and rank the relevance of images with respect to queried keywords based upon associated heat maps.

Image information defining an image may be accessed. The image may include one or more salient objects. A saliency map may be generated based on the image information. The saliency map may include one or more regions corresponding to the one or more salient objects. The one or more regions may be characterized by different levels of intensity than other regions of the saliency map. One or more salient regions around the one or more salient objects may be identified based on the saliency map. A saliency metric for the image may be generated based on one or more of (1) sizes of the one or more salient regions; (2) an amount of the one or more salient regions; and/or (3) histograms within the one or more salient regions.

A method for processing a stream of video images to search for information therein, in particular detect predefined objects and/or a motion, comprising the steps of: a) supplying at least one attention map in at least one space of the positions and of the scales of at least one image of the video stream, b) selecting, in this space, points to be analyzed by making the selection depend at least on the values of the coefficients of the attention map at these points, at least some of the points to be analyzed being selected by random draw with a probability of selection in the draw at a point depending on the value of the attention map at that point, a bias being introduced into the map to give a non-zero probability of selection at any point, c) analyzing the selected points to search therein for said information, d) updating the attention map at least for the processing of the subsequent image, from at least the result of the analysis performed in c), e) reiterating the steps a) to d) for each new image of the video stream and/or for the current image on at least one different scale.

In a method for detecting and assessing reflections on a road (7), a camera (2) produces at least two digital images of at least one point (3) of the road, respectively from different recording perspectives (A, B) of the camera (2). Diffuse reflection and specular reflection of the road (7) are then detected by assessing differences in the appearances of the point (3) of the road in the at least two digital images using digital image processing algorithms. Road reflections are preferably assessed using an approximative approach. An item of road condition information is determined based on the detected reflection, preferably an item of road condition information which states whether the road is dry, wet, snow-covered or icy. Also provided are an apparatus (1) for carrying out the above-mentioned method, and a vehicle having such an apparatus.

If a lesion included in a specification target image is a texture lesion, a probability image calculation unit calculates a probability value indicating a probability that each of a plurality of pixels of the specification target image is included in a lesion area. An output unit calculates, as a candidate area, an area including pixels whose probability values are equal to or larger than a first threshold in a probability image obtained from the probability image calculation unit and, as a modification area, an area including pixels whose probability values are within a certain probability range including the first threshold. An input unit detects an input from a user on a pixel in the modification area. A lesion area specification unit specifies a lesion area on the basis of the probability image, the candidate area, the modification area, and user operation information.

The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.