The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.

A sign language recognizer is configured to detect interest points in an extracted sign language feature, wherein the interest points are localized in space and time in each image acquired from a plurality of frames of a sign language video; apply a filter to determine one or more extrema of a central region of the interest points; associate features with each interest point using a neighboring pixel function; cluster a group of extracted sign language features from the images based on a similarity between the extracted sign language features; represent each image by a histogram of visual words corresponding to the respective image to generate a code book; train a classifier to classify each extracted sign language feature using the code book; detect a posture in each frame of the sign language video using the trained classifier; and construct a sign gesture based on the detected postures.

Independent subspace analysis (ISA) is used to learn (42) filter kernels for CLE images in brain tumor classification. Convolution (46) and stacking are used for unsupervised learning (44, 48) with ISA to derive the filter kernels. A classifier is trained (56) to classify CLE brain images based on features extracted using the filter kernels. The resulting filter kernels and trained classifier are used (60, 64) to assist in diagnosis of occurrence of brain tumors during or as part of neurosurgical resection. The classification may assist a physician in detecting whether CLE examined brain tissue is healthy or not and/or a type of tumor.

Disclosed herein is a medical system (100) comprising a memory (110) storing machine executable instructions (120). The memory (110) further stores a trained first machine learning module (122) trained to output in response to receiving a medical image (124) as input a saliency map (126) as output. The saliency map (126) is predictive of a distribution of user attention over the medical image (124). The medical system (100) further comprises a computational system (104). Execution of the machine executable instructions (120) causes the computational system (104) to receive a medical image (124). The medical image (124) is provided as input to the trained first machine learning module (122). In response to the providing of the medical image (124), a saliency map (126) of the medical image (124) is received as output from the trained first machine learning module (122). The saliency map (126) predicts a distribution of user attention over the medical image (124). The saliency map (126) of the medical image (124) is provided.

Various embodiments provide methods and systems for detecting one or more segments of an image that are related to a particular object in the image (e.g., a logo or trademark) and extracting at least one feature point, each of which is represented by one feature point descriptor, based at least upon a contour curvature of the one or more segments. The at least one feature point descriptor can be converted into one or more codewords to generate a codeword database. A discriminative codebook can then be generated based upon the codeword database and utilized to detect objects and/or features in a query image.

A method of assessing the quality of an retinal image (such as a fundus image) includes selecting at least one region of interest within a retinal image corresponding to a particular structure of the eye (e.g. the optic disc or the macula), and a quality score is calculated in respect of the, or each, region-of-interest. Each region of interest is typically one associated with pathology, as the optic disc and the macula are. Optionally, a quality score may be calculated also in respect of the eye as a whole (i.e. over the entire image, if the entire image corresponds to the retina).

A method of establishing a 3D saliency model based on 3D contrast and depth weight, includes dividing left view of 3D image pair into multiple regions by super-pixel segmentation method, synthesizing a set of features with color and disparity information to describe each region, and using color compactness as weight of disparity in region feature component, calculating feature contrast of a region to surrounding regions; obtaining background prior on depth of disparity map, and improving depth saliency through combining the background prior and the color compactness; taking Gaussian distance between the depth saliency and regions as weight of feature contrast, obtaining initial 3D saliency by adding the weight of the feature contrast; enhancing the initial 3D saliency by 2D saliency and central bias weight.

If a lesion included in a specification target image is a texture lesion, a probability image calculation unit calculates a probability value indicating a probability that each of a plurality of pixels of the specification target image is included in a lesion area. An output unit calculates, as a candidate area, an area including pixels whose probability values are equal to or larger than a first threshold in a probability image obtained from the probability image calculation unit and, as a modification area, an area including pixels whose probability values are within a certain probability range including the first threshold. An input unit detects an input from a user on a pixel in the modification area. A lesion area specification unit specifies a lesion area on the basis of the probability image, the candidate area, the modification area, and user operation information.

A method of establishing a 3D saliency model based on 3D contrast and depth weight, includes dividing left view of 3D image pair into multiple regions by super-pixel segmentation method, synthesizing a set of features with color and disparity information to describe each region, and using color compactness as weight of disparity in region feature component, calculating feature contrast of a region to surrounding regions; obtaining background prior on depth of disparity map, and improving depth saliency through combining the background prior and the color compactness; taking Gaussian distance between the depth saliency and regions as weight of feature contrast, obtaining initial 3D saliency by adding the weight of the feature contrast; enhancing the initial 3D saliency by 2D saliency and central bias weight.

Imagery captured by an autonomous robot is analyzed to discern digital watermark patterns. In some embodiments, identical but geometrically-inconsistent digital watermark patterns are discerned in an image frame, to aid in distinguishing multiple depicted instances of a particular item. In other embodiments, actions of the robot are controlled or altered in accordance with image processing performed by the robot on a digital watermark pattern. The technology is particularly described in the context of retail stores in which the watermark patterns are encoded, e.g., on product packaging, shelving, and shelf labels. A great variety of other features and arrangements are also detailed.